1. Patchew
  2. linux
  3. View series

[PATCH] LoongArch: KVM: Mark hrtimer to expire in hard interrupt context

Huacai Chen posted 1 patch 1 month, 2 weeks ago 
arch/loongarch/kvm/timer.c | 7 ++++---
arch/loongarch/kvm/vcpu.c  | 2 +-
2 files changed, 5 insertions(+), 4 deletions(-)
[PATCH] LoongArch: KVM: Mark hrtimer to expire in hard interrupt context
Posted by Huacai Chen 1 month, 2 weeks ago 
Like commit 2c0d278f3293fc5 ("KVM: LAPIC: Mark hrtimer to expire in hard
interrupt context"), On PREEMPT_RT enabled kernels unmarked hrtimers are
moved into soft interrupt expiry mode by default.

While that's not a functional requirement for the KVM constant timer
emulation, it is a latency issue which can be avoided by marking the
timer so hard interrupt context expiry is enforced.

This fix a "scheduling while atomic" bug for PREEMPT_RT enabled kernels:

 BUG: scheduling while atomic: qemu-system-loo/1011/0x00000002
 Modules linked in: amdgpu rfkill nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ns
 CPU: 1 UID: 0 PID: 1011 Comm: qemu-system-loo Tainted: G        W          6.12.0-rc2+ #1774
 Tainted: [W]=WARN
 Hardware name: Loongson Loongson-3A5000-7A1000-1w-CRB/Loongson-LS3A5000-7A1000-1w-CRB, BIOS vUDK2018-LoongArch-V2.0.0-prebeta9 10/21/2022
 Stack : ffffffffffffffff 0000000000000000 9000000004e3ea38 9000000116744000
         90000001167475a0 0000000000000000 90000001167475a8 9000000005644830
         90000000058dc000 90000000058dbff8 9000000116747420 0000000000000001
         0000000000000001 6a613fc938313980 000000000790c000 90000001001c1140
         00000000000003fe 0000000000000001 000000000000000d 0000000000000003
         0000000000000030 00000000000003f3 000000000790c000 9000000116747830
         90000000057ef000 0000000000000000 9000000005644830 0000000000000004
         0000000000000000 90000000057f4b58 0000000000000001 9000000116747868
         900000000451b600 9000000005644830 9000000003a13998 0000000010000020
         00000000000000b0 0000000000000004 0000000000000000 0000000000071c1d
         ...
 Call Trace:
 [<9000000003a13998>] show_stack+0x38/0x180
 [<9000000004e3ea34>] dump_stack_lvl+0x84/0xc0
 [<9000000003a71708>] __schedule_bug+0x48/0x60
 [<9000000004e45734>] __schedule+0x1114/0x1660
 [<9000000004e46040>] schedule_rtlock+0x20/0x60
 [<9000000004e4e330>] rtlock_slowlock_locked+0x3f0/0x10a0
 [<9000000004e4f038>] rt_spin_lock+0x58/0x80
 [<9000000003b02d68>] hrtimer_cancel_wait_running+0x68/0xc0
 [<9000000003b02e30>] hrtimer_cancel+0x70/0x80
 [<ffff80000235eb70>] kvm_restore_timer+0x50/0x1a0 [kvm]
 [<ffff8000023616c8>] kvm_arch_vcpu_load+0x68/0x2a0 [kvm]
 [<ffff80000234c2d4>] kvm_sched_in+0x34/0x60 [kvm]
 [<9000000003a749a0>] finish_task_switch.isra.0+0x140/0x2e0
 [<9000000004e44a70>] __schedule+0x450/0x1660
 [<9000000004e45cb0>] schedule+0x30/0x180
 [<ffff800002354c70>] kvm_vcpu_block+0x70/0x120 [kvm]
 [<ffff800002354d80>] kvm_vcpu_halt+0x60/0x3e0 [kvm]
 [<ffff80000235b194>] kvm_handle_gspr+0x3f4/0x4e0 [kvm]
 [<ffff80000235f548>] kvm_handle_exit+0x1c8/0x260 [kvm]

Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
---
 arch/loongarch/kvm/timer.c | 7 ++++---
 arch/loongarch/kvm/vcpu.c  | 2 +-
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/arch/loongarch/kvm/timer.c b/arch/loongarch/kvm/timer.c
index 74a4b5c272d6..32dc213374be 100644
--- a/arch/loongarch/kvm/timer.c
+++ b/arch/loongarch/kvm/timer.c
@@ -161,10 +161,11 @@ static void _kvm_save_timer(struct kvm_vcpu *vcpu)
 	if (kvm_vcpu_is_blocking(vcpu)) {
 
 		/*
-		 * HRTIMER_MODE_PINNED is suggested since vcpu may run in
-		 * the same physical cpu in next time
+		 * HRTIMER_MODE_PINNED_HARD is suggested since vcpu may run in
+		 * the same physical cpu in next time, and the timer should run
+		 * in hardirq context even in the PREEMPT_RT case.
 		 */
-		hrtimer_start(&vcpu->arch.swtimer, expire, HRTIMER_MODE_ABS_PINNED);
+		hrtimer_start(&vcpu->arch.swtimer, expire, HRTIMER_MODE_ABS_PINNED_HARD);
 	}
 }
 
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index 0697b1064251..174734a23d0a 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -1457,7 +1457,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	vcpu->arch.vpid = 0;
 	vcpu->arch.flush_gpa = INVALID_GPA;
 
-	hrtimer_init(&vcpu->arch.swtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED);
+	hrtimer_init(&vcpu->arch.swtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED_HARD);
 	vcpu->arch.swtimer.function = kvm_swtimer_wakeup;
 
 	vcpu->arch.handle_exit = kvm_handle_exit;
-- 
2.43.5
Re: [PATCH] LoongArch: KVM: Mark hrtimer to expire in hard interrupt context
Posted by maobibo 1 month, 2 weeks ago 

On 2024/10/13 下午5:01, Huacai Chen wrote:
> Like commit 2c0d278f3293fc5 ("KVM: LAPIC: Mark hrtimer to expire in hard
> interrupt context"), On PREEMPT_RT enabled kernels unmarked hrtimers are
> moved into soft interrupt expiry mode by default.
> 
> While that's not a functional requirement for the KVM constant timer
> emulation, it is a latency issue which can be avoided by marking the
> timer so hard interrupt context expiry is enforced.
> 
> This fix a "scheduling while atomic" bug for PREEMPT_RT enabled kernels:
> 
>   BUG: scheduling while atomic: qemu-system-loo/1011/0x00000002
>   Modules linked in: amdgpu rfkill nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ns
>   CPU: 1 UID: 0 PID: 1011 Comm: qemu-system-loo Tainted: G        W          6.12.0-rc2+ #1774
>   Tainted: [W]=WARN
>   Hardware name: Loongson Loongson-3A5000-7A1000-1w-CRB/Loongson-LS3A5000-7A1000-1w-CRB, BIOS vUDK2018-LoongArch-V2.0.0-prebeta9 10/21/2022
>   Stack : ffffffffffffffff 0000000000000000 9000000004e3ea38 9000000116744000
>           90000001167475a0 0000000000000000 90000001167475a8 9000000005644830
>           90000000058dc000 90000000058dbff8 9000000116747420 0000000000000001
>           0000000000000001 6a613fc938313980 000000000790c000 90000001001c1140
>           00000000000003fe 0000000000000001 000000000000000d 0000000000000003
>           0000000000000030 00000000000003f3 000000000790c000 9000000116747830
>           90000000057ef000 0000000000000000 9000000005644830 0000000000000004
>           0000000000000000 90000000057f4b58 0000000000000001 9000000116747868
>           900000000451b600 9000000005644830 9000000003a13998 0000000010000020
>           00000000000000b0 0000000000000004 0000000000000000 0000000000071c1d
>           ...
>   Call Trace:
>   [<9000000003a13998>] show_stack+0x38/0x180
>   [<9000000004e3ea34>] dump_stack_lvl+0x84/0xc0
>   [<9000000003a71708>] __schedule_bug+0x48/0x60
>   [<9000000004e45734>] __schedule+0x1114/0x1660
>   [<9000000004e46040>] schedule_rtlock+0x20/0x60
>   [<9000000004e4e330>] rtlock_slowlock_locked+0x3f0/0x10a0
>   [<9000000004e4f038>] rt_spin_lock+0x58/0x80
>   [<9000000003b02d68>] hrtimer_cancel_wait_running+0x68/0xc0
>   [<9000000003b02e30>] hrtimer_cancel+0x70/0x80
>   [<ffff80000235eb70>] kvm_restore_timer+0x50/0x1a0 [kvm]
>   [<ffff8000023616c8>] kvm_arch_vcpu_load+0x68/0x2a0 [kvm]
>   [<ffff80000234c2d4>] kvm_sched_in+0x34/0x60 [kvm]
>   [<9000000003a749a0>] finish_task_switch.isra.0+0x140/0x2e0
>   [<9000000004e44a70>] __schedule+0x450/0x1660
>   [<9000000004e45cb0>] schedule+0x30/0x180
>   [<ffff800002354c70>] kvm_vcpu_block+0x70/0x120 [kvm]
>   [<ffff800002354d80>] kvm_vcpu_halt+0x60/0x3e0 [kvm]
>   [<ffff80000235b194>] kvm_handle_gspr+0x3f4/0x4e0 [kvm]
>   [<ffff80000235f548>] kvm_handle_exit+0x1c8/0x260 [kvm]
> 
> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
> ---
>   arch/loongarch/kvm/timer.c | 7 ++++---
>   arch/loongarch/kvm/vcpu.c  | 2 +-
>   2 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/loongarch/kvm/timer.c b/arch/loongarch/kvm/timer.c
> index 74a4b5c272d6..32dc213374be 100644
> --- a/arch/loongarch/kvm/timer.c
> +++ b/arch/loongarch/kvm/timer.c
> @@ -161,10 +161,11 @@ static void _kvm_save_timer(struct kvm_vcpu *vcpu)
>   	if (kvm_vcpu_is_blocking(vcpu)) {
>   
>   		/*
> -		 * HRTIMER_MODE_PINNED is suggested since vcpu may run in
> -		 * the same physical cpu in next time
> +		 * HRTIMER_MODE_PINNED_HARD is suggested since vcpu may run in
> +		 * the same physical cpu in next time, and the timer should run
> +		 * in hardirq context even in the PREEMPT_RT case.
>   		 */
> -		hrtimer_start(&vcpu->arch.swtimer, expire, HRTIMER_MODE_ABS_PINNED);
> +		hrtimer_start(&vcpu->arch.swtimer, expire, HRTIMER_MODE_ABS_PINNED_HARD);
>   	}
>   }
>   
> diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
> index 0697b1064251..174734a23d0a 100644
> --- a/arch/loongarch/kvm/vcpu.c
> +++ b/arch/loongarch/kvm/vcpu.c
> @@ -1457,7 +1457,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
>   	vcpu->arch.vpid = 0;
>   	vcpu->arch.flush_gpa = INVALID_GPA;
>   
> -	hrtimer_init(&vcpu->arch.swtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED);
> +	hrtimer_init(&vcpu->arch.swtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED_HARD);
>   	vcpu->arch.swtimer.function = kvm_swtimer_wakeup;
>   
>   	vcpu->arch.handle_exit = kvm_handle_exit;
> 
Reviewed-by: Bibo Mao <maobibo@loongson.cn>

Patchew 2.1-devel-b67e4c2

© 2016 - 2024 Red Hat, Inc.