[v1] Enable hstateen bits lazily for the KVM RISC-V Guests

[PATCH 4/5] RISC-V: KVM: Enable envcfg and sstateen bits lazily

Posted by Atish Patra 7 months, 2 weeks ago

SENVCFG and SSTATEEN CSRs are controlled by HSENVCFG(62) and
SSTATEEN0(63) bits in hstateen. Enable them lazily at runtime
instead of bootime.

Signed-off-by: Atish Patra <atishp@rivosinc.com>
---
 arch/riscv/kvm/vcpu_insn.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/arch/riscv/kvm/vcpu_insn.c b/arch/riscv/kvm/vcpu_insn.c
index c46907bfe42f..ed6302b1992b 100644
--- a/arch/riscv/kvm/vcpu_insn.c
+++ b/arch/riscv/kvm/vcpu_insn.c
@@ -256,9 +256,37 @@ int kvm_riscv_vcpu_hstateen_lazy_enable(struct kvm_vcpu *vcpu, unsigned int csr_
 	return KVM_INSN_CONTINUE_SAME_SEPC;
 }
 
+static int kvm_riscv_vcpu_hstateen_enable_senvcfg(struct kvm_vcpu *vcpu,
+						  unsigned int csr_num,
+						  unsigned long *val,
+						  unsigned long new_val,
+						  unsigned long wr_mask)
+{
+	return kvm_riscv_vcpu_hstateen_lazy_enable(vcpu, csr_num, SMSTATEEN0_HSENVCFG);
+}
+
+static int kvm_riscv_vcpu_hstateen_enable_stateen(struct kvm_vcpu *vcpu,
+						  unsigned int csr_num,
+						  unsigned long *val,
+						  unsigned long new_val,
+						  unsigned long wr_mask)
+{
+	const unsigned long *isa = vcpu->arch.isa;
+
+	if (riscv_isa_extension_available(isa, SMSTATEEN))
+		return kvm_riscv_vcpu_hstateen_lazy_enable(vcpu, csr_num, SMSTATEEN0_SSTATEEN0);
+	else
+		return KVM_INSN_EXIT_TO_USER_SPACE;
+}
+
+#define KVM_RISCV_VCPU_STATEEN_CSR_FUNCS \
+{ .base = CSR_SENVCFG,    .count = 1, .func = kvm_riscv_vcpu_hstateen_enable_senvcfg }, \
+{ .base = CSR_SSTATEEN0,  .count = 1, .func = kvm_riscv_vcpu_hstateen_enable_stateen },\
+
 static const struct csr_func csr_funcs[] = {
 	KVM_RISCV_VCPU_AIA_CSR_FUNCS
 	KVM_RISCV_VCPU_HPMCOUNTER_CSR_FUNCS
+	KVM_RISCV_VCPU_STATEEN_CSR_FUNCS
 	{ .base = CSR_SEED, .count = 1, .func = seed_csr_rmw },
 #ifdef CONFIG_32BIT
 	KVM_RISCV_VCPU_AIA_CSR_32BIT_FUNCS

-- 
2.43.0

Re: [PATCH 4/5] RISC-V: KVM: Enable envcfg and sstateen bits lazily

Posted by Radim Krčmář 7 months, 1 week ago

2025-05-05T14:39:29-07:00, Atish Patra <atishp@rivosinc.com>:
> SENVCFG and SSTATEEN CSRs are controlled by HSENVCFG(62) and
> SSTATEEN0(63) bits in hstateen. Enable them lazily at runtime
> instead of bootime.
>
> Signed-off-by: Atish Patra <atishp@rivosinc.com>
> ---
> diff --git a/arch/riscv/kvm/vcpu_insn.c b/arch/riscv/kvm/vcpu_insn.c
> @@ -256,9 +256,37 @@ int kvm_riscv_vcpu_hstateen_lazy_enable(struct kvm_vcpu *vcpu, unsigned int csr_
>  	return KVM_INSN_CONTINUE_SAME_SEPC;
>  }
>  
> +static int kvm_riscv_vcpu_hstateen_enable_senvcfg(struct kvm_vcpu *vcpu,
> +						  unsigned int csr_num,
> +						  unsigned long *val,
> +						  unsigned long new_val,
> +						  unsigned long wr_mask)
> +{
> +	return kvm_riscv_vcpu_hstateen_lazy_enable(vcpu, csr_num, SMSTATEEN0_HSENVCFG);
> +}

Basically the same comments as for [1/5]:

Why don't we want to set the ENVCFG bit (62) unconditionally?

It would save us the trap on first access.  We don't get anything from
the trap, so it looks like a net negative to me.

> +
> +static int kvm_riscv_vcpu_hstateen_enable_stateen(struct kvm_vcpu *vcpu,
> +						  unsigned int csr_num,
> +						  unsigned long *val,
> +						  unsigned long new_val,
> +						  unsigned long wr_mask)
> +{
> +	const unsigned long *isa = vcpu->arch.isa;
> +
> +	if (riscv_isa_extension_available(isa, SMSTATEEN))
> +		return kvm_riscv_vcpu_hstateen_lazy_enable(vcpu, csr_num, SMSTATEEN0_SSTATEEN0);
> +	else
> +		return KVM_INSN_EXIT_TO_USER_SPACE;
> +}

The same argument applies to the SE0 bit (63) when the guest has the
sstateen extension.

KVM doesn't want to do anything other than stop trapping and reenter, so
it seems to me we could just not trap in the first place.

Thanks.

Re: [PATCH 4/5] RISC-V: KVM: Enable envcfg and sstateen bits lazily

Posted by Atish Patra 7 months, 1 week ago

On 5/8/25 6:32 AM, Radim Krčmář wrote:
> 2025-05-05T14:39:29-07:00, Atish Patra <atishp@rivosinc.com>:
>> SENVCFG and SSTATEEN CSRs are controlled by HSENVCFG(62) and
>> SSTATEEN0(63) bits in hstateen. Enable them lazily at runtime
>> instead of bootime.
>>
>> Signed-off-by: Atish Patra <atishp@rivosinc.com>
>> ---
>> diff --git a/arch/riscv/kvm/vcpu_insn.c b/arch/riscv/kvm/vcpu_insn.c
>> @@ -256,9 +256,37 @@ int kvm_riscv_vcpu_hstateen_lazy_enable(struct kvm_vcpu *vcpu, unsigned int csr_
>>   	return KVM_INSN_CONTINUE_SAME_SEPC;
>>   }
>>   
>> +static int kvm_riscv_vcpu_hstateen_enable_senvcfg(struct kvm_vcpu *vcpu,
>> +						  unsigned int csr_num,
>> +						  unsigned long *val,
>> +						  unsigned long new_val,
>> +						  unsigned long wr_mask)
>> +{
>> +	return kvm_riscv_vcpu_hstateen_lazy_enable(vcpu, csr_num, SMSTATEEN0_HSENVCFG);
>> +}
> Basically the same comments as for [1/5]:
>
> Why don't we want to set the ENVCFG bit (62) unconditionally?
>
> It would save us the trap on first access.  We don't get anything from
> the trap, so it looks like a net negative to me.

We want to lazy enablement is to make sure that hypervisor is aware of 
the what features
guest is using. We don't want to necessarily enable the architecture 
states for the guest if guest doesn't need it.

We need lazy enablement for CTR like features anyways. This will align 
all the the features controlled
by stateen in the same manner. The cost is just a single trap at the 
boot time.

IMO, it's fair trade off.
>> +
>> +static int kvm_riscv_vcpu_hstateen_enable_stateen(struct kvm_vcpu *vcpu,
>> +						  unsigned int csr_num,
>> +						  unsigned long *val,
>> +						  unsigned long new_val,
>> +						  unsigned long wr_mask)
>> +{
>> +	const unsigned long *isa = vcpu->arch.isa;
>> +
>> +	if (riscv_isa_extension_available(isa, SMSTATEEN))
>> +		return kvm_riscv_vcpu_hstateen_lazy_enable(vcpu, csr_num, SMSTATEEN0_SSTATEEN0);
>> +	else
>> +		return KVM_INSN_EXIT_TO_USER_SPACE;
>> +}
> The same argument applies to the SE0 bit (63) when the guest has the
> sstateen extension.
>
> KVM doesn't want to do anything other than stop trapping and reenter, so
> it seems to me we could just not trap in the first place.
>
> Thanks.

Re: [PATCH 4/5] RISC-V: KVM: Enable envcfg and sstateen bits lazily

Posted by Radim Krčmář 7 months, 1 week ago

2025-05-09T15:38:55-07:00, Atish Patra <atish.patra@linux.dev>:
> On 5/8/25 6:32 AM, Radim Krčmář wrote:
>> 2025-05-05T14:39:29-07:00, Atish Patra <atishp@rivosinc.com>:
>>> SENVCFG and SSTATEEN CSRs are controlled by HSENVCFG(62) and
>>> SSTATEEN0(63) bits in hstateen. Enable them lazily at runtime
>>> instead of bootime.
>>>
>>> Signed-off-by: Atish Patra <atishp@rivosinc.com>
>>> ---
>>> diff --git a/arch/riscv/kvm/vcpu_insn.c b/arch/riscv/kvm/vcpu_insn.c
>>> @@ -256,9 +256,37 @@ int kvm_riscv_vcpu_hstateen_lazy_enable(struct kvm_vcpu *vcpu, unsigned int csr_
>>>   	return KVM_INSN_CONTINUE_SAME_SEPC;
>>>   }
>>>   
>>> +static int kvm_riscv_vcpu_hstateen_enable_senvcfg(struct kvm_vcpu *vcpu,
>>> +						  unsigned int csr_num,
>>> +						  unsigned long *val,
>>> +						  unsigned long new_val,
>>> +						  unsigned long wr_mask)
>>> +{
>>> +	return kvm_riscv_vcpu_hstateen_lazy_enable(vcpu, csr_num, SMSTATEEN0_HSENVCFG);
>>> +}
>> Basically the same comments as for [1/5]:
>>
>> Why don't we want to set the ENVCFG bit (62) unconditionally?
>>
>> It would save us the trap on first access.  We don't get anything from
>> the trap, so it looks like a net negative to me.
>
> We want to lazy enablement is to make sure that hypervisor is aware of 
> the what features
> guest is using. We don't want to necessarily enable the architecture 
> states for the guest if guest doesn't need it.
>
> We need lazy enablement for CTR like features anyways. This will align 
> all the the features controlled
> by stateen in the same manner. The cost is just a single trap at the 
> boot time.
>
> IMO, it's fair trade off.

Yeah, as long as we are doing something with the information from the
trap.