From nobody Sat Oct 25 21:42:57 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521724701333503.204037465847; Thu, 22 Mar 2018 06:18:21 -0700 (PDT) Received: from localhost ([::1]:60767 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ez06m-0007X1-Fj for importer@patchew.org; Thu, 22 Mar 2018 09:18:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37846) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ez02p-00057E-1x for qemu-devel@nongnu.org; Thu, 22 Mar 2018 09:14:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ez02l-0000bg-Sg for qemu-devel@nongnu.org; Thu, 22 Mar 2018 09:14:15 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39480 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ez02l-0000aE-Mx for qemu-devel@nongnu.org; Thu, 22 Mar 2018 09:14:11 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3ED40401DEAB; Thu, 22 Mar 2018 13:14:07 +0000 (UTC) Received: from vitty.brq.redhat.com (unknown [10.43.2.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 09EF1215CDA7; Thu, 22 Mar 2018 13:14:05 +0000 (UTC) From: Vitaly Kuznetsov To: qemu-devel@nongnu.org Date: Thu, 22 Mar 2018 14:13:57 +0100 Message-Id: <20180322131358.15096-2-vkuznets@redhat.com> In-Reply-To: <20180322131358.15096-1-vkuznets@redhat.com> References: <20180322131358.15096-1-vkuznets@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 22 Mar 2018 13:14:07 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 22 Mar 2018 13:14:07 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'vkuznets@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v4 1/2] i386/kvm: add support for Hyper-V reenlightenment MSRs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Marcelo Tosatti , Roman Kagan , Eduardo Habkost , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" KVM recently gained support for Hyper-V Reenlightenment MSRs which are required to make KVM-on-Hyper-V enable TSC page clocksource to its guests when INVTSC is not passed to it (and it is not passed by default in Qemu as it effectively blocks migration). Signed-off-by: Vitaly Kuznetsov --- Changes since v3: - check cpu->hyperv_reenlightenment instead of has_msr_reenlightenment [Eduardo Habkost, Roman Kagan] --- target/i386/cpu.c | 4 +++- target/i386/cpu.h | 4 ++++ target/i386/hyperv-proto.h | 9 ++++++++- target/i386/kvm.c | 38 +++++++++++++++++++++++++++++++++++++- target/i386/machine.c | 24 ++++++++++++++++++++++++ 5 files changed, 76 insertions(+), 3 deletions(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 6bb4ce8719..02579f8234 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -407,7 +407,8 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS]= =3D { NULL /* hv_vpindex_access */, NULL /* hv_msr_reset_access */, NULL /* hv_msr_stats_access */, NULL /* hv_reftsc_access */, NULL /* hv_msr_idle_access */, NULL /* hv_msr_frequency_access= */, - NULL, NULL, NULL, NULL, + NULL /* hv_msr_debug_access */, NULL /* hv_msr_reenlightenment= _access */, + NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, @@ -4764,6 +4765,7 @@ static Property x86_cpu_properties[] =3D { DEFINE_PROP_BOOL("hv-runtime", X86CPU, hyperv_runtime, false), DEFINE_PROP_BOOL("hv-synic", X86CPU, hyperv_synic, false), DEFINE_PROP_BOOL("hv-stimer", X86CPU, hyperv_stimer, false), + DEFINE_PROP_BOOL("hv-reenlightenment", X86CPU, hyperv_reenlightenment,= false), DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true), DEFINE_PROP_BOOL("enforce", X86CPU, enforce_cpuid, false), DEFINE_PROP_BOOL("kvm", X86CPU, expose_kvm, true), diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 2e2bab5ff3..98eed72937 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1174,6 +1174,9 @@ typedef struct CPUX86State { uint64_t msr_hv_synic_sint[HV_SINT_COUNT]; uint64_t msr_hv_stimer_config[HV_STIMER_COUNT]; uint64_t msr_hv_stimer_count[HV_STIMER_COUNT]; + uint64_t msr_hv_reenlightenment_control; + uint64_t msr_hv_tsc_emulation_control; + uint64_t msr_hv_tsc_emulation_status; =20 uint64_t msr_rtit_ctrl; uint64_t msr_rtit_status; @@ -1296,6 +1299,7 @@ struct X86CPU { bool hyperv_runtime; bool hyperv_synic; bool hyperv_stimer; + bool hyperv_reenlightenment; bool check_cpuid; bool enforce_cpuid; bool expose_kvm; diff --git a/target/i386/hyperv-proto.h b/target/i386/hyperv-proto.h index cb4d7f2b7a..93352ebd2a 100644 --- a/target/i386/hyperv-proto.h +++ b/target/i386/hyperv-proto.h @@ -35,7 +35,7 @@ #define HV_RESET_AVAILABLE (1u << 7) #define HV_REFERENCE_TSC_AVAILABLE (1u << 9) #define HV_ACCESS_FREQUENCY_MSRS (1u << 11) - +#define HV_ACCESS_REENLIGHTENMENTS_CONTROL (1u << 13) =20 /* * HV_CPUID_FEATURES.EDX bits @@ -129,6 +129,13 @@ #define HV_X64_MSR_CRASH_CTL 0x40000105 #define HV_CRASH_CTL_NOTIFY (1ull << 63) =20 +/* + * Reenlightenment notification MSRs + */ +#define HV_X64_MSR_REENLIGHTENMENT_CONTROL 0x40000106 +#define HV_X64_MSR_TSC_EMULATION_CONTROL 0x40000107 +#define HV_X64_MSR_TSC_EMULATION_STATUS 0x40000108 + /* * Hypercall status code */ diff --git a/target/i386/kvm.c b/target/i386/kvm.c index d23fff12f5..75f4e1d69e 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -90,6 +90,7 @@ static bool has_msr_hv_runtime; static bool has_msr_hv_synic; static bool has_msr_hv_stimer; static bool has_msr_hv_frequencies; +static bool has_msr_hv_reenlightenment; static bool has_msr_xss; static bool has_msr_spec_ctrl; static bool has_msr_smi_count; @@ -583,7 +584,8 @@ static bool hyperv_enabled(X86CPU *cpu) cpu->hyperv_vpindex || cpu->hyperv_runtime || cpu->hyperv_synic || - cpu->hyperv_stimer); + cpu->hyperv_stimer || + cpu->hyperv_reenlightenment); } =20 static int kvm_arch_set_tsc_khz(CPUState *cs) @@ -654,6 +656,14 @@ static int hyperv_handle_properties(CPUState *cs) env->features[FEAT_HYPERV_EDX] |=3D HV_FREQUENCY_MSRS_AVAILABL= E; } } + if (cpu->hyperv_reenlightenment) { + if (!has_msr_hv_reenlightenment) { + fprintf(stderr, + "Hyper-V Reenlightenment is not supported by kernel\n"= ); + return -ENOSYS; + } + env->features[FEAT_HYPERV_EAX] |=3D HV_ACCESS_REENLIGHTENMENTS_CON= TROL; + } if (cpu->hyperv_crash && has_msr_hv_crash) { env->features[FEAT_HYPERV_EDX] |=3D HV_GUEST_CRASH_MSR_AVAILABLE; } @@ -1185,6 +1195,9 @@ static int kvm_get_supported_msrs(KVMState *s) case HV_X64_MSR_TSC_FREQUENCY: has_msr_hv_frequencies =3D true; break; + case HV_X64_MSR_REENLIGHTENMENT_CONTROL: + has_msr_hv_reenlightenment =3D true; + break; case MSR_IA32_SPEC_CTRL: has_msr_spec_ctrl =3D true; break; @@ -1748,6 +1761,14 @@ static int kvm_put_msrs(X86CPU *cpu, int level) kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC, env->msr_hv_tsc); } + if (cpu->hyperv_reenlightenment) { + kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONTROL, + env->msr_hv_reenlightenment_control); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTROL, + env->msr_hv_tsc_emulation_control); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_STATUS, + env->msr_hv_tsc_emulation_status); + } } if (cpu->hyperv_vapic) { kvm_msr_entry_add(cpu, HV_X64_MSR_APIC_ASSIST_PAGE, @@ -2109,6 +2130,12 @@ static int kvm_get_msrs(X86CPU *cpu) } if (cpu->hyperv_time) { kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC, 0); + + } + if (cpu->hyperv_reenlightenment) { + kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONTROL, 0); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTROL, 0); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_STATUS, 0); } if (has_msr_hv_crash) { int j; @@ -2367,6 +2394,15 @@ static int kvm_get_msrs(X86CPU *cpu) env->msr_hv_stimer_count[(index - HV_X64_MSR_STIMER0_COUNT)/2]= =3D msrs[i].data; break; + case HV_X64_MSR_REENLIGHTENMENT_CONTROL: + env->msr_hv_reenlightenment_control =3D msrs[i].data; + break; + case HV_X64_MSR_TSC_EMULATION_CONTROL: + env->msr_hv_tsc_emulation_control =3D msrs[i].data; + break; + case HV_X64_MSR_TSC_EMULATION_STATUS: + env->msr_hv_tsc_emulation_status =3D msrs[i].data; + break; case MSR_MTRRdefType: env->mtrr_deftype =3D msrs[i].data; break; diff --git a/target/i386/machine.c b/target/i386/machine.c index bd2d82e91b..fd99c0bbb4 100644 --- a/target/i386/machine.c +++ b/target/i386/machine.c @@ -713,6 +713,29 @@ static const VMStateDescription vmstate_msr_hyperv_sti= mer =3D { } }; =20 +static bool hyperv_reenlightenment_enable_needed(void *opaque) +{ + X86CPU *cpu =3D opaque; + CPUX86State *env =3D &cpu->env; + + return env->msr_hv_reenlightenment_control !=3D 0 || + env->msr_hv_tsc_emulation_control !=3D 0 || + env->msr_hv_tsc_emulation_status !=3D 0; +} + +static const VMStateDescription vmstate_msr_hyperv_reenlightenment =3D { + .name =3D "cpu/msr_hyperv_reenlightenment", + .version_id =3D 1, + .minimum_version_id =3D 1, + .needed =3D hyperv_reenlightenment_enable_needed, + .fields =3D (VMStateField[]) { + VMSTATE_UINT64(env.msr_hv_reenlightenment_control, X86CPU), + VMSTATE_UINT64(env.msr_hv_tsc_emulation_control, X86CPU), + VMSTATE_UINT64(env.msr_hv_tsc_emulation_status, X86CPU), + VMSTATE_END_OF_LIST() + } +}; + static bool avx512_needed(void *opaque) { X86CPU *cpu =3D opaque; @@ -1005,6 +1028,7 @@ VMStateDescription vmstate_x86_cpu =3D { &vmstate_msr_hyperv_runtime, &vmstate_msr_hyperv_synic, &vmstate_msr_hyperv_stimer, + &vmstate_msr_hyperv_reenlightenment, &vmstate_avx512, &vmstate_xss, &vmstate_tsc_khz, --=20 2.14.3 From nobody Sat Oct 25 21:42:57 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521724551746285.547347322968; Thu, 22 Mar 2018 06:15:51 -0700 (PDT) Received: from localhost ([::1]:60753 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ez04H-0005qc-Th for importer@patchew.org; Thu, 22 Mar 2018 09:15:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37886) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ez02t-000591-Mr for qemu-devel@nongnu.org; Thu, 22 Mar 2018 09:14:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ez02l-0000bW-SN for qemu-devel@nongnu.org; Thu, 22 Mar 2018 09:14:19 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45504 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ez02l-0000aD-N8 for qemu-devel@nongnu.org; Thu, 22 Mar 2018 09:14:11 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 85F848182D3A; Thu, 22 Mar 2018 13:14:08 +0000 (UTC) Received: from vitty.brq.redhat.com (unknown [10.43.2.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 712F5215CDA7; Thu, 22 Mar 2018 13:14:07 +0000 (UTC) From: Vitaly Kuznetsov To: qemu-devel@nongnu.org Date: Thu, 22 Mar 2018 14:13:58 +0100 Message-Id: <20180322131358.15096-3-vkuznets@redhat.com> In-Reply-To: <20180322131358.15096-1-vkuznets@redhat.com> References: <20180322131358.15096-1-vkuznets@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 22 Mar 2018 13:14:08 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 22 Mar 2018 13:14:08 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'vkuznets@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v4 2/2] i386/kvm: expose Hyper-V frequency MSRs with reenlightenment X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Marcelo Tosatti , Roman Kagan , Eduardo Habkost , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We can also expose Hyper-V frequency MSRs when reenlightenment feature is enabled and TSC frequency is known, Hyper-V on KVM will provide stable TSC page clocksources to its guests. Signed-off-by: Vitaly Kuznetsov --- - Expose frequency MSRs only when either INVTSC or Reenlightenment is provided [Paolo Bonzini] --- target/i386/kvm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 75f4e1d69e..2c3c19d690 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -651,7 +651,8 @@ static int hyperv_handle_properties(CPUState *cs) env->features[FEAT_HYPERV_EAX] |=3D HV_TIME_REF_COUNT_AVAILABLE; env->features[FEAT_HYPERV_EAX] |=3D HV_REFERENCE_TSC_AVAILABLE; =20 - if (has_msr_hv_frequencies && tsc_is_stable_and_known(env)) { + if (has_msr_hv_frequencies && env->tsc_khz && + (tsc_is_stable_and_known(env) || cpu->hyperv_reenlightenment))= { env->features[FEAT_HYPERV_EAX] |=3D HV_ACCESS_FREQUENCY_MSRS; env->features[FEAT_HYPERV_EDX] |=3D HV_FREQUENCY_MSRS_AVAILABL= E; } --=20 2.14.3