From nobody Sat Oct 25 11:08:05 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521219753157564.127661345741; Fri, 16 Mar 2018 10:02:33 -0700 (PDT) Received: from localhost ([::1]:58590 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewskS-0006f7-61 for importer@patchew.org; Fri, 16 Mar 2018 13:02:32 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57095) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewsia-0005dK-Ik for qemu-devel@nongnu.org; Fri, 16 Mar 2018 13:00:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewsiX-0005Xp-Jg for qemu-devel@nongnu.org; Fri, 16 Mar 2018 13:00:36 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:33906 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewsiX-0005X8-Fe for qemu-devel@nongnu.org; Fri, 16 Mar 2018 13:00:33 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 11F7240704BD; Fri, 16 Mar 2018 17:00:28 +0000 (UTC) Received: from vitty.brq.redhat.com (unknown [10.43.2.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id B3C5B202322B; Fri, 16 Mar 2018 17:00:26 +0000 (UTC) From: Vitaly Kuznetsov To: Paolo Bonzini Date: Fri, 16 Mar 2018 18:00:19 +0100 Message-Id: <20180316170020.8212-2-vkuznets@redhat.com> In-Reply-To: <20180316170020.8212-1-vkuznets@redhat.com> References: <20180316170020.8212-1-vkuznets@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Fri, 16 Mar 2018 17:00:28 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Fri, 16 Mar 2018 17:00:28 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'vkuznets@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v2 1/2] i386/kvm: add support for Hyper-V reenlightenment MSRs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, Roman Kagan , Marcelo Tosatti , Eduardo Habkost , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" KVM recently gained support for Hyper-V Reenlightenment MSRs which are required to make KVM-on-Hyper-V enable TSC page clocksource to its guests when INVTSC is not passed to it (and it is not passed by default in Qemu as it effectively blocks migration). Signed-off-by: Vitaly Kuznetsov --- Changes since v1: - add vmstate_msr_hyperv_reenlightenment subsection to vmstate_x86_cpu [Paolo Bonzini] --- target/i386/cpu.h | 3 +++ target/i386/hyperv-proto.h | 9 ++++++++- target/i386/kvm.c | 33 +++++++++++++++++++++++++++++++++ target/i386/machine.c | 24 ++++++++++++++++++++++++ 4 files changed, 68 insertions(+), 1 deletion(-) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 2e2bab5ff3..0b1b556a56 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1174,6 +1174,9 @@ typedef struct CPUX86State { uint64_t msr_hv_synic_sint[HV_SINT_COUNT]; uint64_t msr_hv_stimer_config[HV_STIMER_COUNT]; uint64_t msr_hv_stimer_count[HV_STIMER_COUNT]; + uint64_t msr_hv_reenlightenment_control; + uint64_t msr_hv_tsc_emulation_control; + uint64_t msr_hv_tsc_emulation_status; =20 uint64_t msr_rtit_ctrl; uint64_t msr_rtit_status; diff --git a/target/i386/hyperv-proto.h b/target/i386/hyperv-proto.h index cb4d7f2b7a..93352ebd2a 100644 --- a/target/i386/hyperv-proto.h +++ b/target/i386/hyperv-proto.h @@ -35,7 +35,7 @@ #define HV_RESET_AVAILABLE (1u << 7) #define HV_REFERENCE_TSC_AVAILABLE (1u << 9) #define HV_ACCESS_FREQUENCY_MSRS (1u << 11) - +#define HV_ACCESS_REENLIGHTENMENTS_CONTROL (1u << 13) =20 /* * HV_CPUID_FEATURES.EDX bits @@ -129,6 +129,13 @@ #define HV_X64_MSR_CRASH_CTL 0x40000105 #define HV_CRASH_CTL_NOTIFY (1ull << 63) =20 +/* + * Reenlightenment notification MSRs + */ +#define HV_X64_MSR_REENLIGHTENMENT_CONTROL 0x40000106 +#define HV_X64_MSR_TSC_EMULATION_CONTROL 0x40000107 +#define HV_X64_MSR_TSC_EMULATION_STATUS 0x40000108 + /* * Hypercall status code */ diff --git a/target/i386/kvm.c b/target/i386/kvm.c index d23fff12f5..accf50eac3 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -90,6 +90,7 @@ static bool has_msr_hv_runtime; static bool has_msr_hv_synic; static bool has_msr_hv_stimer; static bool has_msr_hv_frequencies; +static bool has_msr_hv_reenlightenment; static bool has_msr_xss; static bool has_msr_spec_ctrl; static bool has_msr_smi_count; @@ -653,6 +654,11 @@ static int hyperv_handle_properties(CPUState *cs) env->features[FEAT_HYPERV_EAX] |=3D HV_ACCESS_FREQUENCY_MSRS; env->features[FEAT_HYPERV_EDX] |=3D HV_FREQUENCY_MSRS_AVAILABL= E; } + + if (has_msr_hv_reenlightenment) { + env->features[FEAT_HYPERV_EAX] |=3D + HV_ACCESS_REENLIGHTENMENTS_CONTROL; + } } if (cpu->hyperv_crash && has_msr_hv_crash) { env->features[FEAT_HYPERV_EDX] |=3D HV_GUEST_CRASH_MSR_AVAILABLE; @@ -1185,6 +1191,9 @@ static int kvm_get_supported_msrs(KVMState *s) case HV_X64_MSR_TSC_FREQUENCY: has_msr_hv_frequencies =3D true; break; + case HV_X64_MSR_REENLIGHTENMENT_CONTROL: + has_msr_hv_reenlightenment =3D true; + break; case MSR_IA32_SPEC_CTRL: has_msr_spec_ctrl =3D true; break; @@ -1747,6 +1756,15 @@ static int kvm_put_msrs(X86CPU *cpu, int level) if (cpu->hyperv_time) { kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC, env->msr_hv_tsc); + + if (has_msr_hv_reenlightenment) { + kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONT= ROL, + env->msr_hv_reenlightenment_control); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTRO= L, + env->msr_hv_tsc_emulation_control); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_STATUS, + env->msr_hv_tsc_emulation_status); + } } } if (cpu->hyperv_vapic) { @@ -2109,6 +2127,12 @@ static int kvm_get_msrs(X86CPU *cpu) } if (cpu->hyperv_time) { kvm_msr_entry_add(cpu, HV_X64_MSR_REFERENCE_TSC, 0); + + if (has_msr_hv_reenlightenment) { + kvm_msr_entry_add(cpu, HV_X64_MSR_REENLIGHTENMENT_CONTROL, 0); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_CONTROL, 0); + kvm_msr_entry_add(cpu, HV_X64_MSR_TSC_EMULATION_STATUS, 0); + } } if (has_msr_hv_crash) { int j; @@ -2367,6 +2391,15 @@ static int kvm_get_msrs(X86CPU *cpu) env->msr_hv_stimer_count[(index - HV_X64_MSR_STIMER0_COUNT)/2]= =3D msrs[i].data; break; + case HV_X64_MSR_REENLIGHTENMENT_CONTROL: + env->msr_hv_reenlightenment_control =3D msrs[i].data; + break; + case HV_X64_MSR_TSC_EMULATION_CONTROL: + env->msr_hv_tsc_emulation_control =3D msrs[i].data; + break; + case HV_X64_MSR_TSC_EMULATION_STATUS: + env->msr_hv_tsc_emulation_status =3D msrs[i].data; + break; case MSR_MTRRdefType: env->mtrr_deftype =3D msrs[i].data; break; diff --git a/target/i386/machine.c b/target/i386/machine.c index bd2d82e91b..fd99c0bbb4 100644 --- a/target/i386/machine.c +++ b/target/i386/machine.c @@ -713,6 +713,29 @@ static const VMStateDescription vmstate_msr_hyperv_sti= mer =3D { } }; =20 +static bool hyperv_reenlightenment_enable_needed(void *opaque) +{ + X86CPU *cpu =3D opaque; + CPUX86State *env =3D &cpu->env; + + return env->msr_hv_reenlightenment_control !=3D 0 || + env->msr_hv_tsc_emulation_control !=3D 0 || + env->msr_hv_tsc_emulation_status !=3D 0; +} + +static const VMStateDescription vmstate_msr_hyperv_reenlightenment =3D { + .name =3D "cpu/msr_hyperv_reenlightenment", + .version_id =3D 1, + .minimum_version_id =3D 1, + .needed =3D hyperv_reenlightenment_enable_needed, + .fields =3D (VMStateField[]) { + VMSTATE_UINT64(env.msr_hv_reenlightenment_control, X86CPU), + VMSTATE_UINT64(env.msr_hv_tsc_emulation_control, X86CPU), + VMSTATE_UINT64(env.msr_hv_tsc_emulation_status, X86CPU), + VMSTATE_END_OF_LIST() + } +}; + static bool avx512_needed(void *opaque) { X86CPU *cpu =3D opaque; @@ -1005,6 +1028,7 @@ VMStateDescription vmstate_x86_cpu =3D { &vmstate_msr_hyperv_runtime, &vmstate_msr_hyperv_synic, &vmstate_msr_hyperv_stimer, + &vmstate_msr_hyperv_reenlightenment, &vmstate_avx512, &vmstate_xss, &vmstate_tsc_khz, --=20 2.14.3 From nobody Sat Oct 25 11:08:05 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521219749472446.88693892233925; Fri, 16 Mar 2018 10:02:29 -0700 (PDT) Received: from localhost ([::1]:58589 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewskO-0006bo-N0 for importer@patchew.org; Fri, 16 Mar 2018 13:02:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57117) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewsid-0005dS-0t for qemu-devel@nongnu.org; Fri, 16 Mar 2018 13:00:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewsiX-0005Xc-J0 for qemu-devel@nongnu.org; Fri, 16 Mar 2018 13:00:39 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:55936 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewsiX-0005X9-Fg for qemu-devel@nongnu.org; Fri, 16 Mar 2018 13:00:33 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 582B7D1449; Fri, 16 Mar 2018 17:00:29 +0000 (UTC) Received: from vitty.brq.redhat.com (unknown [10.43.2.155]) by smtp.corp.redhat.com (Postfix) with ESMTP id 443C62029296; Fri, 16 Mar 2018 17:00:28 +0000 (UTC) From: Vitaly Kuznetsov To: Paolo Bonzini Date: Fri, 16 Mar 2018 18:00:20 +0100 Message-Id: <20180316170020.8212-3-vkuznets@redhat.com> In-Reply-To: <20180316170020.8212-1-vkuznets@redhat.com> References: <20180316170020.8212-1-vkuznets@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 16 Mar 2018 17:00:29 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 16 Mar 2018 17:00:29 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'vkuznets@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v2 2/2] i386/kvm: lower requirements for Hyper-V frequency MSRs exposure X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, Roman Kagan , Marcelo Tosatti , Eduardo Habkost , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Requiring tsc_is_stable_and_known() is too restrictive: even without INVTCS nested Hyper-V-on-KVM enables TSC pages for its guests e.g. when Reenlightenment MSRs are present. Presence of frequency MSRs doesn't mean these frequencies are stable, it just means they're available for reading. Signed-off-by: Vitaly Kuznetsov --- target/i386/kvm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index accf50eac3..a6d1210f46 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -650,7 +650,7 @@ static int hyperv_handle_properties(CPUState *cs) env->features[FEAT_HYPERV_EAX] |=3D HV_TIME_REF_COUNT_AVAILABLE; env->features[FEAT_HYPERV_EAX] |=3D HV_REFERENCE_TSC_AVAILABLE; =20 - if (has_msr_hv_frequencies && tsc_is_stable_and_known(env)) { + if (has_msr_hv_frequencies && env->tsc_khz) { env->features[FEAT_HYPERV_EAX] |=3D HV_ACCESS_FREQUENCY_MSRS; env->features[FEAT_HYPERV_EDX] |=3D HV_FREQUENCY_MSRS_AVAILABL= E; } --=20 2.14.3