From nobody Mon May 13 21:13:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1651607350; cv=none; d=zohomail.com; s=zohoarc; b=d6OgpWMRS1ec9SO+dNjlo+gUjtygJ46OmxlIzFNughFjaOnGKOF3C2LF0twReof7nXaJBsBJ0YzqXvRgmqbj/xx5mb3OCLV+MvqJZuLAgLA+wNGDLvlF5W9MKJ5Tv1Xz0F2Ur32ZvTbu3MvEptNxRV2or3RRh2Qf2agE6kl0jcA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1651607350; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=k4PPJ/yKOYbYfIlIYWzwGuf2tZLdYakDFvUgbJOwVv8=; b=Jy7QZnkgbJNjVBlcWNBKmdIBo5t7r0aj5FFe5QLPcc3vyIrxqKZopKZd3ESpeY3tY0MasNU1Ppefw52E1b+tSdMcD2ShgJtJ87rKp0lkE8Lp8eCdpV8V8K1Gb9wJilVbXqupswpOaydR9s8aR8x+PUcF4gt3PslihYdsZBOHr2o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1651607350643855.9335599405927; Tue, 3 May 2022 12:49:10 -0700 (PDT) Received: from localhost ([::1]:48892 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nlyWC-0000h6-Tc for importer@patchew.org; Tue, 03 May 2022 15:49:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38098) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nlyTF-0004Sp-0P for qemu-devel@nongnu.org; Tue, 03 May 2022 15:46:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:48419) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nlyT9-0001VJ-Kv for qemu-devel@nongnu.org; Tue, 03 May 2022 15:46:04 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-425-gNgyIkzYNq-aF4t7ecwzXA-1; Tue, 03 May 2022 11:04:01 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 208E11DD7E07 for ; Tue, 3 May 2022 14:49:09 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.192.119]) by smtp.corp.redhat.com (Postfix) with ESMTP id DF0E740CFD19; Tue, 3 May 2022 14:49:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651606945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=k4PPJ/yKOYbYfIlIYWzwGuf2tZLdYakDFvUgbJOwVv8=; b=W6JNBNIG6hcSrpno+DhfAHta3ONcstJexIA9F72QSaC0IXt7fqE9mO2DqdYB7AWkxdGC82 lkvzHt9g/bVoduW1f6qvyoNsPCPFG2ZU6ocdtpnJgakMgMXUOdlYEjjmTvqgyqmG+44DXf 2Nlr0UPMohNCkqLWvHMvLJNfUjdWkgQ= X-MC-Unique: gNgyIkzYNq-aF4t7ecwzXA-1 From: Vitaly Kuznetsov To: qemu-devel@nongnu.org, Paolo Bonzini Subject: [PATCH] i386: docs: Convert hyperv.txt to rST Date: Tue, 3 May 2022 16:49:06 +0200 Message-Id: <20220503144906.3618426-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=vkuznets@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1651607352953100001 Content-Type: text/plain; charset="utf-8" rSTify docs/hyperv.txt and link it from docs/system/target-i386.rst. Signed-off-by: Vitaly Kuznetsov --- - The patch is supposed to be applied on top of "[PATCH v3 0/5] i386: Enable newly introduced KVM Hyper-V enlightenments". --- docs/hyperv.txt | 289 ------------------------------------ docs/system/i386/hyperv.rst | 275 ++++++++++++++++++++++++++++++++++ docs/system/target-i386.rst | 1 + 3 files changed, 276 insertions(+), 289 deletions(-) delete mode 100644 docs/hyperv.txt create mode 100644 docs/system/i386/hyperv.rst diff --git a/docs/hyperv.txt b/docs/hyperv.txt deleted file mode 100644 index 9553e5c03c6b..000000000000 --- a/docs/hyperv.txt +++ /dev/null @@ -1,289 +0,0 @@ -Hyper-V Enlightenments -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - - -1. Description -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -In some cases when implementing a hardware interface in software is slow, = KVM -implements its own paravirtualized interfaces. This works well for Linux as -guest support for such features is added simultaneously with the feature i= tself. -It may, however, be hard-to-impossible to add support for these interfaces= to -proprietary OSes, namely, Microsoft Windows. - -KVM on x86 implements Hyper-V Enlightenments for Windows guests. These fea= tures -make Windows and Hyper-V guests think they're running on top of a Hyper-V -compatible hypervisor and use Hyper-V specific features. - - -2. Setup -=3D=3D=3D=3D=3D=3D=3D=3D=3D -No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In -QEMU, individual enlightenments can be enabled through CPU flags, e.g: - - qemu-system-x86_64 --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time= , ... - -Sometimes there are dependencies between enlightenments, QEMU is supposed = to -check that the supplied configuration is sane. - -When any set of the Hyper-V enlightenments is enabled, QEMU changes hyperv= isor -identification (CPUID 0x40000000..0x4000000A) to Hyper-V. KVM identificati= on -and features are kept in leaves 0x40000100..0x40000101. - - -3. Existing enlightenments -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D - -3.1. hv-relaxed -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -This feature tells guest OS to disable watchdog timeouts as it is running = on a -hypervisor. It is known that some Windows versions will do this even when = they -see 'hypervisor' CPU flag. - -3.2. hv-vapic -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Provides so-called VP Assist page MSR to guest allowing it to work with AP= IC -more efficiently. In particular, this enlightenment allows paravirtualized -(exit-less) EOI processing. - -3.3. hv-spinlocks=3Dxxx -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Enables paravirtualized spinlocks. The parameter indicates how many times -spinlock acquisition should be attempted before indicating the situation t= o the -hypervisor. A special value 0xffffffff indicates "never notify". - -3.4. hv-vpindex -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Provides HV_X64_MSR_VP_INDEX (0x40000002) MSR to the guest which has Virtu= al -processor index information. This enlightenment makes sense in conjunction= with -hv-synic, hv-stimer and other enlightenments which require the guest to kn= ow its -Virtual Processor indices (e.g. when VP index needs to be passed in a -hypercall). - -3.5. hv-runtime -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Provides HV_X64_MSR_VP_RUNTIME (0x40000010) MSR to the guest. The MSR keep= s the -virtual processor run time in 100ns units. This gives guest operating syst= em an -idea of how much time was 'stolen' from it (when the virtual CPU was preem= pted -to perform some other work). - -3.6. hv-crash -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Provides HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 (0x40000100..0x40000105)= and -HV_X64_MSR_CRASH_CTL (0x40000105) MSRs to the guest. These MSRs are writte= n to -by the guest when it crashes, HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 MSRs -contain additional crash information. This information is outputted in QEM= U log -and through QAPI. -Note: unlike under genuine Hyper-V, write to HV_X64_MSR_CRASH_CTL causes g= uest -to shutdown. This effectively blocks crash dump generation by Windows. - -3.7. hv-time -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Enables two Hyper-V-specific clocksources available to the guest: MSR-based -Hyper-V clocksource (HV_X64_MSR_TIME_REF_COUNT, 0x40000020) and Reference = TSC -page (enabled via MSR HV_X64_MSR_REFERENCE_TSC, 0x40000021). Both clocksou= rces -are per-guest, Reference TSC page clocksource allows for exit-less time st= amp -readings. Using this enlightenment leads to significant speedup of all tim= estamp -related operations. - -3.8. hv-synic -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Enables Hyper-V Synthetic interrupt controller - an extension of a local A= PIC. -When enabled, this enlightenment provides additional communication facilit= ies -to the guest: SynIC messages and Events. This is a pre-requisite for -implementing VMBus devices (not yet in QEMU). Additionally, this enlighten= ment -is needed to enable Hyper-V synthetic timers. SynIC is controlled through = MSRs -HV_X64_MSR_SCONTROL..HV_X64_MSR_EOM (0x40000080..0x40000084) and -HV_X64_MSR_SINT0..HV_X64_MSR_SINT15 (0x40000090..0x4000009F) - -Requires: hv-vpindex - -3.9. hv-stimer -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Enables Hyper-V synthetic timers. There are four synthetic timers per virt= ual -CPU controlled through HV_X64_MSR_STIMER0_CONFIG..HV_X64_MSR_STIMER3_COUNT -(0x400000B0..0x400000B7) MSRs. These timers can work either in single-shot= or -periodic mode. It is known that certain Windows versions revert to using H= PET -(or even RTC when HPET is unavailable) extensively when this enlightenment= is -not provided; this can lead to significant CPU consumption, even when virt= ual -CPU is idle. - -Requires: hv-vpindex, hv-synic, hv-time - -3.10. hv-tlbflush -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Enables paravirtualized TLB shoot-down mechanism. On x86 architecture, rem= ote -TLB flush procedure requires sending IPIs and waiting for other CPUs to pe= rform -local TLB flush. In virtualized environment some virtual CPUs may not even= be -scheduled at the time of the call and may not require flushing (or, flushi= ng -may be postponed until the virtual CPU is scheduled). hv-tlbflush enlighte= nment -implements TLB shoot-down through hypervisor enabling the optimization. - -Requires: hv-vpindex - -3.11. hv-ipi -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Enables paravirtualized IPI send mechanism. HvCallSendSyntheticClusterIpi -hypercall may target more than 64 virtual CPUs simultaneously, doing the s= ame -through APIC requires more than one access (and thus exit to the hyperviso= r). - -Requires: hv-vpindex - -3.12. hv-vendor-id=3Dxxx -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -This changes Hyper-V identification in CPUID 0x40000000.EBX-EDX from the d= efault -"Microsoft Hv". The parameter should be no longer than 12 characters. Acco= rding -to the specification, guests shouldn't use this information and it is unkn= own -if there is a Windows version which acts differently. -Note: hv-vendor-id is not an enlightenment and thus doesn't enable Hyper-V -identification when specified without some other enlightenment. - -3.13. hv-reset -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Provides HV_X64_MSR_RESET (0x40000003) MSR to the guest allowing it to res= et -itself by writing to it. Even when this MSR is enabled, it is not a recomm= ended -way for Windows to perform system reboot and thus it may not be used. - -3.14. hv-frequencies -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Provides HV_X64_MSR_TSC_FREQUENCY (0x40000022) and HV_X64_MSR_APIC_FREQUEN= CY -(0x40000023) allowing the guest to get its TSC/APIC frequencies without do= ing -measurements. - -3.15 hv-reenlightenment -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -The enlightenment is nested specific, it targets Hyper-V on KVM guests. Wh= en -enabled, it provides HV_X64_MSR_REENLIGHTENMENT_CONTROL (0x40000106), -HV_X64_MSR_TSC_EMULATION_CONTROL (0x40000107)and HV_X64_MSR_TSC_EMULATION_= STATUS -(0x40000108) MSRs allowing the guest to get notified when TSC frequency ch= anges -(only happens on migration) and keep using old frequency (through emulatio= n in -the hypervisor) until it is ready to switch to the new one. This, in conju= nction -with hv-frequencies, allows Hyper-V on KVM to pass stable clocksource (Ref= erence -TSC page) to its own guests. - -Note, KVM doesn't fully support re-enlightenment notifications and doesn't -emulate TSC accesses after migration so 'tsc-frequency=3D' CPU option also= has to -be specified to make migration succeed. The destination host has to either= have -the same TSC frequency or support TSC scaling CPU feature. - -Recommended: hv-frequencies - -3.16. hv-evmcs -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -The enlightenment is nested specific, it targets Hyper-V on KVM guests. Wh= en -enabled, it provides Enlightened VMCS version 1 feature to the guest. The = feature -implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V) -hypervisors making L2 exits to the hypervisor faster. The feature is Intel= -only. -Note: some virtualization features (e.g. Posted Interrupts) are disabled w= hen -hv-evmcs is enabled. It may make sense to measure your nested workload wit= h and -without the feature to find out if enabling it is beneficial. - -Requires: hv-vapic - -3.17. hv-stimer-direct -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Hyper-V specification allows synthetic timer operation in two modes: "clas= sic", -when expiration event is delivered as SynIC message and "direct", when the= event -is delivered via normal interrupt. It is known that nested Hyper-V can only -use synthetic timers in direct mode and thus 'hv-stimer-direct' needs to be -enabled. - -Requires: hv-vpindex, hv-synic, hv-time, hv-stimer - -3.18. hv-avic (hv-apicv) -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -The enlightenment allows to use Hyper-V SynIC with hardware APICv/AVIC ena= bled. -Normally, Hyper-V SynIC disables these hardware feature and suggests the g= uest -to use paravirtualized AutoEOI feature. -Note: enabling this feature on old hardware (without APICv/AVIC support) m= ay -have negative effect on guest's performance. - -3.19. hv-no-nonarch-coresharing=3Don/off/auto -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -This enlightenment tells guest OS that virtual processors will never share= a -physical core unless they are reported as sibling SMT threads. This inform= ation -is required by Windows and Hyper-V guests to properly mitigate SMT related= CPU -vulnerabilities. -When the option is set to 'auto' QEMU will enable the feature only when KVM -reports that non-architectural coresharing is impossible, this means that -hyper-threading is not supported or completely disabled on the host. This -setting also prevents migration as SMT settings on the destination may dif= fer. -When the option is set to 'on' QEMU will always enable the feature, regard= less -of host setup. To keep guests secure, this can only be used in conjunction= with -exposing correct vCPU topology and vCPU pinning. - -3.20. hv-version-id-{build,major,minor,spack,sbranch,snumber} -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -This changes Hyper-V version identification in CPUID 0x40000002.EAX-EDX fr= om the -default (WS2016). -- hv-version-id-build sets 'Build Number' (32 bits) -- hv-version-id-major sets 'Major Version' (16 bits) -- hv-version-id-minor sets 'Minor Version' (16 bits) -- hv-version-id-spack sets 'Service Pack' (32 bits) -- hv-version-id-sbranch sets 'Service Branch' (8 bits) -- hv-version-id-snumber sets 'Service Number' (24 bits) - -Note: hv-version-id-* are not enlightenments and thus don't enable Hyper-V -identification when specified without any other enlightenments. - -3.21. hv-emsr-bitmap -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -The enlightenment is nested specific, it targets Hyper-V on KVM guests. Wh= en -enabled, it allows L0 (KVM) and L1 (Hyper-V) hypervisors to collaborate to -avoid unnecessary updates to L2 MSR-Bitmap upon vmexits. While the protoco= l is -supported for both VMX (Intel) and SVM (AMD), the VMX implementation requi= res -Enlightened VMCS ('hv-evmcs') feature to also be enabled. - -Recommended: hv-evmcs (Intel) - -3.22. hv-xmm-input -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Hyper-V specification allows to pass parameters for certain hypercalls usi= ng XMM -registers ("XMM Fast Hypercall Input"). When the feature is in use, it all= ows -for faster hypercalls processing as KVM can avoid reading guest's memory. - -3.23. hv-tlbflush-ext -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Allow for extended GVA ranges to be passed to Hyper-V TLB flush hypercalls -(HvFlushVirtualAddressList/HvFlushVirtualAddressListEx). - -Requires: hv-tlbflush - -3.24. hv-tlbflush-direct -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D -The enlightenment is nested specific, it targets Hyper-V on KVM guests. Wh= en -enabled, it allows L0 (KVM) to directly handle TLB flush hypercalls from L2 -guest without the need to exit to L1 (Hyper-V) hypervisor. While the featu= re is -supported for both VMX (Intel) and SVM (AMD), the VMX implementation requi= res -Enlightened VMCS ('hv-evmcs') feature to also be enabled. - -Requires: hv-vapic -Recommended: hv-evmcs (Intel) - -4. Supplementary features -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D - -4.1. hv-passthrough -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -In some cases (e.g. during development) it may make sense to use QEMU in -'pass-through' mode and give Windows guests all enlightenments currently -supported by KVM. This pass-through mode is enabled by "hv-passthrough" CPU -flag. -Note: "hv-passthrough" flag only enables enlightenments which are known to= QEMU -(have corresponding "hv-*" flag) and copies "hv-spinlocks=3D"/"hv-vendor-i= d=3D" -values from KVM to QEMU. "hv-passthrough" overrides all other "hv-*" setti= ngs on -the command line. Also, enabling this flag effectively prevents migration = as the -list of enabled enlightenments may differ between target and destination h= osts. - -4.2. hv-enforce-cpuid -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -By default, KVM allows the guest to use all currently supported Hyper-V -enlightenments when Hyper-V CPUID interface was exposed, regardless of if -some features were not announced in guest visible CPUIDs. 'hv-enforce-cpui= d' -feature alters this behavior and only allows the guest to use exposed Hype= r-V -enlightenments. - - -5. Useful links -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Hyper-V Top Level Functional specification and other information: -https://github.com/MicrosoftDocs/Virtualization-Documentation diff --git a/docs/system/i386/hyperv.rst b/docs/system/i386/hyperv.rst new file mode 100644 index 000000000000..67fffbf06f48 --- /dev/null +++ b/docs/system/i386/hyperv.rst @@ -0,0 +1,275 @@ +Hyper-V Enlightenments +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + + +Description +----------- + +In some cases when implementing a hardware interface in software is slow, = KVM +implements its own paravirtualized interfaces. This works well for Linux as +guest support for such features is added simultaneously with the feature i= tself. +It may, however, be hard-to-impossible to add support for these interfaces= to +proprietary OSes, namely, Microsoft Windows. + +KVM on x86 implements Hyper-V Enlightenments for Windows guests. These fea= tures +make Windows and Hyper-V guests think they're running on top of a Hyper-V +compatible hypervisor and use Hyper-V specific features. + + +Setup +----- + +No Hyper-V enlightenments are enabled by default by either KVM or QEMU. In +QEMU, individual enlightenments can be enabled through CPU flags, e.g: + +.. parsed-literal:: + + |qemu_system| --enable-kvm --cpu host,hv_relaxed,hv_vpindex,hv_time, ... + +Sometimes there are dependencies between enlightenments, QEMU is supposed = to +check that the supplied configuration is sane. + +When any set of the Hyper-V enlightenments is enabled, QEMU changes hyperv= isor +identification (CPUID 0x40000000..0x4000000A) to Hyper-V. KVM identificati= on +and features are kept in leaves 0x40000100..0x40000101. + + +Existing enlightenments +----------------------- + +``hv-relaxed`` + This feature tells guest OS to disable watchdog timeouts as it is runnin= g on a + hypervisor. It is known that some Windows versions will do this even whe= n they + see 'hypervisor' CPU flag. + +``hv-vapic`` + Provides so-called VP Assist page MSR to guest allowing it to work with = APIC + more efficiently. In particular, this enlightenment allows paravirtualiz= ed + (exit-less) EOI processing. + +``hv-spinlocks`` =3D xxx + Enables paravirtualized spinlocks. The parameter indicates how many times + spinlock acquisition should be attempted before indicating the situation= to the + hypervisor. A special value 0xffffffff indicates "never notify". + +``hv-vpindex`` + Provides HV_X64_MSR_VP_INDEX (0x40000002) MSR to the guest which has Vir= tual + processor index information. This enlightenment makes sense in conjuncti= on with + hv-synic, hv-stimer and other enlightenments which require the guest to = know its + Virtual Processor indices (e.g. when VP index needs to be passed in a + hypercall). + +``hv-runtime`` + Provides HV_X64_MSR_VP_RUNTIME (0x40000010) MSR to the guest. The MSR ke= eps the + virtual processor run time in 100ns units. This gives guest operating sy= stem an + idea of how much time was 'stolen' from it (when the virtual CPU was pre= empted + to perform some other work). + +``hv-crash`` + Provides HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 (0x40000100..0x4000010= 5) and + HV_X64_MSR_CRASH_CTL (0x40000105) MSRs to the guest. These MSRs are writ= ten to + by the guest when it crashes, HV_X64_MSR_CRASH_P0..HV_X64_MSR_CRASH_P5 M= SRs + contain additional crash information. This information is outputted in Q= EMU log + and through QAPI. + Note: unlike under genuine Hyper-V, write to HV_X64_MSR_CRASH_CTL causes= guest + to shutdown. This effectively blocks crash dump generation by Windows. + +``hv-time`` + Enables two Hyper-V-specific clocksources available to the guest: MSR-ba= sed + Hyper-V clocksource (HV_X64_MSR_TIME_REF_COUNT, 0x40000020) and Referenc= e TSC + page (enabled via MSR HV_X64_MSR_REFERENCE_TSC, 0x40000021). Both clocks= ources + are per-guest, Reference TSC page clocksource allows for exit-less time = stamp + readings. Using this enlightenment leads to significant speedup of all t= imestamp + related operations. + +``hv-synic`` + Enables Hyper-V Synthetic interrupt controller - an extension of a local= APIC. + When enabled, this enlightenment provides additional communication facil= ities + to the guest: SynIC messages and Events. This is a pre-requisite for + implementing VMBus devices (not yet in QEMU). Additionally, this enlight= enment + is needed to enable Hyper-V synthetic timers. SynIC is controlled throug= h MSRs + HV_X64_MSR_SCONTROL..HV_X64_MSR_EOM (0x40000080..0x40000084) and + HV_X64_MSR_SINT0..HV_X64_MSR_SINT15 (0x40000090..0x4000009F) + + Requires: ``hv-vpindex`` + +``hv-stimer`` + Enables Hyper-V synthetic timers. There are four synthetic timers per vi= rtual + CPU controlled through HV_X64_MSR_STIMER0_CONFIG..HV_X64_MSR_STIMER3_COU= NT + (0x400000B0..0x400000B7) MSRs. These timers can work either in single-sh= ot or + periodic mode. It is known that certain Windows versions revert to using= HPET + (or even RTC when HPET is unavailable) extensively when this enlightenme= nt is + not provided; this can lead to significant CPU consumption, even when vi= rtual + CPU is idle. + + Requires: ``hv-vpindex``, ``hv-synic``, ``hv-time`` + +``hv-tlbflush`` + Enables paravirtualized TLB shoot-down mechanism. On x86 architecture, r= emote + TLB flush procedure requires sending IPIs and waiting for other CPUs to = perform + local TLB flush. In virtualized environment some virtual CPUs may not ev= en be + scheduled at the time of the call and may not require flushing (or, flus= hing + may be postponed until the virtual CPU is scheduled). hv-tlbflush enligh= tenment + implements TLB shoot-down through hypervisor enabling the optimization. + + Requires: ``hv-vpindex`` + +``hv-ipi`` + Enables paravirtualized IPI send mechanism. HvCallSendSyntheticClusterIpi + hypercall may target more than 64 virtual CPUs simultaneously, doing the= same + through APIC requires more than one access (and thus exit to the hypervi= sor). + + Requires: ``hv-vpindex`` + +``hv-vendor-id`` =3D xxx + This changes Hyper-V identification in CPUID 0x40000000.EBX-EDX from the= default + "Microsoft Hv". The parameter should be no longer than 12 characters. Ac= cording + to the specification, guests shouldn't use this information and it is un= known + if there is a Windows version which acts differently. + Note: hv-vendor-id is not an enlightenment and thus doesn't enable Hyper= -V + identification when specified without some other enlightenment. + +``hv-reset`` + Provides HV_X64_MSR_RESET (0x40000003) MSR to the guest allowing it to r= eset + itself by writing to it. Even when this MSR is enabled, it is not a reco= mmended + way for Windows to perform system reboot and thus it may not be used. + +``hv-frequencies`` + Provides HV_X64_MSR_TSC_FREQUENCY (0x40000022) and HV_X64_MSR_APIC_FREQU= ENCY + (0x40000023) allowing the guest to get its TSC/APIC frequencies without = doing + measurements. + +``hv-reenlightenment`` + The enlightenment is nested specific, it targets Hyper-V on KVM guests. = When + enabled, it provides HV_X64_MSR_REENLIGHTENMENT_CONTROL (0x40000106), + HV_X64_MSR_TSC_EMULATION_CONTROL (0x40000107)and HV_X64_MSR_TSC_EMULATIO= N_STATUS + (0x40000108) MSRs allowing the guest to get notified when TSC frequency = changes + (only happens on migration) and keep using old frequency (through emulat= ion in + the hypervisor) until it is ready to switch to the new one. This, in con= junction + with ``hv-frequencies``, allows Hyper-V on KVM to pass stable clocksource + (Reference TSC page) to its own guests. + + Note, KVM doesn't fully support re-enlightenment notifications and doesn= 't + emulate TSC accesses after migration so 'tsc-frequency=3D' CPU option al= so has to + be specified to make migration succeed. The destination host has to eith= er have + the same TSC frequency or support TSC scaling CPU feature. + + Recommended: ``hv-frequencies`` + +``hv-evmcs`` + The enlightenment is nested specific, it targets Hyper-V on KVM guests. = When + enabled, it provides Enlightened VMCS version 1 feature to the guest. Th= e feature + implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V) + hypervisors making L2 exits to the hypervisor faster. The feature is Int= el-only. + + Note: some virtualization features (e.g. Posted Interrupts) are disabled= when + hv-evmcs is enabled. It may make sense to measure your nested workload w= ith and + without the feature to find out if enabling it is beneficial. + + Requires: ``hv-vapic`` + +``hv-stimer-direct`` + Hyper-V specification allows synthetic timer operation in two modes: "cl= assic", + when expiration event is delivered as SynIC message and "direct", when t= he event + is delivered via normal interrupt. It is known that nested Hyper-V can o= nly + use synthetic timers in direct mode and thus ``hv-stimer-direct`` needs = to be + enabled. + + Requires: ``hv-vpindex``, ``hv-synic``, ``hv-time``, ``hv-stimer`` + +``hv-avic`` (``hv-apicv``) + The enlightenment allows to use Hyper-V SynIC with hardware APICv/AVIC e= nabled. + Normally, Hyper-V SynIC disables these hardware feature and suggests the= guest + to use paravirtualized AutoEOI feature. + Note: enabling this feature on old hardware (without APICv/AVIC support)= may + have negative effect on guest's performance. + +``hv-no-nonarch-coresharing`` =3D on/off/auto + This enlightenment tells guest OS that virtual processors will never sha= re a + physical core unless they are reported as sibling SMT threads. This info= rmation + is required by Windows and Hyper-V guests to properly mitigate SMT relat= ed CPU + vulnerabilities. + + When the option is set to 'auto' QEMU will enable the feature only when = KVM + reports that non-architectural coresharing is impossible, this means that + hyper-threading is not supported or completely disabled on the host. This + setting also prevents migration as SMT settings on the destination may d= iffer. + When the option is set to 'on' QEMU will always enable the feature, rega= rdless + of host setup. To keep guests secure, this can only be used in conjuncti= on with + exposing correct vCPU topology and vCPU pinning. + +``hv-version-id-build``, ``hv-version-id-major``, ``hv-version-id-minor``,= ``hv-version-id-spack``, ``hv-version-id-sbranch``, ``hv-version-id-snumbe= r`` + This changes Hyper-V version identification in CPUID 0x40000002.EAX-EDX = from the + default (WS2016). + + - ``hv-version-id-build`` sets 'Build Number' (32 bits) + - ``hv-version-id-major`` sets 'Major Version' (16 bits) + - ``hv-version-id-minor`` sets 'Minor Version' (16 bits) + - ``hv-version-id-spack`` sets 'Service Pack' (32 bits) + - ``hv-version-id-sbranch`` sets 'Service Branch' (8 bits) + - ``hv-version-id-snumber`` sets 'Service Number' (24 bits) + + Note: hv-version-id-* are not enlightenments and thus don't enable Hyper= -V + identification when specified without any other enlightenments. + +``hv-emsr-bitmap`` + The enlightenment is nested specific, it targets Hyper-V on KVM guests. = When + enabled, it allows L0 (KVM) and L1 (Hyper-V) hypervisors to collaborate = to + avoid unnecessary updates to L2 MSR-Bitmap upon vmexits. While the proto= col is + supported for both VMX (Intel) and SVM (AMD), the VMX implementation req= uires + Enlightened VMCS (``hv-evmcs``) feature to also be enabled. + + Recommended: ``hv-evmcs`` (Intel) + +``hv-xmm-input`` + Hyper-V specification allows to pass parameters for certain hypercalls u= sing XMM + registers ("XMM Fast Hypercall Input"). When the feature is in use, it a= llows + for faster hypercalls processing as KVM can avoid reading guest's memory. + +``hv-tlbflush-ext`` + Allow for extended GVA ranges to be passed to Hyper-V TLB flush hypercal= ls + (HvFlushVirtualAddressList/HvFlushVirtualAddressListEx). + + Requires: ``hv-tlbflush`` + +``hv-tlbflush-direct`` + The enlightenment is nested specific, it targets Hyper-V on KVM guests. = When + enabled, it allows L0 (KVM) to directly handle TLB flush hypercalls from= L2 + guest without the need to exit to L1 (Hyper-V) hypervisor. While the fea= ture is + supported for both VMX (Intel) and SVM (AMD), the VMX implementation req= uires + Enlightened VMCS (``hv-evmcs``) feature to also be enabled. + + Requires: ``hv-vapic`` + + Recommended: ``hv-evmcs`` (Intel) + +Supplementary features +---------------------- + +``hv-passthrough`` + In some cases (e.g. during development) it may make sense to use QEMU in + 'pass-through' mode and give Windows guests all enlightenments currently + supported by KVM. This pass-through mode is enabled by "hv-passthrough" = CPU + flag. + + Note: ``hv-passthrough`` flag only enables enlightenments which are know= n to QEMU + (have corresponding 'hv-' flag) and copies ``hv-spinlocks`` and ``hv-ven= dor-id`` + values from KVM to QEMU. ``hv-passthrough`` overrides all other 'hv-' se= ttings on + the command line. Also, enabling this flag effectively prevents migratio= n as the + list of enabled enlightenments may differ between target and destination= hosts. + +``hv-enforce-cpuid`` + By default, KVM allows the guest to use all currently supported Hyper-V + enlightenments when Hyper-V CPUID interface was exposed, regardless of if + some features were not announced in guest visible CPUIDs. ``hv-enforce-c= puid`` + feature alters this behavior and only allows the guest to use exposed Hy= per-V + enlightenments. + + +Useful links +------------ +Hyper-V Top Level Functional specification and other information: + +- https://github.com/MicrosoftDocs/Virtualization-Documentation +- https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/= tlfs + diff --git a/docs/system/target-i386.rst b/docs/system/target-i386.rst index 96bf54889a82..e64c0130772d 100644 --- a/docs/system/target-i386.rst +++ b/docs/system/target-i386.rst @@ -26,6 +26,7 @@ Architectural features :maxdepth: 1 =20 i386/cpu + i386/hyperv i386/kvm-pv i386/sgx i386/amd-memory-encryption --=20 2.35.1