From nobody Tue May 14 22:19:36 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1657639915; cv=none; d=zohomail.com; s=zohoarc; b=ePs9etQeRbU/UcO41e7l82JEW1foXhEzLAg+UW43HK0m9Uf1n1uRGK26sN9McvSoFkT4V5qpalbt+hlcPEGB/hpmXupJ9o60xZv7+Dw7ZqCshHBn74p0HaRK3ffj5eBKx40KFcr5DYWtUf6OldPbDXaV/cY8gf0Lv12RHUJMtvs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1657639915; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=P2MtXK3uYXxinN8LKoPwnVhuceYSaSCrVx/kqPWa9Jc=; b=ld/VqUzn2rFhyfsIBt1aeW3cDBoubKF6cstts2n3SenPoXQyQqCVtx9WnJzpPD7t0OykVb1cSijzZzrMZi3VwuKkMPzQb9r35Ijx2ZBZC+p/14C6fKHbbYrJTokix3bmdK4DiUi4NcR6ukyfNXskPFYPnVEdCOLzS9NHf1ATUEU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1657639915955828.5118597356441; Tue, 12 Jul 2022 08:31:55 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-531-LNgKLYNKMxmB2VDdRu0nLw-1; Tue, 12 Jul 2022 11:31:51 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8686A8037AE; Tue, 12 Jul 2022 15:31:48 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 042CB2026D64; Tue, 12 Jul 2022 15:31:47 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C1815194705E; Tue, 12 Jul 2022 15:31:46 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E05E5194705E for ; Tue, 12 Jul 2022 15:31:45 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id AEF732026D07; Tue, 12 Jul 2022 15:31:45 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2F62F2024CC5; Tue, 12 Jul 2022 15:31:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657639914; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=P2MtXK3uYXxinN8LKoPwnVhuceYSaSCrVx/kqPWa9Jc=; b=aGyZqoOBtz+n3x12fG0eW6Xdm1RTN2zPn7i5Iu4cKblXAjVuEHdhmiPo8KvsIFwJ4q0lwZ VLlWIMivJZ7PwbDUYaNHM/lTa30cgMidoE2KbR9ar4EdoPia0RdjqiAnU6jP2Uf+x4KfVx rJ2Tw5u3ykb2WI0In66VhEaYm5oQzqk= X-MC-Unique: LNgKLYNKMxmB2VDdRu0nLw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Subject: [libvirt PATCH] docs: add info about factors affecting CPU compatibility Date: Tue, 12 Jul 2022 16:31:43 +0100 Message-Id: <20220712153143.1582774-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1657639916878100001 While libvirt solves most of the problem of ensuring compatibility, when there is incompatibility it can be hard for users to track down the cause. Everything knows to check the physical CPU model, but there are a surprisingly large number of other factors influencing compatibility. Signed-off-by: Daniel P. Berrang=C3=A9 --- docs/drvqemu.rst | 80 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/docs/drvqemu.rst b/docs/drvqemu.rst index c174bdb4cf..3344df4fb6 100644 --- a/docs/drvqemu.rst +++ b/docs/drvqemu.rst @@ -416,6 +416,86 @@ following command should be run as root, prior to star= ting libvirtd libvirt will then place each virtual machine in a cgroup at ``/dev/cgroup/libvirt/qemu/$VMNAME/`` =20 + +Live migration compatibility +---------------------------- + +Many factors can affect the ability to live migrate a guest between a pair +of hosts. It is critical that when QEMU is started on the destination host, +the exposed guest machine ABI matches what was exposed by the existing QEMU +process on the source host. To facilitate this, when libvirt receives a +guest configuration document, it will attempt to expand any features that +were not specified, to ensure a stable guest machine ABI. Mostly this invo= lves +adding address information to all devices, and adding controllers to attach +the devices to. + +Certain features that affect the guest ABI, however, may only be known at = the +time the guest is started and can be influenced by features of the host OS +and its hardware. This means that even if the guest XML configuration is t= he +same, it may still be impossible to migrate the guest between two hosts. + +Migration CPU model compatibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The most common problems with migration compatibility surround the use of +the guest CPU ``host-model`` or ``host-passthrough`` modes. Both of these +modes attempt to expose the full host CPU featureset to the guest. The +``host-model`` mode attempts to expose as many features as possible while +retaining the ability to accurately check compatibility between hosts prior +to migration running. The ``host-passthrough`` mode attempts to expose the +host CPU as precisely as possible, but with the cost that it is not possib= le +for libvirt to check compatibility prior to migration. + +If using ``host-model`` the target host hardware and software deployment +must expose a superset of the features of the source host CPU. If using +``host-passthrough`` the target host CPU and software deployment must +always expose a superset of the fetures, however, it is further strongly +recommended that the source and destination hosts be identical in every +way. + +In both cases, there are a number of factors that will influence the CPU +features available to the guest + +* **Physical CPU model** - the core constraint on what features are availa= ble. + Check ``/proc/cpuinfo`` for CPU model name. +* **Firmware revision (BIOS/UEFI/etc)** - firmware updates may bundle micr= ocode + updates which arbitrarily add or remove CPU features, typically in respo= nse + to new hardware vulnerabilities. Check ``dmidecode`` for details on ``x8= 6`` + and ``aarch64`` platforms for firmware version, and ``/proc/cpuinfo`` for + associated microcode version (if not updated by the OS). +* **Firmware settings** - certain firmware settings can affect accessibili= ty of + features. For example, turning on/off SMT/HT not only affects the number + of logical CPUs available to the OS, but can indirectly influence other + factors such as the number of performance counters available for use. Ch= eck + the firmware specific configuration interface. +* **Host kernel version** - the host kernel software version may have a + need to block certain physical CPU features from use in the guest. It can + also emulate certain features that may not exist in the silicon, for exa= mple, + x2apic. Check ``uname -r`` output for kernel version. +* **Host kernel settings** - the kernel command line options can be used to + block certain physical CPU features from use in the guest, for example, + ``tsx=3Doff``, ``l1tf=3D...`` or ``nosmt``. Check ``/proc/cmdline`` and + ``/etc/modprobe.d/*.conf``. +* **microcode update version** - while the firmware will load the initial + microcode in to the CPU, the OS may ship packages providing newer microc= ode + updates since these can be deployed on a more timely manner than firmware + updates. These updates can arbitrarily load add or remove CPU features. + Check ``/proc/cpuinfo`` for microcode version. +* **QEMU version** - even when the kernel supports exposing a CPU feature = to + the guest, an update in the QEMU emulator version will be required to un= lock + its usage with a guest, except with ``host-passthrough``. Check the outp= ut + of ``$QEMU -version``. +* **libvirt version** - even when the kernel and QEMU support exposing a C= PU + feature to the guest, an update in the libvirt version will be required = to + unlock its usage with a guest, except with ``host-passthrough``. Check + ``virsh version``. + +The ``virsh capabilities`` output will provide information on the high lev= el +CPU model, its features, microcode version. Most of the time this will pro= vide +enough information to know whether the CPUs of two hosts will be compatibl= e. +If there are unexpected differences though, checking the above list of +influencing factors can reveal where the difference arises from. + Import and export of libvirt domain XML configs ----------------------------------------------- =20 --=20 2.36.1