From nobody Thu Apr 25 03:54:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1543275774713446.55905124968376; Mon, 26 Nov 2018 15:42:54 -0800 (PST) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 00032307DAB6; Mon, 26 Nov 2018 23:42:50 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F4234608C6; Mon, 26 Nov 2018 23:42:48 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2594A3F7CD; Mon, 26 Nov 2018 23:42:45 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wAQNcAbZ024814 for ; Mon, 26 Nov 2018 18:38:10 -0500 Received: by smtp.corp.redhat.com (Postfix) id DE47F60C47; Mon, 26 Nov 2018 23:38:10 +0000 (UTC) Received: from unknown4CEB42C824F4.redhat.com (ovpn-117-184.phx2.redhat.com [10.3.117.184]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9DFED60BE8 for ; Mon, 26 Nov 2018 23:38:07 +0000 (UTC) From: John Ferlan To: libvir-list@redhat.com Date: Mon, 26 Nov 2018 18:38:06 -0500 Message-Id: <20181126233806.28276-1-jferlan@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2] qemu: Add check for whether KVM nesting is enabled X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Mon, 26 Nov 2018 23:42:53 +0000 (UTC) Content-Type: text/plain; charset="utf-8" Support for nested KVM is handled via a kernel module configuration adjustment which if done after libvirtd is started and/or the last QEMU capabilities adjustment can result in the inability to start a guest and use nested KVM until the capabilities cache is invalidated. This is because without knowing, the CPU settings for a guest may not add the vmx=3Don to/for the guest config. Thus, let's fetch and save the setting during initialization and then when the capabilities are checked for various host related adjustments that could affect whether the capabilities cache is updated add a check whether the nested value was set for Intel, AMD, or s390 to force a refetch of the capabilities. Signed-off-by: John Ferlan --- v1 was part of an RFC: https://www.redhat.com/archives/libvir-list/2018-November/msg00494.html This patch alters that code slightly to add the check Marc Hartmayer requested for S390 and to use "kvm" in the API names and variables to make it clearer that it's not CapsIsNested but CapsKVMIsNested. If it's felt the new check should slide down further in virQEMUCapsIsValid then that's fine - just let me know what it should follow. src/qemu/qemu_capabilities.c | 45 ++++++++++++++++++++++++++++++++++++ src/qemu/qemu_capspriv.h | 2 ++ tests/qemucapabilitiestest.c | 3 +++ 3 files changed, 50 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index fde27010e4..c377733fe6 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -40,6 +40,7 @@ #include "virnodesuspend.h" #include "virnuma.h" #include "virhostcpu.h" +#include "virkmod.h" #include "qemu_monitor.h" #include "virstring.h" #include "qemu_hostdev.h" @@ -557,6 +558,7 @@ struct _virQEMUCaps { virObject parent; =20 bool usedQMP; + bool kvmIsNested; =20 char *binary; time_t ctime; @@ -1528,6 +1530,7 @@ virQEMUCapsPtr virQEMUCapsNewCopy(virQEMUCapsPtr qemu= Caps) return NULL; =20 ret->usedQMP =3D qemuCaps->usedQMP; + ret->kvmIsNested =3D qemuCaps->kvmIsNested; =20 if (VIR_STRDUP(ret->binary, qemuCaps->binary) < 0) goto error; @@ -3587,6 +3590,9 @@ virQEMUCapsLoadCache(virArch hostArch, virQEMUCapsInitHostCPUModel(qemuCaps, hostArch, VIR_DOMAIN_VIRT_KVM); virQEMUCapsInitHostCPUModel(qemuCaps, hostArch, VIR_DOMAIN_VIRT_QEMU); =20 + qemuCaps->kvmIsNested =3D virXPathBoolean("count(./kvmIsNested) > 0", + ctxt) > 0; + ret =3D 0; cleanup: VIR_FREE(str); @@ -3806,6 +3812,9 @@ virQEMUCapsFormatCache(virQEMUCapsPtr qemuCaps) if (qemuCaps->sevCapabilities) virQEMUCapsFormatSEVInfo(qemuCaps, &buf); =20 + if (qemuCaps->kvmIsNested) + virBufferAddLit(&buf, "\n"); + virBufferAdjustIndent(&buf, -2); virBufferAddLit(&buf, "\n"); =20 @@ -3846,6 +3855,30 @@ virQEMUCapsSaveFile(void *data, } =20 =20 +static bool +virQEMUCapsKVMIsNested(void) +{ + VIR_AUTOFREE(char *) kConfig =3D NULL; + + /* Intel, AMD, and s390 related checks */ + if ((kConfig =3D virKModConfig()) && + (strstr(kConfig, "kvm_intel nested=3D1") || + strstr(kConfig, "kvm_amd nested=3D1") || + strstr(kConfig, "kvm nested=3D1"))) + return true; + return false; +} + + +void +virQEMUCapsClearKVMIsNested(virQEMUCapsPtr qemuCaps) +{ + /* For qemucapabilitiestest to avoid printing the on + * hosts with nested set in the kernel */ + qemuCaps->kvmIsNested =3D false; +} + + static bool virQEMUCapsIsValid(void *data, void *privData) @@ -3854,6 +3887,7 @@ virQEMUCapsIsValid(void *data, virQEMUCapsCachePrivPtr priv =3D privData; bool kvmUsable; struct stat sb; + bool kvmIsNested; =20 if (!qemuCaps->binary) return true; @@ -3886,6 +3920,15 @@ virQEMUCapsIsValid(void *data, return false; } =20 + /* Check if someone changed the nested=3D{0|1} value for the kernel fr= om + * the previous time we checked. If so, then refresh the capabilities.= */ + kvmIsNested =3D virQEMUCapsKVMIsNested(); + if (kvmIsNested !=3D qemuCaps->kvmIsNested) { + VIR_WARN("changed kernel nested kvm value was %d", qemuCaps->kvmIs= Nested); + qemuCaps->kvmIsNested =3D kvmIsNested; + return false; + } + if (!virQEMUCapsGuestIsNative(priv->hostArch, qemuCaps->arch)) { VIR_DEBUG("Guest arch (%s) is not native to host arch (%s), " "skipping KVM-related checks", @@ -4472,6 +4515,8 @@ virQEMUCapsInitQMP(virQEMUCapsPtr qemuCaps, if (virQEMUCapsInitQMPMonitor(qemuCaps, cmd->mon) < 0) goto cleanup; =20 + qemuCaps->kvmIsNested =3D virQEMUCapsKVMIsNested(); + if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_KVM)) { virQEMUCapsInitQMPCommandAbort(cmd); if ((rc =3D virQEMUCapsInitQMPCommandRun(cmd, true)) !=3D 0) { diff --git a/src/qemu/qemu_capspriv.h b/src/qemu/qemu_capspriv.h index 8d1a40fe74..edfe2cd6f6 100644 --- a/src/qemu/qemu_capspriv.h +++ b/src/qemu/qemu_capspriv.h @@ -48,6 +48,8 @@ int virQEMUCapsInitQMPMonitor(virQEMUCapsPtr qemuCaps, qemuMonitorPtr mon); =20 +void virQEMUCapsClearKVMIsNested(virQEMUCapsPtr qemuCaps); + int virQEMUCapsInitQMPMonitorTCG(virQEMUCapsPtr qemuCaps, qemuMonitorPtr mon); diff --git a/tests/qemucapabilitiestest.c b/tests/qemucapabilitiestest.c index 8fe5a55e1d..90942c6fce 100644 --- a/tests/qemucapabilitiestest.c +++ b/tests/qemucapabilitiestest.c @@ -63,6 +63,9 @@ testQemuCaps(const void *opaque) qemuMonitorTestGetMonitor(mon)) < 0) goto cleanup; =20 + /* Don't apply what the host has... force clear for testing purposes */ + virQEMUCapsClearKVMIsNested(capsActual); + if (virQEMUCapsGet(capsActual, QEMU_CAPS_KVM)) { qemuMonitorResetCommandID(qemuMonitorTestGetMonitor(mon)); if (virQEMUCapsInitQMPMonitorTCG(capsActual, --=20 2.17.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list