From nobody Thu Apr 18 11:15:16 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1555346163; cv=none; d=zoho.com; s=zohoarc; b=WmwT4FW17roB7gWqNtIylRbB7r4D6ei8y0m2iNUgGLyEB6l+e8XO8pMiF2JoAzDQP+ttxG3+SSCd21YCz9I+PdPeSyN7OKhXXt4xs52H9RHH3/jQWtO/DYRs612xMbIeMBRvvkdfcUd89kLfKHey67CKl3OaxED1OjkY69NaV+s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555346163; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=FxjrHDKdVaBc3HMhnO5iFLHqrNe+uQgCYDbKDsw0NAE=; b=gnJARi3wyNCxJ7uuz3TSYpS/jFm9v8Gx7nmpMeaBMG/4eyc1v4jg8GI+pR4en3iZeSnHUdIGh4tAj+LkwKmRBwhvKdJ6jpuoUZjzk/nBnMEK39tO+xsEN13MlZemBFkJtvHjhJLIr/JXaLECkHG0o7R3VyyjwuHe5iW/AnMC238= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 15553461630951007.8157053684496; Mon, 15 Apr 2019 09:36:03 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BF76730ADBB2; Mon, 15 Apr 2019 16:36:00 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 86A4B6013B; Mon, 15 Apr 2019 16:36:00 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 36EF241F3F; Mon, 15 Apr 2019 16:36:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x3FGZrNW009999 for ; Mon, 15 Apr 2019 12:35:53 -0400 Received: by smtp.corp.redhat.com (Postfix) id A914B5D9D3; Mon, 15 Apr 2019 16:35:53 +0000 (UTC) Received: from moe.brq.redhat.com (unknown [10.43.2.30]) by smtp.corp.redhat.com (Postfix) with ESMTP id 328355D9CA for ; Mon, 15 Apr 2019 16:35:53 +0000 (UTC) From: Michal Privoznik To: libvir-list@redhat.com Date: Mon, 15 Apr 2019 18:35:48 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 1/2] qemu: Set up EMULATOR thread and cpuset.mems before exec()-ing qemu X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Mon, 15 Apr 2019 16:36:01 +0000 (UTC) Content-Type: text/plain; charset="utf-8" It's funny how this went unnoticed for such a long time. Long story short, if a domain is configured with VIR_DOMAIN_NUMATUNE_MEM_STRICT libvirt doesn't really honour that. This is because of 7e72ac787848 after which libvirt allowed qemu to allocate memory just anywhere and only after that it used some magic involving cpuset.memory_migrate and cpuset.mems to move the memory to desired NUMA nodes. This was done in order to work around some KVM bug where KVM would fail if there wasn't a DMA zone available on the NUMA node. Well, while the work around might stopped libvirt tickling the KVM bug it also caused a bug on libvirt side: if there is not enough memory on configured NUMA node(s) then any attempt to start a domain must fail. Because of the way we play with guest memory domains can start just happily. The solution is to move the child we've just forked into emulator cgroup, set up cpuset.mems and exec() qemu only after that. Signed-off-by: Michal Privoznik Reviewed-by: Martin Kletzander --- src/qemu/qemu_process.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index f773aa89b7..746db85631 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6653,6 +6653,10 @@ qemuProcessLaunch(virConnectPtr conn, if (qemuProcessInitCpuAffinity(vm) < 0) goto cleanup; =20 + VIR_DEBUG("Setting emulator tuning/settings"); + if (qemuProcessSetupEmulator(vm) < 0) + goto cleanup; + VIR_DEBUG("Setting cgroup for external devices (if required)"); if (qemuSetupCgroupForExtDevices(vm, driver) < 0) goto cleanup; @@ -6744,10 +6748,6 @@ qemuProcessLaunch(virConnectPtr conn, if (qemuProcessDetectIOThreadPIDs(driver, vm, asyncJob) < 0) goto cleanup; =20 - VIR_DEBUG("Setting emulator tuning/settings"); - if (qemuProcessSetupEmulator(vm) < 0) - goto cleanup; - VIR_DEBUG("Setting global CPU cgroup (if required)"); if (qemuSetupGlobalCpuCgroup(vm) < 0) goto cleanup; --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Thu Apr 18 11:15:16 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1555346159; cv=none; d=zoho.com; s=zohoarc; b=ML8h88xCL3k+aD0nshnsqGUZmSYAtS7f5vbpo9Foe9KcZvq49GsagRFXniTRFN0FDcUFJ+j8V8rXaXh7IsJy2WFxzCf8fYkTw2p5Op6BWe6NAYoq58Xty85e4YUycCGpBxIOhGv96Vr+0+pmuMQxqX4wkvDAfl+r+ebCrt7OETY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1555346159; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=bzy5XchECG0w273D6rvcvjhB9QCkolK9Bc/pvpUB4VE=; b=JIZrv25Kt5eGRJa0srX2GvU9lOVaWHqLu2/R3KBXSFjYAAM6AEM5zZDSoTgzgsuMDbi33T1+IOk76GxmIYixbc+BX3S9xStoLdIicoAP1Fxsvp915BI0Yfq2oJONjFnFCjP/0C4LkBIECS8Fy52RgH+k8ZjjN9AUb5K+G8Yfen8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1555346159460601.3716339523578; Mon, 15 Apr 2019 09:35:59 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CF3743082B40; Mon, 15 Apr 2019 16:35:57 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A8A7B608C1; Mon, 15 Apr 2019 16:35:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 64EA841F3D; Mon, 15 Apr 2019 16:35:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x3FGZswJ010006 for ; Mon, 15 Apr 2019 12:35:54 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7A7FA5D9D3; Mon, 15 Apr 2019 16:35:54 +0000 (UTC) Received: from moe.brq.redhat.com (unknown [10.43.2.30]) by smtp.corp.redhat.com (Postfix) with ESMTP id 024F25D9CA for ; Mon, 15 Apr 2019 16:35:53 +0000 (UTC) From: Michal Privoznik To: libvir-list@redhat.com Date: Mon, 15 Apr 2019 18:35:49 +0200 Message-Id: <84315cdc2f592769c5c5f6072b7c7e6fe5728274.1555346075.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v3 2/2] qemu_cgroup: Remove unused qemuSetupCpusetMems X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Mon, 15 Apr 2019 16:35:58 +0000 (UTC) Content-Type: text/plain; charset="utf-8" This function is not used anymore. Let's remove it. Signed-off-by: Michal Privoznik Reviewed-by: Martin Kletzander --- src/qemu/qemu_cgroup.c | 35 ----------------------------------- src/qemu/qemu_cgroup.h | 1 - 2 files changed, 36 deletions(-) diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index c23f0af2aa..ca76c4fdfa 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -824,41 +824,6 @@ qemuSetupDevicesCgroup(virDomainObjPtr vm) } =20 =20 -int -qemuSetupCpusetMems(virDomainObjPtr vm) -{ - virCgroupPtr cgroup_temp =3D NULL; - qemuDomainObjPrivatePtr priv =3D vm->privateData; - virDomainNumatuneMemMode mode; - char *mem_mask =3D NULL; - int ret =3D -1; - - if (!virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET= )) - return 0; - - if (virDomainNumatuneGetMode(vm->def->numa, -1, &mode) < 0 || - mode !=3D VIR_DOMAIN_NUMATUNE_MEM_STRICT) - return 0; - - if (virDomainNumatuneMaybeFormatNodeset(vm->def->numa, - priv->autoNodeset, - &mem_mask, -1) < 0) - goto cleanup; - - if (mem_mask) - if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_EMULATOR, 0, - false, &cgroup_temp) < 0 || - virCgroupSetCpusetMems(cgroup_temp, mem_mask) < 0) - goto cleanup; - - ret =3D 0; - cleanup: - VIR_FREE(mem_mask); - virCgroupFree(&cgroup_temp); - return ret; -} - - static int qemuSetupCpusetCgroup(virDomainObjPtr vm) { diff --git a/src/qemu/qemu_cgroup.h b/src/qemu/qemu_cgroup.h index dc6d173fce..88ad4a38b6 100644 --- a/src/qemu/qemu_cgroup.h +++ b/src/qemu/qemu_cgroup.h @@ -61,7 +61,6 @@ int qemuConnectCgroup(virDomainObjPtr vm); int qemuSetupCgroup(virDomainObjPtr vm, size_t nnicindexes, int *nicindexes); -int qemuSetupCpusetMems(virDomainObjPtr vm); int qemuSetupCgroupVcpuBW(virCgroupPtr cgroup, unsigned long long period, long long quota); --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list