From nobody Wed May 15 19:23:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655992704; cv=none; d=zohomail.com; s=zohoarc; b=SrUmoUxh9Dr0GmXp1fQAa180B1t7r/i1VZji9VFSBW9ZOLW/Vhs8JATSOPZWjzJg8RQwr4PTzGOZ657VkulIS4KRLubfT7wFQxDg9/yTi6FUvimwSvZ8FkiiZl+yjOKQ6R5TZWgiMmmt98ppOIuaiXUENqmW/NKUXQ9Qab/r+Vo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655992704; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VWMAI2Xe9Tlcy2R9cfsgJkId5mZ/WWq7HCcoEbHg8fs=; b=crOv28L6JnD04M0EoOzinQ2oT24/2+vgvOSYQEc979A0SWzSKEHpl5znamaRa1BEvoOcsFJwEEFTh8kD8Zc1Ac6ZisohetrftSG/zl8yvdyuu8z/ypd6rQXHjhWpj4b/TGRz7T4hOrfS+KlOpu9WFHLEO2ZwFksNAvnG683HTds= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1655992704114766.26126943623; Thu, 23 Jun 2022 06:58:24 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-398-caXptwG-MhOgitl4ZiN4vQ-1; Thu, 23 Jun 2022 09:58:21 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CF57E3816845; Thu, 23 Jun 2022 13:58:19 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB6B018EB2; Thu, 23 Jun 2022 13:58:19 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 31383194B950; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 803CC194B954 for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 5C0BA40D2962; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.227]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2086540D282F for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 174B2241E3D; Thu, 23 Jun 2022 15:58:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655992703; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=VWMAI2Xe9Tlcy2R9cfsgJkId5mZ/WWq7HCcoEbHg8fs=; b=Rs5H0/YL+kUcBqS15O2zs5TEreQ3A/leLeI+xja1iPWYdPfAFsXCd95cZ9/22Dj0vBgYL/ G57jttKdtFHNVIj2QJaaV8SKoLdt1wamdifpQgoqzkZev6BYTExguG0x9DKSBNEOReM/Nt 7xon0S5xkHjsMP1ICcJSNwkQ4RCjNfw= X-MC-Unique: caXptwG-MhOgitl4ZiN4vQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jiri Denemark To: libvir-list@redhat.com Subject: [libvirt PATCH 1/7] qemu: Add qemuDomainSetMaxMemLock helper Date: Thu, 23 Jun 2022 15:58:06 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655992706303100002 Content-Type: text/plain; charset="utf-8" qemuDomainAdjustMaxMemLock combined computing the desired limit with applying it. This patch separates the code to apply a memory locking limit to a new qemuDomainSetMaxMemLock helper for better reusability. Signed-off-by: Jiri Denemark Reviewed-by: J=C3=A1n Tomko --- src/qemu/qemu_domain.c | 95 ++++++++++++++++++++++++++---------------- src/qemu/qemu_domain.h | 3 ++ 2 files changed, 61 insertions(+), 37 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 9769e3bb92..e363993739 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -9459,6 +9459,61 @@ qemuDomainGetMemLockLimitBytes(virDomainDef *def, } =20 =20 +/** + * qemuDomainSetMaxMemLock: + * @vm: domain + * @limit: the desired memory locking limit + * @origPtr: where to store (or load from) the original value of the limit + * + * Set the memory locking limit for @vm unless it's already big enough. If + * @origPtr is non-NULL, the original value of the limit will be store the= re + * and can be restored by calling this function with @limit =3D=3D 0. + * + * Returns: 0 on success, -1 otherwise. + */ +int +qemuDomainSetMaxMemLock(virDomainObj *vm, + unsigned long long limit, + unsigned long long *origPtr) +{ + unsigned long long current =3D 0; + + if (virProcessGetMaxMemLock(vm->pid, ¤t) < 0) + return -1; + + if (limit > 0) { + VIR_DEBUG("Requested memory lock limit: %llu", limit); + /* If the limit is already high enough, we can assume + * that some external process is taking care of managing + * process limits and we shouldn't do anything ourselves: + * we're probably running in a containerized environment + * where we don't have enough privilege anyway */ + if (current >=3D limit) { + VIR_DEBUG("Current limit %llu is big enough", current); + return 0; + } + + /* If this is the first time adjusting the limit, save the current + * value so that we can restore it once memory locking is no longer + * required */ + if (origPtr && *origPtr =3D=3D 0) + *origPtr =3D current; + } else { + /* Once memory locking is no longer required, we can restore the + * original, usually very low, limit. But only if we actually stor= ed + * the original limit before. */ + if (!origPtr || *origPtr =3D=3D 0) + return 0; + + limit =3D *origPtr; + *origPtr =3D 0; + VIR_DEBUG("Resetting memory lock limit back to %llu", limit); + } + + return virProcessSetMaxMemLock(vm->pid, limit); +} + + /** * qemuDomainAdjustMaxMemLock: * @vm: domain @@ -9480,43 +9535,9 @@ int qemuDomainAdjustMaxMemLock(virDomainObj *vm, bool forceVFIO) { - qemuDomainObjPrivate *priv =3D vm->privateData; - unsigned long long currentMemLock =3D 0; - unsigned long long desiredMemLock =3D 0; - - desiredMemLock =3D qemuDomainGetMemLockLimitBytes(vm->def, forceVFIO); - if (virProcessGetMaxMemLock(vm->pid, ¤tMemLock) < 0) - return -1; - - if (desiredMemLock > 0) { - if (currentMemLock < desiredMemLock) { - /* If this is the first time adjusting the limit, save the cur= rent - * value so that we can restore it once memory locking is no l= onger - * required */ - if (priv->originalMemlock =3D=3D 0) { - priv->originalMemlock =3D currentMemLock; - } - } else { - /* If the limit is already high enough, we can assume - * that some external process is taking care of managing - * process limits and we shouldn't do anything ourselves: - * we're probably running in a containerized environment - * where we don't have enough privilege anyway */ - desiredMemLock =3D 0; - } - } else { - /* Once memory locking is no longer required, we can restore the - * original, usually very low, limit */ - desiredMemLock =3D priv->originalMemlock; - priv->originalMemlock =3D 0; - } - - if (desiredMemLock > 0 && - virProcessSetMaxMemLock(vm->pid, desiredMemLock) < 0) { - return -1; - } - - return 0; + return qemuDomainSetMaxMemLock(vm, + qemuDomainGetMemLockLimitBytes(vm->def,= forceVFIO), + &QEMU_DOMAIN_PRIVATE(vm)->originalMemlo= ck); } =20 =20 diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index a87dfff1bb..6d35f61dfd 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -840,6 +840,9 @@ int qemuDomainAdjustMaxMemLock(virDomainObj *vm, bool forceVFIO); int qemuDomainAdjustMaxMemLockHostdev(virDomainObj *vm, virDomainHostdevDef *hostdev); +int qemuDomainSetMaxMemLock(virDomainObj *vm, + unsigned long long limit, + unsigned long long *origPtr); =20 int qemuDomainDefValidateMemoryHotplug(const virDomainDef *def, const virDomainMemoryDef *mem); --=20 2.35.1 From nobody Wed May 15 19:23:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655992714; cv=none; d=zohomail.com; s=zohoarc; b=B5grKRWgEEXrnIwX/PB0rgbWtjsEta2j7bONDk0WzziTokiu+Itmeb2rTPvz1KoLGFcTQwQo7xWtxIN+1EH8lotcNP+JJuWMppjQ8U/MH1Lq/QzR3E2Y3rrbIJtrWvNydeag9KctiF3XG4zBRP67PGNzuguBC3BIz0e8hg+Y8mE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655992714; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+XYTAjzc9d7cWhczld7O5McHsh2b7Kh3oY+g8nnrZ0k=; b=D6xlkQKnkhzgq3+BstP1ymGvnUvCvtPjtG6tclcyeqOsgxHfnMRuxAVC52Mvvw4JvNoLMdLUmqTecRi79pppAiul+4RrseArcP8rm+5NdiLaeZk7WjoifvWMP3QtMAN5ufTpmZQLtV4tGdBjMqsGILf1E8LRAPrtLhfs418Gjw0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 165599271445851.151753063332876; Thu, 23 Jun 2022 06:58:34 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638-XgYfh4RWMCqABMY0sy9EOQ-1; Thu, 23 Jun 2022 09:58:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1E2BF8037B7; Thu, 23 Jun 2022 13:58:26 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 087C217452; Thu, 23 Jun 2022 13:58:26 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 11DE4194B978; Thu, 23 Jun 2022 13:58:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 90CFB194B955 for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 78A742026D2D; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.227]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5959D2026D64 for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 18ED6241E3E; Thu, 23 Jun 2022 15:58:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655992713; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=+XYTAjzc9d7cWhczld7O5McHsh2b7Kh3oY+g8nnrZ0k=; b=YuTSej1cI+VcBtNHw7AIHSc30EvIR6ggUCz7svZ/oMG60AYFTi96mgKNKR9pVASyPbV5z5 xHHGAHjU0MtFdeuXTZlSjpVcl1hS4IJKFA0R8/1511kBksZfmXEhzIQmhF8Bjjunc/8Vk8 ihG7qcZwiIDiI/c76crijguDZBSZHUw= X-MC-Unique: XgYfh4RWMCqABMY0sy9EOQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jiri Denemark To: libvir-list@redhat.com Subject: [libvirt PATCH 2/7] qemu_migration: Use qemuDomainSetMaxMemLock Date: Thu, 23 Jun 2022 15:58:07 +0200 Message-Id: <483e31dec98cd525af8cb48ac2c718981c928e30.1655992592.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655992716323100001 Content-Type: text/plain; charset="utf-8" This helper will not try to set the limit if it is already big enough, which may be useful when libvirt daemon is running in a containerized environment and is not allowed to change memory locking limit. Signed-off-by: Jiri Denemark Reviewed-by: J=C3=A1n Tomko --- src/qemu/qemu_migration.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index d93e7e9c74..fe63f45629 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3181,7 +3181,7 @@ qemuMigrationDstPrepareActive(virQEMUDriver *driver, =20 if (STREQ_NULLABLE(protocol, "rdma") && vm->def->mem.hard_limit > 0 && - virProcessSetMaxMemLock(vm->pid, vm->def->mem.hard_limit << 10) < = 0) { + qemuDomainSetMaxMemLock(vm, vm->def->mem.hard_limit << 10, NULL) <= 0) { goto error; } =20 @@ -4615,7 +4615,7 @@ qemuMigrationSrcStart(virDomainObj *vm, case MIGRATION_DEST_HOST: if (STREQ(spec->dest.host.protocol, "rdma") && vm->def->mem.hard_limit > 0 && - virProcessSetMaxMemLock(vm->pid, vm->def->mem.hard_limit << 10= ) < 0) { + qemuDomainSetMaxMemLock(vm, vm->def->mem.hard_limit << 10, NUL= L) < 0) { return -1; } return qemuMonitorMigrateToHost(priv->mon, migrateFlags, --=20 2.35.1 From nobody Wed May 15 19:23:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655992705; cv=none; d=zohomail.com; s=zohoarc; b=dfXw/GVtw1BdFx27x4hwXvBnZE0zgeY+p582cPD+wTzbBqqBbcAmmYRNQ1irlF1/oeUs5RnD4RHmbec0IFESk0wbx9x6NNWrVnGUyID/CEioWWiqai06H+hVX3UKSQT8Wo39KCofEfA2uvfygz/dm9LvTHY/S3rQ9XSW2omnCko= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655992705; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EsReuu9UC2jY/sUC74d2W+LHFgiE892bR3OxPU3M64s=; b=QOvls57a8tU2hDtQp72b80bXoS7M7i8NoomZ5CT+fKl5R0hcqvlSD+5E82tUkwLhpSmIGUAaCOhPIR0cB61V4H7uXSdBcMKwBBTOXuf+uz+IC70QYWiP/ed/LjdOGE1tlOUsQsILMbWMExVar3gHosDmp8GmBsJY0eXlyMT1XaQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1655992705225720.5331240092589; Thu, 23 Jun 2022 06:58:25 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-283-MBqXXvJ5M4yYHjLnhxo_nw-1; Thu, 23 Jun 2022 09:58:20 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0522F3816843; Thu, 23 Jun 2022 13:58:18 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 217562166B26; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E0D9F194B952; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7AC9C194B950 for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 40AA6141510C; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.227]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 217081415111 for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 1A86E241E3F; Thu, 23 Jun 2022 15:58:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655992704; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=EsReuu9UC2jY/sUC74d2W+LHFgiE892bR3OxPU3M64s=; b=efoKcIFqk2SoXPS2PQOfqWc7q6LwW62xy7oCZXkL55XAlxReNsqASLjacVgDdJVXJnBBAk ZP1pIAFkMmjmXhTapYL6OyIm+7+mK0ipeJRo51KdcrXJgRZEM8xYzLSQyOhbmNI0upSKXh uSbAFmmLSwUTZrElHlkE8GKFfFasEAI= X-MC-Unique: MBqXXvJ5M4yYHjLnhxo_nw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jiri Denemark To: libvir-list@redhat.com Subject: [libvirt PATCH 3/7] qemu_migration: Restore original memory locking limit Date: Thu, 23 Jun 2022 15:58:08 +0200 Message-Id: <698a3e5ee7888148ddfcbf87d86c7b6c5b320ec0.1655992592.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655992706284100001 Content-Type: text/plain; charset="utf-8" For RDMA migration we update memory locking limit, but never set it back once migration finishes (on the destination host) or aborts (on the source host). Signed-off-by: Jiri Denemark Reviewed-by: J=C3=A1n Tomko --- src/qemu/qemu_domain.c | 12 ++++++++++++ src/qemu/qemu_domain.h | 3 +++ src/qemu/qemu_migration.c | 11 ++++++++--- 3 files changed, 23 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index e363993739..60ed358871 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -2413,6 +2413,11 @@ qemuDomainObjPrivateXMLFormat(virBuffer *buf, priv->originalMemlock); } =20 + if (priv->preMigrationMemlock > 0) { + virBufferAsprintf(buf, "%llu\n", + priv->preMigrationMemlock); + } + return 0; } =20 @@ -3139,6 +3144,13 @@ qemuDomainObjPrivateXMLParse(xmlXPathContextPtr ctxt, return -1; } =20 + if (virXPathULongLong("string(./preMigrationMemlock)", ctxt, + &priv->preMigrationMemlock) =3D=3D -2) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("failed to parse pre-migration memlock limit")); + return -1; + } + return 0; } =20 diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 6d35f61dfd..499ad03f91 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -140,6 +140,9 @@ struct _qemuDomainObjPrivate { int nbdPort; /* Port used for migration with NBD */ unsigned short migrationPort; int preMigrationState; + unsigned long long preMigrationMemlock; /* Original RLIMIT_MEMLOCK in = case + it was changed for the curr= ent + migration job. */ =20 virChrdevs *devs; =20 diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index fe63f45629..ffae5576d2 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3181,7 +3181,8 @@ qemuMigrationDstPrepareActive(virQEMUDriver *driver, =20 if (STREQ_NULLABLE(protocol, "rdma") && vm->def->mem.hard_limit > 0 && - qemuDomainSetMaxMemLock(vm, vm->def->mem.hard_limit << 10, NULL) <= 0) { + qemuDomainSetMaxMemLock(vm, vm->def->mem.hard_limit << 10, + &priv->preMigrationMemlock) < 0) { goto error; } =20 @@ -3945,6 +3946,7 @@ qemuMigrationSrcComplete(virQEMUDriver *driver, VIR_DOMAIN_EVENT_STOPPED_MIG= RATED); virObjectEventStateQueue(driver->domainEventState, event); qemuDomainEventEmitJobCompleted(driver, vm); + priv->preMigrationMemlock =3D 0; } =20 =20 @@ -4035,6 +4037,7 @@ qemuMigrationSrcConfirmPhase(virQEMUDriver *driver, =20 qemuMigrationParamsReset(driver, vm, VIR_ASYNC_JOB_MIGRATION_O= UT, jobPriv->migParams, priv->job.apiFlag= s); + qemuDomainSetMaxMemLock(vm, 0, &priv->preMigrationMemlock); } =20 qemuDomainSaveStatus(vm); @@ -4615,7 +4618,8 @@ qemuMigrationSrcStart(virDomainObj *vm, case MIGRATION_DEST_HOST: if (STREQ(spec->dest.host.protocol, "rdma") && vm->def->mem.hard_limit > 0 && - qemuDomainSetMaxMemLock(vm, vm->def->mem.hard_limit << 10, NUL= L) < 0) { + qemuDomainSetMaxMemLock(vm, vm->def->mem.hard_limit << 10, + &priv->preMigrationMemlock) < 0) { return -1; } return qemuMonitorMigrateToHost(priv->mon, migrateFlags, @@ -6155,6 +6159,7 @@ qemuMigrationSrcPerformPhase(virQEMUDriver *driver, qemuMigrationSrcRestoreDomainState(driver, vm); qemuMigrationParamsReset(driver, vm, VIR_ASYNC_JOB_MIGRATION_OUT, jobPriv->migParams, priv->job.apiFlags); + qemuDomainSetMaxMemLock(vm, 0, &priv->preMigrationMemlock); qemuMigrationJobFinish(vm); } else { if (ret < 0) @@ -6410,7 +6415,7 @@ qemuMigrationDstComplete(virQEMUDriver *driver, job->apiFlags); =20 virPortAllocatorRelease(priv->migrationPort); - priv->migrationPort =3D 0; + qemuDomainSetMaxMemLock(vm, 0, &priv->preMigrationMemlock); } =20 =20 --=20 2.35.1 From nobody Wed May 15 19:23:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655992716; cv=none; d=zohomail.com; s=zohoarc; b=JwqZtNtJeXVZaW1wYUWoUjFBbuTI5QzLOj5tj2qcnUv8KptCuoy8XgB0OyGAaTN3ae3qSWaNdfPQHKH6jcM1KwnvOImH01d4/ssGEDnkumrz2XWZAqqD+iT4B0d3tWZ5AJLjYeXe3C5nKnsUQs804wLIc5e95nRgTtMUtUXhYsI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655992716; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CnHriNMe9tmZgdeB3sVBmfaAGW0+hSHfyRGllzSYDMg=; b=HXmpoJhEJDdKGOFqIpqDKc3yzbzgU3NjIaPE6jrglWMwXdkFKo4KETu53W1h1IMzj2Gj1W0XMucSt3/7SND+qZApRcTkMO3hNeBZWFtwXjrtBDCyQojzz0ZXRl8C9VcAJaPNa/PP6/cJ2CxZJEHcaghM/eojOv9iWF/NYQzaHIk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1655992716538555.3749752779737; Thu, 23 Jun 2022 06:58:36 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-453-nhCAbfd5PuKfakqQTabTvg-1; Thu, 23 Jun 2022 09:58:32 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5127F80A0B7; Thu, 23 Jun 2022 13:58:27 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 26AAE2166B26; Thu, 23 Jun 2022 13:58:27 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 77457194E104; Thu, 23 Jun 2022 13:58:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7D7A8194B951 for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 430641121315; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.227]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 23F981121314 for ; Thu, 23 Jun 2022 13:58:15 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 1BCFF241E40; Thu, 23 Jun 2022 15:58:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655992715; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=CnHriNMe9tmZgdeB3sVBmfaAGW0+hSHfyRGllzSYDMg=; b=L4LJLCQsrTybbXpqPzYR5q1TuwNJqxfKt3Hej0oElsKR2fA+v2jU+ZxqjkZwsJQJv2yWpk dCz00+scNMKZFG7RUaaSJuJV/KNJae0WhJkV4HjbXSoGxLAfbdHwNCkDR2CoWTMif1thbn fd/GfI76pbRTu6HjJrAwBS5kxnm5ZYw= X-MC-Unique: nhCAbfd5PuKfakqQTabTvg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jiri Denemark To: libvir-list@redhat.com Subject: [libvirt PATCH 4/7] qemu_migration: Don't set unlimited memlock limit for RDMA Date: Thu, 23 Jun 2022 15:58:09 +0200 Message-Id: <18625b494119a1853fe97215572bd6b0013c50f7.1655992592.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655992718330100005 Content-Type: text/plain; charset="utf-8" Our documentation says RDMA migration requires hard_limit to be set so that we know how big memory locking limit should be set for the domain during migration. But since commit v1.2.13-71-gcf521fc8ba (which changed the default hard_limit value from 0 to VIR_DOMAIN_MEMORY_PARAM_UNLIMITED) we were actually setting memlock limit to unlimited if hard_limit was not set. Signed-off-by: Jiri Denemark Reviewed-by: J=C3=A1n Tomko --- src/qemu/qemu_migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index ffae5576d2..272f1b1b59 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -3180,7 +3180,7 @@ qemuMigrationDstPrepareActive(virQEMUDriver *driver, } =20 if (STREQ_NULLABLE(protocol, "rdma") && - vm->def->mem.hard_limit > 0 && + virMemoryLimitIsSet(vm->def->mem.hard_limit) && qemuDomainSetMaxMemLock(vm, vm->def->mem.hard_limit << 10, &priv->preMigrationMemlock) < 0) { goto error; --=20 2.35.1 From nobody Wed May 15 19:23:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655992748; cv=none; d=zohomail.com; s=zohoarc; b=aDjnB7FO/Pbx8cOhtP0EsGKF0U8Ku7QzpRJ2Qs322aXhCmSP5eT/KDXm6r5fZbk5Jhxj1RJjOZQcrTWyCVg2l6zKBXjGf9vrVwU95FvFdqQlYCrUlv0baPhlDjGMo5y6vUg+pWzI/DrrMowd3Nd7ONQsbdq0vOG9crFjj3it7h8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655992748; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZjSTLWwCvnGbQ3szddDpNHiiLVqfYvmm80MiAUXx49o=; b=EDmui7RoBzaHxhXe0RGz4A7l/LjupLH3fcpbzZQAGjqHH1Ey3Ry4cZykqlG1+4Bjz9/Zz1fX1hbh+g1NNxzIdu5reqjfzOuX8q4oNHIj4fBj+nGRag+fQ56lu7SSV3SfCpeIsLEqrKnN/l57wVFf5VQx44rGUOhOcac2FyRWWXo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1655992748241718.8580483714686; Thu, 23 Jun 2022 06:59:08 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-16-wjDoThC4Mkqf6XDsUNJhNw-1; Thu, 23 Jun 2022 09:58:32 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 10A7C185A7B2; Thu, 23 Jun 2022 13:58:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id F1E871121314; Thu, 23 Jun 2022 13:58:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3DBA0194B95F; Thu, 23 Jun 2022 13:58:21 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7CA3B194B971 for ; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 5B5E11121315; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.227]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3C1161121314 for ; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 1D25F241E41; Thu, 23 Jun 2022 15:58:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655992747; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ZjSTLWwCvnGbQ3szddDpNHiiLVqfYvmm80MiAUXx49o=; b=CSGksyFAHL/RsE6gZzGEOO4XE0EVpzdytEe4Wilo8L1+FpFbs8TpZhIp2LgqmjYgFZHAHU BtzuSgURTLAMAUpDY6dXfQVocTaUp/GhC+W03/IgdGdezrvMv9k2uhcbNId4fSdI/CXfpV Xty53EXx55/K2uzHOwHu0Lt3olk7UtI= X-MC-Unique: wjDoThC4Mkqf6XDsUNJhNw-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jiri Denemark To: libvir-list@redhat.com Subject: [libvirt PATCH 5/7] Add VIR_MIGRATE_ZEROCOPY flag Date: Thu, 23 Jun 2022 15:58:10 +0200 Message-Id: <13a7829f7a64b8ee9ea6d9fec9dd567894fb7fc0.1655992592.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655992748489100001 Content-Type: text/plain; charset="utf-8" The flag can be used to enable zero-copy mechanism for migrating memory pages. Signed-off-by: Jiri Denemark Reviewed-by: J=C3=A1n Tomko --- include/libvirt/libvirt-domain.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index 1ea3284e63..05344aaa95 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1089,6 +1089,15 @@ typedef enum { * Since: 8.5.0 */ VIR_MIGRATE_POSTCOPY_RESUME =3D (1 << 19), + + /* Use zero-copy mechanism for migrating memory pages. For QEMU/KVM th= is + * means QEMU will be temporarily allowed to lock all guest pages in h= ost's + * memory, although only those that are queued for transfer will be lo= cked + * at the same time. + * + * Since: 8.5.0 + */ + VIR_MIGRATE_ZEROCOPY =3D (1 << 20), } virDomainMigrateFlags; =20 =20 --=20 2.35.1 From nobody Wed May 15 19:23:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655992718; cv=none; d=zohomail.com; s=zohoarc; b=NYDtXuceOeEBO/2BooT7ELX+fn8UQE8kfiNNxcugB/TNl5ND0tFi7kRAmFEj3qWYfnYPev0YbeMluwV4fjxml6k2+PcPRW07VUDxtl4M8ncK8W8zELrh6dJN+ton2dpcKFMUSEAEZ24v/DWijkIBe8c78PSH3TdZyPXUJ7501VI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655992718; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GOM7uFHTG4rJF6yyhB4+bEK8VgzTN3QOe86tnfsKsPU=; b=cFJzpyZBPfiijaP6MOb2xP5y0fYBtCIidh6vQOtG9HU6NqGgix/VGBe2//rMSGpVhTxdUqu+T3PU2zVMvm8LmjIorw8+SadmsasAcgxQ3ruZJaqHmEf4LyN9F5nibENh9Q614/RG8mhPYd9QYnRFJGne4jPso05ykEyNdUAorfI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1655992718072880.9334602392948; Thu, 23 Jun 2022 06:58:38 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-595-7X19Ti3tPB68BDVSBIuFrQ-1; Thu, 23 Jun 2022 09:58:32 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B9068181881B; Thu, 23 Jun 2022 13:58:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A50961121315; Thu, 23 Jun 2022 13:58:28 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8D21D1926297; Thu, 23 Jun 2022 13:58:18 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 80162194B972 for ; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 62C3C18EAA; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.227]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 433FA18EB0 for ; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 1E712241E42; Thu, 23 Jun 2022 15:58:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655992716; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=GOM7uFHTG4rJF6yyhB4+bEK8VgzTN3QOe86tnfsKsPU=; b=NWNRCsl/9EcQSYHCCqdjRB9Z1XwvGNVXXsc77TxHO0ppbHggFanRyH6vUhnUkt312IcHaM +LoufvdPUlTlKGcJ/buETp0/20Hqyi5CZYi07uywBNbp7i15/mOXlE53kpGv7sVpjgPQiM qCnNlXzpgyA9T51eR5y7y7I4it9Xnx0= X-MC-Unique: 7X19Ti3tPB68BDVSBIuFrQ-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jiri Denemark To: libvir-list@redhat.com Subject: [libvirt PATCH 6/7] virsh: Add support for VIR_MIGRATE_ZEROCOPY flag Date: Thu, 23 Jun 2022 15:58:11 +0200 Message-Id: <0e1126645ebdff6a320229a8967ef643eae82b09.1655992592.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655992718328100004 Content-Type: text/plain; charset="utf-8" Signed-off-by: Jiri Denemark Reviewed-by: J=C3=A1n Tomko --- docs/manpages/virsh.rst | 7 ++++++- tools/virsh-domain.c | 7 +++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 4701791b04..45469f2f35 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3285,7 +3285,7 @@ migrate [--persistent] [--undefinesource] [--suspend] [--copy-storage-all] [--copy-storage-inc] [--change-protection] [--unsafe] [--verbose] [--rdma-pin-all] [--abort-on-error] [--postcopy] - [--postcopy-after-precopy] [--postcopy-resume] + [--postcopy-after-precopy] [--postcopy-resume] [--zerocopy] domain desturi [migrateuri] [graphicsuri] [listen-address] [dname] [--timeout seconds [--timeout-suspend | --timeout-postcopy]] [--xml file] [--migrate-disks disk-list] [--disks-port port] @@ -3362,6 +3362,11 @@ high (and thus allowing the domain to lock most of t= he host's memory). Doing so may be dangerous to both the domain and the host itself since the host's k= ernel may run out of memory. =20 +*--zerocopy* requests zero-copy mechanism to be used for migrating memory = pages. +For QEMU/KVM this means QEMU will be temporarily allowed to lock all guest +pages in host's memory, although only those that are queued for transfer w= ill +be locked at the same time. + ``Note``: Individual hypervisors usually do not support all possible types= of migration. For example, QEMU does not support direct migration. =20 diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 1d71ac5a67..43034f2f81 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -10890,6 +10890,10 @@ static const vshCmdOptDef opts_migrate[] =3D { .type =3D VSH_OT_BOOL, .help =3D N_("resume failed post-copy migration") }, + {.name =3D "zerocopy", + .type =3D VSH_OT_BOOL, + .help =3D N_("use zero-copy mechanism for migrating memory pages") + }, {.name =3D "migrateuri", .type =3D VSH_OT_STRING, .completer =3D virshCompleteEmpty, @@ -11296,6 +11300,9 @@ doMigrate(void *opaque) if (vshCommandOptBool(cmd, "postcopy-resume")) flags |=3D VIR_MIGRATE_POSTCOPY_RESUME; =20 + if (vshCommandOptBool(cmd, "zerocopy")) + flags |=3D VIR_MIGRATE_ZEROCOPY; + if (vshCommandOptBool(cmd, "tls")) flags |=3D VIR_MIGRATE_TLS; =20 --=20 2.35.1 From nobody Wed May 15 19:23:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1655992717; cv=none; d=zohomail.com; s=zohoarc; b=n8nedh1jh+P52SpNRYzbBgLeAh3osKoqQlbLWPzEo5mxyYD7C36VxGDUemzhLgEIMBBioWOIKKpM1pzhqDqaeNZ1uAduNjVxY/anU1iqo6EmQOtuQkLslEdkmzVLlcbAxkc8VVAf1Cy5LxbQHCZdSbw1oYs2YKZzQLt6KSXs/6M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1655992717; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tX3Zc5kOQbJXAbp1LUds+KjMP8HsDm52VFtUULuhA2Q=; b=CZbG48LwheMC8yo6CuFp+g+lh5zfN1KWw9z1WXIEJVtwswfh8mPyQztpJK91HU1UeX/K+TgJghWDl9VP5gkPT0p++s7AWni800zJa50qif/OOkggu8cqFSvSkYVSBa9vWim40h8YhdWioA8uHGuWVBGt+XBEDntJx0W+VWWV+Ec= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 165599271787554.87835849575117; Thu, 23 Jun 2022 06:58:37 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-653-mX0KMsI7NbmgZiJNyvQ8PA-1; Thu, 23 Jun 2022 09:58:32 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B89F5381685A; Thu, 23 Jun 2022 13:58:27 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A628E2166B2A; Thu, 23 Jun 2022 13:58:27 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 8D0C0194B95A; Thu, 23 Jun 2022 13:58:18 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id A1F18194B978 for ; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 8E1CD40CF8EF; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: from virval.usersys.redhat.com (unknown [10.43.2.227]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3D7A140CF8EE for ; Thu, 23 Jun 2022 13:58:16 +0000 (UTC) Received: by virval.usersys.redhat.com (Postfix, from userid 500) id 1FB44241E43; Thu, 23 Jun 2022 15:58:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655992716; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=tX3Zc5kOQbJXAbp1LUds+KjMP8HsDm52VFtUULuhA2Q=; b=ENirHRkgYAAyimk5oAZZ449I2TVRVmRfkmmGblfYoSfCO+Cs5iPT/YqVuR/NsSYrKHi/8u BubsWc+BMwvuSmihynSk1g6IAmmPhNAQKeAGX3isW/5HiM2mEcS/HYlkgxJhH66AAS1CFT iLD9vdXs0RRmjqNMyQrjr531xEpBq50= X-MC-Unique: mX0KMsI7NbmgZiJNyvQ8PA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jiri Denemark To: libvir-list@redhat.com Subject: [libvirt PATCH 7/7] qemu_migration: Implement VIR_MIGRATE_ZEROCOPY flag Date: Thu, 23 Jun 2022 15:58:12 +0200 Message-Id: <4d7c3df5a18f287a36462b90892700becca0b98f.1655992592.git.jdenemar@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1655992718325100003 Content-Type: text/plain; charset="utf-8" Resolves: https://gitlab.com/libvirt/libvirt/-/issues/306 Signed-off-by: Jiri Denemark Reviewed-by: J=C3=A1n Tomko --- src/qemu/qemu_migration.c | 21 +++++++++++++++++++++ src/qemu/qemu_migration.h | 1 + src/qemu/qemu_migration_params.c | 6 ++++++ src/qemu/qemu_migration_params.h | 1 + 4 files changed, 29 insertions(+) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 272f1b1b59..02a465f6cb 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2634,6 +2634,12 @@ qemuMigrationSrcBeginPhase(virQEMUDriver *driver, } } =20 + if (flags & VIR_MIGRATE_ZEROCOPY && !(flags & VIR_MIGRATE_PARALLEL)) { + virReportError(VIR_ERR_OPERATION_INVALID, "%s", + _("zero-copy is only available for parallel migrati= on")); + return NULL; + } + if (flags & (VIR_MIGRATE_NON_SHARED_DISK | VIR_MIGRATE_NON_SHARED_INC)= ) { if (flags & VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES && !virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_BLOCKDEV)) { @@ -4798,6 +4804,21 @@ qemuMigrationSrcRun(virQEMUDriver *driver, migParams) < 0) goto error; =20 + if (flags & VIR_MIGRATE_ZEROCOPY) { + /* Zero-copy requires pages in transfer to be locked in host memor= y. + * Unfortunately, we have no reliable way of computing how many pa= ges + * will need to be locked at the same time. Thus we set the limit = to + * the whole guest memory and reset it back once migration is done= . */ + unsigned long long limit; + + if (virMemoryLimitIsSet(vm->def->mem.hard_limit)) + limit =3D vm->def->mem.hard_limit; + else + limit =3D virDomainDefGetMemoryTotal(vm->def); + + if (qemuDomainSetMaxMemLock(vm, limit << 10, &priv->preMigrationMe= mlock) < 0) + goto error; + } =20 if (storageMigration) { if (mig->nbd) { diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index fbc0549b34..81cc1e91c0 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -61,6 +61,7 @@ VIR_MIGRATE_PARALLEL | \ VIR_MIGRATE_NON_SHARED_SYNCHRONOUS_WRITES | \ VIR_MIGRATE_POSTCOPY_RESUME | \ + VIR_MIGRATE_ZEROCOPY | \ 0) =20 /* All supported migration parameters and their types. */ diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index 4e83d8d8bd..cc66ed8229 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -94,6 +94,7 @@ VIR_ENUM_IMPL(qemuMigrationCapability, "multifd", "dirty-bitmaps", "return-path", + "zero-copy-send", ); =20 =20 @@ -175,6 +176,11 @@ static const qemuMigrationParamsFlagMapItem qemuMigrat= ionParamsFlagMap[] =3D { VIR_MIGRATE_TUNNELLED, QEMU_MIGRATION_CAP_RETURN_PATH, QEMU_MIGRATION_SOURCE | QEMU_MIGRATION_DESTINATION}, + + {QEMU_MIGRATION_FLAG_REQUIRED, + VIR_MIGRATE_ZEROCOPY, + QEMU_MIGRATION_CAP_ZERO_COPY_SEND, + QEMU_MIGRATION_SOURCE}, }; =20 /* Translation from VIR_MIGRATE_PARAM_* typed parameters to diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index f2f0090343..d1184acded 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -39,6 +39,7 @@ typedef enum { QEMU_MIGRATION_CAP_MULTIFD, QEMU_MIGRATION_CAP_BLOCK_DIRTY_BITMAPS, QEMU_MIGRATION_CAP_RETURN_PATH, + QEMU_MIGRATION_CAP_ZERO_COPY_SEND, =20 QEMU_MIGRATION_CAP_LAST } qemuMigrationCapability; --=20 2.35.1