From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201386; cv=none; d=zohomail.com; s=zohoarc; b=BT34pASKp+TCCIjeOIjbwbX210aPPdeUFsJECh/qe88/BG0SFAlO2vDnKorVbS0NpRbCIV+J5zXu6LeWb3WOR9TLvVe2MX0HKDLLhPc970DJY/Vf3wVSQwgyzVzw/oTNCVDlfLHAdgNyY+zMtmDOSg4Wh3ifnVQwuV7OXDhF9rU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201386; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FAu8ZNkb77ULPLeB6LnX+Y7SwNQUyl6UVC+DMTehx+s=; b=l16+rSzKBQFxvO4lqvdTX7yzyBgCRTdEVSVaSJ8+/+HZt5HyyYdpWMOA1J9IgMmZBLdHPW9CtnDdUtSoUS0ZoE/epP6Z7SY2+FAMllXiRydFBjpdTliLArQO689oCnrbPFALQ1btDSmCOOzYC2pxJepkFAuqZhI6jdVtMaPVx9U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 16782013863171006.5092219829128; Tue, 7 Mar 2023 07:03:06 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-589-lEdJBD2uMOuXs27bUmijTg-1; Tue, 07 Mar 2023 10:03:00 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A0896803D50; Tue, 7 Mar 2023 15:02:44 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 87AEB2166B2A; Tue, 7 Mar 2023 15:02:42 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 3722719465A3; Tue, 7 Mar 2023 15:02:42 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C505919465A3 for ; Tue, 7 Mar 2023 15:02:40 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B0797492C18; Tue, 7 Mar 2023 15:02:40 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A93CF492C14 for ; Tue, 7 Mar 2023 15:02:40 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8CD73857F47 for ; Tue, 7 Mar 2023 15:02:40 +0000 (UTC) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-382-1ehRZxEhMfSsEG1n-ei2fQ-1; Tue, 07 Mar 2023 10:02:36 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4PWJXS2XT8zKq57; Tue, 7 Mar 2023 23:00:24 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:28 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201383; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=FAu8ZNkb77ULPLeB6LnX+Y7SwNQUyl6UVC+DMTehx+s=; b=Yrc9BvhEgx513K9hM5RQAKVFd/5fxtXUCuJ4jFQvkbstR6qKDOK1CBrtMu9v6Grxeur5AI 9GIlHqRlnHyAgusc5NdGE6T19IoPWxKCK7WraYma03LbXmoF9Qh/0zu4MU8S4vmj15GYZy +FMYeDpjPM1yS1eKzfOr3IGmvYS95/c= X-MC-Unique: lEdJBD2uMOuXs27bUmijTg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 1ehRZxEhMfSsEG1n-ei2fQ-1 From: Jiang Jiacheng To: Subject: [PATCH v2 1/8] migration/migration-pin: Implemente the interface of obtaining migration thread information Date: Tue, 7 Mar 2023 22:52:13 +0800 Message-ID: <20230307145220.3767207-2-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201397640100001 Content-Type: text/plain; charset="utf-8" First, we need get migration thread info, implement the related interface to obtain migration threadinfo from qemu using qmp command. Implementation of the Interface for Obtaining Thread Information Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- src/qemu/qemu_domain.c | 24 ++++++++++++++ src/qemu/qemu_domain.h | 11 +++++++ src/qemu/qemu_monitor.c | 25 +++++++++++++++ src/qemu/qemu_monitor.h | 11 +++++++ src/qemu/qemu_monitor_json.c | 62 ++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 5 +++ 6 files changed, 138 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index aa567f652a..ba8d1055ae 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -797,6 +797,28 @@ qemuDomainSecretInfoDestroy(qemuDomainSecretInfo *seci= nfo) qemuDomainSecretInfoClear(secinfo, true); } =20 +static void +qemuDomainMigThreadInfoFree(qemuDomainMigThreadInfo *migthread) +{ + if (!migthread) + return; + + g_free(migthread->thread_name); +} + +void +qemuDomainMigThreadsInfoFree(qemuDomainMigThreadInfo **migthreads, + int nmigthreads) +{ + size_t i; + if (!migthreads) + return; + + for (i =3D 0; i < nmigthreads; i++) { + g_clear_pointer(&migthreads[i], qemuDomainMigThreadInfoFree); + } + g_free(migthreads); +} =20 static virClass *qemuDomainDiskPrivateClass; static void qemuDomainDiskPrivateDispose(void *obj); @@ -1842,6 +1864,8 @@ qemuDomainObjPrivateDataClear(qemuDomainObjPrivate *p= riv) priv->allowReboot =3D VIR_TRISTATE_BOOL_ABSENT; =20 g_clear_pointer(&priv->migrationCaps, virBitmapFree); + qemuDomainMigThreadsInfoFree(priv->migThreadsInfo, priv->migThreadCoun= t); + priv->migThreadCount =3D 0; =20 virHashRemoveAll(priv->blockjobs); =20 diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 550397ee50..42115a20ef 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -97,6 +97,12 @@ struct _qemuDomainSecretInfo { char *ciphertext; /* encoded/encrypted secret */ }; =20 +typedef struct _qemuDomainMigThreadInfo qemuDomainMigThreadInfo; +struct _qemuDomainMigThreadInfo { + char *thread_name; + int thread_id; +}; + typedef struct _qemuDomainObjPrivate qemuDomainObjPrivate; struct _qemuDomainObjPrivate { virQEMUDriver *driver; @@ -138,6 +144,8 @@ struct _qemuDomainObjPrivate { unsigned long migMaxBandwidth; char *origname; int nbdPort; /* Port used for migration with NBD */ + int migThreadCount; + qemuDomainMigThreadInfo **migThreadsInfo; unsigned short migrationPort; int preMigrationState; unsigned long long preMigrationMemlock; /* Original RLIMIT_MEMLOCK in = case @@ -925,6 +933,9 @@ int qemuDomainSecretChardevPrepare(virQEMUDriverConfig = *cfg, ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(4); =20 +void qemuDomainMigThreadsInfoFree(qemuDomainMigThreadInfo **migthreads, + int nmigthreads); + void qemuDomainCleanupStorageSourceFD(virStorageSource *src); =20 void qemuDomainStartupCleanup(virDomainObj *vm); diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index 1fa35f03cc..ad35034db2 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -2222,6 +2222,31 @@ qemuMonitorGetMigrationStats(qemuMonitor *mon, return qemuMonitorJSONGetMigrationStats(mon, stats, error); } =20 +void qemuMonitorMigThreadsInfoFree(qemuMonitorMigThreadInfo **migthreads, + int nmigthreads) +{ + size_t i; + + if (!migthreads) + return; + + for (i =3D 0; i < nmigthreads; i++) { + g_free(migthreads[i]->thread_name); + g_free(migthreads[i]); + } + + g_free(migthreads); +} + +int +qemuMonitorGetMigThreadsInfo(qemuMonitor *mon, + qemuMonitorMigThreadInfo ***migthreads, + int *nmigthreads) +{ + QEMU_CHECK_MONITOR(mon); + + return qemuMonitorJSONGetMigThreadsInfo(mon, migthreads, nmigthreads); +} =20 int qemuMonitorMigrateToFd(qemuMonitor *mon, diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h index 72db0c0838..076e32bdd3 100644 --- a/src/qemu/qemu_monitor.h +++ b/src/qemu/qemu_monitor.h @@ -823,6 +823,17 @@ int qemuMonitorGetMigrationCapabilities(qemuMonitor *m= on, int qemuMonitorSetMigrationCapabilities(qemuMonitor *mon, virJSONValue **caps); =20 +typedef struct _qemuMonitorMigThreadInfo qemuMonitorMigThreadInfo; +struct _qemuMonitorMigThreadInfo { + char *thread_name; + int thread_id; +}; +int qemuMonitorGetMigThreadsInfo(qemuMonitor *mon, + qemuMonitorMigThreadInfo ***migthreads, + int *nmigthreads); +void qemuMonitorMigThreadsInfoFree(qemuMonitorMigThreadInfo **migthreads, + int nmigthreads); + int qemuMonitorGetGICCapabilities(qemuMonitor *mon, virGICCapability **capabilities); =20 diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index ee2ea68acb..4fd381859f 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -3124,6 +3124,68 @@ int qemuMonitorJSONGetMigrationStats(qemuMonitor *mo= n, return 0; } =20 +int +qemuMonitorJSONGetMigThreadsInfo(qemuMonitor *mon, + qemuMonitorMigThreadInfo ***migthreads, + int *nmigthreads) +{ + int ret =3D -1; + g_autoptr(virJSONValue) cmd =3D NULL; + g_autoptr(virJSONValue) reply =3D NULL; + virJSONValue *data =3D NULL; + qemuMonitorMigThreadInfo **infolist =3D NULL; + size_t n =3D 0; + size_t i; + + *migthreads =3D NULL; + + if (!(cmd =3D qemuMonitorJSONMakeCommand("query-migrationthreads", NUL= L))) + return ret; + + if (qemuMonitorJSONCommand(mon, cmd, &reply) < 0) + goto cleanup; + + if (qemuMonitorJSONCheckReply(cmd, reply, VIR_JSON_TYPE_ARRAY) < 0) + goto cleanup; + + data =3D virJSONValueObjectGetArray(reply, "return"); + n =3D virJSONValueArraySize(data); + + infolist =3D g_new0(qemuMonitorMigThreadInfo *, n + 1); + + for (i =3D 0; i < n; i++) { + virJSONValue *child =3D virJSONValueArrayGet(data, i); + const char *tmp; + qemuMonitorMigThreadInfo *info; + + info =3D g_new0(qemuMonitorMigThreadInfo, 1); + + infolist[i] =3D info; + + if (!(tmp =3D virJSONValueObjectGetString(child, "name"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("query-migrationthreads reply data was missi= ng 'name'")); + goto cleanup; + } + info->thread_name =3D g_strdup(tmp); + + if (virJSONValueObjectGetNumberInt(child, "thread-id", + &info->thread_id) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("query-migrationthreads reply has malformed " + "'thread-id' data")); + goto cleanup; + } + } + + *nmigthreads =3D n; + *migthreads =3D g_steal_pointer(&infolist); + ret =3D 0; + + cleanup: + qemuMonitorMigThreadsInfoFree(infolist, n); + return ret; +} =20 int qemuMonitorJSONMigrate(qemuMonitor *mon, unsigned int flags, diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h index 6f376cf9b7..1a0875bd2c 100644 --- a/src/qemu/qemu_monitor_json.h +++ b/src/qemu/qemu_monitor_json.h @@ -156,6 +156,11 @@ int qemuMonitorJSONSetMigrationCapabilities(qemuMonitor *mon, virJSONValue **caps); =20 +int +qemuMonitorJSONGetMigThreadsInfo(qemuMonitor *mon, + qemuMonitorMigThreadInfo ***migthreads, + int *nmigthreads); + int qemuMonitorJSONGetGICCapabilities(qemuMonitor *mon, virGICCapability **capabilities); --=20 2.33.0 From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201433; cv=none; d=zohomail.com; s=zohoarc; b=ZwnPqHm2n3Oa/pFu59+mN1s4SlGN+xVjmNwP9LnLqof7ZC9W/or7INRWGTkUoyi2aY7vfBAAeAaLEH7phIx763EPfSHargFrllCUVZysisanTRohGt764xm7raQQ+Tb7LY4kRLdhIwvtNIRVABJ2tDxX5r1qsBO3ziYuaYrhS3s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201433; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LC4jXHSNv7oJSETUhvXg1bXXGg8oNKOsrLm5VH7AxRg=; b=VlMQNxh/75FpTL2F142W5r2i/LJJ8vNHWj6e2rrV8lJ5Q0t6s3TwFb8i5Uz1IJyTCoLG8uHeiyJQOZzPBVOlONT7qhyCQEMdB3fL+3MNnknNQKAeclEO6vl+pOm3+sTCLYobUYhH6dpPJJbq/PNW2mKncvOavzIBrJj5AICEK58= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1678201433458210.98514628339797; Tue, 7 Mar 2023 07:03:53 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-247-oswwuZr1NeSV6Tz-sgq0rQ-1; Tue, 07 Mar 2023 10:03:41 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A9155280AA2A; Tue, 7 Mar 2023 15:02:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 95DA21410DD9; Tue, 7 Mar 2023 15:02:50 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 7637C19465A3; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id DCDA51946594 for ; Tue, 7 Mar 2023 15:02:44 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id CF536C15BAD; Tue, 7 Mar 2023 15:02:44 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast07.extmail.prod.ext.rdu2.redhat.com [10.11.55.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C7BAAC15BA0 for ; Tue, 7 Mar 2023 15:02:44 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ADE313C0F229 for ; Tue, 7 Mar 2023 15:02:44 +0000 (UTC) Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-248-8BNF8B_8MNuKU3yODsb5xg-1; Tue, 07 Mar 2023 10:02:39 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4PWJWf3v4Xz16P9w; Tue, 7 Mar 2023 22:59:42 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:28 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201432; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=LC4jXHSNv7oJSETUhvXg1bXXGg8oNKOsrLm5VH7AxRg=; b=b3rx1tp+SeSJM2+pg2Wprh86gc9bR5UtyqNHFMlu7MveTVQLumrtwgO1Qhem1GXff5FhSY Dl/cKVNTeM3JoEWAKP4MEq/v/VMXBkPtPrz/+eMjrql93LtEQKuIqKF6kxZEZ+d4gpBXBr nSgA1/diBF5NEEPIUJqXq/dQg3dNU6Q= X-MC-Unique: oswwuZr1NeSV6Tz-sgq0rQ-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 8BNF8B_8MNuKU3yODsb5xg-1 From: Jiang Jiacheng To: Subject: [PATCH v2 2/8] migration/migration-pin: get migration pid for migration pin Date: Tue, 7 Mar 2023 22:52:14 +0800 Message-ID: <20230307145220.3767207-3-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201457421100001 Content-Type: text/plain; charset="utf-8" Second, query migration thread information during migration process proactively. Migration should start after all thread be created, so querying the migration thread infomation before wating for completion and terminate the query in any abnormal situation. Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- src/qemu/qemu_migration.c | 81 ++++++++++++++++++++++++++++++++ src/qemu/qemu_migration_params.c | 19 ++++++++ src/qemu/qemu_migration_params.h | 4 ++ 3 files changed, 104 insertions(+) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 2720f0b083..0cea055eec 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2003,6 +2003,81 @@ qemuMigrationAnyCompleted(virDomainObj *vm, return -1; } =20 +/* Returns 0 on get migration thread info success, + * -1 on unexpected failure + */ +static int +qemuMigrationSrcWaitForThreadInfo(virDomainObj *vm, + qemuMigrationParams *migParams, + unsigned long migflags, + virDomainAsyncJob asyncJob, + virConnectPtr dconn, + unsigned int flags) +{ + int ret =3D -1; + qemuDomainObjPrivate *priv =3D vm->privateData; + qemuMonitorMigThreadInfo **infolist =3D NULL; + qemuDomainMigThreadInfo **migthreads =3D NULL; + int nmigthreads =3D 0; + int rc; + size_t i; + + /* migrationThreadCount =3D live_migration + multifd channels */ + priv->migThreadCount =3D qemuMigrationParamsGetMigThreadCount(migParam= s, + migflags); + while ((rc =3D qemuMigrationAnyCompleted(vm, asyncJob, + dconn, flags)) !=3D 1) { + if (rc < 0) + return rc; + + if (qemuDomainObjEnterMonitorAsync(vm, asyncJob) < 0) + return ret; + rc =3D qemuMonitorGetMigThreadsInfo(priv->mon, &infolist, &nmigthr= eads); + qemuDomainObjExitMonitor(vm); + if (rc < 0) + goto cleanup; + + if (nmigthreads =3D=3D priv->migThreadCount) { + break; + } else if (nmigthreads > priv->migThreadCount) { + VIR_WARN("migration threads is more than expected"); + goto cleanup; + } else { + struct timespec ts =3D { .tv_sec =3D 0, .tv_nsec =3D 10 * 1000= * 1000ull }; + virObjectUnlock(vm); + nanosleep(&ts, NULL); + virObjectLock(vm); + } + } + + migthreads =3D g_new0(qemuDomainMigThreadInfo *, nmigthreads + 1); + + for (i =3D 0; i < nmigthreads; i++) { + qemuDomainMigThreadInfo *migthread; + migthread =3D g_new0(qemuDomainMigThreadInfo, nmigthreads + 1); + + migthread->thread_name =3D g_strdup(infolist[i]->thread_name); + migthread->thread_id =3D infolist[i]->thread_id; + + migthreads[i] =3D migthread; + } + + priv->migThreadsInfo =3D migthreads; + ret =3D 0; + + cleanup: + if (migthreads && (ret < 0)) { + qemuDomainMigThreadsInfoFree(migthreads, priv->migThreadCount); + priv->migThreadCount =3D 0; + } + + if (infolist) { + qemuMonitorMigThreadsInfoFree(infolist, nmigthreads); + } + + return ret; +} + =20 /* Returns 0 on success, -2 when migration needs to be cancelled, or -1 wh= en * QEMU reports failed migration. @@ -4917,6 +4992,12 @@ qemuMigrationSrcRun(virQEMUDriver *driver, if (flags & VIR_MIGRATE_POSTCOPY) waitFlags |=3D QEMU_MIGRATION_COMPLETED_POSTCOPY; =20 + rc =3D qemuMigrationSrcWaitForThreadInfo(vm, migParams, flags, + VIR_ASYNC_JOB_MIGRATION_OUT, + dconn, waitFlags); + if (rc =3D=3D -1) + goto error; + rc =3D qemuMigrationSrcWaitForCompletion(vm, VIR_ASYNC_JOB_MIGRATION_O= UT, dconn, waitFlags); if (rc =3D=3D -2) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index bd09dcfb23..43bed2e618 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -1540,3 +1540,22 @@ qemuMigrationParamsGetTLSHostname(qemuMigrationParam= s *migParams) =20 return migParams->params[QEMU_MIGRATION_PARAM_TLS_HOSTNAME].value.s; } + + +int +qemuMigrationParamsGetMigThreadCount(qemuMigrationParams *migParams, + unsigned long migflags) +{ + int nmigthreads =3D 1; /* live_migtion */ + + if (!(migflags & VIR_MIGRATE_PARALLEL)) { + return nmigthreads; + } else { + if (migParams->params[QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS].set) + nmigthreads +=3D migParams->params[QEMU_MIGRATION_PARAM_MULTIF= D_CHANNELS].value.i; + else + nmigthreads +=3D 2; /* default multifd channel is 2 */ + } + + return nmigthreads; +} diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index e7c65f6a21..975b6691d4 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -166,3 +166,7 @@ qemuMigrationCapsGet(virDomainObj *vm, =20 const char * qemuMigrationParamsGetTLSHostname(qemuMigrationParams *migParams); + +int +qemuMigrationParamsGetMigThreadCount(qemuMigrationParams *migParams, + unsigned long migflags); --=20 2.33.0 From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201435; cv=none; d=zohomail.com; s=zohoarc; b=QRLhLqz/hSHf0tRAT+VTAVHIy2xq2jCtg/OWUK8ddsVpPxGi5Fb34I/vOfnDdjBFac2NrPcF26PanZCQumemt8Q6rTMTeacNfZYZ3U7VsmqWTryWPb9fVfTYybOv8VjypT519uyadKlr2rQp6Te1Es6Vuup5pHi8rzI1rWT6D3g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201435; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Ojsb3YIVbqR+lGcHkHzWE/K0C/3El/INUGIx1+pvPR0=; b=ciVwBG40V0pdPl7p/sUKR0ttPONxW3WlzmdymOqQcoJ0T4/ah+2Vk0yyLzwCcvSfB1bSHpprya9by01D5rYQpuIa7T6AXtyuzChZ1DEBWvHX5/mZLXmqEUH/RapYZIThtLwUWEEkJOkHdgIzG6iY63S5PY9awTVeLRLnVXL3FVw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1678201435619380.92737376311766; Tue, 7 Mar 2023 07:03:55 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-617-K8pJ-pDXOHOwHZaKIpiydg-1; Tue, 07 Mar 2023 10:03:38 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 86DAE38149C4; Tue, 7 Mar 2023 15:03:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B0602166B35; Tue, 7 Mar 2023 15:03:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 4056D19472CA; Tue, 7 Mar 2023 15:02:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0743C1946A76 for ; Tue, 7 Mar 2023 15:02:47 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id E183F4010E36; Tue, 7 Mar 2023 15:02:46 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DA0D74014D1D for ; Tue, 7 Mar 2023 15:02:46 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B84A485CCE5 for ; Tue, 7 Mar 2023 15:02:46 +0000 (UTC) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-XZ4H8VbTOui6MT-RYisRsw-1; Tue, 07 Mar 2023 10:02:40 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4PWJXT34JLzKq5F; Tue, 7 Mar 2023 23:00:25 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:29 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201434; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Ojsb3YIVbqR+lGcHkHzWE/K0C/3El/INUGIx1+pvPR0=; b=Ou0jKTpo0Tb+zAKQTbJ+xg9tGzlCdWP7JHdFkmcG3esaO2UxJ7N2Wq9cj9ewIqvUk7ecP3 Oynaoj32Ci6Zu3DOlwBY6UYur6tTqcXYALnQ1yTRTlRppiK8zNCzj2w5ZGfPiDK/RAj7a3 FtBhs8GoWHllUm5HIAWkg0aPiyyO+VM= X-MC-Unique: K8pJ-pDXOHOwHZaKIpiydg-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: XZ4H8VbTOui6MT-RYisRsw-1 From: Jiang Jiacheng To: Subject: [PATCH v2 3/8] migration/migration-pin: Introduce virDomainPinMigrationThread API Date: Tue, 7 Mar 2023 22:52:15 +0800 Message-ID: <20230307145220.3767207-4-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201465780100001 Content-Type: text/plain; charset="utf-8" Then, pin migration thread with given cpumap. Introduce virDomainPinMigrationThread API, and it will be implement in next patch. Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- include/libvirt/libvirt-domain.h | 5 +++ src/driver-hypervisor.h | 6 +++ src/libvirt-domain.c | 65 ++++++++++++++++++++++++++++++++ src/libvirt_public.syms | 4 ++ src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 13 ++++++- src/remote_protocol-structs | 5 +++ 7 files changed, 98 insertions(+), 1 deletion(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index 5152ed4551..b6c01016d8 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -2429,6 +2429,11 @@ int virDomainGetEmulatorPinInfo = (virDomainPtr domain, int maplen, unsigned int flags); =20 +int virDomainPinMigrationThread (virDomainPtr domain, + unsigned char *cpumap, + int maplen); + + /** * virDomainIOThreadInfo: * diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h index 5219344b72..a8413b1a6c 100644 --- a/src/driver-hypervisor.h +++ b/src/driver-hypervisor.h @@ -1441,6 +1441,11 @@ typedef int int seconds, unsigned int flags); =20 +typedef int +(*virDrvDomainPinMigrationThread)(virDomainPtr domain, + unsigned char *cpumap, + int maplen); + typedef int (*virDrvDomainFDAssociate)(virDomainPtr domain, const char *name, @@ -1720,4 +1725,5 @@ struct _virHypervisorDriver { virDrvDomainGetMessages domainGetMessages; virDrvDomainStartDirtyRateCalc domainStartDirtyRateCalc; virDrvDomainFDAssociate domainFDAssociate; + virDrvDomainPinMigrationThread domainPinMigrationThread; }; diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index 379057d9a7..a18b7753c2 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -14055,3 +14055,68 @@ virDomainFDAssociate(virDomainPtr domain, virDispatchError(conn); return -1; } + +/** + * virDomainPinMigrationThread: + * @domain: a domain object + * @cpumap: pointer to a bit map of real CPUs (in 8-bit bytes) (IN) + * Each bit set to 1 means that corresponding CPU is usable. + * Bytes are stored in little-endian order: CPU0-7, 8-15... + * In each byte, lowest CPU number is least significant bit. + * @maplen: number of bytes in cpumap, from 1 up to size of CPU map in + * underlying virtualization system (Xen...). + * If maplen < size, missing bytes are set to zero. + * If maplen > size, failure code is returned. + * + * Allocate the real CPUs to the migrationThread which will be launched + * at the beginning of migration. + * This allocation can be handled whether before the migration, or during + * the migration which performs as an instant change. + * + * This function may require privileged access to the hypervisor. + * + * Returns 0 in case of success, -1 in case of failure. + * + * Since: 9.1.0 + */ +int +virDomainPinMigrationThread(virDomainPtr domain, + unsigned char *cpumap, + int maplen) +{ + virConnectPtr conn; + g_autofree char *str =3D NULL; + + VIR_DOMAIN_DEBUG(domain, "migration: cpumap=3D%p, maplen=3D%d", + cpumap, maplen); + + virResetLastError(); + + virCheckDomainReturn(domain, -1); + conn =3D domain->conn; + + virCheckNonNullArgGoto(cpumap, error); + virCheckPositiveArgGoto(maplen, error); + + str =3D virBitmapDataFormat(cpumap, maplen); + VIR_INFO("Begin to PinMigrationThread(domain=3D%s): new CPU Affinity:%= s", + NULLSTR(domain->name), NULLSTR(str)); + + if (conn->driver->domainPinMigrationThread) { + int ret; + ret =3D conn->driver->domainPinMigrationThread(domain, cpumap, map= len); + if (ret < 0) { + VIR_ERROR(_("Failed to PinMigrationThread(domain=3D%s), ret=3D= %d"), + NULLSTR(domain->name), ret); + goto error; + } + VIR_INFO("Success to PinMigrationThread(domain=3D%s)", NULLSTR(dom= ain->name)); + return ret; + } + + virReportUnsupportedError(); + + error: + virDispatchError(domain->conn); + return -1; +} diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms index 80742f268e..a5223786ed 100644 --- a/src/libvirt_public.syms +++ b/src/libvirt_public.syms @@ -932,4 +932,8 @@ LIBVIRT_9.0.0 { virDomainFDAssociate; } LIBVIRT_8.5.0; =20 +LIBVIRT_9.1.0 { + global: + virDomainPinMigrationThread; +} LIBVIRT_9.0.0; # .... define new API here using predicted next version number .... diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index a4c60be3d7..54c5099731 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -8126,6 +8126,7 @@ static virHypervisorDriver hypervisor_driver =3D { .domainStartDirtyRateCalc =3D remoteDomainStartDirtyRateCalc, /* 7.2.0= */ .domainSetLaunchSecurityState =3D remoteDomainSetLaunchSecurityState, = /* 8.0.0 */ .domainFDAssociate =3D remoteDomainFDAssociate, /* 9.0.0 */ + .domainPinMigrationThread =3D remoteDomainPinMigrationThread, /* 9.1.0= */ }; =20 static virNetworkDriver network_driver =3D { diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 5d86a51116..358e44bd14 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -3935,6 +3935,11 @@ struct remote_domain_fd_associate_args { remote_nonnull_string name; unsigned int flags; }; + +struct remote_domain_pin_migration_thread_args { + remote_nonnull_domain dom; + opaque cpumap; /* (unsigned char *) */ +}; /*----- Protocol. -----*/ =20 /* Define the program number, protocol version and procedure numbers here.= */ @@ -6974,5 +6979,11 @@ enum remote_procedure { * @generate: none * @acl: domain:write */ - REMOTE_PROC_DOMAIN_FD_ASSOCIATE =3D 443 + REMOTE_PROC_DOMAIN_FD_ASSOCIATE =3D 443, + + /** + * @generate: both + * @acl: domain:write + */ + REMOTE_PROC_DOMAIN_PIN_MIGRATION_THREAD =3D 444 }; diff --git a/src/remote_protocol-structs b/src/remote_protocol-structs index 3c6c230a16..a355f3a76b 100644 --- a/src/remote_protocol-structs +++ b/src/remote_protocol-structs @@ -3273,6 +3273,10 @@ struct remote_domain_fd_associate_args { remote_nonnull_string name; u_int flags; }; +struct remote_domain_pin_migration_thread_args { + remote_nonnull_domain dom; + opaque cpumap; /* (unsigned= char *) */ +}; enum remote_procedure { REMOTE_PROC_CONNECT_OPEN =3D 1, REMOTE_PROC_CONNECT_CLOSE =3D 2, @@ -3717,4 +3721,5 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_RESTORE_PARAMS =3D 441, REMOTE_PROC_DOMAIN_ABORT_JOB_FLAGS =3D 442, REMOTE_PROC_DOMAIN_FD_ASSOCIATE =3D 443, + REMOTE_PROC_DOMAIN_PIN_MIGRATION_THREAD =3D 444, }; --=20 2.33.0 From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201685; cv=none; d=zohomail.com; s=zohoarc; b=VhcjqJEmDRwTkdylKsZSaO0A9AdaUMaOXHM9icFe8YFj5GI3jckw+xQ0s4l3IePGWASQmY2UQHJs+M/r+RNfAKw7LJ5QNAvw5VV6BjIlPoorqd0Hp3ytfuQvBrauclyTJfgzArI3zBvzDPr66xBOMTbjQgHI7gE7U7s7tvoPs2Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201685; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pkuU4psuO8dt7rVd9VP6mBBFSgRe4K2ur/w+lJDxt3E=; b=R4l6e48UOliKc+jR42Au6Pk56I1LfvCEcw5R99BqkGFoUVDnEWK2215OtBLf2+182OjwbDLXSi3JttbUUU0rzaWvpXRTRXnJu7BMXT4A+rtEhiIqvff7/ArfK5HV7CYsjslMANz2qFw8IP1QDhFNwGn7K4vf/ccK6wRoBlrpG64= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1678201685366470.94563373865924; Tue, 7 Mar 2023 07:08:05 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-345-z5Y7i0pYONSXDXI9lUBn8w-1; Tue, 07 Mar 2023 10:05:29 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 301D03815F64; Tue, 7 Mar 2023 15:03:12 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 176E140C10FA; Tue, 7 Mar 2023 15:03:12 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 84A031946594; Tue, 7 Mar 2023 15:02:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 4ECBB19465A3 for ; Tue, 7 Mar 2023 15:02:52 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 42179112132D; Tue, 7 Mar 2023 15:02:52 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3ACD81121314 for ; Tue, 7 Mar 2023 15:02:52 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 171D585CCEC for ; Tue, 7 Mar 2023 15:02:52 +0000 (UTC) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-498-GCXisq6MM6iA9icJRRVzWg-1; Tue, 07 Mar 2023 10:02:37 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PWJZ06rgZzrRt8; Tue, 7 Mar 2023 23:01:44 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:30 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201684; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=pkuU4psuO8dt7rVd9VP6mBBFSgRe4K2ur/w+lJDxt3E=; b=Ytb4T35b/oRLoDAcx3+pvy58zYyt+FkzBz83f4m45UWMCSACLdZsT6eoVXBxxjoh/wogWn GcrdTDcjcNLFExTWzHdgusDn1PUbgOnwyTVulngqjseRkQEj+wZ07Ba2l+P4pY++Zibr4/ /MAu9fCaBfbvWlR0Trlh0VuO8w746Aw= X-MC-Unique: z5Y7i0pYONSXDXI9lUBn8w-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: GCXisq6MM6iA9icJRRVzWg-1 From: Jiang Jiacheng To: Subject: [PATCH v2 4/8] migration/migration-pin: Implement qemuDomainPinMigrationThread Date: Tue, 7 Mar 2023 22:52:16 +0800 Message-ID: <20230307145220.3767207-5-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201687366100001 Content-Type: text/plain; charset="utf-8" Add cgroup for migation thread and implement qemuDomainPinMigrationThread to pin migraiton thread to given cpumap. Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- src/conf/domain_conf.c | 9 ++++++ src/conf/domain_conf.h | 10 +++++++ src/conf/virconftypes.h | 2 ++ src/libvirt_private.syms | 1 + src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 56 ++++++++++++++++++++++++++++++++++++ src/qemu/qemu_process.c | 61 ++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_process.h | 10 +++++++ src/util/vircgroup.c | 3 ++ src/util/vircgroup.h | 1 + 11 files changed, 155 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index f7f9ec3c0a..68200dbbab 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -30883,3 +30883,12 @@ virDomainWatchdogDefFind(const virDomainDef *def, =20 return -1; } + +void +virDomainMigrationIDDefFree(virDomainMigrationIDDef *def) +{ + if (!def) + return; + virBitmapFree(def->cpumask); + VIR_FREE(def); +} diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 21ea4a48b3..fff846f26f 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -4469,3 +4469,13 @@ virDomainObjGetMessages(virDomainObj *vm, =20 bool virDomainDefHasSpiceGraphics(const virDomainDef *def); + +struct _virDomainMigrationIDDef { + bool autofill; + int thread_id; + virBitmap *cpumask; + virDomainThreadSchedParam sched; +}; + +void +virDomainMigrationIDDefFree(virDomainMigrationIDDef *def); diff --git a/src/conf/virconftypes.h b/src/conf/virconftypes.h index e07f967814..84a28a08b8 100644 --- a/src/conf/virconftypes.h +++ b/src/conf/virconftypes.h @@ -261,3 +261,5 @@ typedef struct _virDomainXMLOption virDomainXMLOption; typedef struct _virDomainXMLPrivateDataCallbacks virDomainXMLPrivateDataCa= llbacks; =20 typedef struct _virDomainXenbusControllerOpts virDomainXenbusControllerOpt= s; + +typedef struct _virDomainMigrationIDDef virDomainMigrationIDDef; diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index d5b1b9cb72..b5d9aa62ae 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -525,6 +525,7 @@ virDomainMemoryModelTypeToString; virDomainMemoryRemove; virDomainMemorySourceTypeFromString; virDomainMemorySourceTypeToString; +virDomainMigrationIDDefFree; virDomainMouseModeTypeFromString; virDomainMouseModeTypeToString; virDomainNetAllocateActualDevice; diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index ba8d1055ae..3d497dd5d0 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1853,6 +1853,7 @@ qemuDomainObjPrivateDataClear(qemuDomainObjPrivate *p= riv) /* remove automatic pinning data */ g_clear_pointer(&priv->autoNodeset, virBitmapFree); g_clear_pointer(&priv->autoCpuset, virBitmapFree); + g_clear_pointer(&priv->pcpumap, virBitmapFree); g_clear_pointer(&priv->pciaddrs, virDomainPCIAddressSetFree); g_clear_pointer(&priv->usbaddrs, virDomainUSBAddressSetFree); g_clear_pointer(&priv->origCPU, virCPUDefFree); diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index 42115a20ef..af986b5423 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -171,6 +171,7 @@ struct _qemuDomainObjPrivate { /* Bitmaps below hold data from the auto NUMA feature */ virBitmap *autoNodeset; virBitmap *autoCpuset; + virBitmap *pcpumap; =20 bool signalIOError; /* true if the domain condition should be signalle= d on I/O error */ diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index c546e35953..2b9a952c85 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -20581,6 +20581,61 @@ qemuDomainFDAssociate(virDomainPtr domain, return ret; } =20 +static int +qemuDomainPinMigrationThread(virDomainPtr dom, + unsigned char *cpumap, + int maplen) +{ + int ret =3D -1; + virQEMUDriver *driver =3D dom->conn->privateData; + g_autoptr(virQEMUDriverConfig) cfg =3D NULL; + virDomainObj *vm =3D NULL; + g_autoptr(virBitmap) pcpumap =3D NULL; + qemuDomainObjPrivate *priv =3D NULL; + qemuDomainMigThreadInfo **migthreads =3D NULL; + size_t i; + + cfg =3D virQEMUDriverGetConfig(driver); + + if (!(vm =3D qemuDomainObjFromDomain(dom))) + goto cleanup; + + priv =3D vm->privateData; + + if (virDomainPinMigrationThreadEnsureACL(dom->conn, vm->def) < 0) + goto cleanup; + + if (virDomainObjBeginJob(vm, VIR_JOB_QUERY) < 0) + goto cleanup; + + if (!(pcpumap =3D virBitmapNewData(cpumap, maplen))) + goto endjob; + + if (virBitmapIsAllClear(pcpumap)) { + virReportError(VIR_ERR_INVALID_ARG, "%s", + _("Empty migration thread cpumap list for pinning")= ); + goto endjob; + } + + virBitmapFree(priv->pcpumap); + priv->pcpumap =3D virBitmapNewCopy(pcpumap); + migthreads =3D priv->migThreadsInfo; + + if (migthreads && pcpumap) { + for (i =3D 0; i < priv->migThreadCount; i++) { + qemuProcessSetMigThreadAffinity(priv, vm, migthreads[i]->threa= d_id, pcpumap); + } + } + + ret =3D 0; + + endjob: + virDomainObjEndJob(vm); + + cleanup: + virDomainObjEndAPI(&vm); + return ret; +} =20 static virHypervisorDriver qemuHypervisorDriver =3D { .name =3D QEMU_DRIVER_NAME, @@ -20831,6 +20886,7 @@ static virHypervisorDriver qemuHypervisorDriver =3D= { .domainStartDirtyRateCalc =3D qemuDomainStartDirtyRateCalc, /* 7.2.0 */ .domainSetLaunchSecurityState =3D qemuDomainSetLaunchSecurityState, /*= 8.0.0 */ .domainFDAssociate =3D qemuDomainFDAssociate, /* 9.0.0 */ + .domainPinMigrationThread =3D qemuDomainPinMigrationThread, /* 9.1.0 */ }; =20 =20 diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 9a612ca443..946aeb81b1 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -9547,3 +9547,64 @@ qemuProcessQMPStart(qemuProcessQMP *proc) =20 return 0; } + +int +qemuProcessSetupMigration(virDomainObj *vm, + virDomainMigrationIDDef *migration) +{ + return qemuProcessSetupPid(vm, migration->thread_id, + VIR_CGROUP_THREAD_MIGRATION_THREAD, + 0, + vm->def->cputune.emulatorpin, + vm->def->cputune.emulator_period, + vm->def->cputune.emulator_quota, + &migration->sched); +} + +/* + * In order to set migration thread affinity when vm is migrating, + * we should create the cgroup for migration thread. + */ +void +qemuProcessSetMigThreadAffinity(qemuDomainObjPrivate *priv, + virDomainObj *vm, + int mpid, + virBitmap *pcpumap) +{ + virDomainMigrationIDDef *migration =3D NULL; + int migration_id =3D 0; + virCgroup *cgroup_migthread =3D NULL; + + if (!pcpumap) + return; + + migration =3D g_new0(virDomainMigrationIDDef, 1); + migration->thread_id =3D mpid; + if (qemuProcessSetupMigration(vm, migration) < 0) { + VIR_ERROR(_("fail to setup migration cgroup")); + goto cleanup; + } + + if (virCgroupHasController(priv->cgroup, + VIR_CGROUP_CONTROLLER_CPUSET)) { + if (virCgroupNewThread(priv->cgroup, VIR_CGROUP_THREAD_MIGRATION_T= HREAD, + migration_id, false, &cgroup_migthread) < 0) + goto cleanup; + + if (virDomainCgroupSetupCpusetCpus(cgroup_migthread, pcpumap) < 0)= { + virReportError(VIR_ERR_OPERATION_INVALID, + _("failed to set cpuset.cpus in cgroup" + " for migration%d thread"), migration_id); + goto cleanup; + } + } + + if (virProcessSetAffinity(mpid, pcpumap, false) < 0) + VIR_WARN("failed to set affinity in migration"); + + cleanup: + if (cgroup_migthread) + virCgroupFree(cgroup_migthread); + virDomainMigrationIDDefFree(migration); + return; +} diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index b171f0464c..2af1e2ee87 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -235,3 +235,13 @@ bool qemuProcessRebootAllowed(const virDomainDef *def); =20 void qemuProcessCleanupMigrationJob(virQEMUDriver *driver, virDomainObj *vm); + +int +qemuProcessSetupMigration(virDomainObj *vm, + virDomainMigrationIDDef *migration); + +void +qemuProcessSetMigThreadAffinity(qemuDomainObjPrivate *priv, + virDomainObj *vm, + int mpid, + virBitmap *pcpumap); diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index 73675b4478..d4307c0d72 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -1123,6 +1123,9 @@ virCgroupNewThread(virCgroup *domain, case VIR_CGROUP_THREAD_IOTHREAD: name =3D g_strdup_printf("iothread%d", id); break; + case VIR_CGROUP_THREAD_MIGRATION_THREAD: + name =3D g_strdup_printf("migthread%d", id); + break; case VIR_CGROUP_THREAD_LAST: virReportError(VIR_ERR_INTERNAL_ERROR, _("unexpected name value %d"), nameval); diff --git a/src/util/vircgroup.h b/src/util/vircgroup.h index adf3850b22..c5f6ddd7a6 100644 --- a/src/util/vircgroup.h +++ b/src/util/vircgroup.h @@ -53,6 +53,7 @@ typedef enum { VIR_CGROUP_THREAD_VCPU =3D 0, VIR_CGROUP_THREAD_EMULATOR, VIR_CGROUP_THREAD_IOTHREAD, + VIR_CGROUP_THREAD_MIGRATION_THREAD, =20 VIR_CGROUP_THREAD_LAST } virCgroupThreadName; --=20 2.33.0 From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201416; cv=none; d=zohomail.com; s=zohoarc; b=ACW54A5dTq4BUoqlY7YqcWOecXV0kUTZKWmPIQd3MrOHLshSmXaD3f+knMqcDEr3uKYlGvCzQKuQH6yKnVfvC0uhMJmvsu1JSCKS0it0sYf4iSqmpXVu4M2J7uFuFRbvY8/qu2zbYFgYvow71GCA6JhvdF/vhePy3OhoQWps6ok= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201416; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3dwqiN8iKIGS+2+naTLKBap6+msn9Q7gI4xcgxqmv9s=; b=HBJuGIlaAWiuf4gwjwHrREdroWMmTF7npD3TjQNnP7dWurIx9Sn5l22DJ0Jfgo4xMDqcDRn33UZ5p01ISMlFAbEIp1pEiyGFKiULt+P4/V4yvkUseCGuEwu3S4X3x9trP9ciGObXSkOTZYJkXjMPcyuwKmg1PZvADO9gF+Cj4AI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1678201416117100.34109362455763; Tue, 7 Mar 2023 07:03:36 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-591-BUijhEeNNvSr8FPY1SksQA-1; Tue, 07 Mar 2023 10:03:02 -0500 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 207FF100F913; Tue, 7 Mar 2023 15:02:53 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0C46E492C18; Tue, 7 Mar 2023 15:02:53 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BFAD61946A4B; Tue, 7 Mar 2023 15:02:46 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BE08A1946A4B for ; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id B3155C15BAD; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast08.extmail.prod.ext.rdu2.redhat.com [10.11.55.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AB2A7C15BA0 for ; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9056438149AA for ; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-35-3pg5u3DLOluFwGv7caaOpg-1; Tue, 07 Mar 2023 10:02:39 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4PWJXV3k6TzKq6P; Tue, 7 Mar 2023 23:00:26 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:30 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201414; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3dwqiN8iKIGS+2+naTLKBap6+msn9Q7gI4xcgxqmv9s=; b=crD0vFemAG3L7Xin/t0JELu7cKEn71wUjg5kKB/OoqqS3KaF8AaVSEVV+ESyqYAqIachJW XQPTyQCOdSYf0GkjHtTy9ahJmkvbihsfBPEoAGlpZNTcrbXVvKjQzhsrpvUWIxoTbar3Iq 49p1ETEWqV4K/By47XIRKcjULQK2D3Q= X-MC-Unique: BUijhEeNNvSr8FPY1SksQA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: 3pg5u3DLOluFwGv7caaOpg-1 From: Jiang Jiacheng To: Subject: [PATCH v2 5/8] migration/migration-pin: support migration pin by virsh command Date: Tue, 7 Mar 2023 22:52:17 +0800 Message-ID: <20230307145220.3767207-6-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201426138100001 Content-Type: text/plain; charset="utf-8" Add virsh command 'virsh migrationpin', its usage is like 'virsh migrationpin [vm] [cpulist]'. After using this command, if migration thread infomation have been got, pin it to given cpumap immediately, else wait for next migraion operation and pin migration thread after getting information. Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- tools/virsh-domain.c | 69 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index a4abcb1b03..43ecd651d8 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -13814,6 +13814,69 @@ cmdDomDirtyRateCalc(vshControl *ctl, const vshCmd = *cmd) } =20 =20 +/* + * "migrationpin" command + */ +static const vshCmdInfo info_migrationpin[] =3D { + {.name =3D "help", + .data =3D N_("control domain migrationThreads affinity") + }, + {.name =3D "desc", + .data =3D N_("Pin domain migrationThreads to host physical CPUs.") + }, + {.name =3D NULL} +}; + +static const vshCmdOptDef opts_migrationpin[] =3D { + VIRSH_COMMON_OPT_DOMAIN_FULL(0), + {.name =3D "cpulist", + .type =3D VSH_OT_DATA, + .flags =3D VSH_OFLAG_REQ, + .help =3D N_("host cpu number(s) to set") + }, + {.name =3D NULL} +}; + +static bool +cmdMigrationPin(vshControl *ctl, const vshCmd *cmd) +{ + g_autoptr(virshDomain) dom =3D NULL; + const char *name =3D NULL; + const char *cpulist =3D NULL; + int maxcpu; + g_autofree unsigned char *cpumap =3D NULL; + int cpumaplen; + virshControl *priv =3D ctl->privData; + + dom =3D virshCommandOptDomain(ctl, cmd, NULL); + if (!dom) + return false; + + if (vshCommandOptStringReq(ctl, cmd, "domain", &name) < 0) + return false; + + if (vshCommandOptStringReq(ctl, cmd, "cpulist", &cpulist) < 0) + return false; + + maxcpu =3D virshNodeGetCPUCount(priv->conn); + if (maxcpu < 0) + return false; + + cpumap =3D virshParseCPUList(ctl, &cpumaplen, cpulist, maxcpu); + if (!cpumap) + return false; + + if (virDomainPinMigrationThread(dom, cpumap, + cpumaplen) !=3D 0) + return false; + + vshPrintExtra(ctl, _("Pin domain %s's migration thread " + "to cpulist %s success.\n"), name, cpulist); + + return true; +} + + const vshCmdDef domManagementCmds[] =3D { {.name =3D "attach-device", .handler =3D cmdAttachDevice, @@ -14477,5 +14540,11 @@ const vshCmdDef domManagementCmds[] =3D { .info =3D info_dom_fd_associate, .flags =3D 0 }, + {.name =3D "migrationpin", + .handler =3D cmdMigrationPin, + .opts =3D opts_migrationpin, + .info =3D info_migrationpin, + .flags =3D 0 + }, {.name =3D NULL} }; --=20 2.33.0 From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201438; cv=none; d=zohomail.com; s=zohoarc; b=aYfSSlcWXzdCZHK1W6YavjaUniiGKD9Nf8EsmbON1XAmH/7zhz6bsuVd4jD9V7hWEAlBFR1hsPGZ+rL9k6lL+K6LW4C3MYlkuZ+CIs84aW4nzBEO2xpwpUHlluRSHHjYBWrxeTOZytV0l2sVwrwLzsDkkb3RU2Gq1ltMpHepYzY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201438; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XvoWRSl+6mdQSecRaq5glnCfWJX10tGZJ/uia8CqiEs=; b=DJGV9aoJ+TERP2tiph0nlsYfQwNwai7qyCaLF1foLupEzxYgOzT9SMOV9KYf5k9mE/cEyeNSc3Oo0tHHwdYTGOD0HjvMWDlRQhZ2yYyJJFdy/1AWT2Vt7ikaLyeuQ7NyXTvlhu4a6p1irwz4ZWE2iTl2sh6AC6bZ9owdZFBDTAw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1678201438591619.9931599003553; Tue, 7 Mar 2023 07:03:58 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-488-9sihjN7HPS-K7ZDA5v7pCA-1; Tue, 07 Mar 2023 10:03:33 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DB0133C1024B; Tue, 7 Mar 2023 15:03:03 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id C45FE492C3E; Tue, 7 Mar 2023 15:03:03 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C33161946A51; Tue, 7 Mar 2023 15:02:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id EFD681946A4C for ; Tue, 7 Mar 2023 15:02:53 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id D2359C15BAE; Tue, 7 Mar 2023 15:02:53 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast04.extmail.prod.ext.rdu2.redhat.com [10.11.55.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CAA37C15BA0 for ; Tue, 7 Mar 2023 15:02:53 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AB556100F911 for ; Tue, 7 Mar 2023 15:02:53 +0000 (UTC) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-126-JhdAtblyML6itRuqmf6mPA-1; Tue, 07 Mar 2023 10:02:43 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4PWJWN1Zw1zSkPP; Tue, 7 Mar 2023 22:59:28 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:31 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201437; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=XvoWRSl+6mdQSecRaq5glnCfWJX10tGZJ/uia8CqiEs=; b=ErgGWxT11ZJsa6cMdKrOoSjt5cvR8Chydla6UOtJq4WEgvY1sM5qA9MVGla6l1brcRQbyf K3jKJ7QCmHPjypFDiudOuzlWdWkphE4ULavb44BeFJCMiJ5sVP6I0JDAVdRqko1Ejs+d2o Zh6R1rb10VzQ083EDKHZzL9ShrkccSY= X-MC-Unique: 9sihjN7HPS-K7ZDA5v7pCA-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: JhdAtblyML6itRuqmf6mPA-1 From: Jiang Jiacheng To: Subject: [PATCH v2 6/8] migration/migration-pin: add migrationpin for migration parameters Date: Tue, 7 Mar 2023 22:52:18 +0800 Message-ID: <20230307145220.3767207-7-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201472798100001 Content-Type: text/plain; charset="utf-8" Add 'migrationpin' to migration parameters to provide another method to set the cpulist required for migration thread pin. 'migrationpin' can only be set before migration and won't effect if 'virsh migrationpin' is set. Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- include/libvirt/libvirt-domain.h | 10 ++++++++++ src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain.h | 1 + src/qemu/qemu_migration.c | 2 ++ src/qemu/qemu_migration.h | 33 ++++++++++++++++---------------- src/qemu/qemu_migration_params.c | 21 ++++++++++++++++++++ src/qemu/qemu_migration_params.h | 5 +++++ 7 files changed, 57 insertions(+), 16 deletions(-) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-dom= ain.h index b6c01016d8..5a203d97dc 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -1367,6 +1367,16 @@ typedef enum { */ # define VIR_MIGRATE_PARAM_TLS_DESTINATION "tls.destination" =20 +/** + * VIR_MIGRATE_PARAM_MIGRATIONPIN: + * + * virDomainMigrate* params field: the pin of migration threads for + * migration as VIR_TYPED_PARAM_STRING. + * + * Since: 9.1.0 + */ +# define VIR_MIGRATE_PARAM_MIGRATIONPIN "migration.pin" + /* Domain migration. */ virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 3d497dd5d0..04e89bae95 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1903,6 +1903,7 @@ qemuDomainObjPrivateFree(void *data) virObjectUnref(priv->monConfig); g_free(priv->lockState); g_free(priv->origname); + g_free(priv->migrationThreadPinList); =20 virChrdevFree(priv->devs); =20 diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h index af986b5423..f52f2c9957 100644 --- a/src/qemu/qemu_domain.h +++ b/src/qemu/qemu_domain.h @@ -146,6 +146,7 @@ struct _qemuDomainObjPrivate { int nbdPort; /* Port used for migration with NBD */ int migThreadCount; qemuDomainMigThreadInfo **migThreadsInfo; + char *migrationThreadPinList; unsigned short migrationPort; int preMigrationState; unsigned long long preMigrationMemlock; /* Original RLIMIT_MEMLOCK in = case diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 0cea055eec..1fb091255f 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -4891,6 +4891,8 @@ qemuMigrationSrcRun(virQEMUDriver *driver, priv->migMaxBandwidth * 1024 * 1024) < 0) goto error; =20 + qemuMigrationMigrationParamsToVM(migParams, vm); + if (qemuMigrationParamsApply(vm, VIR_ASYNC_JOB_MIGRATION_OUT, migParams, flags) < 0) goto error; diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index d21b6f67e8..f416462c9d 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -66,28 +66,29 @@ =20 /* All supported migration parameters and their types. */ #define QEMU_MIGRATION_PARAMETERS \ - VIR_MIGRATE_PARAM_URI, VIR_TYPED_PARAM_STRING, \ - VIR_MIGRATE_PARAM_DEST_NAME, VIR_TYPED_PARAM_STRING, \ - VIR_MIGRATE_PARAM_DEST_XML, VIR_TYPED_PARAM_STRING, \ - VIR_MIGRATE_PARAM_BANDWIDTH, VIR_TYPED_PARAM_ULLONG, \ - VIR_MIGRATE_PARAM_GRAPHICS_URI, VIR_TYPED_PARAM_STRING, \ - VIR_MIGRATE_PARAM_LISTEN_ADDRESS, VIR_TYPED_PARAM_STRING, \ - VIR_MIGRATE_PARAM_MIGRATE_DISKS, VIR_TYPED_PARAM_STRING | \ - VIR_TYPED_PARAM_MULTIPLE, \ - VIR_MIGRATE_PARAM_DISKS_PORT, VIR_TYPED_PARAM_INT, \ - VIR_MIGRATE_PARAM_COMPRESSION, VIR_TYPED_PARAM_STRING | \ - VIR_TYPED_PARAM_MULTIPLE, \ + VIR_MIGRATE_PARAM_URI, VIR_TYPED_PARAM_STRING= , \ + VIR_MIGRATE_PARAM_DEST_NAME, VIR_TYPED_PARAM_STRING= , \ + VIR_MIGRATE_PARAM_DEST_XML, VIR_TYPED_PARAM_STRING= , \ + VIR_MIGRATE_PARAM_BANDWIDTH, VIR_TYPED_PARAM_ULLONG= , \ + VIR_MIGRATE_PARAM_GRAPHICS_URI, VIR_TYPED_PARAM_STRING= , \ + VIR_MIGRATE_PARAM_LISTEN_ADDRESS, VIR_TYPED_PARAM_STRING= , \ + VIR_MIGRATE_PARAM_MIGRATE_DISKS, VIR_TYPED_PARAM_STRING= | \ + VIR_TYPED_PARAM_MULTIP= LE, \ + VIR_MIGRATE_PARAM_DISKS_PORT, VIR_TYPED_PARAM_INT, \ + VIR_MIGRATE_PARAM_COMPRESSION, VIR_TYPED_PARAM_STRING= | \ + VIR_TYPED_PARAM_MULTIP= LE, \ VIR_MIGRATE_PARAM_COMPRESSION_MT_LEVEL, VIR_TYPED_PARAM_INT, \ VIR_MIGRATE_PARAM_COMPRESSION_MT_THREADS, VIR_TYPED_PARAM_INT, \ VIR_MIGRATE_PARAM_COMPRESSION_MT_DTHREADS, VIR_TYPED_PARAM_INT, \ VIR_MIGRATE_PARAM_COMPRESSION_XBZRLE_CACHE, VIR_TYPED_PARAM_ULLONG= , \ - VIR_MIGRATE_PARAM_PERSIST_XML, VIR_TYPED_PARAM_STRING, \ + VIR_MIGRATE_PARAM_PERSIST_XML, VIR_TYPED_PARAM_STRING= , \ VIR_MIGRATE_PARAM_AUTO_CONVERGE_INITIAL, VIR_TYPED_PARAM_INT, \ VIR_MIGRATE_PARAM_AUTO_CONVERGE_INCREMENT, VIR_TYPED_PARAM_INT, \ - VIR_MIGRATE_PARAM_BANDWIDTH_POSTCOPY, VIR_TYPED_PARAM_ULLONG, \ - VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS, VIR_TYPED_PARAM_INT, \ - VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING, \ - VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING, \ + VIR_MIGRATE_PARAM_BANDWIDTH_POSTCOPY, VIR_TYPED_PARAM_ULLONG= , \ + VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS, VIR_TYPED_PARAM_INT, \ + VIR_MIGRATE_PARAM_TLS_DESTINATION, VIR_TYPED_PARAM_STRING= , \ + VIR_MIGRATE_PARAM_DISKS_URI, VIR_TYPED_PARAM_STRING= , \ + VIR_MIGRATE_PARAM_MIGRATIONPIN, VIR_TYPED_PARAM_STRING= , \ NULL =20 =20 diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_par= ams.c index 43bed2e618..7852f3e295 100644 --- a/src/qemu/qemu_migration_params.c +++ b/src/qemu/qemu_migration_params.c @@ -114,6 +114,7 @@ VIR_ENUM_IMPL(qemuMigrationParam, "xbzrle-cache-size", "max-postcopy-bandwidth", "multifd-channels", + "migrationpin", ); =20 typedef struct _qemuMigrationParamsAlwaysOnItem qemuMigrationParamsAlwaysO= nItem; @@ -228,6 +229,10 @@ static const qemuMigrationParamsTPMapItem qemuMigratio= nParamsTPMap[] =3D { {.typedParam =3D VIR_MIGRATE_PARAM_TLS_DESTINATION, .param =3D QEMU_MIGRATION_PARAM_TLS_HOSTNAME, .party =3D QEMU_MIGRATION_SOURCE}, + + {.typedParam =3D VIR_MIGRATE_PARAM_MIGRATIONPIN, + .param =3D QEMU_MIGRATION_PARAM_MIGRATIONPIN, + .party =3D QEMU_MIGRATION_SOURCE}, }; =20 static const qemuMigrationParamInfoItem qemuMigrationParamInfo[] =3D { @@ -271,6 +276,9 @@ static const qemuMigrationParamInfoItem qemuMigrationPa= ramInfo[] =3D { [QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS] =3D { .type =3D QEMU_MIGRATION_PARAM_TYPE_INT, }, + [QEMU_MIGRATION_PARAM_MIGRATIONPIN] =3D { + .type =3D QEMU_MIGRATION_PARAM_TYPE_STRING, + }, }; G_STATIC_ASSERT(G_N_ELEMENTS(qemuMigrationParamInfo) =3D=3D QEMU_MIGRATION= _PARAM_LAST); =20 @@ -798,6 +806,10 @@ qemuMigrationParamsToJSON(qemuMigrationParams *migPara= ms, if (!pv->set) continue; =20 + if (i =3D=3D QEMU_MIGRATION_PARAM_MIGRATIONPIN) { + continue; + } + if (postcopyResume && !qemuMigrationParamInfo[i].applyOnPostcopyRe= sume) continue; =20 @@ -1559,3 +1571,12 @@ qemuMigrationParamsGetMigThreadCount(qemuMigrationPa= rams *migParams, =20 return nmigthreads; } + +void +qemuMigrationMigrationParamsToVM(const qemuMigrationParams *migParams, con= st virDomainObj *vm) +{ + if (migParams && migParams->params[QEMU_MIGRATION_PARAM_MIGRATIONPIN].= set) { + qemuDomainObjPrivate *priv =3D vm->privateData; + priv->migrationThreadPinList =3D g_strdup(migParams->params[QEMU_M= IGRATION_PARAM_MIGRATIONPIN].value.s); + } +} diff --git a/src/qemu/qemu_migration_params.h b/src/qemu/qemu_migration_par= ams.h index 975b6691d4..d8321cace8 100644 --- a/src/qemu/qemu_migration_params.h +++ b/src/qemu/qemu_migration_params.h @@ -59,6 +59,7 @@ typedef enum { QEMU_MIGRATION_PARAM_XBZRLE_CACHE_SIZE, QEMU_MIGRATION_PARAM_MAX_POSTCOPY_BANDWIDTH, QEMU_MIGRATION_PARAM_MULTIFD_CHANNELS, + QEMU_MIGRATION_PARAM_MIGRATIONPIN, =20 QEMU_MIGRATION_PARAM_LAST } qemuMigrationParam; @@ -170,3 +171,7 @@ qemuMigrationParamsGetTLSHostname(qemuMigrationParams *= migParams); int qemuMigrationParamsGetMigThreadCount(qemuMigrationParams *migParams, unsigned long migflags); + +void +qemuMigrationMigrationParamsToVM(const qemuMigrationParams *migParams, + const virDomainObj *vm); --=20 2.33.0 From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201966; cv=none; d=zohomail.com; s=zohoarc; b=WwzfHqvbyg0n9tMIWNo6/vty1qJ0sd+4CWU2ncPbxqf9aWDtuEMiwPbg8g/iL1YhRESJlJ2bWSz8lRP6UQvgBsBW6Mcd1qR4Nt+8nANpsVZqI9eRifB2EG9A4fBVDNfGegoo/G3VK19qW9GzqFXzT5HeCAcv+p/ofPeb1S2fgk4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201966; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7FbPBa5t6A51CVEXR/BkmiSnnpb3bYmbMyCgyO4a8xM=; b=A7IC/Z7ssFXDXE85erpKCE7qa4F1hejlDPeRD04mQps8dfNFEbvEtLkzkcmvx5dP5e1pqf8GjpyMN8bWylqYlCMWVgqNYXBMNp7+64Sy15r/TZYEoOffZn7amdHxveUuetLEKHXjn5fK83V1FC7L6ozgia3TrMUuNWn0ch1PX7c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1678201966213994.8469822119844; Tue, 7 Mar 2023 07:12:46 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-329-mcG5Q3q7OH6MdKuQtBcrww-1; Tue, 07 Mar 2023 10:08:43 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 35ECE85CCEA; Tue, 7 Mar 2023 15:03:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C6C414171B6; Tue, 7 Mar 2023 15:03:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 018CE1946A72; Tue, 7 Mar 2023 15:02:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0769819465BA for ; Tue, 7 Mar 2023 15:02:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id F0B8D40B40E4; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast08.extmail.prod.ext.rdu2.redhat.com [10.11.55.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E900D401B290 for ; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CD10B38149A3 for ; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-651-fsCrU2ofNKCna5_-Jpx_vg-1; Tue, 07 Mar 2023 10:02:40 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PWJWh3VCmznWWL; Tue, 7 Mar 2023 22:59:44 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:31 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201965; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=7FbPBa5t6A51CVEXR/BkmiSnnpb3bYmbMyCgyO4a8xM=; b=HiI8x9tbnQgSHfqDLnjVCgZ9IQowL2Z/kEAGg0jw3q51Ps+RgZrYlOGAxww0ETcQ0+f8g8 r6DQvAH5Ms3GqfRsRTqywaWkrgO0174Pmhvuh8u+rMzhEtn1LmekezZKnYxwf3aNA0820j Wh5raMJXCbQhFqy0dqGbNWA4jRoBF+Q= X-MC-Unique: mcG5Q3q7OH6MdKuQtBcrww-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: fsCrU2ofNKCna5_-Jpx_vg-1 From: Jiang Jiacheng To: Subject: [PATCH v2 7/8] migration/migration-pin: pin migration thread immediately after get thread info Date: Tue, 7 Mar 2023 22:52:19 +0800 Message-ID: <20230307145220.3767207-8-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201967814100001 Content-Type: text/plain; charset="utf-8" If pcpumap is set before migration, pin migration thread to expected cpulist immediately after get thread info during migration. Add 'qemuProcessGetPcpumap' to get pcpumap, try get the result of 'virsh migratepin' first, and if it's not set, try to get the pcpumap from migration parameter. Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- src/qemu/qemu_migration.c | 28 ++++++++++++++ src/qemu/qemu_process.c | 81 +++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_process.h | 3 ++ 3 files changed, 112 insertions(+) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 1fb091255f..30f41e8d25 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2003,6 +2003,23 @@ qemuMigrationAnyCompleted(virDomainObj *vm, return -1; } =20 + +static void +qemuMigrationSrcThreadPin(virDomainObj *vm, + virBitmap *pcpumap, + int nmigthreads) +{ + qemuDomainObjPrivate *priv =3D vm->privateData; + qemuDomainMigThreadInfo **migthreads =3D priv->migThreadsInfo; + size_t i; + + + for (i =3D 0; i < nmigthreads; i++) { + qemuProcessSetMigThreadAffinity(priv, vm, migthreads[i]->thread_id= , pcpumap); + } +} + + /* Returns 0 on get migration thread info success, * -1 on unexpected failure */ @@ -2018,6 +2035,7 @@ qemuMigrationSrcWaitForThreadInfo(virDomainObj *vm, qemuDomainObjPrivate *priv =3D vm->privateData; qemuMonitorMigThreadInfo **infolist =3D NULL; qemuDomainMigThreadInfo **migthreads =3D NULL; + virBitmap *pcpumap =3D NULL; int nmigthreads =3D 0; int rc; size_t i; @@ -2065,7 +2083,17 @@ qemuMigrationSrcWaitForThreadInfo(virDomainObj *vm, priv->migThreadsInfo =3D migthreads; ret =3D 0; =20 + /* Try get pcpumap for migration thread pin, + * but thread pin failed shouldn't effect migration */ + if (!(pcpumap =3D qemuProcessGetPcpumap(priv))) + goto cleanup; + + qemuMigrationSrcThreadPin(vm, pcpumap, nmigthreads); + cleanup: + if (pcpumap !=3D priv->pcpumap) + virBitmapFree(pcpumap); + if (migthreads && (ret < 0)) { qemuDomainMigThreadsInfoFree(migthreads, priv->migThreadCount); priv->migThreadCount =3D 0; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 946aeb81b1..e787db8b24 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -9608,3 +9608,84 @@ qemuProcessSetMigThreadAffinity(qemuDomainObjPrivate= *priv, virDomainMigrationIDDefFree(migration); return; } + + +static unsigned char * +virParseCPUList(int *cpumaplen, const char *cpulist, int maxcpu) +{ + int lastcpu; + unsigned char *cpumap =3D NULL; + virBitmap *map =3D NULL; + + if (cpulist[0] =3D=3D 'r') { + map =3D virBitmapNew(maxcpu); + if (!map) + return NULL; + virBitmapSetAll(map); + } else { + if (virBitmapParse(cpulist, &map, 1024) < 0 || + virBitmapIsAllClear(map)) { + goto cleanup; + } + + lastcpu =3D virBitmapLastSetBit(map); + if (lastcpu >=3D maxcpu) + goto cleanup; + } + + if (virBitmapToData(map, &cpumap, cpumaplen) < 0) + VIR_ERROR(_("Bitmap to data failure")); + + cleanup: + virBitmapFree(map); + return cpumap; +} + + +/* + * The value of "virsh migrationpin" is saved to priv->pcpumap + * If priv->pcpumap is NULL, it means migrationpin command is not called, + * otherwise we set the affinity of migration thread by migrationpin + */ +virBitmap* +qemuProcessGetPcpumap(qemuDomainObjPrivate *priv) +{ + int cpumaplen =3D 0; + int maxcpu =3D 0; + virBitmap *pcpumap =3D NULL; + g_autofree unsigned char *cpumap =3D NULL; + + /* priv->pcpumap =3D NULL means migrationpin is not set, try to get pc= pumap + * from migration parameter. */ + if (priv->pcpumap) + return priv->pcpumap; + + if (!(priv->migrationThreadPinList) || STREQ(priv->migrationThreadPinL= ist, "")) { + VIR_ERROR(_("didn't set the migratin thread pin")); + return NULL; + } + + /* judge whether set_migration_pin is default value or not */ + if (STREQ(priv->migrationThreadPinList, "none")) + return NULL; + + maxcpu =3D virHostCPUGetCount(); + if (maxcpu < 0) { + VIR_ERROR(_("get the cpu count of host failure")); + return NULL; + } + + cpumap =3D virParseCPUList(&cpumaplen, priv->migrationThreadPinList, m= axcpu); + if (!cpumap) { + VIR_ERROR(_("parse migration.pin param failure : migration.pin =3D= %s"), + priv->migrationThreadPinList); + return NULL; + } + + if (!(pcpumap =3D virBitmapNewData(cpumap, cpumaplen))) { + VIR_ERROR(_("Bitmap data failure")); + return NULL; + } + + return pcpumap; +} diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index 2af1e2ee87..74e96ccb07 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -245,3 +245,6 @@ qemuProcessSetMigThreadAffinity(qemuDomainObjPrivate *p= riv, virDomainObj *vm, int mpid, virBitmap *pcpumap); + +virBitmap* +qemuProcessGetPcpumap(qemuDomainObjPrivate *priv); --=20 2.33.0 From nobody Sat Apr 27 04:32:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201420; cv=none; d=zohomail.com; s=zohoarc; b=fYKMpA5ye+WIO5IyTH/BHp+DNtK36vk9MJw7FUL7Hu+5e6NPYZVDayuX0QfQHMli+TCTtbLbTc8fJ2Eg4hOhq/xrcQTgxpy4c0kvsowlADFqaWvZST8znAV/kyvA3cFUvtUijZy5jB1T68y7lFy8yUsBL9xCO1FCcoLntP2UEno= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201420; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EQv4BExwAEtynWILfvd/xMbE970K6dRw+3RaIExjwq0=; b=G9AYdK2y4Bb7aRv+dYkBCUduxxmJK7CjD+jQBJl5DeyA99DGsynskmhva316z0/LOXqn+3ZRNm72gI4Wk5dn8QUW4ve1o37ojgWCc+71RsCo6+jrALnMx2RJjKq2LMQpMCxRHekWQK91Qow8zbFfJjdIbooX+z1MMojhhVHj9dY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 167820142025759.34331495747563; Tue, 7 Mar 2023 07:03:40 -0800 (PST) Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638--TQjYDBLMZuMy54_Ih419g-1; Tue, 07 Mar 2023 10:03:29 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4ADD11C0A58A; Tue, 7 Mar 2023 15:03:06 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 372F81121314; Tue, 7 Mar 2023 15:03:06 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0ED6F1946A46; Tue, 7 Mar 2023 15:02:58 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 807E61946A4C for ; Tue, 7 Mar 2023 15:02:54 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 74B2014171C4; Tue, 7 Mar 2023 15:02:54 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast02.extmail.prod.ext.rdu2.redhat.com [10.11.55.18]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6BC8E14171C7 for ; Tue, 7 Mar 2023 15:02:54 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4F4DC803CBB for ; Tue, 7 Mar 2023 15:02:54 +0000 (UTC) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-660-FESoB3uPNxarJzY89jxaoA-1; Tue, 07 Mar 2023 10:02:41 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PWJZ30wVLzrSJK; Tue, 7 Mar 2023 23:01:47 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:32 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201419; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=EQv4BExwAEtynWILfvd/xMbE970K6dRw+3RaIExjwq0=; b=G6BeoAANUz4D4favRhOUrqIzWmDitD2iYMa5PspUU+31Vg2K5MQ7lA65SdkFau32mxNP9o Hc3LRv4y/iyqHObw/dgp+97wHH931tWjgoLc6ejUReLDlwwFAxDtxiq15dVqR1rb+lFcLs QLnPYolIpIxBG9LuvdcZsuVu3GQ0Wko= X-MC-Unique: -TQjYDBLMZuMy54_Ih419g-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: FESoB3uPNxarJzY89jxaoA-1 From: Jiang Jiacheng To: Subject: [PATCH v2 8/8] migration/migration-pin/multifd-pin: add migration pin status handle Date: Tue, 7 Mar 2023 22:52:20 +0800 Message-ID: <20230307145220.3767207-9-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201449502100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- src/qemu/qemu_process.c | 56 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index e787db8b24..85a22c7e51 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -1438,6 +1438,61 @@ qemuProcessHandleSpiceMigrated(qemuMonitor *mon G_GN= UC_UNUSED, } =20 =20 +static void +qemuProcessHandleMigrationPinStatus(virDomainObj *vm, int status) +{ + qemuDomainObjPrivate *priv =3D vm->privateData; + + if (vm->job->asyncJob !=3D VIR_ASYNC_JOB_MIGRATION_OUT) + return; + + switch (status) { + case QEMU_MONITOR_MIGRATION_STATUS_INACTIVE: + case QEMU_MONITOR_MIGRATION_STATUS_SETUP: + case QEMU_MONITOR_MIGRATION_STATUS_ACTIVE: + case QEMU_MONITOR_MIGRATION_STATUS_PRE_SWITCHOVER: + case QEMU_MONITOR_MIGRATION_STATUS_DEVICE: + case QEMU_MONITOR_MIGRATION_STATUS_POSTCOPY: + case QEMU_MONITOR_MIGRATION_STATUS_POSTCOPY_PAUSED: + case QEMU_MONITOR_MIGRATION_STATUS_POSTCOPY_RECOVER: + case QEMU_MONITOR_MIGRATION_STATUS_CANCELLING: + case QEMU_MONITOR_MIGRATION_STATUS_COMPLETED: + case QEMU_MONITOR_MIGRATION_STATUS_WAIT_UNPLUG: + break; + case QEMU_MONITOR_MIGRATION_STATUS_ERROR: + /* + * migration thread is still running, + * so we can't delete migration Cgroup. + */ + qemuDomainMigThreadsInfoFree(priv->migThreadsInfo, priv->migThread= Count); + priv->migThreadsInfo =3D NULL; + priv->migThreadCount =3D 0; + g_free(priv->migrationThreadPinList); + virBitmapFree(priv->pcpumap); + priv->pcpumap =3D NULL; + break; + case QEMU_MONITOR_MIGRATION_STATUS_CANCELLED: + qemuDomainMigThreadsInfoFree(priv->migThreadsInfo, priv->migThread= Count); + priv->migThreadsInfo =3D NULL; + priv->migThreadCount =3D 0; + g_free(priv->migrationThreadPinList); + virBitmapFree(priv->pcpumap); + priv->pcpumap =3D NULL; + if (virCgroupDelThread(priv->cgroup, + VIR_CGROUP_THREAD_MIGRATION_THREAD, 0) < 0) + VIR_WARN("Failed to delete migration thread Cgroup!"); + VIR_INFO("success to free pcpumap and migrationPids"); + break; + default: + VIR_WARN("got unknown migration status'%s'", + qemuMonitorMigrationStatusTypeToString(status)); + break; + } + + return; +} + + static void qemuProcessHandleMigrationStatus(qemuMonitor *mon G_GNUC_UNUSED, virDomainObj *vm, @@ -1552,6 +1607,7 @@ qemuProcessHandleMigrationStatus(qemuMonitor *mon G_G= NUC_UNUSED, default: break; } + qemuProcessHandleMigrationPinStatus(vm, status); =20 cleanup: virObjectUnlock(vm); --=20 2.33.0