From nobody Fri Oct 18 08:36:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=quarantine dis=quarantine) header.from=huawei.com ARC-Seal: i=1; a=rsa-sha256; t=1678201966; cv=none; d=zohomail.com; s=zohoarc; b=WwzfHqvbyg0n9tMIWNo6/vty1qJ0sd+4CWU2ncPbxqf9aWDtuEMiwPbg8g/iL1YhRESJlJ2bWSz8lRP6UQvgBsBW6Mcd1qR4Nt+8nANpsVZqI9eRifB2EG9A4fBVDNfGegoo/G3VK19qW9GzqFXzT5HeCAcv+p/ofPeb1S2fgk4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678201966; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7FbPBa5t6A51CVEXR/BkmiSnnpb3bYmbMyCgyO4a8xM=; b=A7IC/Z7ssFXDXE85erpKCE7qa4F1hejlDPeRD04mQps8dfNFEbvEtLkzkcmvx5dP5e1pqf8GjpyMN8bWylqYlCMWVgqNYXBMNp7+64Sy15r/TZYEoOffZn7amdHxveUuetLEKHXjn5fK83V1FC7L6ozgia3TrMUuNWn0ch1PX7c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail header.from= (p=quarantine dis=quarantine) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1678201966213994.8469822119844; Tue, 7 Mar 2023 07:12:46 -0800 (PST) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-329-mcG5Q3q7OH6MdKuQtBcrww-1; Tue, 07 Mar 2023 10:08:43 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 35ECE85CCEA; Tue, 7 Mar 2023 15:03:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C6C414171B6; Tue, 7 Mar 2023 15:03:01 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 018CE1946A72; Tue, 7 Mar 2023 15:02:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0769819465BA for ; Tue, 7 Mar 2023 15:02:46 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id F0B8D40B40E4; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast08.extmail.prod.ext.rdu2.redhat.com [10.11.55.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E900D401B290 for ; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CD10B38149A3 for ; Tue, 7 Mar 2023 15:02:45 +0000 (UTC) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-651-fsCrU2ofNKCna5_-Jpx_vg-1; Tue, 07 Mar 2023 10:02:40 -0500 Received: from dggpeml500022.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PWJWh3VCmznWWL; Tue, 7 Mar 2023 22:59:44 +0800 (CST) Received: from localhost.localdomain (10.175.124.27) by dggpeml500022.china.huawei.com (7.185.36.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Tue, 7 Mar 2023 23:02:31 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678201965; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=7FbPBa5t6A51CVEXR/BkmiSnnpb3bYmbMyCgyO4a8xM=; b=HiI8x9tbnQgSHfqDLnjVCgZ9IQowL2Z/kEAGg0jw3q51Ps+RgZrYlOGAxww0ETcQ0+f8g8 r6DQvAH5Ms3GqfRsRTqywaWkrgO0174Pmhvuh8u+rMzhEtn1LmekezZKnYxwf3aNA0820j Wh5raMJXCbQhFqy0dqGbNWA4jRoBF+Q= X-MC-Unique: mcG5Q3q7OH6MdKuQtBcrww-1 X-Original-To: libvir-list@listman.corp.redhat.com X-MC-Unique: fsCrU2ofNKCna5_-Jpx_vg-1 From: Jiang Jiacheng To: Subject: [PATCH v2 7/8] migration/migration-pin: pin migration thread immediately after get thread info Date: Tue, 7 Mar 2023 22:52:19 +0800 Message-ID: <20230307145220.3767207-8-jiangjiacheng@huawei.com> In-Reply-To: <20230307145220.3767207-1-jiangjiacheng@huawei.com> References: <20230307145220.3767207-1-jiangjiacheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500022.china.huawei.com (7.185.36.66) X-CFilter-Loop: Reflected X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yubihong@huawei.com, quintela@redhat.com, linyilu@huawei.com, jiangjiacheng@huawei.com, dgilbert@redhat.com, xiexiangyou@huawei.com, zhengchuan@huawei.com, caozhongwang1@huawei.com, wanghao232@huawei.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1678201967814100001 Content-Type: text/plain; charset="utf-8" If pcpumap is set before migration, pin migration thread to expected cpulist immediately after get thread info during migration. Add 'qemuProcessGetPcpumap' to get pcpumap, try get the result of 'virsh migratepin' first, and if it's not set, try to get the pcpumap from migration parameter. Signed-off-by: zhengchuan Signed-off-by: Jiang Jiacheng --- src/qemu/qemu_migration.c | 28 ++++++++++++++ src/qemu/qemu_process.c | 81 +++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_process.h | 3 ++ 3 files changed, 112 insertions(+) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 1fb091255f..30f41e8d25 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2003,6 +2003,23 @@ qemuMigrationAnyCompleted(virDomainObj *vm, return -1; } =20 + +static void +qemuMigrationSrcThreadPin(virDomainObj *vm, + virBitmap *pcpumap, + int nmigthreads) +{ + qemuDomainObjPrivate *priv =3D vm->privateData; + qemuDomainMigThreadInfo **migthreads =3D priv->migThreadsInfo; + size_t i; + + + for (i =3D 0; i < nmigthreads; i++) { + qemuProcessSetMigThreadAffinity(priv, vm, migthreads[i]->thread_id= , pcpumap); + } +} + + /* Returns 0 on get migration thread info success, * -1 on unexpected failure */ @@ -2018,6 +2035,7 @@ qemuMigrationSrcWaitForThreadInfo(virDomainObj *vm, qemuDomainObjPrivate *priv =3D vm->privateData; qemuMonitorMigThreadInfo **infolist =3D NULL; qemuDomainMigThreadInfo **migthreads =3D NULL; + virBitmap *pcpumap =3D NULL; int nmigthreads =3D 0; int rc; size_t i; @@ -2065,7 +2083,17 @@ qemuMigrationSrcWaitForThreadInfo(virDomainObj *vm, priv->migThreadsInfo =3D migthreads; ret =3D 0; =20 + /* Try get pcpumap for migration thread pin, + * but thread pin failed shouldn't effect migration */ + if (!(pcpumap =3D qemuProcessGetPcpumap(priv))) + goto cleanup; + + qemuMigrationSrcThreadPin(vm, pcpumap, nmigthreads); + cleanup: + if (pcpumap !=3D priv->pcpumap) + virBitmapFree(pcpumap); + if (migthreads && (ret < 0)) { qemuDomainMigThreadsInfoFree(migthreads, priv->migThreadCount); priv->migThreadCount =3D 0; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 946aeb81b1..e787db8b24 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -9608,3 +9608,84 @@ qemuProcessSetMigThreadAffinity(qemuDomainObjPrivate= *priv, virDomainMigrationIDDefFree(migration); return; } + + +static unsigned char * +virParseCPUList(int *cpumaplen, const char *cpulist, int maxcpu) +{ + int lastcpu; + unsigned char *cpumap =3D NULL; + virBitmap *map =3D NULL; + + if (cpulist[0] =3D=3D 'r') { + map =3D virBitmapNew(maxcpu); + if (!map) + return NULL; + virBitmapSetAll(map); + } else { + if (virBitmapParse(cpulist, &map, 1024) < 0 || + virBitmapIsAllClear(map)) { + goto cleanup; + } + + lastcpu =3D virBitmapLastSetBit(map); + if (lastcpu >=3D maxcpu) + goto cleanup; + } + + if (virBitmapToData(map, &cpumap, cpumaplen) < 0) + VIR_ERROR(_("Bitmap to data failure")); + + cleanup: + virBitmapFree(map); + return cpumap; +} + + +/* + * The value of "virsh migrationpin" is saved to priv->pcpumap + * If priv->pcpumap is NULL, it means migrationpin command is not called, + * otherwise we set the affinity of migration thread by migrationpin + */ +virBitmap* +qemuProcessGetPcpumap(qemuDomainObjPrivate *priv) +{ + int cpumaplen =3D 0; + int maxcpu =3D 0; + virBitmap *pcpumap =3D NULL; + g_autofree unsigned char *cpumap =3D NULL; + + /* priv->pcpumap =3D NULL means migrationpin is not set, try to get pc= pumap + * from migration parameter. */ + if (priv->pcpumap) + return priv->pcpumap; + + if (!(priv->migrationThreadPinList) || STREQ(priv->migrationThreadPinL= ist, "")) { + VIR_ERROR(_("didn't set the migratin thread pin")); + return NULL; + } + + /* judge whether set_migration_pin is default value or not */ + if (STREQ(priv->migrationThreadPinList, "none")) + return NULL; + + maxcpu =3D virHostCPUGetCount(); + if (maxcpu < 0) { + VIR_ERROR(_("get the cpu count of host failure")); + return NULL; + } + + cpumap =3D virParseCPUList(&cpumaplen, priv->migrationThreadPinList, m= axcpu); + if (!cpumap) { + VIR_ERROR(_("parse migration.pin param failure : migration.pin =3D= %s"), + priv->migrationThreadPinList); + return NULL; + } + + if (!(pcpumap =3D virBitmapNewData(cpumap, cpumaplen))) { + VIR_ERROR(_("Bitmap data failure")); + return NULL; + } + + return pcpumap; +} diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index 2af1e2ee87..74e96ccb07 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -245,3 +245,6 @@ qemuProcessSetMigThreadAffinity(qemuDomainObjPrivate *p= riv, virDomainObj *vm, int mpid, virBitmap *pcpumap); + +virBitmap* +qemuProcessGetPcpumap(qemuDomainObjPrivate *priv); --=20 2.33.0