From nobody Mon Nov 25 05:35:14 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1717503306; cv=none; d=zohomail.com; s=zohoarc; b=n1U3bZF5nb7Mej7UUnm1qeDeYwweAXMUbNVh+Zgb+1gwu0P4pQ8uXkBVsCCNNFUF2MBNryX6+jhm97H64DtZ8JJlSORNQJcKRF/JVRlednRcfsR5s13AwwCsL6/nXLhLucDY9aTFZaDVreD2NiCx7TBKbEA5unkDhKRxGFu/y/s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1717503306; h=Content-Type:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=XZ1uiPQmQrPdYPLVweGoc/EHbMf8brAvj3mTAO0shuY=; b=QU4+MFbEWc1xwwYZUvp0ZlUBKjKyr94rZ+O8ob8tSRNMV7iQrBp3+Uz728FIJTY3OEp2B3Jdy7dZrWruwbO0FtTbqZSU3rq3eJlEX6TtP7H7/7ISr9EyN65fuG4Y4Tel5FZbVVlJWyIkO9/yCAAsPHdm1RY4FiVKiMTJYr4gPAk= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1717503306814942.0355163505163; Tue, 4 Jun 2024 05:15:06 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sET3q-0003iw-0Z; Tue, 04 Jun 2024 08:14:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sET3l-0003gk-Rx for qemu-devel@nongnu.org; Tue, 04 Jun 2024 08:14:37 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sET3g-0006xF-Qf for qemu-devel@nongnu.org; Tue, 04 Jun 2024 08:14:37 -0400 Received: from mail.maildlp.com (unknown [172.19.88.214]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4VtqDf2DGjz2Cjml; Tue, 4 Jun 2024 20:10:42 +0800 (CST) Received: from dggpemf200006.china.huawei.com (unknown [7.185.36.61]) by mail.maildlp.com (Postfix) with ESMTPS id DA27A1A016F; Tue, 4 Jun 2024 20:14:22 +0800 (CST) Received: from DESKTOP-8LI8G6S.china.huawei.com (10.173.124.235) by dggpemf200006.china.huawei.com (7.185.36.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 4 Jun 2024 20:14:22 +0800 To: CC: , , , , , , , , , , , , , , , Jialin Wang Subject: [PATCH 5/6] migration: introduce new RDMA live migration Date: Tue, 4 Jun 2024 20:14:11 +0800 Message-ID: <1717503252-51884-6-git-send-email-arei.gonglei@huawei.com> X-Mailer: git-send-email 2.8.2.windows.1 In-Reply-To: <1717503252-51884-1-git-send-email-arei.gonglei@huawei.com> References: <1717503252-51884-1-git-send-email-arei.gonglei@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.173.124.235] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemf200006.china.huawei.com (7.185.36.61) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.190; envelope-from=arei.gonglei@huawei.com; helo=szxga04-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Gonglei From: Gonglei via Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1717503307955100001 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jialin Wang Signed-off-by: Jialin Wang Signed-off-by: Gonglei --- migration/meson.build | 2 + migration/migration.c | 11 +++++- migration/rdma.c | 88 +++++++++++++++++++++++++++++++++++++++++++ migration/rdma.h | 24 ++++++++++++ 4 files changed, 124 insertions(+), 1 deletion(-) create mode 100644 migration/rdma.c create mode 100644 migration/rdma.h diff --git a/migration/meson.build b/migration/meson.build index 4e8a9ccf3e..04e2e16239 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -42,3 +42,5 @@ system_ss.add(when: zstd, if_true: files('multifd-zstd.c'= )) specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_true: files('ram.c', 'target.c')) + +system_ss.add(when: rdma, if_true: files('rdma.c')) diff --git a/migration/migration.c b/migration/migration.c index 6b9ad4ff5f..77c301d351 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -25,6 +25,7 @@ #include "sysemu/runstate.h" #include "sysemu/sysemu.h" #include "sysemu/cpu-throttle.h" +#include "rdma.h" #include "ram.h" #include "migration/global_state.h" #include "migration/misc.h" @@ -145,7 +146,7 @@ static bool transport_supports_multi_channels(Migration= Address *addr) } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_FILE) { return migrate_mapped_ram(); } else { - return false; + return addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_RDMA; } } =20 @@ -644,6 +645,10 @@ static void qemu_start_incoming_migration(const char *= uri, bool has_channels, } else if (saddr->type =3D=3D SOCKET_ADDRESS_TYPE_FD) { fd_start_incoming_migration(saddr->u.fd.str, errp); } +#ifdef CONFIG_RDMA + } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_RDMA) { + rdma_start_incoming_migration(&addr->u.rdma, errp); +#endif } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_EXEC) { exec_start_incoming_migration(addr->u.exec.args, errp); } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_FILE) { @@ -2046,6 +2051,10 @@ void qmp_migrate(const char *uri, bool has_channels, } else if (saddr->type =3D=3D SOCKET_ADDRESS_TYPE_FD) { fd_start_outgoing_migration(s, saddr->u.fd.str, &local_err); } +#ifdef CONFIG_RDMA + } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_RDMA) { + rdma_start_outgoing_migration(s, &addr->u.rdma, &local_err); +#endif } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_EXEC) { exec_start_outgoing_migration(s, addr->u.exec.args, &local_err); } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_FILE) { diff --git a/migration/rdma.c b/migration/rdma.c new file mode 100644 index 0000000000..09a4de7f59 --- /dev/null +++ b/migration/rdma.c @@ -0,0 +1,88 @@ +/* + * QEMU live migration via RDMA + * + * Copyright (c) 2024 HUAWEI TECHNOLOGIES CO., LTD. + * + * Authors: + * Jialin Wang + * Gonglei + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "io/channel-rdma.h" +#include "io/channel.h" +#include "qapi/clone-visitor.h" +#include "qapi/qapi-types-sockets.h" +#include "qapi/qapi-visit-sockets.h" +#include "channel.h" +#include "migration.h" +#include "rdma.h" +#include "trace.h" +#include + +static struct RDMAOutgoingArgs { + InetSocketAddress *addr; +} outgoing_args; + +static void rdma_outgoing_migration(QIOTask *task, gpointer opaque) +{ + MigrationState *s =3D opaque; + QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(qio_task_get_source(task)); + + migration_channel_connect(s, QIO_CHANNEL(rioc), outgoing_args.addr->ho= st, + NULL); + object_unref(OBJECT(rioc)); +} + +void rdma_start_outgoing_migration(MigrationState *s, InetSocketAddress *i= addr, + Error **errp) +{ + QIOChannelRDMA *rioc =3D qio_channel_rdma_new(); + + /* in case previous migration leaked it */ + qapi_free_InetSocketAddress(outgoing_args.addr); + outgoing_args.addr =3D QAPI_CLONE(InetSocketAddress, iaddr); + + qio_channel_set_name(QIO_CHANNEL(rioc), "migration-rdma-outgoing"); + qio_channel_rdma_connect_async(rioc, iaddr, rdma_outgoing_migration, s, + NULL, NULL); +} + +static void coroutine_fn rdma_accept_incoming_migration(void *opaque) +{ + QIOChannelRDMA *rioc =3D opaque; + QIOChannelRDMA *cioc; + + while (!migration_has_all_channels()) { + cioc =3D qio_channel_rdma_accept(rioc, NULL); + + qio_channel_set_name(QIO_CHANNEL(cioc), "migration-rdma-incoming"); + migration_channel_process_incoming(QIO_CHANNEL(cioc)); + object_unref(OBJECT(cioc)); + } +} + +void rdma_start_incoming_migration(InetSocketAddress *addr, Error **errp) +{ + QIOChannelRDMA *rioc =3D qio_channel_rdma_new(); + MigrationIncomingState *mis =3D migration_incoming_get_current(); + Coroutine *co; + int num =3D 1; + + qio_channel_set_name(QIO_CHANNEL(rioc), "migration-rdma-listener"); + + if (qio_channel_rdma_listen_sync(rioc, addr, num, errp) < 0) { + object_unref(OBJECT(rioc)); + return; + } + + mis->transport_data =3D rioc; + mis->transport_cleanup =3D object_unref; + + qio_channel_set_blocking(QIO_CHANNEL(rioc), false, NULL); + co =3D qemu_coroutine_create(rdma_accept_incoming_migration, rioc); + aio_co_schedule(qemu_get_current_aio_context(), co); +} diff --git a/migration/rdma.h b/migration/rdma.h new file mode 100644 index 0000000000..4c3eb9a972 --- /dev/null +++ b/migration/rdma.h @@ -0,0 +1,24 @@ +/* + * QEMU live migration via RDMA + * + * Copyright (c) 2024 HUAWEI TECHNOLOGIES CO., LTD. + * + * Authors: + * Jialin Wang + * Gonglei + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_MIGRATION_RDMA_H +#define QEMU_MIGRATION_RDMA_H + +#include "qemu/sockets.h" + +void rdma_start_outgoing_migration(MigrationState *s, InetSocketAddress *a= ddr, + Error **errp); + +void rdma_start_incoming_migration(InetSocketAddress *addr, Error **errp); + +#endif /* QEMU_MIGRATION_RDMA_H */ --=20 2.43.0