From nobody Sun May 19 16:27:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1670968036; cv=none; d=zohomail.com; s=zohoarc; b=knKUs/nIqcqwBl96PVVulFTMLetv3fd3d/qjDsRbzWy2rX7VSvgjK9hXAh13btwgXYmxvvtoNTjJUAIvLMa5vu9gmG5yeKhEMdqrgXFYXqaPN4+uYOtjp6PbLEQyn3s+AhBV/VsFTiV0HKm84Jpt8H6uCi6AsRMSNkwKB76ZwMg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670968036; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5SsDJHjbmYgWOI28JIdJ4nkCfxRhomPn/I0rSYWBMco=; b=GqXVDQ28tjoSjKngtTPRbItK5CiHUQspQQ0tafSj6I+/rCGQPtUVrVYE+hHxOMMRLYihaxfXBCHjn1Gt+ed10zMgI+GWFJ1PYNjZPxdsOvMN67JrrDCd89T++McyYkU0BKmm3UO38W57ZwzlEjS20Iu/bZO/bxXhA3I0vx+Ubs8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670968036053187.5717136464707; Tue, 13 Dec 2022 13:47:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p5CzV-0000FY-UA; Tue, 13 Dec 2022 16:39:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzN-0000Eb-NH for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzL-0007eI-Ke for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:01 -0500 Received: from mail-ua1-f72.google.com (mail-ua1-f72.google.com [209.85.222.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-10-P6o-09mOOPGroYav0IHKfg-1; Tue, 13 Dec 2022 16:38:57 -0500 Received: by mail-ua1-f72.google.com with SMTP id z42-20020ab0492d000000b00423b333ff7dso5303728uac.22 for ; Tue, 13 Dec 2022 13:38:57 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020a05620a288c00b006feb158e5e7sm8607487qkp.70.2022.12.13.13.38.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 13:38:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670967539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5SsDJHjbmYgWOI28JIdJ4nkCfxRhomPn/I0rSYWBMco=; b=RTefwoXwUuv7GSqvHPye8XA5lMYWvPuuNUrMP9wYoelGIZ4q67R0yvdlMLZaakanJSsVoh h0/+5zpVZDsC/DywIDypYxviQA/Q+prYdnqy1Zqtgz/4vbI3ijrxqOwR1x/9wdvca6HFge lsIDovdDBcPMdmgGdNd/UOFsAJq26OA= X-MC-Unique: P6o-09mOOPGroYav0IHKfg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5SsDJHjbmYgWOI28JIdJ4nkCfxRhomPn/I0rSYWBMco=; b=w49Kui3WRmWSGCNBghKuOglX/wKh4gLR1Eh3iJfIunKbBOpjBjCUOB3/zLUV0V7VxO 3ztbCs/uVRXfCr2hHN1mO+NlvqT/+zZpITnmWH/fgI6gVapAmzkTiCOVCwX7VC1AoKC1 MzbVQOiApHGH6jWlscmVJ3dD5liB1nBc6tQ2viGFH+gZgudhrwfTOjiB1LfArHxacm9f lfiCwGPH1ydbNqWKC1RHqHmAKNUEBlg6fUMalEcYeh2VmDKviD4hEvxtWWvGao6dod0b 5EQrXdsoElNewnaUh4US+f3Wb6z3G2MByjkuvpoQmpFx+IC14xUUELAFfNYYf32KaW3y WRYw== X-Gm-Message-State: ANoB5pn7MtKXslgJUBGLVxOyNTBmQv/X0NlcQiCAYUsaS8W7MeLKZeBz QiD63JYUQHJwrv0LgklmyC2m9/UdqHV1PvF8PuAVM3R+oQ55PjfuXzE+2nKQSAXLq12U274omrX PZwBPZy3gdkTH5ppz5kuNhGOaqkqibCWxwyshOc8vNiFBweQ586sLJwp3rjPJHfSn X-Received: by 2002:a05:6102:548f:b0:3b5:1803:80e4 with SMTP id bk15-20020a056102548f00b003b5180380e4mr4023538vsb.18.1670967534769; Tue, 13 Dec 2022 13:38:54 -0800 (PST) X-Google-Smtp-Source: AA0mqf741BDBmJqv80o4ZvycveeF9JJb2/sqdMEr9kv5w5GMDKEeJyYP8t1jAVviz+XO1g+oPphvug== X-Received: by 2002:a05:6102:548f:b0:3b5:1803:80e4 with SMTP id bk15-20020a056102548f00b003b5180380e4mr4023505vsb.18.1670967534329; Tue, 13 Dec 2022 13:38:54 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , "Daniel P . Berrange" , Leonardo Bras Soares Passos , Manish Mishra , "Dr . David Alan Gilbert" , peterx@redhat.com Subject: [PATCH 1/5] io: Add support for MSG_PEEK for socket channel Date: Tue, 13 Dec 2022 16:38:46 -0500 Message-Id: <20221213213850.1481858-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221213213850.1481858-1-peterx@redhat.com> References: <20221213213850.1481858-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-type: text/plain Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PP_MIME_FAKE_ASCII_TEXT=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1670968038197100003 From: "manish.mishra" MSG_PEEK reads from the peek of channel, The data is treated as unread and the next read shall still return this data. This support is currently added only for socket class. Extra parameter 'flags' is added to io_readv calls to pass extra read flags like MSG_PEEK. Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Peter Xu --- chardev/char-socket.c | 4 ++-- include/io/channel.h | 6 ++++++ io/channel-buffer.c | 1 + io/channel-command.c | 1 + io/channel-file.c | 1 + io/channel-null.c | 1 + io/channel-socket.c | 17 ++++++++++++++++- io/channel-tls.c | 1 + io/channel-websock.c | 1 + io/channel.c | 16 ++++++++++++---- migration/channel-block.c | 1 + migration/rdma.c | 1 + scsi/qemu-pr-helper.c | 2 +- tests/qtest/tpm-emu.c | 2 +- tests/unit/test-io-channel-socket.c | 1 + util/vhost-user-server.c | 2 +- 16 files changed, 48 insertions(+), 10 deletions(-) diff --git a/chardev/char-socket.c b/chardev/char-socket.c index 879564aa8a..5afce9a464 100644 --- a/chardev/char-socket.c +++ b/chardev/char-socket.c @@ -283,11 +283,11 @@ static ssize_t tcp_chr_recv(Chardev *chr, char *buf, = size_t len) if (qio_channel_has_feature(s->ioc, QIO_CHANNEL_FEATURE_FD_PASS)) { ret =3D qio_channel_readv_full(s->ioc, &iov, 1, &msgfds, &msgfds_num, - NULL); + 0, NULL); } else { ret =3D qio_channel_readv_full(s->ioc, &iov, 1, NULL, NULL, - NULL); + 0, NULL); } =20 if (msgfds_num) { diff --git a/include/io/channel.h b/include/io/channel.h index c680ee7480..716235d496 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -34,6 +34,8 @@ OBJECT_DECLARE_TYPE(QIOChannel, QIOChannelClass, =20 #define QIO_CHANNEL_WRITE_FLAG_ZERO_COPY 0x1 =20 +#define QIO_CHANNEL_READ_FLAG_MSG_PEEK 0x1 + typedef enum QIOChannelFeature QIOChannelFeature; =20 enum QIOChannelFeature { @@ -41,6 +43,7 @@ enum QIOChannelFeature { QIO_CHANNEL_FEATURE_SHUTDOWN, QIO_CHANNEL_FEATURE_LISTEN, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY, + QIO_CHANNEL_FEATURE_READ_MSG_PEEK, }; =20 =20 @@ -114,6 +117,7 @@ struct QIOChannelClass { size_t niov, int **fds, size_t *nfds, + int flags, Error **errp); int (*io_close)(QIOChannel *ioc, Error **errp); @@ -188,6 +192,7 @@ void qio_channel_set_name(QIOChannel *ioc, * @niov: the length of the @iov array * @fds: pointer to an array that will received file handles * @nfds: pointer filled with number of elements in @fds on return + * @flags: read flags (QIO_CHANNEL_READ_FLAG_*) * @errp: pointer to a NULL-initialized error object * * Read data from the IO channel, storing it in the @@ -224,6 +229,7 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp); =20 =20 diff --git a/io/channel-buffer.c b/io/channel-buffer.c index bf52011be2..8096180f85 100644 --- a/io/channel-buffer.c +++ b/io/channel-buffer.c @@ -54,6 +54,7 @@ static ssize_t qio_channel_buffer_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelBuffer *bioc =3D QIO_CHANNEL_BUFFER(ioc); diff --git a/io/channel-command.c b/io/channel-command.c index 74516252ba..e7edd091af 100644 --- a/io/channel-command.c +++ b/io/channel-command.c @@ -203,6 +203,7 @@ static ssize_t qio_channel_command_readv(QIOChannel *io= c, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelCommand *cioc =3D QIO_CHANNEL_COMMAND(ioc); diff --git a/io/channel-file.c b/io/channel-file.c index b67687c2aa..d76663e6ae 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -86,6 +86,7 @@ static ssize_t qio_channel_file_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelFile *fioc =3D QIO_CHANNEL_FILE(ioc); diff --git a/io/channel-null.c b/io/channel-null.c index 75e3781507..4fafdb770d 100644 --- a/io/channel-null.c +++ b/io/channel-null.c @@ -60,6 +60,7 @@ qio_channel_null_readv(QIOChannel *ioc, size_t niov, int **fds G_GNUC_UNUSED, size_t *nfds G_GNUC_UNUSED, + int flags, Error **errp) { QIOChannelNull *nioc =3D QIO_CHANNEL_NULL(ioc); diff --git a/io/channel-socket.c b/io/channel-socket.c index b76dca9cc1..dfb8cb6c40 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -173,6 +173,8 @@ int qio_channel_socket_connect_sync(QIOChannelSocket *i= oc, } #endif =20 + qio_channel_set_feature(QIO_CHANNEL(ioc), QIO_CHANNEL_FEATURE_READ_MSG= _PEEK); + return 0; } =20 @@ -406,6 +408,8 @@ qio_channel_socket_accept(QIOChannelSocket *ioc, } #endif /* WIN32 */ =20 + qio_channel_set_feature(QIO_CHANNEL(cioc), QIO_CHANNEL_FEATURE_READ_MS= G_PEEK); + trace_qio_channel_socket_accept_complete(ioc, cioc, cioc->fd); return cioc; =20 @@ -496,6 +500,7 @@ static ssize_t qio_channel_socket_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelSocket *sioc =3D QIO_CHANNEL_SOCKET(ioc); @@ -517,6 +522,10 @@ static ssize_t qio_channel_socket_readv(QIOChannel *io= c, =20 } =20 + if (flags & QIO_CHANNEL_READ_FLAG_MSG_PEEK) { + sflags |=3D MSG_PEEK; + } + retry: ret =3D recvmsg(sioc->fd, &msg, sflags); if (ret < 0) { @@ -624,11 +633,17 @@ static ssize_t qio_channel_socket_readv(QIOChannel *i= oc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelSocket *sioc =3D QIO_CHANNEL_SOCKET(ioc); ssize_t done =3D 0; ssize_t i; + int sflags =3D 0; + + if (flags & QIO_CHANNEL_READ_FLAG_MSG_PEEK) { + sflags |=3D MSG_PEEK; + } =20 for (i =3D 0; i < niov; i++) { ssize_t ret; @@ -636,7 +651,7 @@ static ssize_t qio_channel_socket_readv(QIOChannel *ioc, ret =3D recv(sioc->fd, iov[i].iov_base, iov[i].iov_len, - 0); + sflags); if (ret < 0) { if (errno =3D=3D EAGAIN) { if (done) { diff --git a/io/channel-tls.c b/io/channel-tls.c index 4ce890a538..c730cb8ec5 100644 --- a/io/channel-tls.c +++ b/io/channel-tls.c @@ -260,6 +260,7 @@ static ssize_t qio_channel_tls_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelTLS *tioc =3D QIO_CHANNEL_TLS(ioc); diff --git a/io/channel-websock.c b/io/channel-websock.c index fb4932ade7..a12acc27cf 100644 --- a/io/channel-websock.c +++ b/io/channel-websock.c @@ -1081,6 +1081,7 @@ static ssize_t qio_channel_websock_readv(QIOChannel *= ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelWebsock *wioc =3D QIO_CHANNEL_WEBSOCK(ioc); diff --git a/io/channel.c b/io/channel.c index 0640941ac5..a8c7f11649 100644 --- a/io/channel.c +++ b/io/channel.c @@ -52,6 +52,7 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelClass *klass =3D QIO_CHANNEL_GET_CLASS(ioc); @@ -63,7 +64,14 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc, return -1; } =20 - return klass->io_readv(ioc, iov, niov, fds, nfds, errp); + if ((flags & QIO_CHANNEL_READ_FLAG_MSG_PEEK) && + !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { + error_setg_errno(errp, EINVAL, + "Channel does not support peek read"); + return -1; + } + + return klass->io_readv(ioc, iov, niov, fds, nfds, flags, errp); } =20 =20 @@ -146,7 +154,7 @@ int qio_channel_readv_full_all_eof(QIOChannel *ioc, while ((nlocal_iov > 0) || local_fds) { ssize_t len; len =3D qio_channel_readv_full(ioc, local_iov, nlocal_iov, local_f= ds, - local_nfds, errp); + local_nfds, 0, errp); if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { if (qemu_in_coroutine()) { qio_channel_yield(ioc, G_IO_IN); @@ -284,7 +292,7 @@ ssize_t qio_channel_readv(QIOChannel *ioc, size_t niov, Error **errp) { - return qio_channel_readv_full(ioc, iov, niov, NULL, NULL, errp); + return qio_channel_readv_full(ioc, iov, niov, NULL, NULL, 0, errp); } =20 =20 @@ -303,7 +311,7 @@ ssize_t qio_channel_read(QIOChannel *ioc, Error **errp) { struct iovec iov =3D { .iov_base =3D buf, .iov_len =3D buflen }; - return qio_channel_readv_full(ioc, &iov, 1, NULL, NULL, errp); + return qio_channel_readv_full(ioc, &iov, 1, NULL, NULL, 0, errp); } =20 =20 diff --git a/migration/channel-block.c b/migration/channel-block.c index f4ab53acdb..b7374363c3 100644 --- a/migration/channel-block.c +++ b/migration/channel-block.c @@ -53,6 +53,7 @@ qio_channel_block_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelBlock *bioc =3D QIO_CHANNEL_BLOCK(ioc); diff --git a/migration/rdma.c b/migration/rdma.c index 94a55dd95b..d8b4632094 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2854,6 +2854,7 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc, size_t niov, int **fds, size_t *nfds, + int flags, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c index 196b78c00d..199227a556 100644 --- a/scsi/qemu-pr-helper.c +++ b/scsi/qemu-pr-helper.c @@ -614,7 +614,7 @@ static int coroutine_fn prh_read(PRHelperClient *client= , void *buf, int sz, iov.iov_base =3D buf; iov.iov_len =3D sz; n_read =3D qio_channel_readv_full(QIO_CHANNEL(client->ioc), &iov, = 1, - &fds, &nfds, errp); + &fds, &nfds, 0, errp); =20 if (n_read =3D=3D QIO_CHANNEL_ERR_BLOCK) { qio_channel_yield(QIO_CHANNEL(client->ioc), G_IO_IN); diff --git a/tests/qtest/tpm-emu.c b/tests/qtest/tpm-emu.c index 2994d1cf42..3cf1acaf7d 100644 --- a/tests/qtest/tpm-emu.c +++ b/tests/qtest/tpm-emu.c @@ -106,7 +106,7 @@ void *tpm_emu_ctrl_thread(void *data) int *pfd =3D NULL; size_t nfd =3D 0; =20 - qio_channel_readv_full(ioc, &iov, 1, &pfd, &nfd, &error_abort); + qio_channel_readv_full(ioc, &iov, 1, &pfd, &nfd, 0, &error_abort); cmd =3D be32_to_cpu(cmd); g_assert_cmpint(cmd, =3D=3D, CMD_SET_DATAFD); g_assert_cmpint(nfd, =3D=3D, 1); diff --git a/tests/unit/test-io-channel-socket.c b/tests/unit/test-io-chann= el-socket.c index b36a5d972a..b964bb202d 100644 --- a/tests/unit/test-io-channel-socket.c +++ b/tests/unit/test-io-channel-socket.c @@ -460,6 +460,7 @@ static void test_io_channel_unix_fd_pass(void) G_N_ELEMENTS(iorecv), &fdrecv, &nfdrecv, + 0, &error_abort); =20 g_assert(nfdrecv =3D=3D G_N_ELEMENTS(fdsend)); diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c index 232984ace6..145eb17c08 100644 --- a/util/vhost-user-server.c +++ b/util/vhost-user-server.c @@ -116,7 +116,7 @@ vu_message_read(VuDev *vu_dev, int conn_fd, VhostUserMs= g *vmsg) * qio_channel_readv_full may have short reads, keeping calling it * until getting VHOST_USER_HDR_SIZE or 0 bytes in total */ - rc =3D qio_channel_readv_full(ioc, &iov, 1, &fds, &nfds, &local_er= r); + rc =3D qio_channel_readv_full(ioc, &iov, 1, &fds, &nfds, 0, &local= _err); if (rc < 0) { if (rc =3D=3D QIO_CHANNEL_ERR_BLOCK) { assert(local_err =3D=3D NULL); --=20 2.37.3 From nobody Sun May 19 16:27:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1670967864; cv=none; d=zohomail.com; s=zohoarc; b=lExuTkQA/UAm8U8LhjwRj9ET41D+tW4MxgEE1a8M3g4+xhG0hUoiujysrnVYNsT+BTIjJWUhAgGcCEdunm6mNhffodiKCI/O8seYLCSKOB7ayp2gjZHgrs44pzxjmt3fqQYoNFFt35pdZF4vOmmjMmpqx09QNDdE2VAPKIs53Bo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670967864; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AZuvU4QeBogKbaQl5ZIKjeZ9crvrQIVntMvHBMji+Pk=; b=SlJ/aryhBjzv19sD+zhVBlWKpfe24AcNyJ3CYNv5XIQ/sap8XvWJV62qLD3ucaLdB0Xf8LgShU93M0Ze39KyiMy71ztbrAe2qSe5H1HlT7ljmwoE8odYfrS/oxBs3FhOh+Ew/2n/QzV+dNiKrwOfnCyhsB7gvtSiJKEHX2AWQQE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670967864304671.0897983757882; Tue, 13 Dec 2022 13:44:24 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p5Czd-0000IW-LM; Tue, 13 Dec 2022 16:39:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzP-0000FN-K9 for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzN-0007eV-DG for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:03 -0500 Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-235-LIrI21TSNBavYdqWT0gX7A-1; Tue, 13 Dec 2022 16:38:59 -0500 Received: by mail-qk1-f200.google.com with SMTP id h13-20020a05620a244d00b006fb713618b8so1194318qkn.0 for ; Tue, 13 Dec 2022 13:38:59 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020a05620a288c00b006feb158e5e7sm8607487qkp.70.2022.12.13.13.38.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 13:38:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670967540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AZuvU4QeBogKbaQl5ZIKjeZ9crvrQIVntMvHBMji+Pk=; b=MsWntry+uf+S3HCp4fLcKuT1L4mn3/VDWoEiuNXzU1MhSCuvd+/UhMOJF4yrPAIHhOM6s9 +aAjJ75pATJKp2oKklLlamncU6sxFII2MsJgYir5anGooA0Dzg5rv1JGUvFkxyYYmWSHil 20gV6LhqG1Ra6KSex2Z9l51lkB2+tCI= X-MC-Unique: LIrI21TSNBavYdqWT0gX7A-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AZuvU4QeBogKbaQl5ZIKjeZ9crvrQIVntMvHBMji+Pk=; b=fTyA9oDanqag6FJk51+dbRg2mY+x0Q7cMwvhD7s2Dw4bviyKE2UdgvbyHnotngnbrj bwyILGp7CBkZB3E9xqrGGqeJl0X9qCewMv5lei1Pu+7G6Yo2H606WK9dOb2rx1pbfzcU RdLHTvGYwOCgaRdQdg2hCiJ0MHMkQ7TSzep8EVSH4NKLDPDT75CIqfYBNBJ/SS2riHsU iA/p3Su1VtQp22ohgvd+ip3Hd02NlJNVQ21fFTg/Z4rBwLmxuRJlCQsQfi9EwHFEkks3 FUjj2X5Bcxqip4snbMlySr8eNmHvvroCdwrXrCNZGs4redublyx3DiyTRj6brEdBrWOz W7Ug== X-Gm-Message-State: ANoB5pk/GetfVbge3lewaRDAVaBUwPV99ATIe3HZJdtnMxk3M85TH3uu ij6ihVG0IjlI5ZyzzegTzWAWNfAPVe+lLtjc5V0kSPRWeodeG5KxHrOi87KQTncsBvCPeE6o8Qj 1R9pGfnzQGNzXRN1p/N0A8LVyo9r+LNDcuWaMETfMB7JYmLfTRv6jwifjg7TCjUnl X-Received: by 2002:a05:622a:5da9:b0:3a8:28a4:c4be with SMTP id fu41-20020a05622a5da900b003a828a4c4bemr2082435qtb.0.1670967537671; Tue, 13 Dec 2022 13:38:57 -0800 (PST) X-Google-Smtp-Source: AA0mqf6+PwCcF8YVKrp2VD/kBYL9U2QqyxSwoRXJHYk4BhEhrQpapVEbMqgDGgBMsfeH3bjNJ0/OBQ== X-Received: by 2002:a05:622a:5da9:b0:3a8:28a4:c4be with SMTP id fu41-20020a05622a5da900b003a828a4c4bemr2082402qtb.0.1670967537256; Tue, 13 Dec 2022 13:38:57 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , "Daniel P . Berrange" , Leonardo Bras Soares Passos , Manish Mishra , "Dr . David Alan Gilbert" , peterx@redhat.com Subject: [PATCH 2/5] migration: check magic value for deciding the mapping of channels Date: Tue, 13 Dec 2022 16:38:47 -0500 Message-Id: <20221213213850.1481858-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221213213850.1481858-1-peterx@redhat.com> References: <20221213213850.1481858-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-type: text/plain Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PP_MIME_FAKE_ASCII_TEXT=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1670967865499100005 From: "manish.mishra" Current logic assumes that channel connections on the destination side are always established in the same order as the source and the first one will always be the main channel followed by the multifid or post-copy preemption channel. This may not be always true, as even if a channel has a connection established on the source side it can be in the pending state on the destination side and a newer connection can be established first. Basically causing out of order mapping of channels on the destination side. Currently, all channels except post-copy preempt send a magic number, this patch uses that magic number to decide the type of channel. This logic is applicable only for precopy(multifd) live migration, as mentioned, the post-copy preempt channel does not send any magic number. Also, tls live migrations already does tls handshake before creating other channels, so this issue is not possible with tls, hence this logic is avoided for tls live migrations. This patch uses read peek to check the magic number of channels so that current data/control stream management remains un-effected. Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Signed-off-by: Peter Xu --- migration/channel.c | 45 ++++++++++++++++++++++++++++++++++++++++ migration/channel.h | 5 +++++ migration/migration.c | 45 +++++++++++++++++++++++++++++----------- migration/multifd.c | 12 ++++------- migration/multifd.h | 2 +- migration/postcopy-ram.c | 5 +---- migration/postcopy-ram.h | 2 +- 7 files changed, 90 insertions(+), 26 deletions(-) diff --git a/migration/channel.c b/migration/channel.c index 1b0815039f..270b7acca2 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -92,3 +92,48 @@ void migration_channel_connect(MigrationState *s, migrate_fd_connect(s, error); error_free(error); } + + +/** + * @migration_channel_read_peek - Read from the peek of migration channel, + * without actually removing it from channel buffer. + * + * @ioc: the channel object + * @buf: the memory region to read data into + * @buflen: the number of bytes to read in @buf + * @errp: pointer to a NULL-initialized error object + * + * Returns 0 if successful, returns -1 and sets @errp if fails. + */ +int migration_channel_read_peek(QIOChannel *ioc, + const char *buf, + const size_t buflen, + Error **errp) +{ + ssize_t len =3D 0; + struct iovec iov =3D { .iov_base =3D (char *)buf, .iov_len =3D buflen = }; + + while (true) { + len =3D qio_channel_readv_full(ioc, &iov, 1, NULL, + NULL, QIO_CHANNEL_READ_FLAG_MSG_PEEK,= errp); + + if (len <=3D 0 && len !=3D QIO_CHANNEL_ERR_BLOCK) { + error_setg(errp, + "Failed to read peek of channel"); + return -1; + } + + if (len =3D=3D buflen) { + break; + } + + /* 1ms sleep. */ + if (qemu_in_coroutine()) { + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 1000000); + } else { + g_usleep(1000); + } + } + + return 0; +} diff --git a/migration/channel.h b/migration/channel.h index 67a461c28a..5bdb8208a7 100644 --- a/migration/channel.h +++ b/migration/channel.h @@ -24,4 +24,9 @@ void migration_channel_connect(MigrationState *s, QIOChannel *ioc, const char *hostname, Error *error_in); + +int migration_channel_read_peek(QIOChannel *ioc, + const char *buf, + const size_t buflen, + Error **errp); #endif diff --git a/migration/migration.c b/migration/migration.c index c3490c495d..35b43d0108 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -31,6 +31,7 @@ #include "migration.h" #include "savevm.h" #include "qemu-file.h" +#include "channel.h" #include "migration/vmstate.h" #include "block/block.h" #include "qapi/error.h" @@ -733,31 +734,51 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); Error *local_err =3D NULL; - bool start_migration; QEMUFile *f; + bool default_channel =3D true; + uint32_t channel_magic =3D 0; + int ret =3D 0; =20 - if (!mis->from_src_file) { - /* The first connection (multifd may have multiple) */ + if (migrate_use_multifd() && !migrate_postcopy_ram() && + qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_READ_MSG_PEEK)) { + /* + * With multiple channels, it is possible that we receive channels + * out of order on destination side, causing incorrect mapping of + * source channels on destination side. Check channel MAGIC to + * decide type of channel. Please note this is best effort, postco= py + * preempt channel does not send any magic number so avoid it for + * postcopy live migration. Also tls live migration already does + * tls handshake while initializing main channel so with tls this + * issue is not possible. + */ + ret =3D migration_channel_read_peek(ioc, (void *)&channel_magic, + sizeof(channel_magic), &local_er= r); + + if (ret !=3D 0) { + error_propagate(errp, local_err); + return; + } + + default_channel =3D (channel_magic =3D=3D cpu_to_be32(QEMU_VM_FILE= _MAGIC)); + } else { + default_channel =3D !mis->from_src_file; + } + + if (default_channel) { f =3D qemu_file_new_input(ioc); =20 if (!migration_incoming_setup(f, errp)) { return; } - - /* - * Common migration only needs one channel, so we can start - * right now. Some features need more than one channel, we wait. - */ - start_migration =3D !migration_needs_multiple_sockets(); } else { /* Multiple connections */ assert(migration_needs_multiple_sockets()); if (migrate_use_multifd()) { - start_migration =3D multifd_recv_new_channel(ioc, &local_err); + multifd_recv_new_channel(ioc, &local_err); } else { assert(migrate_postcopy_preempt()); f =3D qemu_file_new_input(ioc); - start_migration =3D postcopy_preempt_new_channel(mis, f); + postcopy_preempt_new_channel(mis, f); } if (local_err) { error_propagate(errp, local_err); @@ -765,7 +786,7 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Er= ror **errp) } } =20 - if (start_migration) { + if (migration_has_all_channels()) { /* If it's a recovery, we're done */ if (postcopy_try_recover()) { return; diff --git a/migration/multifd.c b/migration/multifd.c index 17f5730257..4f6028997d 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1228,11 +1228,9 @@ bool multifd_recv_all_channels_created(void) =20 /* * Try to receive all multifd channels to get ready for the migration. - * - Return true and do not set @errp when correctly receiving all channel= s; - * - Return false and do not set @errp when correctly receiving the curren= t one; - * - Return false and set @errp when failing to receive the current channe= l. + * Sets @errp when failing to receive the current channel. */ -bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp) +void multifd_recv_new_channel(QIOChannel *ioc, Error **errp) { MultiFDRecvParams *p; Error *local_err =3D NULL; @@ -1245,7 +1243,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) "failed to receive packet" " via multifd channel %d: ", qatomic_read(&multifd_recv_state->count)); - return false; + return; } trace_multifd_recv_new_channel(id); =20 @@ -1255,7 +1253,7 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) id); multifd_recv_terminate_threads(local_err); error_propagate(errp, local_err); - return false; + return; } p->c =3D ioc; object_ref(OBJECT(ioc)); @@ -1266,6 +1264,4 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error = **errp) qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p, QEMU_THREAD_JOINABLE); qatomic_inc(&multifd_recv_state->count); - return qatomic_read(&multifd_recv_state->count) =3D=3D - migrate_multifd_channels(); } diff --git a/migration/multifd.h b/migration/multifd.h index 519f498643..913e4ba274 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -18,7 +18,7 @@ void multifd_save_cleanup(void); int multifd_load_setup(Error **errp); int multifd_load_cleanup(Error **errp); bool multifd_recv_all_channels_created(void); -bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp); +void multifd_recv_new_channel(QIOChannel *ioc, Error **errp); void multifd_recv_sync_main(void); int multifd_send_sync_main(QEMUFile *f); int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index b9a37ef255..f84f783ab4 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1539,7 +1539,7 @@ void postcopy_unregister_shared_ufd(struct PostCopyFD= *pcfd) } } =20 -bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) +void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) { /* * The new loading channel has its own threads, so it needs to be @@ -1548,9 +1548,6 @@ bool postcopy_preempt_new_channel(MigrationIncomingSt= ate *mis, QEMUFile *file) qemu_file_set_blocking(file, true); mis->postcopy_qemufile_dst =3D file; trace_postcopy_preempt_new_channel(); - - /* Start the migration immediately */ - return true; } =20 /* diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 6147bf7d1d..25881c4127 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -190,7 +190,7 @@ enum PostcopyChannels { RAM_CHANNEL_MAX, }; =20 -bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); +void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); int postcopy_preempt_setup(MigrationState *s, Error **errp); int postcopy_preempt_wait_channel(MigrationState *s); =20 --=20 2.37.3 From nobody Sun May 19 16:27:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1670967882; cv=none; d=zohomail.com; s=zohoarc; b=XQveTTI6w67x1zLmlHlYtq3ZKtjPUGJVyEkPgMzrJqwUA7YXwOtm/AlBe692tMVCiV5jP0vUjERtrwOZGDA7ckGiKosP2DG7f23zlWUR4m8u9EoEBDRdiZGCgT+uIhp+9MzkV2PFwXZD+ic5Bw517c/cd8JB6Im9j64N2coqMm4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670967882; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aaLqkomZXef9Fs5hzdtdAF5w6gy6tt9W28FJzupeb24=; b=XAI0w6GtS9E/6blpL10X7mf3w065dl4y1m5HJ6qAjFIwd+C1dDIAWn8YMMstZgI7sZksbAPr4PeDxrc0U51PHoCMe6c7q5h6fcC/UrhN46RYOm0VZRdbSO3Sp3KhMYVvwabcGfqyQd9SfGA8bWbP8Btp13CJPxVMzxjW+Yd1W0Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670967882198881.3013279239751; Tue, 13 Dec 2022 13:44:42 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p5Czn-0000Tx-Oe; Tue, 13 Dec 2022 16:39:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzS-0000G1-NU for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzQ-0007f7-LK for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:06 -0500 Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-626-xnRZTw79NDW_NnQHli65Rw-1; Tue, 13 Dec 2022 16:39:01 -0500 Received: by mail-qv1-f71.google.com with SMTP id y11-20020ad457cb000000b004c6fafdde42so15545523qvx.5 for ; Tue, 13 Dec 2022 13:39:01 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020a05620a288c00b006feb158e5e7sm8607487qkp.70.2022.12.13.13.38.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 13:38:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670967543; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aaLqkomZXef9Fs5hzdtdAF5w6gy6tt9W28FJzupeb24=; b=KOHbOkiiW2so1NnIpzsCOidPdA2lwY/tsuep+BtSnPvO3QTo30zZmPykzNrD8vOfbDdjSZ Jcc3fle02xvZS1N2c4od4IZuUbMnQ+fZQFub/bkug/pqSWnJv70fmRFbMhLQGVbdOMMYpW ndL+bmfqhNrMDbWXiWo4ms8amnZ1pTY= X-MC-Unique: xnRZTw79NDW_NnQHli65Rw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aaLqkomZXef9Fs5hzdtdAF5w6gy6tt9W28FJzupeb24=; b=kEs/Gyr71K6DaIxB5WXdegaa8LAcsihXSAAdr/m6/PMlCRFXmIxeCKvF6HRf/d6zKb cI8jwiM0ptavKSShBLVLQ7g/NJcmvL5Jzz7+75cqt5fcS2JbXR5b55p8vQ5iYqJ4RmCE P0rGRd2MmenG5NRUgcawakS6n5u9BCHBJaGq6UexY+PhVihTBeQskfVzHumgYnldL2Du BbwRWLtVIjG+wfoWRiXp2pPWIFvuM/Rl2mOpvLxMADeJxVHJJ0fFuDtrU/go6PuUJGwH T31wHkwlOzABZ9No30fLRizJxGNNMoRgu4PC2mNIQvEfYKSw1uSv1qb3sdEz/VviimFf QYCA== X-Gm-Message-State: ANoB5pn8NyiCYSTe+dCXpEwBS1unwWKzkZImUuXFewgaeQAk1gXg9l2Z ChSbRxOPbJN1f8PEW29pF1BCKv5qgJShfmERRmbhku4mxZnwLK5bQKMwXb9UaFHq5TJIdlnzosq NCkc6Ji5MHvrhs0lvM8otowKgLFVBj2dN4Ul8AE975PJmHQQRQgMqIRJzE9jjWLZF X-Received: by 2002:a05:622a:408e:b0:3a5:c775:59ec with SMTP id cg14-20020a05622a408e00b003a5c77559ecmr32809846qtb.28.1670967540427; Tue, 13 Dec 2022 13:39:00 -0800 (PST) X-Google-Smtp-Source: AA0mqf5HegN+x6AF97VHt+utlzS98bVXxCilDIHFeR1PD4mZg3584Ce4jXjFrQAbfNPivOojGfoNGA== X-Received: by 2002:a05:622a:408e:b0:3a5:c775:59ec with SMTP id cg14-20020a05622a408e00b003a5c77559ecmr32809816qtb.28.1670967540128; Tue, 13 Dec 2022 13:39:00 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , "Daniel P . Berrange" , Leonardo Bras Soares Passos , Manish Mishra , "Dr . David Alan Gilbert" , peterx@redhat.com Subject: [PATCH 3/5] migration: Rework multi-channel checks on URI Date: Tue, 13 Dec 2022 16:38:48 -0500 Message-Id: <20221213213850.1481858-4-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221213213850.1481858-1-peterx@redhat.com> References: <20221213213850.1481858-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1670967883444100003 Content-Type: text/plain; charset="utf-8" The whole idea of multi-channel checks was not properly done, IMHO. Currently we check multi-channel in a lot of places, but actually that's not needed because we only need to check it right after we get the URI and that should be it. If the URI check succeeded, we should never need to check it again because we must have it. If it check fails, we should fail immediately on either the qmp_migrate or qmp_migrate_incoming, instead of failingg it later after the connection established. Neither should we fail any set capabiliities like what we used to do here: 5ad15e8614 ("migration: allow enabling mutilfd for specific protocol only",= 2021-10-19) Because logically the URI will only be set later after the capability is set, so it doesn't make a lot of sense to check the URI type when setting the capability, because we're checking the cap with an old URI passed in, and that may not even be the URI we're going to use later. This patch mostly reverted all such checks for before, dropping the variable migrate_allow_multi_channels and helpers. Instead, add a common helper to check URI for multi-channels for either qmp_migrate and qmp_migrate_incoming and that should do all the proper checks. The failure will only trigger with the "migrate" or "migrate_incoming" command, or when user specified "-incoming xxx" where "xxx" is not "defer". With that, make postcopy_preempt_setup() as simple as creating the channel. Signed-off-by: Peter Xu --- migration/migration.c | 56 +++++++++++++++++++--------------------- migration/migration.h | 3 --- migration/multifd.c | 12 ++------- migration/postcopy-ram.c | 14 +--------- migration/postcopy-ram.h | 2 +- 5 files changed, 31 insertions(+), 56 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 35b43d0108..898bc9fe5c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -182,16 +182,26 @@ static int migration_maybe_pause(MigrationState *s, int new_state); static void migrate_fd_cancel(MigrationState *s); =20 -static bool migrate_allow_multi_channels =3D true; +static bool migration_needs_multiple_sockets(void) +{ + return migrate_use_multifd() || migrate_postcopy_preempt(); +} =20 -void migrate_protocol_allow_multi_channels(bool allow) +static bool uri_supports_multi_channels(const char *uri) { - migrate_allow_multi_channels =3D allow; + return strstart(uri, "tcp:", NULL) || strstart(uri, "unix:", NULL) || + strstart(uri, "vsock:", NULL); } =20 -bool migrate_multi_channels_is_allowed(void) +static bool migration_uri_validate(const char *uri, Error **errp) { - return migrate_allow_multi_channels; + if (migration_needs_multiple_sockets() && + !uri_supports_multi_channels(uri)) { + error_setg(errp, "Migration requires multi-channel URIs (e.g. tcp)= "); + return false; + } + + return true; } =20 static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) @@ -491,12 +501,15 @@ static void qemu_start_incoming_migration(const char = *uri, Error **errp) { const char *p =3D NULL; =20 - migrate_protocol_allow_multi_channels(false); /* reset it anyway */ + /* URI is not suitable for migration? */ + if (!migration_uri_validate(uri, errp)) { + return; + } + qapi_event_send_migration(MIGRATION_STATUS_SETUP); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multi_channels(true); socket_start_incoming_migration(p ? p : uri, errp); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { @@ -725,11 +738,6 @@ void migration_fd_process_incoming(QEMUFile *f, Error = **errp) migration_incoming_process(); } =20 -static bool migration_needs_multiple_sockets(void) -{ - return migrate_use_multifd() || migrate_postcopy_preempt(); -} - void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); @@ -1344,15 +1352,6 @@ static bool migrate_caps_check(bool *cap_list, } #endif =20 - - /* incoming side only */ - if (runstate_check(RUN_STATE_INMIGRATE) && - !migrate_multi_channels_is_allowed() && - cap_list[MIGRATION_CAPABILITY_MULTIFD]) { - error_setg(errp, "multifd is not supported by current protocol"); - return false; - } - if (cap_list[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT]) { if (!cap_list[MIGRATION_CAPABILITY_POSTCOPY_RAM]) { error_setg(errp, "Postcopy preempt requires postcopy-ram"); @@ -2428,6 +2427,11 @@ void qmp_migrate(const char *uri, bool has_blk, bool= blk, MigrationState *s =3D migrate_get_current(); const char *p =3D NULL; =20 + /* URI is not suitable for migration? */ + if (!migration_uri_validate(uri, errp)) { + return; + } + if (!migrate_prepare(s, has_blk && blk, has_inc && inc, has_resume && resume, errp)) { /* Error detected, put into errp */ @@ -2440,11 +2444,9 @@ void qmp_migrate(const char *uri, bool has_blk, bool= blk, } } =20 - migrate_protocol_allow_multi_channels(false); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multi_channels(true); socket_start_outgoing_migration(s, p ? p : uri, &local_err); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { @@ -4291,12 +4293,8 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) } =20 /* This needs to be done before resuming a postcopy */ - if (postcopy_preempt_setup(s, &local_err)) { - error_report_err(local_err); - migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, - MIGRATION_STATUS_FAILED); - migrate_fd_cleanup(s); - return; + if (migrate_postcopy_preempt()) { + postcopy_preempt_setup(s); } =20 if (resume) { diff --git a/migration/migration.h b/migration/migration.h index ae4ffd3454..ea8d083a51 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -470,7 +470,4 @@ void migration_cancel(const Error *error); void populate_vfio_info(MigrationInfo *info); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); =20 -bool migrate_multi_channels_is_allowed(void); -void migrate_protocol_allow_multi_channels(bool allow); - #endif diff --git a/migration/multifd.c b/migration/multifd.c index 4f6028997d..15d2540b88 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -517,7 +517,7 @@ void multifd_save_cleanup(void) { int i; =20 - if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { + if (!migrate_use_multifd()) { return; } multifd_send_terminate_threads(NULL); @@ -915,10 +915,6 @@ int multifd_save_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } - if (!migrate_multi_channels_is_allowed()) { - error_setg(errp, "multifd is not supported by current protocol"); - return -1; - } =20 thread_count =3D migrate_multifd_channels(); multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); @@ -1021,7 +1017,7 @@ int multifd_load_cleanup(Error **errp) { int i; =20 - if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { + if (!migrate_use_multifd()) { return 0; } multifd_recv_terminate_threads(NULL); @@ -1170,10 +1166,6 @@ int multifd_load_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } - if (!migrate_multi_channels_is_allowed()) { - error_setg(errp, "multifd is not supported by current protocol"); - return -1; - } thread_count =3D migrate_multifd_channels(); multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index f84f783ab4..7872b4fdd3 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1632,22 +1632,10 @@ int postcopy_preempt_wait_channel(MigrationState *s) return s->postcopy_qemufile_src ? 0 : -1; } =20 -int postcopy_preempt_setup(MigrationState *s, Error **errp) +void postcopy_preempt_setup(MigrationState *s) { - if (!migrate_postcopy_preempt()) { - return 0; - } - - if (!migrate_multi_channels_is_allowed()) { - error_setg(errp, "Postcopy preempt is not supported as current " - "migration stream does not support multi-channels."); - return -1; - } - /* Kick an async task to connect */ socket_send_channel_create(postcopy_preempt_send_channel_new, s); - - return 0; } =20 static void postcopy_pause_ram_fast_load(MigrationIncomingState *mis) diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 25881c4127..d5604cbcf1 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -191,7 +191,7 @@ enum PostcopyChannels { }; =20 void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); -int postcopy_preempt_setup(MigrationState *s, Error **errp); +void postcopy_preempt_setup(MigrationState *s); int postcopy_preempt_wait_channel(MigrationState *s); =20 #endif --=20 2.37.3 From nobody Sun May 19 16:27:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1670967789; cv=none; d=zohomail.com; s=zohoarc; b=eer9OX1lOmYZcZC6hmcYfBT9NhdQ6rNi/Hwoh5y6ePa287zC40dH6goeHGUnv32oZITEI2d1AaK/ZEMAqHCMJRrsZEBUG9G5RbGHrmLG0MgkNXprGfbeV/l73tMnmasM+TNXZoxghWUNrNLD7Z3MGv31zC4H2sARfaeWbvkVd+Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670967789; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nlUuDhuH2MYPNNQx7CecYXVPXua+Kg8ijPATqs4gGgU=; b=a1r7XlBI76GAm6vlwoegIDtyFVycVKQcXkHgEuCkK1aCeplWUlw1fgBoAWzEhpJsq0Vp12RcxBQNlWSBZTuJDEGDBabQucQhgYqUmo2YgDQ8U8KWXAo4eaNmea3Q4eDZt2r3wldIjiO0XtFAl4D9MqT48N2AYKeZIlO2Je2tV+Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670967789621627.4344972572142; Tue, 13 Dec 2022 13:43:09 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p5Czd-0000Im-Le; Tue, 13 Dec 2022 16:39:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzT-0000G6-EM for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzR-0007fK-UF for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:07 -0500 Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-74-D5H7ta-ZMf62xi-O5xsKcQ-1; Tue, 13 Dec 2022 16:39:04 -0500 Received: by mail-qk1-f197.google.com with SMTP id h13-20020a05620a244d00b006fb713618b8so1194554qkn.0 for ; Tue, 13 Dec 2022 13:39:04 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020a05620a288c00b006feb158e5e7sm8607487qkp.70.2022.12.13.13.39.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 13:39:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670967545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nlUuDhuH2MYPNNQx7CecYXVPXua+Kg8ijPATqs4gGgU=; b=cughDSN0hEDd/x3k1BAMV8DtdXUl5KpvR1/jk/64xJss+AMfPHQuSZW0/PxIzmWHbQ6+AA z//DiuKYhwU0p2KBuN0aicds8+9V2cyCyRKWRGal3jrSl8QoXy44WyjKBf7QuTceCuHsDM IMc4GPi5N2hVOaiHhpu5QbjqEW/tLbE= X-MC-Unique: D5H7ta-ZMf62xi-O5xsKcQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nlUuDhuH2MYPNNQx7CecYXVPXua+Kg8ijPATqs4gGgU=; b=PQEQH/WBKWxFCFdO8jCWRysICrYMvz/paExiXzrXgKTTFzbZA+J2AJW/evL4B1lwMc aE9SBsyTlQyjNMD9n9N7c+gTyzswUL65qiQE1mzdDIcMH1X/zhuN2nz98xYLmyLRsZlu tf3qXibyJ4p9DBsfP7Zj1kLhP+2KLbG4uxtMuXrQcnzZFMdNI8DxmIHGJ/hBqMv7s92q UhJ5umiW+4rlxNtQU6jOHrdD484yjNdavy5SylSLObxwz4jdPbxcDGehIPdvQTaImKM8 gwUdukQqoUsi1na25sWA4pdnllWohrH1nR0Y5+bCMUIYiSZjpSrgLY1mQXzgltFkn65g 8TGQ== X-Gm-Message-State: ANoB5plxp+9xFuNyEdYmXu4xfsp38AvTniEdMjvNqshEEhqzk7jXJo55 3EBkbzXRoRIA0TJvkjq21PTP/xqGEhAVOmR+bP5xouVgLoDILjnSXekd6rcRQplyJ26rN34FfDE oBLejRW52NeTBieVTfIBGqdim/GoKAd/Sd/7taA6MN9RSUP+90UqklDxIQhUVbjHI X-Received: by 2002:ad4:4d45:0:b0:4c7:6ee5:257c with SMTP id m5-20020ad44d45000000b004c76ee5257cmr27167296qvm.51.1670967542522; Tue, 13 Dec 2022 13:39:02 -0800 (PST) X-Google-Smtp-Source: AA0mqf5PW7bY76+Q43XHeZnrplCcueI5UZbYMCnFFWCoZOU/XqabFTNB/atFGT3l+7/PIbfpgqgChg== X-Received: by 2002:ad4:4d45:0:b0:4c7:6ee5:257c with SMTP id m5-20020ad44d45000000b004c76ee5257cmr27167272qvm.51.1670967542238; Tue, 13 Dec 2022 13:39:02 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , "Daniel P . Berrange" , Leonardo Bras Soares Passos , Manish Mishra , "Dr . David Alan Gilbert" , peterx@redhat.com Subject: [PATCH 4/5] migration: Add a semaphore to count PONGs Date: Tue, 13 Dec 2022 16:38:49 -0500 Message-Id: <20221213213850.1481858-5-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221213213850.1481858-1-peterx@redhat.com> References: <20221213213850.1481858-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1670967790966100003 Content-Type: text/plain; charset="utf-8" This is mostly useless, but useful for us to know whether the main channel is correctly established without changing the migration protocol. Signed-off-by: Peter Xu --- migration/migration.c | 3 +++ migration/migration.h | 6 ++++++ 2 files changed, 9 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 898bc9fe5c..fa59846717 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2981,6 +2981,7 @@ retry: case MIG_RP_MSG_PONG: tmp32 =3D ldl_be_p(buf); trace_source_return_path_thread_pong(tmp32); + qemu_sem_post(&ms->rp_state.rp_pong_acks); break; =20 case MIG_RP_MSG_REQ_PAGES: @@ -4470,6 +4471,7 @@ static void migration_instance_finalize(Object *obj) qemu_sem_destroy(&ms->postcopy_pause_sem); qemu_sem_destroy(&ms->postcopy_pause_rp_sem); qemu_sem_destroy(&ms->rp_state.rp_sem); + qemu_sem_destroy(&ms->rp_state.rp_pong_acks); qemu_sem_destroy(&ms->postcopy_qemufile_src_sem); error_free(ms->error); } @@ -4519,6 +4521,7 @@ static void migration_instance_init(Object *obj) qemu_sem_init(&ms->postcopy_pause_sem, 0); qemu_sem_init(&ms->postcopy_pause_rp_sem, 0); qemu_sem_init(&ms->rp_state.rp_sem, 0); + qemu_sem_init(&ms->rp_state.rp_pong_acks, 0); qemu_sem_init(&ms->rate_limit_sem, 0); qemu_sem_init(&ms->wait_unplug_sem, 0); qemu_sem_init(&ms->postcopy_qemufile_src_sem, 0); diff --git a/migration/migration.h b/migration/migration.h index ea8d083a51..61dc2c81df 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -275,6 +275,12 @@ struct MigrationState { */ bool rp_thread_created; QemuSemaphore rp_sem; + /* + * We post to this when we got one PONG from dest. So far it's an + * easy way to know the main channel has successfully established + * on dest QEMU. + */ + QemuSemaphore rp_pong_acks; } rp_state; =20 double mbps; --=20 2.37.3 From nobody Sun May 19 16:27:43 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1670967872; cv=none; d=zohomail.com; s=zohoarc; b=X+YXc2DDYMvYEGdUpJgtiX/zGLhdC7FpWMGvXOqU6T+TppelYYrw2lcEplxBHhsYh/NJ8pLHfZCfk4UymMsoTe9H4v/E5OaRk7InpIBMS/c2TdmmQrxhSh9TZXRILyxKz8gxeQtDRmgTIg2ACYAPDNGwaFWrIAhbPZ2wsLX2GFE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670967872; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AlvnR1gWb8O/mfxcecLMZW7H10/RtoTDPsSF5rFLWVk=; b=f0xWACLKHlSDGavSmYyYDLjcpDw74bPLS/dm/Sfc9UVDsR+mfd5sas8nvGfEy4OEJ+1LsX4C3vnLnioGTJPzawuqTLMmVfOuQEKoGuAwepjS0vYmi4WYYR5ZtZlUTzd91ALXrVb/h48ebGI2pH9EqiHqDxgKjWY006bHyHFEhm0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1670967872018159.40346722733898; Tue, 13 Dec 2022 13:44:32 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p5Czj-0000NI-1P; Tue, 13 Dec 2022 16:39:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzY-0000Hd-MU for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5CzW-0007fn-OG for qemu-devel@nongnu.org; Tue, 13 Dec 2022 16:39:12 -0500 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-133-j18HXkIsN96uGW8t508YgA-1; Tue, 13 Dec 2022 16:39:08 -0500 Received: by mail-qv1-f69.google.com with SMTP id jh2-20020a0562141fc200b004c74bbb0affso15862513qvb.21 for ; Tue, 13 Dec 2022 13:39:08 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020a05620a288c00b006feb158e5e7sm8607487qkp.70.2022.12.13.13.39.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 13:39:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670967550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AlvnR1gWb8O/mfxcecLMZW7H10/RtoTDPsSF5rFLWVk=; b=fyaPdo+GVsJydqzJkpN3PQjXKUF6YsUHvd81utmqQlbKXc037joUQfpH2dSCPPKw2Q4oub SEFsCMpQN7zCbAiVDmKYt9w1Aarj0AeR3j2phbPRmpcFTaw+0y+mmyfq/JoTEBza1D/tsD sRcvuuxTt5yIsVKoniG2UKGqgh6Ox/U= X-MC-Unique: j18HXkIsN96uGW8t508YgA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AlvnR1gWb8O/mfxcecLMZW7H10/RtoTDPsSF5rFLWVk=; b=glvxj7TbXmZkaJ2QNrGwiE/ePmKG0IjUujXVT+NGNFik+mRHowLjuBTqs+Q2kNZy71 Vhb3F+6kuIGqFOhgiJ1xsvVNoZK9c5T1MBZGNc5V2GWc1bSnqnCUaIX2NNd31vGYNtB+ b6n8Kuq4fuiBCrEwnvOaMKu5V6L8t8DQaHW3zJx6xTxK52G+F/TH398a21nzxCAgK6pE ww2D3uAS3mWR3sufpQlckFGYJKvCGz6E/K+gtFNh0wFQZbWc65LDOsNzVbanOJ/r6Ml8 GWlBsHXOtPZuIsbl+/LLaeis4b2Mm7C85jYAG32j6PsafYDM0LdAv0DDg6KsEV5/2GA+ e4yA== X-Gm-Message-State: ANoB5pmRg9+fcSP9elsjgMcXClUJ34+v92MyHevd4cHdFZDdIGwnDOw4 TWPThRmUvAGoNlMMVwp+Fiz/tbsBvbRcirdCNRt8wf2TKi0NGevWCJQ+1SvNQpVxMc9EAr8+PQ0 4IgQk8dvpr4pjYQYc2sf9l46JV3G/peO0c4JoOvsV3sDpfc8201Zma8pj+ZUNwUe5 X-Received: by 2002:a05:622a:ce:b0:3a5:fe93:7df9 with SMTP id p14-20020a05622a00ce00b003a5fe937df9mr40478974qtw.31.1670967545569; Tue, 13 Dec 2022 13:39:05 -0800 (PST) X-Google-Smtp-Source: AA0mqf6/2POGgkWcpBOkk6QtNLdw+Yh18tWi4+WBM/nkal7TuZ0gXESr5y/fDXg8AC2ZxuOJep406w== X-Received: by 2002:a05:622a:ce:b0:3a5:fe93:7df9 with SMTP id p14-20020a05622a00ce00b003a5fe937df9mr40478935qtw.31.1670967545145; Tue, 13 Dec 2022 13:39:05 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , "Daniel P . Berrange" , Leonardo Bras Soares Passos , Manish Mishra , "Dr . David Alan Gilbert" , peterx@redhat.com Subject: [PATCH 5/5] migration: Postpone postcopy preempt channel to be after main Date: Tue, 13 Dec 2022 16:38:50 -0500 Message-Id: <20221213213850.1481858-6-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221213213850.1481858-1-peterx@redhat.com> References: <20221213213850.1481858-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1670967873410100003 Content-Type: text/plain; charset="utf-8" Postcopy with preempt-mode enabled needs two channels to communicate. The order of channel establishment is not guaranteed. It can happen that the dest QEMU got the preempt channel connection request before the main channel is established, then the migration may make no progress even during precopy due to the wrong order. To fix it, create the preempt channel only if we know the main channel is established. For a general postcopy migration, we delay it until postcopy_start(), that's where we already went through some part of precopy on the main channel. To make sure dest QEMU has already established the channel, we wait until we got the first PONG received. That's something we do at the start of precopy when postcopy enabled so it's guaranteed to happen sooner or later. For a postcopy recovery, we delay it to qemu_savevm_state_resume_prepare() where we'll have round trips of data on bitmap synchronizations, which means the main channel must have been established. Signed-off-by: Peter Xu --- migration/migration.c | 72 ++++++++++++++++++++++++++++++---------- migration/migration.h | 6 ++++ migration/postcopy-ram.c | 17 ++++++++-- migration/postcopy-ram.h | 2 +- migration/savevm.c | 6 +++- 5 files changed, 82 insertions(+), 21 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index fa59846717..a85f0a092d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -232,6 +232,8 @@ void migration_object_init(void) qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fast_load, 0); + qemu_sem_init(¤t_incoming->postcopy_qemufile_dst_done, 0); + qemu_mutex_init(¤t_incoming->page_request_mutex); current_incoming->page_requested =3D g_tree_new(page_request_addr_cmp); =20 @@ -738,6 +740,31 @@ void migration_fd_process_incoming(QEMUFile *f, Error = **errp) migration_incoming_process(); } =20 +/* + * Returns true when we want to start a new incoming migration process, + * false otherwise. + */ +static bool migration_should_start_incoming(bool main_channel) +{ + /* Multifd doesn't start unless all channels are established */ + if (migrate_use_multifd()) { + return migration_has_all_channels(); + } + + /* Preempt channel only starts when the main channel is created */ + if (migrate_postcopy_preempt()) { + return main_channel; + } + + /* + * For all the rest types of migration, we should only reach here when + * it's the main channel that's being created, and we should always + * proceed with this channel. + */ + assert(main_channel); + return true; +} + void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); @@ -794,7 +821,7 @@ void migration_ioc_process_incoming(QIOChannel *ioc, Er= ror **errp) } } =20 - if (migration_has_all_channels()) { + if (migration_should_start_incoming(default_channel)) { /* If it's a recovery, we're done */ if (postcopy_try_recover()) { return; @@ -3115,6 +3142,13 @@ static int await_return_path_close_on_source(Migrati= onState *ms) return ms->rp_state.error; } =20 +static inline void +migration_wait_main_channel(MigrationState *ms) +{ + /* Wait until one PONG message received */ + qemu_sem_wait(&ms->rp_state.rp_pong_acks); +} + /* * Switch from normal iteration to postcopy * Returns non-0 on error @@ -3129,9 +3163,12 @@ static int postcopy_start(MigrationState *ms) bool restart_block =3D false; int cur_state =3D MIGRATION_STATUS_ACTIVE; =20 - if (postcopy_preempt_wait_channel(ms)) { - migrate_set_state(&ms->state, ms->state, MIGRATION_STATUS_FAILED); - return -1; + if (migrate_postcopy_preempt()) { + migration_wait_main_channel(ms); + if (postcopy_preempt_establish_channel(ms)) { + migrate_set_state(&ms->state, ms->state, MIGRATION_STATUS_FAIL= ED); + return -1; + } } =20 if (!migrate_pause_before_switchover()) { @@ -3542,6 +3579,20 @@ static int postcopy_do_resume(MigrationState *s) return ret; } =20 + /* + * If preempt is enabled, re-establish the preempt channel. Note that + * we do it after resume prepare to make sure the main channel will be + * created before the preempt channel. E.g. with weak network, the + * dest QEMU may get messed up with the preempt and main channels on + * the order of connection setup. This guarantees the correct order. + */ + ret =3D postcopy_preempt_establish_channel(s); + if (ret) { + error_report("%s: postcopy_preempt_establish_channel(): %d", + __func__, ret); + return ret; + } + /* * Last handshake with destination on the resume (destination will * switch to postcopy-active afterwards) @@ -3603,14 +3654,6 @@ static MigThrError postcopy_pause(MigrationState *s) if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { /* Woken up by a recover procedure. Give it a shot */ =20 - if (postcopy_preempt_wait_channel(s)) { - /* - * Preempt enabled, and new channel create failed; loop - * back to wait for another recovery. - */ - continue; - } - /* * Firstly, let's wake up the return path now, with a new * return path channel. @@ -4293,11 +4336,6 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) } } =20 - /* This needs to be done before resuming a postcopy */ - if (migrate_postcopy_preempt()) { - postcopy_preempt_setup(s); - } - if (resume) { /* Wakeup the main migration thread to do the recovery */ migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_PAUSED, diff --git a/migration/migration.h b/migration/migration.h index 61dc2c81df..5674a13876 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -115,6 +115,12 @@ struct MigrationIncomingState { unsigned int postcopy_channels; /* QEMUFile for postcopy only; it'll be handled by a separate thread */ QEMUFile *postcopy_qemufile_dst; + /* + * When postcopy_qemufile_dst is properly setup, this sem is posted. + * One can wait on this semaphore to wait until the preempt channel is + * properly setup. + */ + QemuSemaphore postcopy_qemufile_dst_done; /* Postcopy priority thread is used to receive postcopy requested page= s */ QemuThread postcopy_prio_thread; bool postcopy_prio_thread_created; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 7872b4fdd3..2c86bfc091 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1200,6 +1200,11 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) } =20 if (migrate_postcopy_preempt()) { + /* + * The preempt channel is established in asynchronous way. Wait + * for its completion. + */ + qemu_sem_wait(&mis->postcopy_qemufile_dst_done); /* * This thread needs to be created after the temp pages because * it'll fetch RAM_CHANNEL_POSTCOPY PostcopyTmpPage immediately. @@ -1547,6 +1552,7 @@ void postcopy_preempt_new_channel(MigrationIncomingSt= ate *mis, QEMUFile *file) */ qemu_file_set_blocking(file, true); mis->postcopy_qemufile_dst =3D file; + qemu_sem_post(&mis->postcopy_qemufile_dst_done); trace_postcopy_preempt_new_channel(); } =20 @@ -1615,14 +1621,21 @@ out: postcopy_preempt_send_channel_done(s, ioc, local_err); } =20 -/* Returns 0 if channel established, -1 for error. */ -int postcopy_preempt_wait_channel(MigrationState *s) +/* + * This function will kick off an async task to establish the preempt + * channel, and wait until the connection setup completed. Returns 0 if + * channel established, -1 for error. + */ +int postcopy_preempt_establish_channel(MigrationState *s) { /* If preempt not enabled, no need to wait */ if (!migrate_postcopy_preempt()) { return 0; } =20 + /* Kick off async task to establish preempt channel */ + postcopy_preempt_setup(s); + /* * We need the postcopy preempt channel to be established before * starting doing anything. diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index d5604cbcf1..b4867a32d5 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -192,6 +192,6 @@ enum PostcopyChannels { =20 void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); void postcopy_preempt_setup(MigrationState *s); -int postcopy_preempt_wait_channel(MigrationState *s); +int postcopy_preempt_establish_channel(MigrationState *s); =20 #endif diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..881dce4647 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2118,7 +2118,11 @@ static int loadvm_postcopy_handle_resume(MigrationIn= comingState *mis) qemu_sem_post(&mis->postcopy_pause_sem_fault); =20 if (migrate_postcopy_preempt()) { - /* The channel should already be setup again; make sure of it */ + /* + * The preempt channel will be created in async manner, now let's + * wait for it and make sure it's created. + */ + qemu_sem_wait(&mis->postcopy_qemufile_dst_done); assert(mis->postcopy_qemufile_dst); /* Kick the fast ram load thread too */ qemu_sem_post(&mis->postcopy_pause_sem_fast_load); --=20 2.37.3