From nobody Wed Oct 29 17:10:19 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525531065431507.3356364823526; Sat, 5 May 2018 07:37:45 -0700 (PDT) Received: from localhost ([::1]:39037 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyJk-0000mW-5v for importer@patchew.org; Sat, 05 May 2018 10:37:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58389) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyI9-0008Mk-08 for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fEyI7-00084m-UQ for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:05 -0400 Received: from mail-pg0-x243.google.com ([2607:f8b0:400e:c05::243]:37170) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fEyI7-00084C-Nj for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:03 -0400 Received: by mail-pg0-x243.google.com with SMTP id a13-v6so17326052pgu.4 for ; Sat, 05 May 2018 07:36:03 -0700 (PDT) Received: from VM_127_91_centos.localdomain ([119.28.195.137]) by smtp.gmail.com with ESMTPSA id n67sm38529640pfh.188.2018.05.05.07.36.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 May 2018 07:36:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mf5U6AdnNQLsermZiUHmfOKFPT6CfqKr1GXfJOxDllU=; b=acMXzLLTY5SWzegsFO/u7lurnV0XOJEucmLcKm9ZA3tZhf9G4J6FfnqG07ikgnQjBQ kFPpIQMh/IqZP8BNJe0QjxslRbzDNl12uOWqT8RyBzsSz0/5zmXdE7M/nHhw6xPmytPC 9B63nvF36YDH2nlglWjEZ19NWxgIU6JmSE1cpz1bhAFNUhcYPIpyIGQO0qmTrWjki8SC V5UA1zP5MoA+afz9h56xx5br8xwsgRCOPE0DY4RPuC1bhO2vKjieYlhdAHf+FG80Jzno 7GIAs1UFD5B2OH8i6cDujvNZzx7ihj/qPI36um13XzgQbx8zuOXIdVNtcDQkeviqMsu3 yVBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mf5U6AdnNQLsermZiUHmfOKFPT6CfqKr1GXfJOxDllU=; b=TVQpaxcD0bpi/3XKdsvX5n8Th7ooMnn1uOqdPtg7qiG2ClAAPZchz211IYb7MT0HXw 2HIQpIaNIi3jontnjWxZ+m7jZ+PTG8UJTTEqByH3o2dbZj1NGkBNR7LGKCbcXGo88K5y zq52XS34U68/5HODXzcRqLr1Qqmeq7LokXIZKvq6mIBNAnfqxC0KxPidkg+fuYJ9Y2Z+ 3xt1p6ZWkrl3b/D2lvX5BH54abnuCCnwvLkbFgjKEQFoNste948Yo8avTynNhsJvSG9h 27ZnKOO12ENBEIwt6J2GY3p7zAnpvfM522r34vxcxASF8eK6evRq+sUOHa1+Sq9J/hn/ tfTg== X-Gm-Message-State: ALQs6tDD05Bv7W1d2rSOQ85TUXfUn++gIF8pWvJwOl+o8AO/518Sx/7B sxkYC+g0v+BlOFp/kEFfhLVaJA== X-Google-Smtp-Source: AB8JxZqhs1PDsRDScgbynm8/Yb7q3o8kKKm2qWyueABxXIMC2j2Y4SnKG5pJaZhhlGA0Bh6eW6zfjA== X-Received: by 10.98.69.137 with SMTP id n9mr29538245pfi.158.1525530962977; Sat, 05 May 2018 07:36:02 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: quintela@redhat.com, dgilbert@redhat.com, berrange@redhat.com Date: Sat, 5 May 2018 22:35:31 +0800 Message-Id: <1525530936-21835-2-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> References: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::243 Subject: [Qemu-devel] [PATCH v3 1/6] migration: disable RDMA WRITE after postcopy started X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: adido@mellanox.com, galsha@mellanox.com, aviadye@mellanox.com, qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" RDMA WRITE operations are performed with no notification to the destination qemu, then the destination qemu can not wakeup. This patch disable RDMA WRI= TE after postcopy started. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/qemu-file.c | 8 ++++++-- migration/rdma.c | 12 ++++++++++++ 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 0463f4c..977b9ae 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -253,8 +253,12 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t b= lock_offset, if (f->hooks && f->hooks->save_page) { int ret =3D f->hooks->save_page(f, f->opaque, block_offset, offset, size, bytes_sent); - f->bytes_xfer +=3D size; - if (ret !=3D RAM_SAVE_CONTROL_DELAYED) { + if (ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { + f->bytes_xfer +=3D size; + } + + if (ret !=3D RAM_SAVE_CONTROL_DELAYED && + ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { if (bytes_sent && *bytes_sent > 0) { qemu_update_position(f, *bytes_sent); } else if (ret < 0) { diff --git a/migration/rdma.c b/migration/rdma.c index da474fc..a22be43 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2927,6 +2927,10 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void = *opaque, =20 CHECK_ERROR_STATE(); =20 + if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + return RAM_SAVE_CONTROL_NOT_SUPP; + } + qemu_fflush(f); =20 if (size > 0) { @@ -3482,6 +3486,10 @@ static int qemu_rdma_registration_start(QEMUFile *f,= void *opaque, =20 CHECK_ERROR_STATE(); =20 + if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + return 0; + } + trace_qemu_rdma_registration_start(flags); qemu_put_be64(f, RAM_SAVE_FLAG_HOOK); qemu_fflush(f); @@ -3504,6 +3512,10 @@ static int qemu_rdma_registration_stop(QEMUFile *f, = void *opaque, =20 CHECK_ERROR_STATE(); =20 + if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + return 0; + } + qemu_fflush(f); ret =3D qemu_rdma_drain_cq(f, rdma); =20 --=20 1.8.3.1 From nobody Wed Oct 29 17:10:19 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525531176440325.29184936159436; Sat, 5 May 2018 07:39:36 -0700 (PDT) Received: from localhost ([::1]:39044 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyLX-0002CY-Iy for importer@patchew.org; Sat, 05 May 2018 10:39:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58400) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyIB-0008OP-Ig for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fEyIA-000876-BE for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:07 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:46792) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fEyIA-00086b-2e for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:06 -0400 Received: by mail-pf0-x243.google.com with SMTP id p12so19564886pff.13 for ; Sat, 05 May 2018 07:36:05 -0700 (PDT) Received: from VM_127_91_centos.localdomain ([119.28.195.137]) by smtp.gmail.com with ESMTPSA id n67sm38529640pfh.188.2018.05.05.07.36.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 May 2018 07:36:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wAd4xypdYL/wi/5ZPgwiQvTdyXGrJSPu1qDPeE2wRUo=; b=Z6DhfLT9VUs/3SZW/51nmzuWWFRpQOa2mrCf4jEJpUH1Iw9WaSWOZIkfe3AjxEAthJ bugVfPnH1D3YCXvrLAZrwgCP3oJnX5+bh2JrvfaBGO98Jnov1ePwgFxz2bqv/I3uG0/l Ty0eaLN/zTnGI5KBK3yp+al2YZQzsd4La90ckH1zpSlOAmVF5knEGBYCFeb2nmu2b8XV 7C4vdKPLBg+8BUd5kfSbLGHLohlN2cpjGljKUAybNbJwspv3uZxQfXjD2nQJBCNzjF4p hsI2IuCFe+VJyG/r678kb3dRuR02D4VNhwP8sa6nGP8yv1OfDzOJtfmfR6rtS6vuIOac Vicw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wAd4xypdYL/wi/5ZPgwiQvTdyXGrJSPu1qDPeE2wRUo=; b=EhBOImxr66Wd4WrQFzWj2HwvfBJ+mJ5O7SxyX3ZtLOyz/6rj6Dcg6bpNMMB5IYD1dL iXEUi5LeMmFML5GmxbQlnGImLYxbLXn0O8FIxUk+C0CSj4C3n3sqEJjs3fsT0KEIDirw VcP0mddawGCVu51AsSAIoZzc3PAH2vM8kyG0LV7qrnIcW9JVP6jNcCFKse68PVdbcleI z8DkwN5v03oggeF+BqhQ7J41UGSW7NSKZhy8LJNxec963bwryBU7Wp6UYrzjMae24FZw cIsUKn6mdIV/yrW5tw8Va7ScjZuj3JtqJKjtoAvBNvRSZ3TWqXLqvs5t3ZmTKCLOHrxT pYaw== X-Gm-Message-State: ALQs6tCEL33YfbjAExDLWKqOLy6LovMH1S/SGamMZJD9Scz+AJ6xCGvp 1a9wc80x9cReXWSborl0CiY= X-Google-Smtp-Source: AB8JxZq9eMc/SAd17OEZADomR9z3QI2TEKyd2AK5Nuc6oQ70HG9TZ7tHyjt+icJ/gHe2ldOjjH+0jA== X-Received: by 10.98.200.29 with SMTP id z29mr30639672pff.81.1525530965099; Sat, 05 May 2018 07:36:05 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: quintela@redhat.com, dgilbert@redhat.com, berrange@redhat.com Date: Sat, 5 May 2018 22:35:32 +0800 Message-Id: <1525530936-21835-3-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> References: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PATCH v3 2/6] migration: create a dedicated connection for rdma return path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: adido@mellanox.com, galsha@mellanox.com, aviadye@mellanox.com, qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If start a RDMA migration with postcopy enabled, the source qemu establish a dedicated connection for return path. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++= ++-- 1 file changed, 91 insertions(+), 3 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index a22be43..c745427 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -387,6 +387,10 @@ typedef struct RDMAContext { uint64_t unregistrations[RDMA_SIGNALED_SEND_MAX]; =20 GHashTable *blockmap; + + /* the RDMAContext for return path */ + struct RDMAContext *return_path; + bool is_return_path; } RDMAContext; =20 #define TYPE_QIO_CHANNEL_RDMA "qio-channel-rdma" @@ -2329,10 +2333,22 @@ static void qemu_rdma_cleanup(RDMAContext *rdma) rdma_destroy_id(rdma->cm_id); rdma->cm_id =3D NULL; } + + /* the destination side, listen_id and channel is shared */ if (rdma->listen_id) { - rdma_destroy_id(rdma->listen_id); + if (!rdma->is_return_path) { + rdma_destroy_id(rdma->listen_id); + } rdma->listen_id =3D NULL; + + if (rdma->channel) { + if (!rdma->is_return_path) { + rdma_destroy_event_channel(rdma->channel); + } + rdma->channel =3D NULL; + } } + if (rdma->channel) { rdma_destroy_event_channel(rdma->channel); rdma->channel =3D NULL; @@ -2561,6 +2577,25 @@ err_dest_init_create_listen_id: =20 } =20 +static void qemu_rdma_return_path_dest_init(RDMAContext *rdma_return_path, + RDMAContext *rdma) +{ + int idx; + + for (idx =3D 0; idx < RDMA_WRID_MAX; idx++) { + rdma_return_path->wr_data[idx].control_len =3D 0; + rdma_return_path->wr_data[idx].control_curr =3D NULL; + } + + /*the CM channel and CM id is shared*/ + rdma_return_path->channel =3D rdma->channel; + rdma_return_path->listen_id =3D rdma->listen_id; + + rdma->return_path =3D rdma_return_path; + rdma_return_path->return_path =3D rdma; + rdma_return_path->is_return_path =3D true; +} + static void *qemu_rdma_data_init(const char *host_port, Error **errp) { RDMAContext *rdma =3D NULL; @@ -3018,6 +3053,8 @@ err: return ret; } =20 +static void rdma_accept_incoming_migration(void *opaque); + static int qemu_rdma_accept(RDMAContext *rdma) { RDMACapabilities cap; @@ -3112,7 +3149,14 @@ static int qemu_rdma_accept(RDMAContext *rdma) } } =20 - qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL); + /* Accept the second connection request for return path */ + if (migrate_postcopy() && !rdma->is_return_path) { + qemu_set_fd_handler(rdma->channel->fd, rdma_accept_incoming_migrat= ion, + NULL, + (void *)(intptr_t)rdma->return_path); + } else { + qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL); + } =20 ret =3D rdma_accept(rdma->cm_id, &conn_param); if (ret) { @@ -3693,6 +3737,10 @@ static void rdma_accept_incoming_migration(void *opa= que) =20 trace_qemu_rdma_accept_incoming_migration_accepted(); =20 + if (rdma->is_return_path) { + return; + } + f =3D qemu_fopen_rdma(rdma, "rb"); if (f =3D=3D NULL) { ERROR(errp, "could not qemu_fopen_rdma!"); @@ -3707,7 +3755,7 @@ static void rdma_accept_incoming_migration(void *opaq= ue) void rdma_start_incoming_migration(const char *host_port, Error **errp) { int ret; - RDMAContext *rdma; + RDMAContext *rdma, *rdma_return_path; Error *local_err =3D NULL; =20 trace_rdma_start_incoming_migration(); @@ -3734,12 +3782,24 @@ void rdma_start_incoming_migration(const char *host= _port, Error **errp) =20 trace_rdma_start_incoming_migration_after_rdma_listen(); =20 + /* initialize the RDMAContext for return path */ + if (migrate_postcopy()) { + rdma_return_path =3D qemu_rdma_data_init(host_port, &local_err); + + if (rdma_return_path =3D=3D NULL) { + goto err; + } + + qemu_rdma_return_path_dest_init(rdma_return_path, rdma); + } + qemu_set_fd_handler(rdma->channel->fd, rdma_accept_incoming_migration, NULL, (void *)(intptr_t)rdma); return; err: error_propagate(errp, local_err); g_free(rdma); + g_free(rdma_return_path); } =20 void rdma_start_outgoing_migration(void *opaque, @@ -3747,6 +3807,7 @@ void rdma_start_outgoing_migration(void *opaque, { MigrationState *s =3D opaque; RDMAContext *rdma =3D qemu_rdma_data_init(host_port, errp); + RDMAContext *rdma_return_path =3D NULL; int ret =3D 0; =20 if (rdma =3D=3D NULL) { @@ -3767,6 +3828,32 @@ void rdma_start_outgoing_migration(void *opaque, goto err; } =20 + /* RDMA postcopy need a seprate queue pair for return path */ + if (migrate_postcopy()) { + rdma_return_path =3D qemu_rdma_data_init(host_port, errp); + + if (rdma_return_path =3D=3D NULL) { + goto err; + } + + ret =3D qemu_rdma_source_init(rdma_return_path, + s->enabled_capabilities[MIGRATION_CAPABILITY_RDMA_PIN_ALL], er= rp); + + if (ret) { + goto err; + } + + ret =3D qemu_rdma_connect(rdma_return_path, errp); + + if (ret) { + goto err; + } + + rdma->return_path =3D rdma_return_path; + rdma_return_path->return_path =3D rdma; + rdma_return_path->is_return_path =3D true; + } + trace_rdma_start_outgoing_migration_after_rdma_connect(); =20 s->to_dst_file =3D qemu_fopen_rdma(rdma, "wb"); @@ -3774,4 +3861,5 @@ void rdma_start_outgoing_migration(void *opaque, return; err: g_free(rdma); + g_free(rdma_return_path); } --=20 1.8.3.1 From nobody Wed Oct 29 17:10:19 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525531072036884.8354071708653; Sat, 5 May 2018 07:37:52 -0700 (PDT) Received: from localhost ([::1]:39038 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyJr-0000rm-5U for importer@patchew.org; Sat, 05 May 2018 10:37:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58414) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyIF-0008Qs-BB for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fEyIC-000896-7A for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:11 -0400 Received: from mail-pf0-x242.google.com ([2607:f8b0:400e:c00::242]:40465) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fEyIC-00087t-1U for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:08 -0400 Received: by mail-pf0-x242.google.com with SMTP id f189so19575124pfa.7 for ; Sat, 05 May 2018 07:36:07 -0700 (PDT) Received: from VM_127_91_centos.localdomain ([119.28.195.137]) by smtp.gmail.com with ESMTPSA id n67sm38529640pfh.188.2018.05.05.07.36.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 May 2018 07:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KI7yza/CX80Qu1nB/Rg7RbScxhHCIvUGBOeuuYrplns=; b=bpyDzY6Oc2dgBa5cwKr0EOnU4rGrWDfwlgx17Y6C4VgSDW/zZGLkLiwoLsyJpUGKiv HqEVTDC7i1+Ntis2znU75Y8wxcp0lkSbinGH1m0YYOFWHWj6G/+Se2hymtIEHOlXCcOX DlCqXzHOcD0DxL1XQV9jRmfnMZVsK8zez0TzBzx97eYwAEjOqHQ9YSw7j9Yf8Dz47pWT X/u5NFbxSAcEiyOoGItX4nxY+y+t5uOj01W0mtkp2SKt6jRnMuYSHp49rKG63d6Oru2R lBxnl5xx0d9LvJ5fs3qJwo7IUHs6e8JwVH+/dTnrswt+MsMv+d4t4UNR3LvymsuTmxq2 zcPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KI7yza/CX80Qu1nB/Rg7RbScxhHCIvUGBOeuuYrplns=; b=AGuWRmPuZ35gkhYG1EYK+FwcRMSuV+N1NtPR0tMgpPCcH/de464+ZyNDCvZoyUODKv pZ39BOdJCin3bkWbtu1DuespfDHu/xYQbekhwQwP2FaEU7Tk7WmcD099QQn4zYMnUQcT 4+c//QgQtX/sbsKueJcEohJTAv6J/Wldx2xiqOWjOxHFQzdIWxVa5DxfJ/EGS85Liew1 +mjI3AK/+0kVOqc/B/TPw4Gq4ZMFiP5qzflexeZxKz53Ciqc8cALNP5t4eBJZvsN3eT1 Y1JVVbmsyuQBMMF9yWG8zI8Lh7Zf4Pd1CLUN+WDvmpDJO/3DDmt604JFkOwgbIU3HZi7 5GGQ== X-Gm-Message-State: ALQs6tCb2ZpDg4ydrxJ5ko0RclRPUyV9qGj+aKoJksO2z77kSxK21Vz9 XdO2jwxCemi5+kKYox6Ahv4= X-Google-Smtp-Source: AB8JxZqGg+tzc7HyS32HL2m6fmLTh0QDfUcBchsTTGLPHz3Wm+uU4CRZjkyQ0O4rCXslQkoqF72fYQ== X-Received: by 10.98.76.202 with SMTP id e71mr30321261pfj.171.1525530967223; Sat, 05 May 2018 07:36:07 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: quintela@redhat.com, dgilbert@redhat.com, berrange@redhat.com Date: Sat, 5 May 2018 22:35:33 +0800 Message-Id: <1525530936-21835-4-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> References: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::242 Subject: [Qemu-devel] [PATCH v3 3/6] migration: remove unnecessary variables len in QIOChannelRDMA X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: adido@mellanox.com, galsha@mellanox.com, aviadye@mellanox.com, qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Because qio_channel_rdma_writev and qio_channel_rdma_readv maybe invoked by different threads concurrently, this patch removes unnecessary variables len in QIOChannelRDMA and use local variable instead. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Daniel P. Berrang=C3=A9berrange@redhat.com> --- migration/rdma.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index c745427..f5c1d02 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -404,7 +404,6 @@ struct QIOChannelRDMA { QIOChannel parent; RDMAContext *rdma; QEMUFile *file; - size_t len; bool blocking; /* XXX we don't actually honour this yet */ }; =20 @@ -2640,6 +2639,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, int ret; ssize_t done =3D 0; size_t i; + size_t len =3D 0; =20 CHECK_ERROR_STATE(); =20 @@ -2659,10 +2659,10 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *= ioc, while (remaining) { RDMAControlHeader head; =20 - rioc->len =3D MIN(remaining, RDMA_SEND_INCREMENT); - remaining -=3D rioc->len; + len =3D MIN(remaining, RDMA_SEND_INCREMENT); + remaining -=3D len; =20 - head.len =3D rioc->len; + head.len =3D len; head.type =3D RDMA_CONTROL_QEMU_FILE; =20 ret =3D qemu_rdma_exchange_send(rdma, &head, data, NULL, NULL,= NULL); @@ -2672,8 +2672,8 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, return ret; } =20 - data +=3D rioc->len; - done +=3D rioc->len; + data +=3D len; + done +=3D len; } } =20 @@ -2768,8 +2768,7 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc, } } } - rioc->len =3D done; - return rioc->len; + return done; } =20 /* --=20 1.8.3.1 From nobody Wed Oct 29 17:10:19 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525531278065414.2626501248569; Sat, 5 May 2018 07:41:18 -0700 (PDT) Received: from localhost ([::1]:39057 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyNB-0003e6-DL for importer@patchew.org; Sat, 05 May 2018 10:41:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58415) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyIF-0008Qt-BM for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fEyIE-0008Cl-AH for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:11 -0400 Received: from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]:40382) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fEyIE-0008BN-3R for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:10 -0400 Received: by mail-pg0-x242.google.com with SMTP id l2-v6so17312785pgc.7 for ; Sat, 05 May 2018 07:36:09 -0700 (PDT) Received: from VM_127_91_centos.localdomain ([119.28.195.137]) by smtp.gmail.com with ESMTPSA id n67sm38529640pfh.188.2018.05.05.07.36.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 May 2018 07:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2uQY15QStn6Po6OCGcb7M+5H4LvOklMk0X5aqrbF9/Y=; b=ceS0qhOfYTK+2cHiq3+WVTnWgZpv/1irdN9P7zJfgCoKZAJrZ5ifloEIwGtu+mbWuS PHqfHkQuF6HdkeeY2nwmSjecSIR2d3zVl6KpIxMRWbWUWTcP7njXyvyk/m19cglpVhPZ 3zGrI+xp4aozraqsfxfwDH+kgFZU7SmIInj2ZuoNoObzpwn1q2kEKZWsh+dLrcx3dsNn 5ZVOUg9YahHraWmz2R1HDRWzpTB4EwmgFTdI0VznDpTH+Mam7qQZHPrN0TUwDlyKCwow 60b9z7c8NRGtoXzEnjBgekXdbtZ6b29QNvEns8JwtDAFLDSCsPlBDv/Xv0Y1zrtUperJ SPAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2uQY15QStn6Po6OCGcb7M+5H4LvOklMk0X5aqrbF9/Y=; b=CPMtiSeI42ZWsT3WLraaYB1i46smwH4WM9rGFUCcgX9LxT55jQXiuAK7H4/2F3AcO9 wlOUS+etF1bWZbC50VPMnP/mnVgkb8Ivee30/lBztC6zXoYXTyxVNO0z4wdUq6pdIG5e q3U3v5/SuYJL76pI/dq5boeK+0FXukz91lrCVaxhFDcMPgsXJaet6ubyO6o1Zu644dEP WIHj0OEyGSF415+ipAJj4n4KlSf/t2dRiFRH2ZlTn/rdKLT/ZK/84DjFU2sMRVnqPtbJ 7zFILbkhL5WmXaxJgotd3HDwhFvZIB3AJGi3Dxe6uo1B3u34Af2Zo8nvlOc7UnJQgxmZ ZnBA== X-Gm-Message-State: ALQs6tDmapezR43nkt36pRUF4q6iBP5EaYZhQ+ac566bjRGWbFL3Lq9d tHmkzh+vbbYNroX+37ouqWw= X-Google-Smtp-Source: AB8JxZroet8Vp+O3K5h5DUHDvzuaqcelrvLjT98Sg3oNuOWujfjrWVCytRwWbGibPRv8iooMeFFIuA== X-Received: by 2002:a63:41c5:: with SMTP id o188-v6mr24679560pga.7.1525530969275; Sat, 05 May 2018 07:36:09 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: quintela@redhat.com, dgilbert@redhat.com, berrange@redhat.com Date: Sat, 5 May 2018 22:35:34 +0800 Message-Id: <1525530936-21835-5-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> References: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::242 Subject: [Qemu-devel] [PATCH v3 4/6] migration: avoid concurrent invoke channel_close by different threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: adido@mellanox.com, galsha@mellanox.com, aviadye@mellanox.com, qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The channel_close maybe invoked by different threads. For example, source qemu invokes qemu_fclose in main thread, migration thread and return path thread. Destination qemu invokes qemu_fclose in main thread, listen thread and COLO incoming thread. Add a mutex in QEMUFile struct to avoid concurrent invoke channel_close. Signed-off-by: Lidong Chen --- migration/qemu-file.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 977b9ae..87d0f05 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -52,6 +52,7 @@ struct QEMUFile { unsigned int iovcnt; =20 int last_error; + QemuMutex lock; }; =20 /* @@ -96,6 +97,7 @@ QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps = *ops) =20 f =3D g_new0(QEMUFile, 1); =20 + qemu_mutex_init(&f->lock); f->opaque =3D opaque; f->ops =3D ops; return f; @@ -328,7 +330,9 @@ int qemu_fclose(QEMUFile *f) ret =3D qemu_file_get_error(f); =20 if (f->ops->close) { + qemu_mutex_lock(&f->lock); int ret2 =3D f->ops->close(f->opaque); + qemu_mutex_unlock(&f->lock); if (ret >=3D 0) { ret =3D ret2; } @@ -339,6 +343,7 @@ int qemu_fclose(QEMUFile *f) if (f->last_error) { ret =3D f->last_error; } + qemu_mutex_destroy(&f->lock); g_free(f); trace_qemu_file_fclose(); return ret; --=20 1.8.3.1 From nobody Wed Oct 29 17:10:19 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525531181777577.4424626718691; Sat, 5 May 2018 07:39:41 -0700 (PDT) Received: from localhost ([::1]:39045 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyLc-0002Ge-UO for importer@patchew.org; Sat, 05 May 2018 10:39:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58444) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyIJ-0008T7-JC for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fEyIH-0008Ew-3k for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:15 -0400 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:39642) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fEyIG-0008EP-Q3 for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:13 -0400 Received: by mail-pf0-x244.google.com with SMTP id a22so5483179pfn.6 for ; Sat, 05 May 2018 07:36:12 -0700 (PDT) Received: from VM_127_91_centos.localdomain ([119.28.195.137]) by smtp.gmail.com with ESMTPSA id n67sm38529640pfh.188.2018.05.05.07.36.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 May 2018 07:36:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8f5o+M3IMKSkf+u7xy0Pn2f+U92/TorLyNLqHaa75dY=; b=E83OGyk26Plx7ctPqCbqNtE0lEM6c9dmE4/JxaPiovqc0NHIDZ8ickn+FUh154Tft0 nk5tcXLYQq39t01LZbkZb5Auf11s9LTZkuTC5crTvpzVjYvqc8p+DWVpRjvFdMQ4KFsS GXK7bm2ooYy3seX3/w5EkftKdQ+m4EHKW3qcrab0xWszZSxrfMK0nnIfUJfqdAFELDW4 rd0ZS5BBkYZ8FDubafr6+6seBSpJkHbqTRi5GxTRmMSoQ+k6djfZRjymD4IbskjEIzUy ZDlxJ6m35SHtuOHtpgTKY3Yfk9in1zYE6WNZ/HqAMfLwjqkQwfzQ1Xq95FAwhmU00a6E 2e/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8f5o+M3IMKSkf+u7xy0Pn2f+U92/TorLyNLqHaa75dY=; b=r1M51jbaR0BMxmiodeWa5W6a26pg3BGKKO5t5sX/EEzGexFsdKNKPBiCXQKrKYf5kK eIwCOEndY/bvh2y3xfByDc6QyJsUVpquVK4DmEf7NQfSLdA1I9Fj+7K500DNYuT2Jb2u bzi730UZvaNys2/FM9ev1pfwFuAgxzFMjY2EPDXH2QS1r/TlbTH4sKvaMZH9ZTDJ6qxw 43rpjdxtD1LGMgFm4dWOnljSKiuIY+GOnQ4unVqVUK4LfYBXfXF0H5ZQEPqP9HrMRI0J PtYZ0IJwRNs2bHZRu1S5dgIwPyqOOXxjiolLz+ddZscZtCqImicmpTTTECyf6efhomPr 3Gig== X-Gm-Message-State: ALQs6tC3Rul8ulp3AqpLVtSyCCDJFCnjnqmR4FmgaKQF4O5vv7yR+V9F Jv9OkCiusVZcxkykl2oGrko= X-Google-Smtp-Source: AB8JxZrblBLPFgsR90FBKc0x78gi1IkkpN4eer3g3OYdTLwbLmevx5NQ5bf4/EnIGtGvcHhISVtJgg== X-Received: by 2002:a65:4601:: with SMTP id v1-v6mr13670772pgq.237.1525530971665; Sat, 05 May 2018 07:36:11 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: quintela@redhat.com, dgilbert@redhat.com, berrange@redhat.com Date: Sat, 5 May 2018 22:35:35 +0800 Message-Id: <1525530936-21835-6-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> References: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::244 Subject: [Qemu-devel] [PATCH v3 5/6] migration: implement bi-directional RDMA QIOChannel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: adido@mellanox.com, galsha@mellanox.com, aviadye@mellanox.com, qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel currently, this patch use RCU to protect = it. Signed-off-by: Lidong Chen --- migration/colo.c | 2 + migration/migration.c | 2 + migration/postcopy-ram.c | 2 + migration/ram.c | 4 + migration/rdma.c | 196 ++++++++++++++++++++++++++++++++++++++++---= ---- migration/savevm.c | 3 + 6 files changed, 183 insertions(+), 26 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index 4381067..88936f5 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -534,6 +534,7 @@ void *colo_process_incoming_thread(void *opaque) uint64_t value; Error *local_err =3D NULL; =20 + rcu_register_thread(); qemu_sem_init(&mis->colo_incoming_sem, 0); =20 migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, @@ -666,5 +667,6 @@ out: } migration_incoming_exit_colo(); =20 + rcu_unregister_thread(); return NULL; } diff --git a/migration/migration.c b/migration/migration.c index 0bdb28e..584666b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1787,6 +1787,7 @@ static void *source_return_path_thread(void *opaque) int res; =20 trace_source_return_path_thread_entry(); + rcu_register_thread(); while (!ms->rp_state.error && !qemu_file_get_error(rp) && migration_is_setup_or_active(ms->state)) { trace_source_return_path_thread_loop_top(); @@ -1887,6 +1888,7 @@ static void *source_return_path_thread(void *opaque) out: ms->rp_state.from_dst_file =3D NULL; qemu_fclose(rp); + rcu_unregister_thread(); return NULL; } =20 diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 8ceeaa2..4e05966 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -842,6 +842,7 @@ static void *postcopy_ram_fault_thread(void *opaque) RAMBlock *rb =3D NULL; =20 trace_postcopy_ram_fault_thread_entry(); + rcu_register_thread(); mis->last_rb =3D NULL; /* last RAMBlock we sent part of */ qemu_sem_post(&mis->fault_thread_sem); =20 @@ -1013,6 +1014,7 @@ static void *postcopy_ram_fault_thread(void *opaque) } } } + rcu_unregister_thread(); trace_postcopy_ram_fault_thread_exit(); g_free(pfd); return NULL; diff --git a/migration/ram.c b/migration/ram.c index 912810c..9bc92fc 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -491,6 +491,7 @@ static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; =20 + rcu_register_thread(); while (true) { qemu_mutex_lock(&p->mutex); if (p->quit) { @@ -500,6 +501,7 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_unlock(&p->mutex); qemu_sem_wait(&p->sem); } + rcu_unregister_thread(); =20 return NULL; } @@ -592,6 +594,7 @@ static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; =20 + rcu_register_thread(); while (true) { qemu_mutex_lock(&p->mutex); if (p->quit) { @@ -601,6 +604,7 @@ static void *multifd_recv_thread(void *opaque) qemu_mutex_unlock(&p->mutex); qemu_sem_wait(&p->sem); } + rcu_unregister_thread(); =20 return NULL; } diff --git a/migration/rdma.c b/migration/rdma.c index f5c1d02..854f355 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -86,6 +86,7 @@ static uint32_t known_capabilities =3D RDMA_CAPABILITY_PI= N_ALL; " to abort!"); \ rdma->error_reported =3D 1; \ } \ + rcu_read_unlock(); \ return rdma->error_state; \ } \ } while (0) @@ -402,7 +403,8 @@ typedef struct QIOChannelRDMA QIOChannelRDMA; =20 struct QIOChannelRDMA { QIOChannel parent; - RDMAContext *rdma; + RDMAContext *rdmain; + RDMAContext *rdmaout; QEMUFile *file; bool blocking; /* XXX we don't actually honour this yet */ }; @@ -2635,12 +2637,20 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *= ioc, { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); QEMUFile *f =3D rioc->file; - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; int ret; ssize_t done =3D 0; size_t i; size_t len =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 /* @@ -2650,6 +2660,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, ret =3D qemu_rdma_write_flush(f, rdma); if (ret < 0) { rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -2669,6 +2680,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, =20 if (ret < 0) { rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -2677,6 +2689,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, } } =20 + rcu_read_unlock(); return done; } =20 @@ -2710,12 +2723,20 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *i= oc, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; RDMAControlHeader head; int ret =3D 0; ssize_t i; size_t done =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmain); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 for (i =3D 0; i < niov; i++) { @@ -2727,7 +2748,7 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc, * were given and dish out the bytes until we run * out of bytes. */ - ret =3D qemu_rdma_fill(rioc->rdma, data, want, 0); + ret =3D qemu_rdma_fill(rdma, data, want, 0); done +=3D ret; want -=3D ret; /* Got what we needed, so go to next iovec */ @@ -2749,25 +2770,28 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *i= oc, =20 if (ret < 0) { rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 /* * SEND was received with new bytes, now try again. */ - ret =3D qemu_rdma_fill(rioc->rdma, data, want, 0); + ret =3D qemu_rdma_fill(rdma, data, want, 0); done +=3D ret; want -=3D ret; =20 /* Still didn't get enough, so lets just return */ if (want) { if (done =3D=3D 0) { + rcu_read_unlock(); return QIO_CHANNEL_ERR_BLOCK; } else { break; } } } + rcu_read_unlock(); return done; } =20 @@ -2819,15 +2843,29 @@ qio_channel_rdma_source_prepare(GSource *source, gint *timeout) { QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma =3D rsource->rioc->rdma; + RDMAContext *rdma; GIOCondition cond =3D 0; *timeout =3D -1; =20 + rcu_read_lock(); + if (rsource->condition =3D=3D G_IO_IN) { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmain); + } else { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmaout); + } + + if (!rdma) { + error_report("RDMAContext is NULL when prepare Gsource"); + rcu_read_unlock(); + return FALSE; + } + if (rdma->wr_data[0].control_len) { cond |=3D G_IO_IN; } cond |=3D G_IO_OUT; =20 + rcu_read_unlock(); return cond & rsource->condition; } =20 @@ -2835,14 +2873,28 @@ static gboolean qio_channel_rdma_source_check(GSource *source) { QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma =3D rsource->rioc->rdma; + RDMAContext *rdma; GIOCondition cond =3D 0; =20 + rcu_read_lock(); + if (rsource->condition =3D=3D G_IO_IN) { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmain); + } else { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmaout); + } + + if (!rdma) { + error_report("RDMAContext is NULL when check Gsource"); + rcu_read_unlock(); + return FALSE; + } + if (rdma->wr_data[0].control_len) { cond |=3D G_IO_IN; } cond |=3D G_IO_OUT; =20 + rcu_read_unlock(); return cond & rsource->condition; } =20 @@ -2853,14 +2905,28 @@ qio_channel_rdma_source_dispatch(GSource *source, { QIOChannelFunc func =3D (QIOChannelFunc)callback; QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma =3D rsource->rioc->rdma; + RDMAContext *rdma; GIOCondition cond =3D 0; =20 + rcu_read_lock(); + if (rsource->condition =3D=3D G_IO_IN) { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmain); + } else { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmaout); + } + + if (!rdma) { + error_report("RDMAContext is NULL when dispatch Gsource"); + rcu_read_unlock(); + return FALSE; + } + if (rdma->wr_data[0].control_len) { cond |=3D G_IO_IN; } cond |=3D G_IO_OUT; =20 + rcu_read_unlock(); return (*func)(QIO_CHANNEL(rsource->rioc), (cond & rsource->condition), user_data); @@ -2905,15 +2971,32 @@ static int qio_channel_rdma_close(QIOChannel *ioc, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); + RDMAContext *rdmain, *rdmaout; trace_qemu_rdma_close(); - if (rioc->rdma) { - if (!rioc->rdma->error_state) { - rioc->rdma->error_state =3D qemu_file_get_error(rioc->file); - } - qemu_rdma_cleanup(rioc->rdma); - g_free(rioc->rdma); - rioc->rdma =3D NULL; + + rdmain =3D rioc->rdmain; + if (rdmain) { + atomic_rcu_set(&rioc->rdmain, NULL); + } + + rdmaout =3D rioc->rdmaout; + if (rdmaout) { + atomic_rcu_set(&rioc->rdmaout, NULL); } + + synchronize_rcu(); + + if (rdmain) { + qemu_rdma_cleanup(rdmain); + } + + if (rdmaout) { + qemu_rdma_cleanup(rdmaout); + } + + g_free(rdmain); + g_free(rdmaout); + return 0; } =20 @@ -2956,12 +3039,21 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void= *opaque, size_t size, uint64_t *bytes_sent) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; int ret; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + rcu_read_unlock(); return RAM_SAVE_CONTROL_NOT_SUPP; } =20 @@ -3046,9 +3138,11 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void = *opaque, } } =20 + rcu_read_unlock(); return RAM_SAVE_CONTROL_DELAYED; err: rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -3224,8 +3318,8 @@ static int qemu_rdma_registration_handle(QEMUFile *f,= void *opaque) RDMAControlHeader blocks =3D { .type =3D RDMA_CONTROL_RAM_BLOCKS_RESUL= T, .repeat =3D 1 }; QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; - RDMALocalBlocks *local =3D &rdma->local_ram_blocks; + RDMAContext *rdma; + RDMALocalBlocks *local; RDMAControlHeader head; RDMARegister *reg, *registers; RDMACompress *comp; @@ -3238,8 +3332,17 @@ static int qemu_rdma_registration_handle(QEMUFile *f= , void *opaque) int count =3D 0; int i =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmain); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 + local =3D &rdma->local_ram_blocks; do { trace_qemu_rdma_registration_handle_wait(); =20 @@ -3469,6 +3572,7 @@ out: if (ret < 0) { rdma->error_state =3D ret; } + rcu_read_unlock(); return ret; } =20 @@ -3482,10 +3586,18 @@ out: static int rdma_block_notification_handle(QIOChannelRDMA *rioc, const char *name) { - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; int curr; int found =3D -1; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmain); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + /* Find the matching RAMBlock in our local list */ for (curr =3D 0; curr < rdma->local_ram_blocks.nb_blocks; curr++) { if (!strcmp(rdma->local_ram_blocks.block[curr].block_name, name)) { @@ -3496,6 +3608,7 @@ rdma_block_notification_handle(QIOChannelRDMA *rioc, = const char *name) =20 if (found =3D=3D -1) { error_report("RAMBlock '%s' not found on destination", name); + rcu_read_unlock(); return -ENOENT; } =20 @@ -3503,6 +3616,7 @@ rdma_block_notification_handle(QIOChannelRDMA *rioc, = const char *name) trace_rdma_block_notification_handle(name, rdma->next_src_index); rdma->next_src_index++; =20 + rcu_read_unlock(); return 0; } =20 @@ -3525,11 +3639,19 @@ static int qemu_rdma_registration_start(QEMUFile *f= , void *opaque, uint64_t flags, void *data) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; + + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } =20 CHECK_ERROR_STATE(); =20 if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + rcu_read_unlock(); return 0; } =20 @@ -3537,6 +3659,7 @@ static int qemu_rdma_registration_start(QEMUFile *f, = void *opaque, qemu_put_be64(f, RAM_SAVE_FLAG_HOOK); qemu_fflush(f); =20 + rcu_read_unlock(); return 0; } =20 @@ -3549,13 +3672,21 @@ static int qemu_rdma_registration_stop(QEMUFile *f,= void *opaque, { Error *local_err =3D NULL, **errp =3D &local_err; QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; RDMAControlHeader head =3D { .len =3D 0, .repeat =3D 1 }; int ret =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + rcu_read_unlock(); return 0; } =20 @@ -3587,6 +3718,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, v= oid *opaque, qemu_rdma_reg_whole_ram_blocks : NULL); if (ret < 0) { ERROR(errp, "receiving remote info!"); + rcu_read_unlock(); return ret; } =20 @@ -3610,6 +3742,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, v= oid *opaque, "not identical on both the source and destination.= ", local->nb_blocks, nb_dest_blocks); rdma->error_state =3D -EINVAL; + rcu_read_unlock(); return -EINVAL; } =20 @@ -3626,6 +3759,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, v= oid *opaque, local->block[i].length, rdma->dest_blocks[i].length); rdma->error_state =3D -EINVAL; + rcu_read_unlock(); return -EINVAL; } local->block[i].remote_host_addr =3D @@ -3643,9 +3777,11 @@ static int qemu_rdma_registration_stop(QEMUFile *f, = void *opaque, goto err; } =20 + rcu_read_unlock(); return 0; err: rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -3663,10 +3799,15 @@ static const QEMUFileHooks rdma_write_hooks =3D { static void qio_channel_rdma_finalize(Object *obj) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(obj); - if (rioc->rdma) { - qemu_rdma_cleanup(rioc->rdma); - g_free(rioc->rdma); - rioc->rdma =3D NULL; + if (rioc->rdmain) { + qemu_rdma_cleanup(rioc->rdmain); + g_free(rioc->rdmain); + rioc->rdmain =3D NULL; + } + if (rioc->rdmaout) { + qemu_rdma_cleanup(rioc->rdmaout); + g_free(rioc->rdmaout); + rioc->rdmaout =3D NULL; } } =20 @@ -3706,13 +3847,16 @@ static QEMUFile *qemu_fopen_rdma(RDMAContext *rdma,= const char *mode) } =20 rioc =3D QIO_CHANNEL_RDMA(object_new(TYPE_QIO_CHANNEL_RDMA)); - rioc->rdma =3D rdma; =20 if (mode[0] =3D=3D 'w') { rioc->file =3D qemu_fopen_channel_output(QIO_CHANNEL(rioc)); + rioc->rdmaout =3D rdma; + rioc->rdmain =3D rdma->return_path; qemu_file_set_hooks(rioc->file, &rdma_write_hooks); } else { rioc->file =3D qemu_fopen_channel_input(QIO_CHANNEL(rioc)); + rioc->rdmain =3D rdma; + rioc->rdmaout =3D rdma->return_path; qemu_file_set_hooks(rioc->file, &rdma_read_hooks); } =20 diff --git a/migration/savevm.c b/migration/savevm.c index e2be02a..45ec809 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1573,6 +1573,7 @@ static void *postcopy_ram_listen_thread(void *opaque) qemu_sem_post(&mis->listen_thread_sem); trace_postcopy_ram_listen_thread_start(); =20 + rcu_register_thread(); /* * Because we're a thread and not a coroutine we can't yield * in qemu_file, and thus we must be blocking now. @@ -1605,6 +1606,7 @@ static void *postcopy_ram_listen_thread(void *opaque) * to leave the guest running and fire MCEs for pages that never * arrived as a desperate recovery step. */ + rcu_unregister_thread(); exit(EXIT_FAILURE); } =20 @@ -1619,6 +1621,7 @@ static void *postcopy_ram_listen_thread(void *opaque) migration_incoming_state_destroy(); qemu_loadvm_state_cleanup(); =20 + rcu_unregister_thread(); return NULL; } =20 --=20 1.8.3.1 From nobody Wed Oct 29 17:10:19 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1525531366749134.01775501835402; Sat, 5 May 2018 07:42:46 -0700 (PDT) Received: from localhost ([::1]:39061 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyOb-0004Rq-R2 for importer@patchew.org; Sat, 05 May 2018 10:42:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58445) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fEyIJ-0008T9-JG for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fEyII-0008GM-Q0 for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:15 -0400 Received: from mail-pg0-x244.google.com ([2607:f8b0:400e:c05::244]:44958) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fEyII-0008G5-Jd for qemu-devel@nongnu.org; Sat, 05 May 2018 10:36:14 -0400 Received: by mail-pg0-x244.google.com with SMTP id 82-v6so17312447pge.11 for ; Sat, 05 May 2018 07:36:14 -0700 (PDT) Received: from VM_127_91_centos.localdomain ([119.28.195.137]) by smtp.gmail.com with ESMTPSA id n67sm38529640pfh.188.2018.05.05.07.36.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 May 2018 07:36:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PyWc5EFvCalprNQtF0ueBfM34TbNe8VfPGXTG01pGZ0=; b=OMiON8aB9ufQ3IQipquhy3Qg4qQ7n0tWu7Eq1zAyXiP4nwKDiqDCupz/tYrWwTSjbB n/aEADWKiEHUFTyg2sQH7tBfUCobuRGkeTiQ7BBzRDqNHMkkjMTVv4OLIihKPVPiFgrk g/IeFdkkl1UuXJSASxwkqFUr1fDmA46DTtgMFW8AUuahpdEVvl7/9QQm0jnRbuv00KDL LdoxPVaQqGcJ8SRqmiB+iWRAmP0+F/KqSnfzEohX+782O2rjBlzj+7S8xdSAyvlVPhAJ hPKfzHIT3CMsGD4sdDTBG1kBYuAVTHujTv35y25FI2fLKdKVib+iRqiht+17yUagATal 6zkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PyWc5EFvCalprNQtF0ueBfM34TbNe8VfPGXTG01pGZ0=; b=Qq7LaaTociqTlqcM6ki8CjVYLWFoQhG6lKsaAMYt0a+dyfDbDjIU0q0xSNXV8u0tW8 oRWAC+KwjWK2vK023sHVBaK4v3DZz9XLndfwqX9gTPgd25VNFjDtTdTWwTfmmiygz+mL lV3U45+iAIlWUPqXoFK07XIlADKvUptHH4ZyMyeGVYa0syFf01dAN6qEvtUP/O6xHiBy TQMcNRlgo1/jJBLuGTTopqvM4I17mH55xpLDhURSyeNtJkJIQZeHB/mgq5BzXijTuMVj UtfwqHxG5CJ6q+9pOi64lgXizIt6zzIlvH6tSk6XI3KsGdOYIqNz7p49LCeaSTwskbQF v3ww== X-Gm-Message-State: ALQs6tBupJA8s+bi1Q8Nuq4sp1XGMS5ulIogFn/GnLA4ZECApsQ8JJWj 8i97BwGgG+srF8/N6GI+3jQZIQ== X-Google-Smtp-Source: AB8JxZrouq7Mgrhts3K2gbYK+QoBoU2SJgZvKiapo6NQTZ32c64m1Fgk+VNnMfzEmNoQmRo0NN79kw== X-Received: by 2002:a17:902:bb06:: with SMTP id l6-v6mr32016806pls.255.1525530973786; Sat, 05 May 2018 07:36:13 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: quintela@redhat.com, dgilbert@redhat.com, berrange@redhat.com Date: Sat, 5 May 2018 22:35:36 +0800 Message-Id: <1525530936-21835-7-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> References: <1525530936-21835-1-git-send-email-lidongchen@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::244 Subject: [Qemu-devel] [PATCH v3 6/6] migration: Stop rdma yielding during incoming postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: adido@mellanox.com, galsha@mellanox.com, aviadye@mellanox.com, qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" During incoming postcopy, the destination qemu will invoke qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma yield, and poll the completion channel fd instead. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index 854f355..ed9cfb1 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1490,11 +1490,13 @@ static int qemu_rdma_wait_comp_channel(RDMAContext = *rdma) * Coroutine doesn't start until migration_fd_process_incoming() * so don't yield unless we know we're running inside of a coroutine. */ - if (rdma->migration_started_on_destination) { + if (rdma->migration_started_on_destination && + migration_incoming_get_current()->state =3D=3D MIGRATION_STATUS_AC= TIVE) { yield_until_fd_readable(rdma->comp_channel->fd); } else { /* This is the source side, we're in a separate thread * or destination prior to migration_fd_process_incoming() + * after postcopy, the destination also in a seprate thread. * we can't yield; so we have to poll the fd. * But we need to be able to handle 'cancel' or an error * without hanging forever. --=20 1.8.3.1