From nobody Wed Nov 5 07:52:36 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=gmail.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1533287842575767.160645286179; Fri, 3 Aug 2018 02:17:22 -0700 (PDT) Received: from localhost ([::1]:49750 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1flWD3-0003yq-7W for importer@patchew.org; Fri, 03 Aug 2018 05:17:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52795) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1flW9x-0001RR-AT for qemu-devel@nongnu.org; Fri, 03 Aug 2018 05:14:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1flW9u-0001Nv-Kn for qemu-devel@nongnu.org; Fri, 03 Aug 2018 05:14:09 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]:36318) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1flW9u-0001Mt-Ai for qemu-devel@nongnu.org; Fri, 03 Aug 2018 05:14:06 -0400 Received: by mail-pf1-x432.google.com with SMTP id b11-v6so2930815pfo.3 for ; Fri, 03 Aug 2018 02:14:06 -0700 (PDT) Received: from VM_120_46_centos.localdomain ([119.28.87.64]) by smtp.gmail.com with ESMTPSA id e21-v6sm7991352pfl.187.2018.08.03.02.14.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Aug 2018 02:14:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ve+ewzePoLVm5jRNrYZ1DCfvOb5UloP+RAKRNeBmMoQ=; b=Hbnedu8JVlx8w5YEJIwEZwpRms5masX2kgYeGb6sP10wcfby+fP3Xcq55ohcoj8SZ0 KxN6X+pb/ypXo8RyKdkNCtCycGoqztl6uGOAeSxy4NBpcYIIIbCEM1PzR5mGBd7VNbf9 wEgP+hGuQomzMJG+YlvcI8APvA9wn4XATezy6VYIkDnq+gFLaT8i4nCrKyArm7wCWZ26 euNIcw/kvP0X5TEsGQccr1CCTDpDik1PLv9CKScR6L+1VGU4EMwIP9UUU/15AQd+zJDH ysGj2s3xv8AcwR9DnNO6twnSdwLMtKDH9mna2DznyYvjugA1IJFmfHatmEg2hCZixs7J PoIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ve+ewzePoLVm5jRNrYZ1DCfvOb5UloP+RAKRNeBmMoQ=; b=mRGLaX16o4f9C22gMNmm4oqsUTHjusF0sZC7tiDXKlN8HPnQ6LtGR+5Pah6iuVYc/l PNYBP81UAvNwlrumrIZTmv/8f4NKVHLC6OCPWVYK6+FPzeiu0AaR9DZ/vHPzAEU2OJHT k+QH/lee5+g1oLzPv7kG9GKlwI2qYOhrL/j5ct5BDjC6vHCzfKb2yAIFByB5VKQh1Ch3 WKHB7CiGg9/W/eFhHuoqXGpskJEZZNJe3qoqDVaN1Kwr0nGjTHQbfmDA4cLNCGcMYorv jDYJBpc5+MRZ7qUvwsX26BwIDGvPErZs9XoA/GSyapFCMoi/yJcFk0WxV1zqtU0KdiMr efjA== X-Gm-Message-State: AOUpUlHrH3tCIaFfuQsF2XjE5q5sH/y6Bi4HuzX0299QJcfL7nQzHZrH qM2r2rnCL69p+74DF38aN/4= X-Google-Smtp-Source: AAOMgpemJmnUtJGg43Ymv8gG25xK9n6vQ66w9quE1GwOCCkt5957gE4O2qeC0Qlp1d9/zYZfXXOaKg== X-Received: by 2002:a63:9f0a:: with SMTP id g10-v6mr2903640pge.324.1533287645328; Fri, 03 Aug 2018 02:14:05 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: zhang.zhanghailiang@huawei.com, quintela@redhat.com, dgilbert@redhat.com Date: Fri, 3 Aug 2018 17:13:42 +0800 Message-Id: <1533287630-4221-5-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1533287630-4221-1-git-send-email-lidongchen@tencent.com> References: <1533287630-4221-1-git-send-email-lidongchen@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::432 Subject: [Qemu-devel] [PATCH v6 04/12] migration: implement bi-directional RDMA QIOChannel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lidong Chen , qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDMRC_1 RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Lidong Chen This patch implements bi-directional RDMA QIOChannel. Because different threads may access RDMAQIOChannel currently, this patch use RCU to protect = it. Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 2 + migration/migration.c | 2 + migration/postcopy-ram.c | 2 + migration/ram.c | 4 + migration/rdma.c | 196 ++++++++++++++++++++++++++++++++++++++++---= ---- migration/savevm.c | 3 + 6 files changed, 183 insertions(+), 26 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index 4381067..88936f5 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -534,6 +534,7 @@ void *colo_process_incoming_thread(void *opaque) uint64_t value; Error *local_err =3D NULL; =20 + rcu_register_thread(); qemu_sem_init(&mis->colo_incoming_sem, 0); =20 migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, @@ -666,5 +667,6 @@ out: } migration_incoming_exit_colo(); =20 + rcu_unregister_thread(); return NULL; } diff --git a/migration/migration.c b/migration/migration.c index a3a0756..f190964 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2104,6 +2104,7 @@ static void *source_return_path_thread(void *opaque) int res; =20 trace_source_return_path_thread_entry(); + rcu_register_thread(); =20 retry: while (!ms->rp_state.error && !qemu_file_get_error(rp) && @@ -2243,6 +2244,7 @@ out: trace_source_return_path_thread_end(); ms->rp_state.from_dst_file =3D NULL; qemu_fclose(rp); + rcu_unregister_thread(); return NULL; } =20 diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 932f188..3952d78 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -853,6 +853,7 @@ static void *postcopy_ram_fault_thread(void *opaque) RAMBlock *rb =3D NULL; =20 trace_postcopy_ram_fault_thread_entry(); + rcu_register_thread(); mis->last_rb =3D NULL; /* last RAMBlock we sent part of */ qemu_sem_post(&mis->fault_thread_sem); =20 @@ -1059,6 +1060,7 @@ retry: } } } + rcu_unregister_thread(); trace_postcopy_ram_fault_thread_exit(); g_free(pfd); return NULL; diff --git a/migration/ram.c b/migration/ram.c index 24dea27..4da0930 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -988,6 +988,7 @@ static void *multifd_send_thread(void *opaque) int ret; =20 trace_multifd_send_thread_start(p->id); + rcu_register_thread(); =20 if (multifd_send_initial_packet(p, &local_err) < 0) { goto out; @@ -1050,6 +1051,7 @@ out: p->running =3D false; qemu_mutex_unlock(&p->mutex); =20 + rcu_unregister_thread(); trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages); =20 return NULL; @@ -1219,6 +1221,7 @@ static void *multifd_recv_thread(void *opaque) int ret; =20 trace_multifd_recv_thread_start(p->id); + rcu_register_thread(); =20 while (true) { uint32_t used; @@ -1265,6 +1268,7 @@ static void *multifd_recv_thread(void *opaque) p->running =3D false; qemu_mutex_unlock(&p->mutex); =20 + rcu_unregister_thread(); trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages); =20 return NULL; diff --git a/migration/rdma.c b/migration/rdma.c index 57af5ed..a5535fb 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -86,6 +86,7 @@ static uint32_t known_capabilities =3D RDMA_CAPABILITY_PI= N_ALL; " to abort!"); \ rdma->error_reported =3D 1; \ } \ + rcu_read_unlock(); \ return rdma->error_state; \ } \ } while (0) @@ -402,7 +403,8 @@ typedef struct QIOChannelRDMA QIOChannelRDMA; =20 struct QIOChannelRDMA { QIOChannel parent; - RDMAContext *rdma; + RDMAContext *rdmain; + RDMAContext *rdmaout; QEMUFile *file; bool blocking; /* XXX we don't actually honour this yet */ }; @@ -2630,12 +2632,20 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *= ioc, { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); QEMUFile *f =3D rioc->file; - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; int ret; ssize_t done =3D 0; size_t i; size_t len =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 /* @@ -2645,6 +2655,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, ret =3D qemu_rdma_write_flush(f, rdma); if (ret < 0) { rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -2664,6 +2675,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, =20 if (ret < 0) { rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -2672,6 +2684,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, } } =20 + rcu_read_unlock(); return done; } =20 @@ -2705,12 +2718,20 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *i= oc, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; RDMAControlHeader head; int ret =3D 0; ssize_t i; size_t done =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmain); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 for (i =3D 0; i < niov; i++) { @@ -2722,7 +2743,7 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc, * were given and dish out the bytes until we run * out of bytes. */ - ret =3D qemu_rdma_fill(rioc->rdma, data, want, 0); + ret =3D qemu_rdma_fill(rdma, data, want, 0); done +=3D ret; want -=3D ret; /* Got what we needed, so go to next iovec */ @@ -2744,25 +2765,28 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *i= oc, =20 if (ret < 0) { rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 /* * SEND was received with new bytes, now try again. */ - ret =3D qemu_rdma_fill(rioc->rdma, data, want, 0); + ret =3D qemu_rdma_fill(rdma, data, want, 0); done +=3D ret; want -=3D ret; =20 /* Still didn't get enough, so lets just return */ if (want) { if (done =3D=3D 0) { + rcu_read_unlock(); return QIO_CHANNEL_ERR_BLOCK; } else { break; } } } + rcu_read_unlock(); return done; } =20 @@ -2814,15 +2838,29 @@ qio_channel_rdma_source_prepare(GSource *source, gint *timeout) { QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma =3D rsource->rioc->rdma; + RDMAContext *rdma; GIOCondition cond =3D 0; *timeout =3D -1; =20 + rcu_read_lock(); + if (rsource->condition =3D=3D G_IO_IN) { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmain); + } else { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmaout); + } + + if (!rdma) { + error_report("RDMAContext is NULL when prepare Gsource"); + rcu_read_unlock(); + return FALSE; + } + if (rdma->wr_data[0].control_len) { cond |=3D G_IO_IN; } cond |=3D G_IO_OUT; =20 + rcu_read_unlock(); return cond & rsource->condition; } =20 @@ -2830,14 +2868,28 @@ static gboolean qio_channel_rdma_source_check(GSource *source) { QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma =3D rsource->rioc->rdma; + RDMAContext *rdma; GIOCondition cond =3D 0; =20 + rcu_read_lock(); + if (rsource->condition =3D=3D G_IO_IN) { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmain); + } else { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmaout); + } + + if (!rdma) { + error_report("RDMAContext is NULL when check Gsource"); + rcu_read_unlock(); + return FALSE; + } + if (rdma->wr_data[0].control_len) { cond |=3D G_IO_IN; } cond |=3D G_IO_OUT; =20 + rcu_read_unlock(); return cond & rsource->condition; } =20 @@ -2848,14 +2900,28 @@ qio_channel_rdma_source_dispatch(GSource *source, { QIOChannelFunc func =3D (QIOChannelFunc)callback; QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma =3D rsource->rioc->rdma; + RDMAContext *rdma; GIOCondition cond =3D 0; =20 + rcu_read_lock(); + if (rsource->condition =3D=3D G_IO_IN) { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmain); + } else { + rdma =3D atomic_rcu_read(&rsource->rioc->rdmaout); + } + + if (!rdma) { + error_report("RDMAContext is NULL when dispatch Gsource"); + rcu_read_unlock(); + return FALSE; + } + if (rdma->wr_data[0].control_len) { cond |=3D G_IO_IN; } cond |=3D G_IO_OUT; =20 + rcu_read_unlock(); return (*func)(QIO_CHANNEL(rsource->rioc), (cond & rsource->condition), user_data); @@ -2900,15 +2966,32 @@ static int qio_channel_rdma_close(QIOChannel *ioc, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); + RDMAContext *rdmain, *rdmaout; trace_qemu_rdma_close(); - if (rioc->rdma) { - if (!rioc->rdma->error_state) { - rioc->rdma->error_state =3D qemu_file_get_error(rioc->file); - } - qemu_rdma_cleanup(rioc->rdma); - g_free(rioc->rdma); - rioc->rdma =3D NULL; + + rdmain =3D rioc->rdmain; + if (rdmain) { + atomic_rcu_set(&rioc->rdmain, NULL); + } + + rdmaout =3D rioc->rdmaout; + if (rdmaout) { + atomic_rcu_set(&rioc->rdmaout, NULL); } + + synchronize_rcu(); + + if (rdmain) { + qemu_rdma_cleanup(rdmain); + } + + if (rdmaout) { + qemu_rdma_cleanup(rdmaout); + } + + g_free(rdmain); + g_free(rdmaout); + return 0; } =20 @@ -2951,12 +3034,21 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void= *opaque, size_t size, uint64_t *bytes_sent) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; int ret; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + rcu_read_unlock(); return RAM_SAVE_CONTROL_NOT_SUPP; } =20 @@ -3041,9 +3133,11 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void = *opaque, } } =20 + rcu_read_unlock(); return RAM_SAVE_CONTROL_DELAYED; err: rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -3219,8 +3313,8 @@ static int qemu_rdma_registration_handle(QEMUFile *f,= void *opaque) RDMAControlHeader blocks =3D { .type =3D RDMA_CONTROL_RAM_BLOCKS_RESUL= T, .repeat =3D 1 }; QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; - RDMALocalBlocks *local =3D &rdma->local_ram_blocks; + RDMAContext *rdma; + RDMALocalBlocks *local; RDMAControlHeader head; RDMARegister *reg, *registers; RDMACompress *comp; @@ -3233,8 +3327,17 @@ static int qemu_rdma_registration_handle(QEMUFile *f= , void *opaque) int count =3D 0; int i =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmain); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 + local =3D &rdma->local_ram_blocks; do { trace_qemu_rdma_registration_handle_wait(); =20 @@ -3468,6 +3571,7 @@ out: if (ret < 0) { rdma->error_state =3D ret; } + rcu_read_unlock(); return ret; } =20 @@ -3481,10 +3585,18 @@ out: static int rdma_block_notification_handle(QIOChannelRDMA *rioc, const char *name) { - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; int curr; int found =3D -1; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmain); + + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + /* Find the matching RAMBlock in our local list */ for (curr =3D 0; curr < rdma->local_ram_blocks.nb_blocks; curr++) { if (!strcmp(rdma->local_ram_blocks.block[curr].block_name, name)) { @@ -3495,6 +3607,7 @@ rdma_block_notification_handle(QIOChannelRDMA *rioc, = const char *name) =20 if (found =3D=3D -1) { error_report("RAMBlock '%s' not found on destination", name); + rcu_read_unlock(); return -ENOENT; } =20 @@ -3502,6 +3615,7 @@ rdma_block_notification_handle(QIOChannelRDMA *rioc, = const char *name) trace_rdma_block_notification_handle(name, rdma->next_src_index); rdma->next_src_index++; =20 + rcu_read_unlock(); return 0; } =20 @@ -3524,11 +3638,19 @@ static int qemu_rdma_registration_start(QEMUFile *f= , void *opaque, uint64_t flags, void *data) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; + + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } =20 CHECK_ERROR_STATE(); =20 if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + rcu_read_unlock(); return 0; } =20 @@ -3536,6 +3658,7 @@ static int qemu_rdma_registration_start(QEMUFile *f, = void *opaque, qemu_put_be64(f, RAM_SAVE_FLAG_HOOK); qemu_fflush(f); =20 + rcu_read_unlock(); return 0; } =20 @@ -3548,13 +3671,21 @@ static int qemu_rdma_registration_stop(QEMUFile *f,= void *opaque, { Error *local_err =3D NULL, **errp =3D &local_err; QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(opaque); - RDMAContext *rdma =3D rioc->rdma; + RDMAContext *rdma; RDMAControlHeader head =3D { .len =3D 0, .repeat =3D 1 }; int ret =3D 0; =20 + rcu_read_lock(); + rdma =3D atomic_rcu_read(&rioc->rdmaout); + if (!rdma) { + rcu_read_unlock(); + return -EIO; + } + CHECK_ERROR_STATE(); =20 if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTI= VE) { + rcu_read_unlock(); return 0; } =20 @@ -3586,6 +3717,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, v= oid *opaque, qemu_rdma_reg_whole_ram_blocks : NULL); if (ret < 0) { ERROR(errp, "receiving remote info!"); + rcu_read_unlock(); return ret; } =20 @@ -3609,6 +3741,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, v= oid *opaque, "not identical on both the source and destination.= ", local->nb_blocks, nb_dest_blocks); rdma->error_state =3D -EINVAL; + rcu_read_unlock(); return -EINVAL; } =20 @@ -3625,6 +3758,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, v= oid *opaque, local->block[i].length, rdma->dest_blocks[i].length); rdma->error_state =3D -EINVAL; + rcu_read_unlock(); return -EINVAL; } local->block[i].remote_host_addr =3D @@ -3642,9 +3776,11 @@ static int qemu_rdma_registration_stop(QEMUFile *f, = void *opaque, goto err; } =20 + rcu_read_unlock(); return 0; err: rdma->error_state =3D ret; + rcu_read_unlock(); return ret; } =20 @@ -3662,10 +3798,15 @@ static const QEMUFileHooks rdma_write_hooks =3D { static void qio_channel_rdma_finalize(Object *obj) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(obj); - if (rioc->rdma) { - qemu_rdma_cleanup(rioc->rdma); - g_free(rioc->rdma); - rioc->rdma =3D NULL; + if (rioc->rdmain) { + qemu_rdma_cleanup(rioc->rdmain); + g_free(rioc->rdmain); + rioc->rdmain =3D NULL; + } + if (rioc->rdmaout) { + qemu_rdma_cleanup(rioc->rdmaout); + g_free(rioc->rdmaout); + rioc->rdmaout =3D NULL; } } =20 @@ -3705,13 +3846,16 @@ static QEMUFile *qemu_fopen_rdma(RDMAContext *rdma,= const char *mode) } =20 rioc =3D QIO_CHANNEL_RDMA(object_new(TYPE_QIO_CHANNEL_RDMA)); - rioc->rdma =3D rdma; =20 if (mode[0] =3D=3D 'w') { rioc->file =3D qemu_fopen_channel_output(QIO_CHANNEL(rioc)); + rioc->rdmaout =3D rdma; + rioc->rdmain =3D rdma->return_path; qemu_file_set_hooks(rioc->file, &rdma_write_hooks); } else { rioc->file =3D qemu_fopen_channel_input(QIO_CHANNEL(rioc)); + rioc->rdmain =3D rdma; + rioc->rdmaout =3D rdma->return_path; qemu_file_set_hooks(rioc->file, &rdma_read_hooks); } =20 diff --git a/migration/savevm.c b/migration/savevm.c index 7f92567..13e51f0 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1622,6 +1622,7 @@ static void *postcopy_ram_listen_thread(void *opaque) qemu_sem_post(&mis->listen_thread_sem); trace_postcopy_ram_listen_thread_start(); =20 + rcu_register_thread(); /* * Because we're a thread and not a coroutine we can't yield * in qemu_file, and thus we must be blocking now. @@ -1662,6 +1663,7 @@ static void *postcopy_ram_listen_thread(void *opaque) * to leave the guest running and fire MCEs for pages that never * arrived as a desperate recovery step. */ + rcu_unregister_thread(); exit(EXIT_FAILURE); } =20 @@ -1676,6 +1678,7 @@ static void *postcopy_ram_listen_thread(void *opaque) migration_incoming_state_destroy(); qemu_loadvm_state_cleanup(); =20 + rcu_unregister_thread(); return NULL; } =20 --=20 1.8.3.1