From nobody Sat May 18 23:23:28 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1690987144; cv=none; d=zohomail.com; s=zohoarc; b=bfa6qQlbHdcy0k3NRCaxEZsipESbthI8SBY4cX7ZqR8RgmDwxwXR1clItmkcpRTDcHVzADVBMFQY0GyIbCmRTod5mKk1I8q6NcfEW1VhwX2VVTkLPTx2jDeDRN+HNLZf9Jd3zRLHFI4Pwlc93awE5ERf7hvlZyuBIOiDBp3P5QI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690987144; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XMb3qAivY1c37CdOHWbc5oVyEYgvE5PM07Pa/uSzoM4=; b=dUKgPo4RfJs2aOxfSupg5OD4awPX5vcn37lk7HdLogz4wTO6/m4jP+QMym8u0xUvdNyaXs1v3wWo2Bia1OWlXyUQkxAjsEJcph7D/SxWrzThJx8vrKnFclf/tSRzmspsn9UIac9XM7W2jzSAexkJR0JOO8UBgotSzQnBZZI/KRI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690987144523823.514155534729; Wed, 2 Aug 2023 07:39:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qRCyk-0006bR-QB; Wed, 02 Aug 2023 10:37:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRCyC-0005By-E3 for qemu-devel@nongnu.org; Wed, 02 Aug 2023 10:37:00 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qRCy3-00072D-WE for qemu-devel@nongnu.org; Wed, 02 Aug 2023 10:36:58 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7AF8A1F889; Wed, 2 Aug 2023 14:36:50 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 00C0813909; Wed, 2 Aug 2023 14:36:48 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id qB4ALwBqymSmEgAAMHmgww (envelope-from ); Wed, 02 Aug 2023 14:36:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1690987010; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XMb3qAivY1c37CdOHWbc5oVyEYgvE5PM07Pa/uSzoM4=; b=xo/+g996iUrVEqKdZBxUWUOZyeazVp7/NHvWzgkFNNbjWKo93qlthyZL2fbSVhEs0XxcJR 7w5bZ0w2vmTab+5NbrYXfT7CSkJvHEnT8B1SxsPcZvgRIz2rRlbN0IJ8oHZvUl4d8aCtB1 946/Ag1kIFMdPE4/Bo10+j3t6Rpa0Yw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1690987010; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XMb3qAivY1c37CdOHWbc5oVyEYgvE5PM07Pa/uSzoM4=; b=CnL36d7nLXs1yno4bih5n5ldz9Lv4rGbNVRrKzgg275x2+y0yx1FtoWBvZ/6gl1b194koq Kag0JFEYsbhefNAg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Wei Wang , Leonardo Bras Subject: [PATCH v2 1/2] migration: Split await_return_path_close_on_source Date: Wed, 2 Aug 2023 11:36:43 -0300 Message-Id: <20230802143644.7534-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230802143644.7534-1-farosas@suse.de> References: <20230802143644.7534-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1690990003562100001 Content-Type: text/plain; charset="utf-8" This function currently has a straight-forward part which is waiting for the thread to join and a complicated part which is doing a qemu_file_shutdown() on the return path file. The shutdown is tricky because all calls to qemu_file_shutdown() set f->last_error to -EIO, which means we can never know if an error is an actual error or if we cleanly shutdown the file previously. This is particularly bothersome for postcopy because it would send the return path thread into the retry routine which would wait on the postcopy_pause_rp_sem and consequently block the main thread. We haven't had reports of this so I must presume we never reach here with postcopy. The shutdown call is also racy because since it doesn't take the qemu_file_lock, it could NULL-dereference if the return path thread happens to be in the middle of the critical region at migration_release_dst_files(). Move this more complicated part of the code to a separate routine so we can wait on the thread without all of this baggage. Signed-off-by: Fabiano Rosas --- migration/migration.c | 46 +++++++++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 19 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 91bba630a8..58f09275a8 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2038,6 +2038,25 @@ static int open_return_path_on_source(MigrationState= *ms, /* Returns 0 if the RP was ok, otherwise there was an error on the RP */ static int await_return_path_close_on_source(MigrationState *ms) { + if (!ms->rp_state.rp_thread_created) { + return 0; + } + + trace_await_return_path_close_on_source_joining(); + qemu_thread_join(&ms->rp_state.rp_thread); + ms->rp_state.rp_thread_created =3D false; + trace_await_return_path_close_on_source_close(); + return ms->rp_state.error; +} + +static int close_return_path_on_source(MigrationState *ms) +{ + int ret; + + if (!ms->rp_state.rp_thread_created) { + return 0; + } + /* * If this is a normal exit then the destination will send a SHUT and = the * rp_thread will exit, however if there's an error we need to cause @@ -2051,11 +2070,12 @@ static int await_return_path_close_on_source(Migrat= ionState *ms) qemu_file_shutdown(ms->rp_state.from_dst_file); mark_source_rp_bad(ms); } - trace_await_return_path_close_on_source_joining(); - qemu_thread_join(&ms->rp_state.rp_thread); - ms->rp_state.rp_thread_created =3D false; - trace_await_return_path_close_on_source_close(); - return ms->rp_state.error; + + trace_migration_return_path_end_before(); + ret =3D await_return_path_close_on_source(ms); + trace_migration_return_path_end_after(ret); + + return ret; } =20 static inline void @@ -2351,20 +2371,8 @@ static void migration_completion(MigrationState *s) goto fail; } =20 - /* - * If rp was opened we must clean up the thread before - * cleaning everything else up (since if there are no failures - * it will wait for the destination to send it's status in - * a SHUT command). - */ - if (s->rp_state.rp_thread_created) { - int rp_error; - trace_migration_return_path_end_before(); - rp_error =3D await_return_path_close_on_source(s); - trace_migration_return_path_end_after(rp_error); - if (rp_error) { - goto fail; - } + if (close_return_path_on_source(s)) { + goto fail; } =20 if (qemu_file_get_error(s->to_dst_file)) { --=20 2.35.3 From nobody Sat May 18 23:23:28 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1690987164; cv=none; d=zohomail.com; s=zohoarc; b=TYash6Mn6cQukfZL8iUxjl2tjKh5dSW5Zb9R1DzGYpUtGEMA2o2MS+GrO2AzqzjJcu+mRQQmRcF7uMWZIhr/ehu+TorQJSrLbpdeNLT/ivvYGdePQ4IAXql/bNMlBqs2WjEsLYXU2vf3yQErMql/S7MpQugzt13bjELcQ0ZZpj4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690987164; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=X/MsHE4VGoqUse5DmXgBt6l+2Kk57ZHXdT0jeHtAPd8=; b=O9gxI4L5MUOugDXwYa3gVqW4pd55o93CFP5IJbG9fvq1KCDSgleUs4nrFqt/TCkXwouhmPJYFCsGgtFJHo2skTB/hVxJ8uo2f/6rJH8jE9YRxdivp5VC3/W+j9Xk05qyO/lYZmc4R/qggq8G9D9g0eFCdgfO8RuVuEizDMBMe6c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690987164841104.02520515438573; Wed, 2 Aug 2023 07:39:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qRCzh-0000al-Rx; Wed, 02 Aug 2023 10:38:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRCyD-0005KG-LA for qemu-devel@nongnu.org; Wed, 02 Aug 2023 10:37:01 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qRCy5-00073A-Vo for qemu-devel@nongnu.org; Wed, 02 Aug 2023 10:37:01 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7109521A2A; Wed, 2 Aug 2023 14:36:52 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E77F713909; Wed, 2 Aug 2023 14:36:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id ML3mKwJqymSmEgAAMHmgww (envelope-from ); Wed, 02 Aug 2023 14:36:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1690987012; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X/MsHE4VGoqUse5DmXgBt6l+2Kk57ZHXdT0jeHtAPd8=; b=zk7kAGE72/kmiFB5bBLZs+9nFl2zZyuVCd+3znQ8yPW/zU/lXIaQgxesSxGmXhg/nXNK8U KKgrA+O4NApMV864aKA8h6B6ydZX2sZbKVQRMH/C4pHot4vQ+72kNwVRzycAa7US6hi/BR 0Qbgyv4kupGl3c6QJQY0dLPuFGpJsEY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1690987012; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X/MsHE4VGoqUse5DmXgBt6l+2Kk57ZHXdT0jeHtAPd8=; b=wMWJLHo1IE6h21C5B00tez/TtmDmmHT5WfKVcliPlgvn7pYtGSadTTTosyxlxFcqqQnkuk EXRWySYjYdOisPCA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Wei Wang , Leonardo Bras Subject: [PATCH v2 2/2] migration: Replace the return path retry logic Date: Wed, 2 Aug 2023 11:36:44 -0300 Message-Id: <20230802143644.7534-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230802143644.7534-1-farosas@suse.de> References: <20230802143644.7534-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1690990139023100001 Content-Type: text/plain; charset="utf-8" Replace the return path retry logic with finishing and restarting the thread. This fixes a race when resuming the migration that leads to a segfault. Currently when doing postcopy we consider that an IO error on the return path file could be due to a network intermittency. We then keep the thread alive but have it do cleanup of the 'from_dst_file' and wait on the 'postcopy_pause_rp' semaphore. When the user issues a migrate resume, a new return path is opened and the thread is allowed to continue. There's a race condition in the above mechanism. It is possible for the new return path file to be setup *before* the cleanup code in the return path thread has had a chance to run, leading to the *new* file being closed and the pointer set to NULL. When the thread is released after the resume, it tries to dereference 'from_dst_file' and crashes: Thread 7 "return path" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffd1dbf700 (LWP 9611)] 0x00005555560e4893 in qemu_file_get_error_obj (f=3D0x0, errp=3D0x0) at ../m= igration/qemu-file.c:154 154 return f->last_error; (gdb) bt #0 0x00005555560e4893 in qemu_file_get_error_obj (f=3D0x0, errp=3D0x0) at= ../migration/qemu-file.c:154 #1 0x00005555560e4983 in qemu_file_get_error (f=3D0x0) at ../migration/qe= mu-file.c:206 #2 0x0000555555b9a1df in source_return_path_thread (opaque=3D0x555556e060= 00) at ../migration/migration.c:1876 #3 0x000055555602e14f in qemu_thread_start (args=3D0x55555782e780) at ../= util/qemu-thread-posix.c:541 #4 0x00007ffff38d76ea in start_thread (arg=3D0x7fffd1dbf700) at pthread_c= reate.c:477 #5 0x00007ffff35efa6f in clone () at ../sysdeps/unix/sysv/linux/x86_64/cl= one.S:95 Here's the race (important bit is open_return_path happening before migration_release_dst_files): migration | qmp | return path --------------------------+-----------------------------+------------------= --------------- qmp_migrate_pause() shutdown(ms->to_dst_file) f->last_error =3D -EIO migrate_detect_error() postcopy_pause() set_state(PAUSED) wait(postcopy_pause_sem) qmp_migrate(resume) migrate_fd_connect() resume =3D state =3D=3D PAUSED open_return_path <-- TOO SOON! set_state(RECOVER) post(postcopy_pause_sem) (incoming closes to_src_file) res =3D qemu_file_get_error(rp) migration_release_dst_files() ms->rp_state.from_dst_file =3D NULL post(postcopy_pause_rp_sem) postcopy_pause_return_path_thread() wait(postcopy_pause_rp_sem) rp =3D ms->rp_state.from_dst_file goto retry qemu_file_get_error(rp) SIGSEGV ---------------------------------------------------------------------------= ---------------- We can keep the retry logic without having the thread alive and waiting. The only piece of data used by it is the 'from_dst_file' and it is only allowed to proceed after a migrate resume is issued and the semaphore released at migrate_fd_connect(). Move the retry logic to outside the thread by having open_return_path_on_source() wait for the thread to finish before creating a new one with the updated 'from_dst_file'. Signed-off-by: Fabiano Rosas --- migration/migration.c | 64 +++++++++++------------------------------- migration/migration.h | 1 - migration/trace-events | 1 + 3 files changed, 17 insertions(+), 49 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 58f09275a8..1356269122 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1764,18 +1764,6 @@ static void migrate_handle_rp_req_pages(MigrationSta= te *ms, const char* rbname, } } =20 -/* Return true to retry, false to quit */ -static bool postcopy_pause_return_path_thread(MigrationState *s) -{ - trace_postcopy_pause_return_path(); - - qemu_sem_wait(&s->postcopy_pause_rp_sem); - - trace_postcopy_pause_return_path_continued(); - - return true; -} - static int migrate_handle_rp_recv_bitmap(MigrationState *s, char *block_na= me) { RAMBlock *block =3D qemu_ram_block_by_name(block_name); @@ -1859,7 +1847,6 @@ static void *source_return_path_thread(void *opaque) trace_source_return_path_thread_entry(); rcu_register_thread(); =20 -retry: while (!ms->rp_state.error && !qemu_file_get_error(rp) && migration_is_setup_or_active(ms->state)) { trace_source_return_path_thread_loop_top(); @@ -1981,28 +1968,18 @@ retry: } =20 out: - res =3D qemu_file_get_error(rp); - if (res) { - if (res && migration_in_postcopy()) { + if (qemu_file_get_error(rp)) { + if (migration_in_postcopy()) { /* - * Maybe there is something we can do: it looks like a - * network down issue, and we pause for a recovery. + * This could be a network issue that would have been + * detected by the main migration thread and caused the + * migration to pause. Do cleanup and finish. */ - migration_release_dst_files(ms); - rp =3D NULL; - if (postcopy_pause_return_path_thread(ms)) { - /* - * Reload rp, reset the rest. Referencing it is safe since - * it's reset only by us above, or when migration completes - */ - rp =3D ms->rp_state.from_dst_file; - ms->rp_state.error =3D false; - goto retry; - } + ms->rp_state.error =3D false; + } else { + trace_source_return_path_thread_bad_end(); + mark_source_rp_bad(ms); } - - trace_source_return_path_thread_bad_end(); - mark_source_rp_bad(ms); } =20 trace_source_return_path_thread_end(); @@ -2011,8 +1988,7 @@ out: return NULL; } =20 -static int open_return_path_on_source(MigrationState *ms, - bool create_thread) +static int open_return_path_on_source(MigrationState *ms) { ms->rp_state.from_dst_file =3D qemu_file_get_return_path(ms->to_dst_fi= le); if (!ms->rp_state.from_dst_file) { @@ -2021,11 +1997,6 @@ static int open_return_path_on_source(MigrationState= *ms, =20 trace_open_return_path_on_source(); =20 - if (!create_thread) { - /* We're done */ - return 0; - } - qemu_thread_create(&ms->rp_state.rp_thread, "return path", source_return_path_thread, ms, QEMU_THREAD_JOINABLE= ); ms->rp_state.rp_thread_created =3D true; @@ -2549,6 +2520,11 @@ static MigThrError postcopy_pause(MigrationState *s) qemu_file_shutdown(file); qemu_fclose(file); =20 + if (await_return_path_close_on_source(s)) { + trace_migration_return_path_pause_err(); + return MIG_THR_ERR_FATAL; + } + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_POSTCOPY_PAUSED); =20 @@ -2566,12 +2542,6 @@ static MigThrError postcopy_pause(MigrationState *s) if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { /* Woken up by a recover procedure. Give it a shot */ =20 - /* - * Firstly, let's wake up the return path now, with a new - * return path channel. - */ - qemu_sem_post(&s->postcopy_pause_rp_sem); - /* Do the resume logic */ if (postcopy_do_resume(s) =3D=3D 0) { /* Let's continue! */ @@ -3259,7 +3229,7 @@ void migrate_fd_connect(MigrationState *s, Error *err= or_in) * QEMU uses the return path. */ if (migrate_postcopy_ram() || migrate_return_path()) { - if (open_return_path_on_source(s, !resume)) { + if (open_return_path_on_source(s)) { error_report("Unable to open return-path for postcopy"); migrate_set_state(&s->state, s->state, MIGRATION_STATUS_FAILED= ); migrate_fd_cleanup(s); @@ -3320,7 +3290,6 @@ static void migration_instance_finalize(Object *obj) qemu_sem_destroy(&ms->rate_limit_sem); qemu_sem_destroy(&ms->pause_sem); qemu_sem_destroy(&ms->postcopy_pause_sem); - qemu_sem_destroy(&ms->postcopy_pause_rp_sem); qemu_sem_destroy(&ms->rp_state.rp_sem); qemu_sem_destroy(&ms->rp_state.rp_pong_acks); qemu_sem_destroy(&ms->postcopy_qemufile_src_sem); @@ -3340,7 +3309,6 @@ static void migration_instance_init(Object *obj) migrate_params_init(&ms->parameters); =20 qemu_sem_init(&ms->postcopy_pause_sem, 0); - qemu_sem_init(&ms->postcopy_pause_rp_sem, 0); qemu_sem_init(&ms->rp_state.rp_sem, 0); qemu_sem_init(&ms->rp_state.rp_pong_acks, 0); qemu_sem_init(&ms->rate_limit_sem, 0); diff --git a/migration/migration.h b/migration/migration.h index b7c8b67542..e78db5361c 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -382,7 +382,6 @@ struct MigrationState { =20 /* Needed by postcopy-pause state */ QemuSemaphore postcopy_pause_sem; - QemuSemaphore postcopy_pause_rp_sem; /* * Whether we abort the migration if decompression errors are * detected at the destination. It is left at false for qemu diff --git a/migration/trace-events b/migration/trace-events index 5259c1044b..19ec649d1d 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -164,6 +164,7 @@ migration_rate_limit_pre(int ms) "%d ms" migration_rate_limit_post(int urgent) "urgent: %d" migration_return_path_end_before(void) "" migration_return_path_end_after(int rp_error) "%d" +migration_return_path_pause_err(void) "" migration_thread_after_loop(void) "" migration_thread_file_err(void) "" migration_thread_setup_complete(void) "" --=20 2.35.3