From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452457; cv=none; d=zohomail.com; s=zohoarc; b=RLA/KvIGRgeuNbOwROykEQZRkESeVGwCf6SRqnr+V7f9VXH9ee7cH8DfdHXXey0dh87sZqiaDj3/u9wqJS9kcvaIhDoVSfV740rjTJ1OES3t2yLDgDSJBgsRFBgw+2IYktx9Diae/CPxX2lhq2FGcfLEKlfncbTvUJcuWpewwls= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452457; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gQVDJc/VWRXwS0mHPY1vDLVZJ1suUUTu444PbUJHcVk=; b=PLRkzCs57hetP4d4+NpqA6sKAfCAC46TxH/0alZ2ELk98bHLkC2Et2K6LvTC9mjlO9VLXwZ6/jjJsLmqIrJm2VFySCaFt6pQakFqTohmtW7DoXlPjirgd69SloXt9Vwkc069v+6TkseYxFkGLfc9sPo4EEF3HDB+2T2/KeH28Hs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452457634498.2657855184716; Mon, 11 Sep 2023 10:14:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTi-0002lf-6u; Mon, 11 Sep 2023 13:13:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTg-0002kt-Av for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:36 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTa-00008e-DP for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:35 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 249BA2183F; Mon, 11 Sep 2023 17:13:27 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 86CBC139CC; Mon, 11 Sep 2023 17:13:25 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id EN1MEbVK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452407; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gQVDJc/VWRXwS0mHPY1vDLVZJ1suUUTu444PbUJHcVk=; b=A/vuFtjF9XzJgrFMawwx6lgzutPb+V2K3h8WxrLoskFZlEPIG93NK7rib6tuu4eViFSC5r khHpQfsvAjA0Dsd7CIQI58ARjwZDyVn/Sn3qMMICutMtqSR4m/z1VludkgPRM0/06FWbv4 EFCSZfIy3+Bbvpc/AKiAUNhSbCPoYYU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452407; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gQVDJc/VWRXwS0mHPY1vDLVZJ1suUUTu444PbUJHcVk=; b=lWFAXV6df54Iud3eOxl7Qh9mQglpVD8cSvfvydsTXvxdR1zDPLTgeQNdQ1Vrd3YvIFF6nk bzkQTEhSk/+ZGpDw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 01/10] migration: Fix possible race when setting rp_state.error Date: Mon, 11 Sep 2023 14:13:11 -0300 Message-Id: <20230911171320.24372-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452500018100003 Content-Type: text/plain; charset="utf-8" We don't need to set the rp_state.error right after a shutdown because qemu_file_shutdown() always sets the QEMUFile error, so the return path thread would have seen it and set the rp error itself. Setting the error outside of the thread is also racy because the thread could clear it after we set it. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 5528acb65e..f88c86079c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2062,7 +2062,6 @@ static int await_return_path_close_on_source(Migratio= nState *ms) * waiting for the destination. */ qemu_file_shutdown(ms->rp_state.from_dst_file); - mark_source_rp_bad(ms); } trace_await_return_path_close_on_source_joining(); qemu_thread_join(&ms->rp_state.rp_thread); --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452518; cv=none; d=zohomail.com; s=zohoarc; b=e8g6sH9J/zexYodrTCbXUaaR9eyg2BXgQvxWXBlGMC19Go99z4jlKRCuQS6bmuo64KfHAlbzif3fd+wvSmMjI89BsozHhu8QpAsJ8XAJknkPnB7qL5p4+41jU1WQvIMnF5qv/zqox/JNA34jyUWuZnTbEWcHlCTg8Pus7W6K96M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452518; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QCN0lzhzxfoxI0dPzlD2+/IaDtzUb62BaW0g8BRMoDE=; b=XFIc0/pvpee/g7xJ1YcBDiyChaDGHKtB2gRDpfgClA+T52MSgSdWr3sRZZ5sschQkmaNTIPvTxPFOp3FhUE0vS+EJBu9GOTuzam1Zyboy3sf1ONfJ2y3kQKVaWsgkZ0pRfJ/ygdcncVPEevKl/8MIp1NRtsBWRbXAiUfxhRjPbs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452518619875.0259436856875; Mon, 11 Sep 2023 10:15:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTh-0002lb-GX; Mon, 11 Sep 2023 13:13:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTg-0002ku-BO for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:36 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTa-000091-D2 for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:35 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4F9E72185A; Mon, 11 Sep 2023 17:13:29 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id A7C0F139CC; Mon, 11 Sep 2023 17:13:27 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id OBBEGbdK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452409; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QCN0lzhzxfoxI0dPzlD2+/IaDtzUb62BaW0g8BRMoDE=; b=EeiPuUPw5AyYQzsViD2QzWhc0/wbSutAoUh9BzE0WbUW6wTpA6LGwbf7cfGF13Z5mQEU7j /sFslMNno0WTUci2c/qCQ1tFh9khjPUtNyEi0+VS5802zCfxZ8DvYo4CrNuQz4a/bUeeNV L9mfiHcqT9zdXraeho0/+0zBXDs2hYk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452409; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QCN0lzhzxfoxI0dPzlD2+/IaDtzUb62BaW0g8BRMoDE=; b=l03uQr9D77QU9zvy/df7DIWIPoE/H8KL8HllCcBUzYLHTQQri1yo0c0iWqr+FdzyfJNRZ4 RMaUTpv7H9Ro4kBw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 02/10] migration: Fix possible races when shutting down the return path Date: Mon, 11 Sep 2023 14:13:12 -0300 Message-Id: <20230911171320.24372-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452540969100001 Content-Type: text/plain; charset="utf-8" We cannot call qemu_file_shutdown() on the return path file without taking the file lock. The return path thread could be running it's cleanup code and have just cleared the from_dst_file pointer. Checking ms->to_dst_file for errors could also race with migrate_fd_cleanup() which clears the to_dst_file pointer. Protect both accesses by taking the file lock. This was caught by inspection, it should be rare, but the next patches will start calling this code from other places, so let's do the correct thing. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index f88c86079c..85c171f32c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2052,17 +2052,18 @@ static int open_return_path_on_source(MigrationStat= e *ms, static int await_return_path_close_on_source(MigrationState *ms) { /* - * If this is a normal exit then the destination will send a SHUT and = the - * rp_thread will exit, however if there's an error we need to cause - * it to exit. + * If this is a normal exit then the destination will send a SHUT + * and the rp_thread will exit, however if there's an error we + * need to cause it to exit. shutdown(2), if we have it, will + * cause it to unblock if it's stuck waiting for the destination. */ - if (qemu_file_get_error(ms->to_dst_file) && ms->rp_state.from_dst_file= ) { - /* - * shutdown(2), if we have it, will cause it to unblock if it's st= uck - * waiting for the destination. - */ - qemu_file_shutdown(ms->rp_state.from_dst_file); + WITH_QEMU_LOCK_GUARD(&ms->qemu_file_lock) { + if (ms->to_dst_file && ms->rp_state.from_dst_file && + qemu_file_get_error(ms->to_dst_file)) { + qemu_file_shutdown(ms->rp_state.from_dst_file); + } } + trace_await_return_path_close_on_source_joining(); qemu_thread_join(&ms->rp_state.rp_thread); ms->rp_state.rp_thread_created =3D false; --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452517; cv=none; d=zohomail.com; s=zohoarc; b=AEB7R+T4QuUHONi/r7GRz/3acOGSu+cV3x3UlhBapdS2lzgFuKLj/x3ClJTZQAcB7sYCYnqPTfIGN76yebvNt2zUwaqZjuS10YpLRaUifNG459+JRTeneMafAjF9P4PxbOFWi0VlhTq7bgEEToIiBpT4zXCauxbNqMnZhBXLVk8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452517; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ratvuBX3qQ91vHO0LxE42YTYPtTJVqto9t43u0ml8kE=; b=Qg+0p0vita08skXKBa1jBFcp93Db2FHjuhd+yP/2ewbCqQnRTDRVyuFuPJd/Vc/dnGs7XfH+ElAX5GyryNSF/1+7Zqm0/38QRhNPK31Xk9QT//PBVdYGvrEHC26C4MdaMni2RiXzYXYJ45TBevJ+AwAMRpZRdbqtUfSPrhl4DXg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 169445251771569.22472808827206; Mon, 11 Sep 2023 10:15:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTm-0002mz-1W; Mon, 11 Sep 2023 13:13:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTk-0002mh-Nt for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:40 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTc-00009j-FQ for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:40 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5DD9321860; Mon, 11 Sep 2023 17:13:31 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C5BC3139CC; Mon, 11 Sep 2023 17:13:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6CbGILlK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452411; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ratvuBX3qQ91vHO0LxE42YTYPtTJVqto9t43u0ml8kE=; b=RA8+wNYMyGLJM6odTLQY9gvtqnhQt1tzx27oKif49ocJYohGIPXMnG36fmpru1wGJMi9Ed OXSBbVo/hreZBRiAwe+iiY2aFSWl75ptuqiiK3uZksEGrrMCi53wxK/yHOCzAsaS1HFAkI gLZSb7UDKSJojadL7aaHytKIXlf9sT8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452411; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ratvuBX3qQ91vHO0LxE42YTYPtTJVqto9t43u0ml8kE=; b=r//NEdKFlVpq8jCfWRBjzRlMlBPxcxsSocu4u0WK4ONXiCiFo65uLNK03YpjvR1rlUhj6U ux75easfabIl/eCA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 03/10] migration: Fix possible race when shutting down to_dst_file Date: Mon, 11 Sep 2023 14:13:13 -0300 Message-Id: <20230911171320.24372-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452530466100003 Content-Type: text/plain; charset="utf-8" It's not safe to call qemu_file_shutdown() on the to_dst_file without first checking for the file's presence under the lock. The cleanup of this file happens at postcopy_pause() and migrate_fd_cleanup() which are not necessarily running in the same thread as migrate_fd_cancel(). Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 85c171f32c..5e6a766235 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1245,7 +1245,7 @@ static void migrate_fd_error(MigrationState *s, const= Error *error) static void migrate_fd_cancel(MigrationState *s) { int old_state ; - QEMUFile *f =3D migrate_get_current()->to_dst_file; + trace_migrate_fd_cancel(); =20 WITH_QEMU_LOCK_GUARD(&s->qemu_file_lock) { @@ -1271,11 +1271,13 @@ static void migrate_fd_cancel(MigrationState *s) * If we're unlucky the migration code might be stuck somewhere in a * send/write while the network has failed and is waiting to timeout; * if we've got shutdown(2) available then we can force it to quit. - * The outgoing qemu file gets closed in migrate_fd_cleanup that is - * called in a bh, so there is no race against this cancel. */ - if (s->state =3D=3D MIGRATION_STATUS_CANCELLING && f) { - qemu_file_shutdown(f); + if (s->state =3D=3D MIGRATION_STATUS_CANCELLING) { + WITH_QEMU_LOCK_GUARD(&s->qemu_file_lock) { + if (s->to_dst_file) { + qemu_file_shutdown(s->to_dst_file); + } + } } if (s->state =3D=3D MIGRATION_STATUS_CANCELLING && s->block_inactive) { Error *local_err =3D NULL; @@ -1519,12 +1521,14 @@ void qmp_migrate_pause(Error **errp) { MigrationState *ms =3D migrate_get_current(); MigrationIncomingState *mis =3D migration_incoming_get_current(); - int ret; + int ret =3D 0; =20 if (ms->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE) { /* Source side, during postcopy */ qemu_mutex_lock(&ms->qemu_file_lock); - ret =3D qemu_file_shutdown(ms->to_dst_file); + if (ms->to_dst_file) { + ret =3D qemu_file_shutdown(ms->to_dst_file); + } qemu_mutex_unlock(&ms->qemu_file_lock); if (ret) { error_setg(errp, "Failed to pause source migration"); --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452510; cv=none; d=zohomail.com; s=zohoarc; b=LnTyYasLMn5lX/QPCeVneyc7oRmqwwnmkBU9vsw72hJuw9PAzFGapNz5mE5irYJluQFJca9QJh/bek6hm0LtQcjrArg1YR+O7NsrczTI24qzTPhGoJAKZ373fMOHRQ1sUFlonSkstPwa/FDxUGHQ7Td1V9ABNyIs9ap3cPjvZGE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452510; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xmbFysED0AHDJV1mZ+g/WSZUPpQFtPzNFvXjvePLOVI=; b=WqgJ8J8EoR5xvyGLo5oGsc/4KvGWjw401rYx04BTDApd8OnVKf0ta++OA62PpWUu391+PCNSKSuOyYmm2xP/dlSh02CUNxtVZxdwH5jkjDUc2DQrLmEtctXJg9tdIOjEMD3n+HMRu2tHr05D70U9I/LnbVDp6X/ckNdXvuCPLgg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452510438485.0372270666413; Mon, 11 Sep 2023 10:15:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTs-0002oq-3g; Mon, 11 Sep 2023 13:13:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTq-0002oO-Gi for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:46 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTg-00009x-3C for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:46 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7FD1C1F88C; Mon, 11 Sep 2023 17:13:33 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DF2F8139CC; Mon, 11 Sep 2023 17:13:31 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6CweJ7tK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452413; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xmbFysED0AHDJV1mZ+g/WSZUPpQFtPzNFvXjvePLOVI=; b=Her8+oyml3bkwUFpfobg8hhakLQ4QYT+94X6qCER/GD3IzaamgoeSWX9/poBkLy1YvCASB bL9Y34M3R7SJWLiML86/V7ghHeCOeIpAx9GLnATr9Lk8AfB+3NFQ6bY8UjjP+teKN4LNMV zMX7drAF+T2B3jbTFFMaTRqhhtufAlU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452413; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xmbFysED0AHDJV1mZ+g/WSZUPpQFtPzNFvXjvePLOVI=; b=h+bPZ8WYWrSNC6/J7E3EcdqkG3g58Xlems7ATwPs/TevwBpSIUxlrs3YdbDfjGNeehZ6Pf YmsTeSGdKeZlhODQ== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 04/10] migration: Remove redundant cleanup of postcopy_qemufile_src Date: Mon, 11 Sep 2023 14:13:14 -0300 Message-Id: <20230911171320.24372-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452523875100003 Content-Type: text/plain; charset="utf-8" This file is owned by the return path thread which is already doing cleanup. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 5e6a766235..195726eb4a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1177,12 +1177,6 @@ static void migrate_fd_cleanup(MigrationState *s) qemu_fclose(tmp); } =20 - if (s->postcopy_qemufile_src) { - migration_ioc_unregister_yank_from_file(s->postcopy_qemufile_src); - qemu_fclose(s->postcopy_qemufile_src); - s->postcopy_qemufile_src =3D NULL; - } - assert(!migration_is_active(s)); =20 if (s->state =3D=3D MIGRATION_STATUS_CANCELLING) { --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452513; cv=none; d=zohomail.com; s=zohoarc; b=NkqUCQ22dmIGvLIFTuj6t9rNDDoTrtAATvb7sTwWIdnm5zeUjwpPerh6J75z2WTvUROhL0ttCP5LoYPIjsplEd4/4XMHV2pnd04g66OkHwCE6NOQO0pJ4laqMP403r8JPTjrdf2cwZfLMUEMczvCd1sjuWkDkDsGdEBx/FKI+S4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452513; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=M4F3Z9FAUn93sfwbK8FJ9e5DMMvpXA/8ArfN50R/0eg=; b=LC0QX+YF2KPrXN0LOBBOJWKa4PWSEEScF6bbGlSdApZtdL3MlGBnIZgb17hMmVtmYSc9gCgQc+sHbcobGSypj1AUIgk/K3Orbt9Q5Ibo3En1edXUuWKuZh88zGHWFZ8C7tujsoP216nINpvlSRY6tCa+DJea6GjPPTYz2GUd2w8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452513162902.7510221064006; Mon, 11 Sep 2023 10:15:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTl-0002mp-3F; Mon, 11 Sep 2023 13:13:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTk-0002mT-65 for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:40 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTi-0000AE-0G for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:39 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9D5A421875; Mon, 11 Sep 2023 17:13:35 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0E42D139CC; Mon, 11 Sep 2023 17:13:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id iPmKL71K/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452415; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M4F3Z9FAUn93sfwbK8FJ9e5DMMvpXA/8ArfN50R/0eg=; b=xMfAarmjooA7RcdpfBeXAAEqZjbRIuUM2tuC+Wnekwx3KQRJVZzbH0S8wfWTgYZzd96HNy OH5sOdFgNJwQuZxdttCRm7vhwmOW3RqfcUq+v5X+CuvPzj5ABvK95rP4sCApWMkafgS9KN ne/7GeaFsTHdB7Um/bcxSvtX+yyjeQQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452415; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M4F3Z9FAUn93sfwbK8FJ9e5DMMvpXA/8ArfN50R/0eg=; b=GME/daoo0Dz5+FuOi1SbdhjDM2OfLhzGML9G2vgMbEgDicYPFOYgT6pV8z5C8uK47GCfXd K81c1zi/8ePb+CAg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 05/10] migration: Consolidate return path closing code Date: Mon, 11 Sep 2023 14:13:15 -0300 Message-Id: <20230911171320.24372-6-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452532916100001 Content-Type: text/plain; charset="utf-8" We'll start calling the await_return_path_close_on_source() function from other parts of the code, so move all of the related checks and tracepoints into it. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 195726eb4a..4edbee3a5d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2049,6 +2049,14 @@ static int open_return_path_on_source(MigrationState= *ms, /* Returns 0 if the RP was ok, otherwise there was an error on the RP */ static int await_return_path_close_on_source(MigrationState *ms) { + int ret; + + if (!ms->rp_state.rp_thread_created) { + return 0; + } + + trace_migration_return_path_end_before(); + /* * If this is a normal exit then the destination will send a SHUT * and the rp_thread will exit, however if there's an error we @@ -2066,7 +2074,10 @@ static int await_return_path_close_on_source(Migrati= onState *ms) qemu_thread_join(&ms->rp_state.rp_thread); ms->rp_state.rp_thread_created =3D false; trace_await_return_path_close_on_source_close(); - return ms->rp_state.error; + + ret =3D ms->rp_state.error; + trace_migration_return_path_end_after(ret); + return ret; } =20 static inline void @@ -2362,20 +2373,8 @@ static void migration_completion(MigrationState *s) goto fail; } =20 - /* - * If rp was opened we must clean up the thread before - * cleaning everything else up (since if there are no failures - * it will wait for the destination to send it's status in - * a SHUT command). - */ - if (s->rp_state.rp_thread_created) { - int rp_error; - trace_migration_return_path_end_before(); - rp_error =3D await_return_path_close_on_source(s); - trace_migration_return_path_end_after(rp_error); - if (rp_error) { - goto fail; - } + if (await_return_path_close_on_source(s)) { + goto fail; } =20 if (qemu_file_get_error(s->to_dst_file)) { --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452475; cv=none; d=zohomail.com; s=zohoarc; b=kDXLOy4WHGl5xBE+TDFzIDu/ONkbu0QEBrerQeoeEFDDurAkrEpr6bsu+B8LbpiJzH4pJNuKlBD8EE4t89n8yhcv9NuoZtk67Md/2pXhixy/0cKcnBhbocpj1MxtklZwe6hTRGHF4Te3j2AEb7HW3K6NPSsxfPff9hPzsteGiMc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452475; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nsROk6EVy4STMdPRXEStC03FsfSuYWtANPTfYLuwSvk=; b=nhqpnctiM4YEjHjTc2KoxtQ9eqr2d/D6WYx6Os97KXQJq6NVLU6ABlE0VWKQFF4X6iC9QGbtpz3IGff9fiTFqihiMze2bmvlKtMJgyb709sewSKDb6Vwv/lWadGiY1WxxBHLZy8SNHWMA6KbybodkhFB6PE+JKaDRKUSrUF6bL0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452475659547.8773798974578; Mon, 11 Sep 2023 10:14:35 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTo-0002ns-Qg; Mon, 11 Sep 2023 13:13:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTl-0002mr-KV for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:41 -0400 Received: from smtp-out1.suse.de ([2001:67c:2178:6::1c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTi-0000Ax-RU for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:41 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C2D022183F; Mon, 11 Sep 2023 17:13:37 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2E7CA139CC; Mon, 11 Sep 2023 17:13:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id eHNVN79K/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452417; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nsROk6EVy4STMdPRXEStC03FsfSuYWtANPTfYLuwSvk=; b=qD1skM7d8QQ8Ocl9ghRE2vquQ70laO1notkffxocKhLWiVFtwRnsOfEsvKREq2QW1Du8AX IY4V3lWkFjq1LjrKWWNxQ0VDFKogI4YPGkYxxN7vi1oTnGUNdO2RlAlCF5AhBIuBtWm927 RAUH8it/PG4smrvi9RIny+jVUeCsPQI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452417; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nsROk6EVy4STMdPRXEStC03FsfSuYWtANPTfYLuwSvk=; b=2v67M5xAvN4ieQD0MFTCVaU09qTjjC130SNWfNvGX5+/yjDbAhWAMDh7v1zWFR7qNylmGE o7NqKKsVCm2u/pCg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 06/10] migration: Replace the return path retry logic Date: Mon, 11 Sep 2023 14:13:16 -0300 Message-Id: <20230911171320.24372-7-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1c; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452476801100002 Content-Type: text/plain; charset="utf-8" Replace the return path retry logic with finishing and restarting the thread. This fixes a race when resuming the migration that leads to a segfault. Currently when doing postcopy we consider that an IO error on the return path file could be due to a network intermittency. We then keep the thread alive but have it do cleanup of the 'from_dst_file' and wait on the 'postcopy_pause_rp' semaphore. When the user issues a migrate resume, a new return path is opened and the thread is allowed to continue. There's a race condition in the above mechanism. It is possible for the new return path file to be setup *before* the cleanup code in the return path thread has had a chance to run, leading to the *new* file being closed and the pointer set to NULL. When the thread is released after the resume, it tries to dereference 'from_dst_file' and crashes: Thread 7 "return path" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffd1dbf700 (LWP 9611)] 0x00005555560e4893 in qemu_file_get_error_obj (f=3D0x0, errp=3D0x0) at ../m= igration/qemu-file.c:154 154 return f->last_error; (gdb) bt #0 0x00005555560e4893 in qemu_file_get_error_obj (f=3D0x0, errp=3D0x0) at= ../migration/qemu-file.c:154 #1 0x00005555560e4983 in qemu_file_get_error (f=3D0x0) at ../migration/qe= mu-file.c:206 #2 0x0000555555b9a1df in source_return_path_thread (opaque=3D0x555556e060= 00) at ../migration/migration.c:1876 #3 0x000055555602e14f in qemu_thread_start (args=3D0x55555782e780) at ../= util/qemu-thread-posix.c:541 #4 0x00007ffff38d76ea in start_thread (arg=3D0x7fffd1dbf700) at pthread_c= reate.c:477 #5 0x00007ffff35efa6f in clone () at ../sysdeps/unix/sysv/linux/x86_64/cl= one.S:95 Here's the race (important bit is open_return_path happening before migration_release_dst_files): migration | qmp | return path Reviewed-by: Peter Xu --------------------------+-----------------------------+------------------= --------------- qmp_migrate_pause() shutdown(ms->to_dst_file) f->last_error =3D -EIO migrate_detect_error() postcopy_pause() set_state(PAUSED) wait(postcopy_pause_sem) qmp_migrate(resume) migrate_fd_connect() resume =3D state =3D=3D PAUSED open_return_path <-- TOO SOON! set_state(RECOVER) post(postcopy_pause_sem) (incoming closes to_src_file) res =3D qemu_file_get_error(rp) migration_release_dst_files() ms->rp_state.from_dst_file =3D NULL post(postcopy_pause_rp_sem) postcopy_pause_return_path_thread() wait(postcopy_pause_rp_sem) rp =3D ms->rp_state.from_dst_file goto retry qemu_file_get_error(rp) SIGSEGV ---------------------------------------------------------------------------= ---------------- We can keep the retry logic without having the thread alive and waiting. The only piece of data used by it is the 'from_dst_file' and it is only allowed to proceed after a migrate resume is issued and the semaphore released at migrate_fd_connect(). Move the retry logic to outside the thread by waiting for the thread to finish before pausing the migration. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 60 ++++++++----------------------------------- migration/migration.h | 1 - 2 files changed, 11 insertions(+), 50 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 4edbee3a5d..7dfcbc3634 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1775,18 +1775,6 @@ static void migrate_handle_rp_req_pages(MigrationSta= te *ms, const char* rbname, } } =20 -/* Return true to retry, false to quit */ -static bool postcopy_pause_return_path_thread(MigrationState *s) -{ - trace_postcopy_pause_return_path(); - - qemu_sem_wait(&s->postcopy_pause_rp_sem); - - trace_postcopy_pause_return_path_continued(); - - return true; -} - static int migrate_handle_rp_recv_bitmap(MigrationState *s, char *block_na= me) { RAMBlock *block =3D qemu_ram_block_by_name(block_name); @@ -1870,7 +1858,6 @@ static void *source_return_path_thread(void *opaque) trace_source_return_path_thread_entry(); rcu_register_thread(); =20 -retry: while (!ms->rp_state.error && !qemu_file_get_error(rp) && migration_is_setup_or_active(ms->state)) { trace_source_return_path_thread_loop_top(); @@ -1992,26 +1979,7 @@ retry: } =20 out: - res =3D qemu_file_get_error(rp); - if (res) { - if (res && migration_in_postcopy()) { - /* - * Maybe there is something we can do: it looks like a - * network down issue, and we pause for a recovery. - */ - migration_release_dst_files(ms); - rp =3D NULL; - if (postcopy_pause_return_path_thread(ms)) { - /* - * Reload rp, reset the rest. Referencing it is safe since - * it's reset only by us above, or when migration completes - */ - rp =3D ms->rp_state.from_dst_file; - ms->rp_state.error =3D false; - goto retry; - } - } - + if (qemu_file_get_error(rp)) { trace_source_return_path_thread_bad_end(); mark_source_rp_bad(ms); } @@ -2022,8 +1990,7 @@ out: return NULL; } =20 -static int open_return_path_on_source(MigrationState *ms, - bool create_thread) +static int open_return_path_on_source(MigrationState *ms) { ms->rp_state.from_dst_file =3D qemu_file_get_return_path(ms->to_dst_fi= le); if (!ms->rp_state.from_dst_file) { @@ -2032,11 +1999,6 @@ static int open_return_path_on_source(MigrationState= *ms, =20 trace_open_return_path_on_source(); =20 - if (!create_thread) { - /* We're done */ - return 0; - } - qemu_thread_create(&ms->rp_state.rp_thread, "return path", source_return_path_thread, ms, QEMU_THREAD_JOINABLE= ); ms->rp_state.rp_thread_created =3D true; @@ -2076,6 +2038,7 @@ static int await_return_path_close_on_source(Migratio= nState *ms) trace_await_return_path_close_on_source_close(); =20 ret =3D ms->rp_state.error; + ms->rp_state.error =3D false; trace_migration_return_path_end_after(ret); return ret; } @@ -2551,6 +2514,13 @@ static MigThrError postcopy_pause(MigrationState *s) qemu_file_shutdown(file); qemu_fclose(file); =20 + /* + * We're already pausing, so ignore any errors on the return + * path and just wait for the thread to finish. It will be + * re-created when we resume. + */ + await_return_path_close_on_source(s); + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_POSTCOPY_PAUSED); =20 @@ -2568,12 +2538,6 @@ static MigThrError postcopy_pause(MigrationState *s) if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { /* Woken up by a recover procedure. Give it a shot */ =20 - /* - * Firstly, let's wake up the return path now, with a new - * return path channel. - */ - qemu_sem_post(&s->postcopy_pause_rp_sem); - /* Do the resume logic */ if (postcopy_do_resume(s) =3D=3D 0) { /* Let's continue! */ @@ -3263,7 +3227,7 @@ void migrate_fd_connect(MigrationState *s, Error *err= or_in) * QEMU uses the return path. */ if (migrate_postcopy_ram() || migrate_return_path()) { - if (open_return_path_on_source(s, !resume)) { + if (open_return_path_on_source(s)) { error_setg(&local_err, "Unable to open return-path for postcop= y"); migrate_set_state(&s->state, s->state, MIGRATION_STATUS_FAILED= ); migrate_set_error(s, local_err); @@ -3327,7 +3291,6 @@ static void migration_instance_finalize(Object *obj) qemu_sem_destroy(&ms->rate_limit_sem); qemu_sem_destroy(&ms->pause_sem); qemu_sem_destroy(&ms->postcopy_pause_sem); - qemu_sem_destroy(&ms->postcopy_pause_rp_sem); qemu_sem_destroy(&ms->rp_state.rp_sem); qemu_sem_destroy(&ms->rp_state.rp_pong_acks); qemu_sem_destroy(&ms->postcopy_qemufile_src_sem); @@ -3347,7 +3310,6 @@ static void migration_instance_init(Object *obj) migrate_params_init(&ms->parameters); =20 qemu_sem_init(&ms->postcopy_pause_sem, 0); - qemu_sem_init(&ms->postcopy_pause_rp_sem, 0); qemu_sem_init(&ms->rp_state.rp_sem, 0); qemu_sem_init(&ms->rp_state.rp_pong_acks, 0); qemu_sem_init(&ms->rate_limit_sem, 0); diff --git a/migration/migration.h b/migration/migration.h index 6eea18db36..36eb5ba70b 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -382,7 +382,6 @@ struct MigrationState { =20 /* Needed by postcopy-pause state */ QemuSemaphore postcopy_pause_sem; - QemuSemaphore postcopy_pause_rp_sem; /* * Whether we abort the migration if decompression errors are * detected at the destination. It is left at false for qemu --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452514; cv=none; d=zohomail.com; s=zohoarc; b=MMTQSghY5Y2osEtg0ZuqbKviqFWzXef8fOE1FZS5EDw2CCNpG8CvwhYU7mPH9XeKw2M5HpDAw6I+lOOx8m6k6fAS9r25/Uz3QMa9M2531KGTnPunBHAjmlGK1pPQoo87/kaKyIp1jUz5yX6WKIszNG6y3y8Z73ZtU2zPMdlgpmY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452514; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DGhGE717fUJFvyUY5vhbyY/GLiHbUzH5b0cd6j63xVg=; b=JKFO9ZyDOi3TTaYDVVGjq7Q+MbzkcBDwJmBDalUzQsrU9YvrVHFh0D4j7o2WB9i8TrCu+0PB8iYH5pPXt6IgflMTov9FD/z+Y3j6FfO23Y4icvV3Jc4eEOd4MXfS9NWxnljnz6qBE/dpE2GekLZJ1kGWampHm0lI33zJHlcylBY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452514691443.8451368475886; Mon, 11 Sep 2023 10:15:14 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTp-0002nx-Li; Mon, 11 Sep 2023 13:13:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTn-0002nV-GN for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:43 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTl-0000BL-Ez for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:43 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E4E9D1F8B3; Mon, 11 Sep 2023 17:13:39 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 51062139CC; Mon, 11 Sep 2023 17:13:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id WAwpBMJK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:38 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452419; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DGhGE717fUJFvyUY5vhbyY/GLiHbUzH5b0cd6j63xVg=; b=J4IAGJjTD30qvGty/hkpPcKJC2gJ4vBEFH26c54SnChaOvioXC2QSjTaEhEAOB1IBs+o0B rypW3722VODRMpHPL8LnB+ric5/HTIj4+uSNPKYZIaZ1IpgsrJReBkTeylfHcmUMVfonlC QZFVoW0ecRqSdUw8I9NfQ9THQ9eMmhM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452419; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DGhGE717fUJFvyUY5vhbyY/GLiHbUzH5b0cd6j63xVg=; b=IhDUWFv3yAckTDx50sZOcXTsOVoWyfJkU98O7T83krP/OHGPqk6hRQEshNbUapJoCgHhq5 CtQ03x9NJ4kvrlCA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 07/10] migration: Move return path cleanup to main migration thread Date: Mon, 11 Sep 2023 14:13:17 -0300 Message-Id: <20230911171320.24372-8-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452528498100001 Content-Type: text/plain; charset="utf-8" Now that the return path thread is allowed to finish during a paused migration, we can move the cleanup of the QEMUFiles to the main migration thread. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/migration.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 7dfcbc3634..7fec57ad7f 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -98,6 +98,7 @@ static int migration_maybe_pause(MigrationState *s, int *current_active_state, int new_state); static void migrate_fd_cancel(MigrationState *s); +static int await_return_path_close_on_source(MigrationState *s); =20 static bool migration_needs_multiple_sockets(void) { @@ -1177,6 +1178,12 @@ static void migrate_fd_cleanup(MigrationState *s) qemu_fclose(tmp); } =20 + /* + * We already cleaned up to_dst_file, so errors from the return + * path might be due to that, ignore them. + */ + await_return_path_close_on_source(s); + assert(!migration_is_active(s)); =20 if (s->state =3D=3D MIGRATION_STATUS_CANCELLING) { @@ -1985,7 +1992,6 @@ out: } =20 trace_source_return_path_thread_end(); - migration_release_dst_files(ms); rcu_unregister_thread(); return NULL; } @@ -2039,6 +2045,9 @@ static int await_return_path_close_on_source(Migratio= nState *ms) =20 ret =3D ms->rp_state.error; ms->rp_state.error =3D false; + + migration_release_dst_files(ms); + trace_migration_return_path_end_after(ret); return ret; } --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452498; cv=none; d=zohomail.com; s=zohoarc; b=MO7p1s3GcrsjFf9fgnmtLoKAGt44/nV4n5a6yMUHQE9hPjK5a3fOuk7IY1yXYfSyNm4/aXqrFbw08rBS0k471Vbdb5yZi8BhBTmWmtBZ1o9mU71uVtLCJTmF9HyVShJ4TE+D1+rxwFiLXy7P/AD7a08oDzMsPYDxXIE1DIKjl6c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452498; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UGGxjAB3Urj0ONFUV5hITxPJOdr38wwUKzL/jNXoWEg=; b=aS0nIcEsg/dVarLRvv6bZf4y2kijzRSkbJ9PGMgtjkZa0BKmxCB/aJWG0gll/B52mwhSMAVUUkUMttPBqknq+vvealkD8GQl/LXB6r1JszDBOAHNEvxa7KA+z9WgQsE/gESA2FMHW+Qoc7FCnAIRKpT+S/ikyLV4z3mnq8UcQrU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 169445249805111.747008650723842; Mon, 11 Sep 2023 10:14:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTq-0002oP-JO; Mon, 11 Sep 2023 13:13:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTp-0002nu-HG for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:45 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTn-0000Bw-D2 for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:45 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4F96A1F8BB; Mon, 11 Sep 2023 17:13:42 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7300A139CC; Mon, 11 Sep 2023 17:13:40 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2KVSDMRK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452422; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UGGxjAB3Urj0ONFUV5hITxPJOdr38wwUKzL/jNXoWEg=; b=tsmb8PPjP45TPN2siHZud1J2W8QTAnW35az3StrPHvHsZNNvojVmedfwF4VksPUgZ6rxKL jeA9j4Tna2jczMQ7kfsvey84eZZfsOPwJDFsutevT4hwZ/Q5FGu8qHHN9zshS/Sje7NN7a mPYxKh9sZqIpkW3ibuRwgi+Jv+jkX3Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452422; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UGGxjAB3Urj0ONFUV5hITxPJOdr38wwUKzL/jNXoWEg=; b=Lp+y+BPmHYFuPRB52wNDlKGyaKH8tjv5HUhyYdgQEbErASLhgYpeF1PU+YbGJDc7q9LlDp SZS1U32U97t+KaAw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Lukas Straub , Leonardo Bras Subject: [PATCH v6 08/10] migration/yank: Use channel features Date: Mon, 11 Sep 2023 14:13:18 -0300 Message-Id: <20230911171320.24372-9-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452516423100003 Content-Type: text/plain; charset="utf-8" Stop using outside knowledge about the io channels when registering yank functions. Query for features instead. The yank method for all channels used with migration code currently is to call the qio_channel_shutdown() function, so query for QIO_CHANNEL_FEATURE_SHUTDOWN. We could add a separate feature in the future for indicating whether a channel supports yanking, but that seems overkill at the moment. Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- migration/yank_functions.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/migration/yank_functions.c b/migration/yank_functions.c index d5a710a3f2..979e60c762 100644 --- a/migration/yank_functions.c +++ b/migration/yank_functions.c @@ -8,12 +8,9 @@ */ =20 #include "qemu/osdep.h" -#include "qapi/error.h" #include "io/channel.h" #include "yank_functions.h" #include "qemu/yank.h" -#include "io/channel-socket.h" -#include "io/channel-tls.h" #include "qemu-file.h" =20 void migration_yank_iochannel(void *opaque) @@ -26,8 +23,7 @@ void migration_yank_iochannel(void *opaque) /* Return whether yank is supported on this ioc */ static bool migration_ioc_yank_supported(QIOChannel *ioc) { - return object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_SOCKET) || - object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS); + return qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_SHUTDOWN); } =20 void migration_ioc_register_yank(QIOChannel *ioc) --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452565; cv=none; d=zohomail.com; s=zohoarc; b=Np5YGlH5nxLhRbZNFmHsGmHpecdaaxGCERrxJbAKGhmvvJp+tZszGX7ZKdvm9JoqEOTDpqbA0kE3W0cZa3Ff6dFfvWdedTT2hYMfETbGhArENSHjaWZuQWbSkDLEnV8MgDDnVmloGT01WAHO1o4QwIKZfkfZBFLDCm85z919hz0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452565; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=t+yRV2zV71YcJVy41VS6eqUMCMiCNBCaZK6ydOiBfpk=; b=UN702yuq/qVSRTKd3Yb7DOJ+XnqVx+fERlxSp68y8+l3gWxKHlxTWSgEF3/3aD6W2mccmJ51T37ql+xWbbK5L7zV13Xwr1b9aHkNtof1Mf1udQKmYY0TNxnu3Zx6ObBawCDrXW2A1cFiMzqWk7GzMGWNLNYB/rVh/ImubIM5G34= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452565601444.7275300413046; Mon, 11 Sep 2023 10:16:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTt-0002pC-Ah; Mon, 11 Sep 2023 13:13:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTs-0002p3-Fj for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:48 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTp-0000CY-Oc for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:48 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AD41D1F8B3; Mon, 11 Sep 2023 17:13:44 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D2A4E139CC; Mon, 11 Sep 2023 17:13:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id aODeI8ZK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452424; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t+yRV2zV71YcJVy41VS6eqUMCMiCNBCaZK6ydOiBfpk=; b=vW447TdZJdezib8NsP+DDCGMSnL57heOcKo0NgcJl7XlnNhVcfSF2/r0/NdZ+bsTb+VLsa C65V4dJYqrGAYullL5xlxerbe721ILbyA83kOuu5fFupqT8Cmw20MSX0lFNMBpf6PeIUjL 9Peej9JLbRQKsz/kqVSXwr3Rh7JGR6s= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452424; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t+yRV2zV71YcJVy41VS6eqUMCMiCNBCaZK6ydOiBfpk=; b=FrjhfSCoq17DAHWhQ+BXdUD2BK3VYBvetg66RZlQYpOAR0jWuXfbSVMs6T2VB/5RJ02mbN RqMdc81HqrDnXXBg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Lukas Straub , Leonardo Bras Subject: [PATCH v6 09/10] migration/yank: Keep track of registered yank instances Date: Mon, 11 Sep 2023 14:13:19 -0300 Message-Id: <20230911171320.24372-10-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.29; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452566955100001 Content-Type: text/plain; charset="utf-8" The core yank code is strict about balanced registering and unregistering of yank functions. This creates a difficulty because the migration code registers one yank function per QIOChannel, but each QIOChannel can be referenced by more than one QEMUFile. The yank function should not be removed until all QEMUFiles have been closed. Keep a reference count of how many QEMUFiles are using a QIOChannel that has a yank function. Only unregister the yank function when all QEMUFiles have been closed. This improves the current code by removing the need for the programmer to know which QEMUFile is the last one to be cleaned up and fixes the theoretical issue of removing the yank function while another QEMUFile could still be using the ioc and require a yank. Signed-off-by: Fabiano Rosas --- migration/yank_functions.c | 81 ++++++++++++++++++++++++++++++++++---- migration/yank_functions.h | 8 ++++ 2 files changed, 81 insertions(+), 8 deletions(-) diff --git a/migration/yank_functions.c b/migration/yank_functions.c index 979e60c762..afdeef30ff 100644 --- a/migration/yank_functions.c +++ b/migration/yank_functions.c @@ -10,9 +10,32 @@ #include "qemu/osdep.h" #include "io/channel.h" #include "yank_functions.h" +#include "qemu/lockable.h" #include "qemu/yank.h" #include "qemu-file.h" =20 +static QemuMutex ioc_list_lock; +static QLIST_HEAD(, Yankable) yankable_ioc_list + =3D QLIST_HEAD_INITIALIZER(yankable_ioc_list); + +static void __attribute__((constructor)) ioc_list_lock_init(void) +{ + qemu_mutex_init(&ioc_list_lock); +} + +static void yankable_ref(Yankable *yankable) +{ + assert(yankable->refcnt > 0); + yankable->refcnt++; + assert(yankable->refcnt < INT_MAX); +} + +static void yankable_unref(Yankable *yankable) +{ + assert(yankable->refcnt > 0); + yankable->refcnt--; +} + void migration_yank_iochannel(void *opaque) { QIOChannel *ioc =3D QIO_CHANNEL(opaque); @@ -28,20 +51,62 @@ static bool migration_ioc_yank_supported(QIOChannel *io= c) =20 void migration_ioc_register_yank(QIOChannel *ioc) { - if (migration_ioc_yank_supported(ioc)) { - yank_register_function(MIGRATION_YANK_INSTANCE, - migration_yank_iochannel, - ioc); + Yankable *new, *entry; + + if (!ioc || !migration_ioc_yank_supported(ioc)) { + return; } + + WITH_QEMU_LOCK_GUARD(&ioc_list_lock) { + QLIST_FOREACH(entry, &yankable_ioc_list, next) { + if (entry->opaque =3D=3D ioc) { + yankable_ref(entry); + return; + } + } + + new =3D g_new0(Yankable, 1); + new->refcnt =3D 1; + new->opaque =3D ioc; + object_ref(ioc); + + QLIST_INSERT_HEAD(&yankable_ioc_list, new, next); + } + + yank_register_function(MIGRATION_YANK_INSTANCE, + migration_yank_iochannel, + ioc); } =20 void migration_ioc_unregister_yank(QIOChannel *ioc) { - if (migration_ioc_yank_supported(ioc)) { - yank_unregister_function(MIGRATION_YANK_INSTANCE, - migration_yank_iochannel, - ioc); + Yankable *entry, *tmp; + + if (!ioc || !migration_ioc_yank_supported(ioc)) { + return; } + + WITH_QEMU_LOCK_GUARD(&ioc_list_lock) { + QLIST_FOREACH_SAFE(entry, &yankable_ioc_list, next, tmp) { + if (entry->opaque =3D=3D ioc) { + yankable_unref(entry); + + if (!entry->refcnt) { + QLIST_REMOVE(entry, next); + g_free(entry); + goto unreg; + } + } + } + } + + return; + +unreg: + yank_unregister_function(MIGRATION_YANK_INSTANCE, + migration_yank_iochannel, + ioc); + object_unref(ioc); } =20 void migration_ioc_unregister_yank_from_file(QEMUFile *file) diff --git a/migration/yank_functions.h b/migration/yank_functions.h index a7577955ed..c928a46f68 100644 --- a/migration/yank_functions.h +++ b/migration/yank_functions.h @@ -7,6 +7,14 @@ * See the COPYING file in the top-level directory. */ =20 +struct Yankable { + void *opaque; + int refcnt; + QLIST_ENTRY(Yankable) next; +}; + +typedef struct Yankable Yankable; + /** * migration_yank_iochannel: yank function for iochannel * --=20 2.35.3 From nobody Thu May 9 22:05:23 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1694452457; cv=none; d=zohomail.com; s=zohoarc; b=GD14+JfN7FVDOQcqSjGyZs+7V3ksgUuRxiQCYAov4tlzruGacxcp5+bnJ3Y6Qx/3iMuNHoVJ28V9Nsd3xgJb+fHyBdKtm9jRI8Id46Q3QCVrlJbEUkrfH9jc1mrKvPqKC539BEXy34XPiU4EQfbth7NnqvPKdO0p/1iD9eyTPjg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694452457; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=86Hnc8VCm1b08YFeJ9MLYUHFLCAO81JcW8bL6osu5cc=; b=HtciuBApyaCVlTvfC5G0ADHv43qLoP8cUGJYcFL+dEArKP+YI+TnrxWsRSQR5qfZteHE0yLGXC1xTIB4nUB7AM7WxqDTVWV3tsEXgw8ooaZSagfzzAXhowfLhrNDsF0naWlFkhjxJehFpJqNyirGJqul9KUS7VGQVRCv+tH+VWs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1694452457676118.42514095591753; Mon, 11 Sep 2023 10:14:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfkTw-0002rT-Tb; Mon, 11 Sep 2023 13:13:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfkTu-0002pY-Qe for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:50 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfkTs-0000D9-7n for qemu-devel@nongnu.org; Mon, 11 Sep 2023 13:13:50 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id CD2871F88C; Mon, 11 Sep 2023 17:13:46 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 398DC139CC; Mon, 11 Sep 2023 17:13:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 8MpGOshK/2QyIAAAMHmgww (envelope-from ); Mon, 11 Sep 2023 17:13:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1694452426; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=86Hnc8VCm1b08YFeJ9MLYUHFLCAO81JcW8bL6osu5cc=; b=K/puJbCPKRac9ij2IyIMhXgGV0KBQ2Kdtr6wPodim38COEYN71tc6WzPENCxkQZlHzN1rw DMwcL10UvqWAUrRkO4WqKIDKFjGmUPsW/WNpfkBFXZCz1cS392xG5TqpY/StJ7jhUW/jeO rq3YXU59aXaBCiGWH0269oLEH0rW+m8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1694452426; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=86Hnc8VCm1b08YFeJ9MLYUHFLCAO81JcW8bL6osu5cc=; b=JQk5g0TW1i4lD7Kr0GmOevQVo4YA6eWdwrJtU9KN9JAu4r7rVy0SXWzuBzoPM9y3SdSPFl 3G0e6x/8WXRXrfCA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Juan Quintela , Peter Xu , Leonardo Bras Subject: [PATCH v6 10/10] migration: Add a wrapper to cleanup migration files Date: Mon, 11 Sep 2023 14:13:20 -0300 Message-Id: <20230911171320.24372-11-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230911171320.24372-1-farosas@suse.de> References: <20230911171320.24372-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1694452458844100002 Content-Type: text/plain; charset="utf-8" We currently have a pattern for cleaning up a migration QEMUFile: qemu_mutex_lock(&s->qemu_file_lock); file =3D s->file_name; s->file_name =3D NULL; qemu_mutex_unlock(&s->qemu_file_lock); migration_ioc_unregister_yank_from_file(file); qemu_file_shutdown(file); qemu_fclose(file); This sequence requires some consideration about locking to avoid TOC/TOU bugs and avoid passing NULL into the functions that don't expect it. There's no need to call a shutdown() right before a close() and a shutdown() in another thread being issued as a means to unblock a file should not collide with this close(). Create a wrapper function to make sure the locking is being done properly. Remove the extra shutdown(). Signed-off-by: Fabiano Rosas --- migration/migration.c | 89 ++++++++++++------------------------------- 1 file changed, 25 insertions(+), 64 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 7fec57ad7f..3e94521321 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -217,6 +217,26 @@ MigrationIncomingState *migration_incoming_get_current= (void) return current_incoming; } =20 +static void migration_file_release(QEMUFile **file) +{ + MigrationState *ms =3D migrate_get_current(); + QEMUFile *tmp; + + /* + * Reset the pointer before releasing it to avoid holding the lock + * for too long. + */ + WITH_QEMU_LOCK_GUARD(&ms->qemu_file_lock) { + tmp =3D *file; + *file =3D NULL; + } + + if (tmp) { + migration_ioc_unregister_yank_from_file(tmp); + qemu_fclose(tmp); + } +} + void migration_incoming_transport_cleanup(MigrationIncomingState *mis) { if (mis->socket_address_list) { @@ -1155,8 +1175,6 @@ static void migrate_fd_cleanup(MigrationState *s) qemu_savevm_state_cleanup(); =20 if (s->to_dst_file) { - QEMUFile *tmp; - trace_migrate_fd_cleanup(); qemu_mutex_unlock_iothread(); if (s->migration_thread_running) { @@ -1166,16 +1184,7 @@ static void migrate_fd_cleanup(MigrationState *s) qemu_mutex_lock_iothread(); =20 multifd_save_cleanup(); - qemu_mutex_lock(&s->qemu_file_lock); - tmp =3D s->to_dst_file; - s->to_dst_file =3D NULL; - qemu_mutex_unlock(&s->qemu_file_lock); - /* - * Close the file handle without the lock to make sure the - * critical section won't block for long. - */ - migration_ioc_unregister_yank_from_file(tmp); - qemu_fclose(tmp); + migration_file_release(&s->to_dst_file); } =20 /* @@ -1815,38 +1824,6 @@ static int migrate_handle_rp_resume_ack(MigrationSta= te *s, uint32_t value) return 0; } =20 -/* - * Release ms->rp_state.from_dst_file (and postcopy_qemufile_src if - * existed) in a safe way. - */ -static void migration_release_dst_files(MigrationState *ms) -{ - QEMUFile *file; - - WITH_QEMU_LOCK_GUARD(&ms->qemu_file_lock) { - /* - * Reset the from_dst_file pointer first before releasing it, as we - * can't block within lock section - */ - file =3D ms->rp_state.from_dst_file; - ms->rp_state.from_dst_file =3D NULL; - } - - /* - * Do the same to postcopy fast path socket too if there is. No - * locking needed because this qemufile should only be managed by - * return path thread. - */ - if (ms->postcopy_qemufile_src) { - migration_ioc_unregister_yank_from_file(ms->postcopy_qemufile_src); - qemu_file_shutdown(ms->postcopy_qemufile_src); - qemu_fclose(ms->postcopy_qemufile_src); - ms->postcopy_qemufile_src =3D NULL; - } - - qemu_fclose(file); -} - /* * Handles messages sent on the return path towards the source VM * @@ -2046,7 +2023,8 @@ static int await_return_path_close_on_source(Migratio= nState *ms) ret =3D ms->rp_state.error; ms->rp_state.error =3D false; =20 - migration_release_dst_files(ms); + migration_file_release(&ms->rp_state.from_dst_file); + migration_file_release(&ms->postcopy_qemufile_src); =20 trace_migration_return_path_end_after(ret); return ret; @@ -2502,26 +2480,9 @@ static MigThrError postcopy_pause(MigrationState *s) assert(s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); =20 while (true) { - QEMUFile *file; - - /* - * Current channel is possibly broken. Release it. Note that this= is - * guaranteed even without lock because to_dst_file should only be - * modified by the migration thread. That also guarantees that the - * unregister of yank is safe too without the lock. It should be = safe - * even to be within the qemu_file_lock, but we didn't do that to = avoid - * taking more mutex (yank_lock) within qemu_file_lock. TL;DR: we= make - * the qemu_file_lock critical section as small as possible. - */ + /* Current channel is possibly broken. Release it. */ assert(s->to_dst_file); - migration_ioc_unregister_yank_from_file(s->to_dst_file); - qemu_mutex_lock(&s->qemu_file_lock); - file =3D s->to_dst_file; - s->to_dst_file =3D NULL; - qemu_mutex_unlock(&s->qemu_file_lock); - - qemu_file_shutdown(file); - qemu_fclose(file); + migration_file_release(&s->to_dst_file); =20 /* * We're already pausing, so ignore any errors on the return --=20 2.35.3