From nobody Sat Feb 7 06:39:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1626189928; cv=none; d=zohomail.com; s=zohoarc; b=azwmJ1tMwsHV2s+IOYgCvJQk5vgL26FmXsVTwlFJKpr1kL/bERw825vDEx5gnwSmL2jsk5yxWdSjOMrgmfibZ9QMWoVTURPux7d7X9b1TKRgp8PffNb4iCxoG2lgGcpVbGGIQguRVAppQDi73DMes2qRN5bVP0i4Awa//CY6bug= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1626189928; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=YYeaLDFCI+qBQWjEoAPvCNuevMUuWd8mkDHFtWClOn8=; b=RCxjF/Ezydyt44J5ard0TsoGrszvarCRlK8Bhaff0EM1t7REgTDniyu0X7+FxAl1ZhhtJDaU0ji6plNapFg0EJJSLP+sABcXZAAmHjF0PHjvoCOCAixegQi3eY9SsoNXvLO0eXh8QSE8bMfq0E7QFljZMzhZpKSCxwPOFVK2700= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626189928394830.3878215255517; Tue, 13 Jul 2021 08:25:28 -0700 (PDT) Received: from localhost ([::1]:55968 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3KHn-0004yS-D3 for importer@patchew.org; Tue, 13 Jul 2021 11:25:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52912) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGM-0002Or-Ci for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:23:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30208) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGK-0001nS-TV for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:23:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-182-2Nhl2Y6xMB-RHG5JXaDpuw-1; Tue, 13 Jul 2021 11:23:52 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8B50A80414A; Tue, 13 Jul 2021 15:23:51 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-214.ams2.redhat.com [10.36.114.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD1FC5DF21; Tue, 13 Jul 2021 15:23:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626189836; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YYeaLDFCI+qBQWjEoAPvCNuevMUuWd8mkDHFtWClOn8=; b=OP7l12stm0E3NTYYlJEUq1rF5/pQsSZJjD/SGvRD84ahQmjq1J4fI1bYkbwWBNMu5ZjcVc 2cjnPdXpipsCeNTeZ3YzxhvomXb07oeOTfoUm5x4+38B9Xwi5agPNe+Y0qHKP0U2We74/I pxRSrHgtVdOy0GQtuPZg4GZ+ywvy5mk= X-MC-Unique: 2Nhl2Y6xMB-RHG5JXaDpuw-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, lizhijian@cn.fujitsu.com, lvivier@redhat.com, peterx@redhat.com Subject: [PULL 1/6] migration/rdma: prevent from double free the same mr Date: Tue, 13 Jul 2021 16:23:19 +0100 Message-Id: <20210713152324.217255-2-dgilbert@redhat.com> In-Reply-To: <20210713152324.217255-1-dgilbert@redhat.com> References: <20210713152324.217255-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1626189930243100001 Content-Type: text/plain; charset="utf-8" From: Li Zhijian backtrace: '0x00007ffff5f44ec2 in __ibv_dereg_mr_1_1 (mr=3D0x7fff1007d390) at /home/li= zhijian/rdma-core/libibverbs/verbs.c:478 478 void *addr =3D mr->addr; (gdb) bt #0 0x00007ffff5f44ec2 in __ibv_dereg_mr_1_1 (mr=3D0x7fff1007d390) at /hom= e/lizhijian/rdma-core/libibverbs/verbs.c:478 #1 0x0000555555891fcc in rdma_delete_block (block=3D, rdma= =3D0x7fff38176010) at ../migration/rdma.c:691 #2 qemu_rdma_cleanup (rdma=3D0x7fff38176010) at ../migration/rdma.c:2365 #3 0x00005555558925b0 in qio_channel_rdma_close_rcu (rcu=3D0x555556b8b6c0= ) at ../migration/rdma.c:3073 #4 0x0000555555d652a3 in call_rcu_thread (opaque=3Dopaque@entry=3D0x0) at= ../util/rcu.c:281 #5 0x0000555555d5edf9 in qemu_thread_start (args=3D0x7fffe88bb4d0) at ../= util/qemu-thread-posix.c:541 #6 0x00007ffff54c73f9 in start_thread () at /lib64/libpthread.so.0 #7 0x00007ffff53f3b03 in clone () at /lib64/libc.so.6 ' Signed-off-by: Li Zhijian Message-Id: <20210708144521.1959614-1-lizhijian@cn.fujitsu.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/rdma.c b/migration/rdma.c index 38a099f7ee..5c2d113aa9 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1143,6 +1143,7 @@ static int qemu_rdma_reg_whole_ram_blocks(RDMAContext= *rdma) =20 for (i--; i >=3D 0; i--) { ibv_dereg_mr(local->block[i].mr); + local->block[i].mr =3D NULL; rdma->total_registrations--; } =20 --=20 2.31.1 From nobody Sat Feb 7 06:39:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1626190059; cv=none; d=zohomail.com; s=zohoarc; b=Sq+K0+0MLW8/sJI2r+GTmu5kf2WEN6eTw1WgoNOolZg3cqP9EWeqsJDX0VvjhJ8NejA0XxJTLVs30Sbwk0bnZibce7l2EWsLxwgTsDG1o+6pD4YUhq3op6XNg7rIcq8ZqPV3GPzPo6EQEd1Z4QUM6ZTOfLLTPRwn0rRQ7XC51qk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1626190059; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2KoVCc6jUhG+Bk0Xar0TTM2OqPHLVk686edut6IUc4E=; b=ZxCLqA2qoQFPIubXrrNYSIpqzTVLmnsmSH4Y6mtDUqplOEUeYuA5uj1tsTA+l5yOEnF3Gs2q53Etvjdf5dMDUOjgv1DtnVGjvCIe4geU6mq7gaAUHX2b6rVKw/i454cK1Mh+Q4dk47QH9o9HmwbBLzVm9uj1P3cp3gov2ZWAxdk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626190059068609.9784857203123; Tue, 13 Jul 2021 08:27:39 -0700 (PDT) Received: from localhost ([::1]:34304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3KJu-0001A1-1u for importer@patchew.org; Tue, 13 Jul 2021 11:27:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52980) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGV-0002vJ-D9 for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:38836) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGT-0001tR-QQ for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-55-wAJjnaEDM4qsUvvQWASBZg-1; Tue, 13 Jul 2021 11:24:03 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 455901080A64; Tue, 13 Jul 2021 15:24:02 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-214.ams2.redhat.com [10.36.114.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id D49E55DAA5; Tue, 13 Jul 2021 15:23:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626189845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2KoVCc6jUhG+Bk0Xar0TTM2OqPHLVk686edut6IUc4E=; b=fAJT2uVUFMU0LEBANG0ulvRnsVxCe5HJYSXMrOwwVjUDrK3ZPWn2MtKOG+YvdkGpqjFqdn izXB5n0dmODB4rlAAvJ/rY8iYVIMOwxqLoeRsaeX3vFLxpzrpSdAetnrGn1WBAhPn/D9k0 fGJ+1WmDqcpWFUErv/fppwCha4Dq3g4= X-MC-Unique: wAJjnaEDM4qsUvvQWASBZg-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, lizhijian@cn.fujitsu.com, lvivier@redhat.com, peterx@redhat.com Subject: [PULL 2/6] migration: failover: emit a warning when the card is not fully unplugged Date: Tue, 13 Jul 2021 16:23:20 +0100 Message-Id: <20210713152324.217255-3-dgilbert@redhat.com> In-Reply-To: <20210713152324.217255-1-dgilbert@redhat.com> References: <20210713152324.217255-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1626190059721100001 Content-Type: text/plain; charset="utf-8" From: Laurent Vivier When the migration fails or is canceled we wait the end of the unplug operation to be able to plug it back. But if the unplug operation is never finished we stop to wait and QEMU emits a warning to inform the user. Based-on: 20210629155007.629086-1-lvivier@redhat.com Signed-off-by: Laurent Vivier Message-Id: <20210701131458.112036-1-lvivier@redhat.com> Reviewed-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 5ff7ba9d5c..d717cd089a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3701,6 +3701,10 @@ static void qemu_savevm_wait_unplug(MigrationState *= s, int old_state, while (timeout-- && qemu_savevm_state_guest_unplug_pending()) { qemu_sem_timedwait(&s->wait_unplug_sem, 250); } + if (qemu_savevm_state_guest_unplug_pending()) { + warn_report("migration: partially unplugged device on " + "failure"); + } } =20 migrate_set_state(&s->state, MIGRATION_STATUS_WAIT_UNPLUG, new_sta= te); --=20 2.31.1 From nobody Sat Feb 7 06:39:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1626190091; cv=none; d=zohomail.com; s=zohoarc; b=TOYVwUyNsBkNyVcni6yoZjatf8dG7hmW53Jf2Jo9Va3bWjX06Zfg3kA6wmMJwtjpySHdK7H2etfCXmIlV9KASP9pYInonmRiz6A9shZyRHGbIE+njHyh2g0/rg4orFTT4Gyk6z5AYpaPcB3alQhpwq7SeEFuqKo0G+kWMsxPo9M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1626190091; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0snJlGCX5abLMsB6dvL+8UI1xERqaWyAo489B2AWz0I=; b=mVoe4qRBOFvgCosRrH3WHDMQHZBgS/UR1hMhyF8EkO7A5Gg1l5FEYBCTHPE8ghlL++vk8FyAZ3XmLx5+3Xp9KkfE8R8q1PbBxULpuRXH5Y/kUOG1GT5LKX13KOKaqyUzyP4h8PyBvaA7qtP6WdC+d2CKZDiCwr1qWOoZYZcrX34= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626190091168194.41270825180152; Tue, 13 Jul 2021 08:28:11 -0700 (PDT) Received: from localhost ([::1]:36386 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3KKQ-0002ZH-4e for importer@patchew.org; Tue, 13 Jul 2021 11:28:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53000) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGW-000316-Sv for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45250) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGV-0001uN-7L for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:08 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-574-J4ydi55hPUiRdT5Ulap2CQ-1; Tue, 13 Jul 2021 11:24:04 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B1A12804140; Tue, 13 Jul 2021 15:24:03 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-214.ams2.redhat.com [10.36.114.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8EBDE5D9CA; Tue, 13 Jul 2021 15:24:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626189846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0snJlGCX5abLMsB6dvL+8UI1xERqaWyAo489B2AWz0I=; b=d5W9CAwvUu9+E29xGlvGkM/vBQFUk66x6mv7ATBntrIqFWzokGUS4mEygWqOtY0tPmNqfQ 17kHhj2UIhIAcMcAJKvyLURHg9AJQqRUHPK8uXgAW79G53IgN3Rpp870/44SZL8TaYczNA 1/Slmb+39S8g3kPAxtYEy5FeTdylUa4= X-MC-Unique: J4ydi55hPUiRdT5Ulap2CQ-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, lizhijian@cn.fujitsu.com, lvivier@redhat.com, peterx@redhat.com Subject: [PULL 3/6] migration: Release return path early for paused postcopy Date: Tue, 13 Jul 2021 16:23:21 +0100 Message-Id: <20210713152324.217255-4-dgilbert@redhat.com> In-Reply-To: <20210713152324.217255-1-dgilbert@redhat.com> References: <20210713152324.217255-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1626190091688100001 Content-Type: text/plain; charset="utf-8" From: Peter Xu When postcopy pause triggered, we rely on the migration thread to cleanup t= he to_dst_file handle, and the return path thread to cleanup the from_dst_file handle (which is stored in the local variable "rp"). Within the process, from_dst_file cleanup (qemu_fclose) is postponed until = it's setup again due to a postcopy recovery. It used to work before yank was born; after yank is introduced we rely on t= he refcount of IOC to correctly unregister yank function in channel_close(). = If without the early and on-time release of from_dst_file handle the yank func= tion will be leftover during paused postcopy. Without this patch, below steps (quoted from Xiaohui) could trigger qemu src crash: 1.Boot vm on src host 2.Boot vm on dst host 3.Enable postcopy on src&dst host 4.Load stressapptest in vm and set postcopy speed to 50M 5.Start migration from src to dst host, change into postcopy mode when mi= gration is active. 6.When postcopy is active, down the network card(do migration via this ne= twork) on dst host. 7.Wait untill postcopy is paused on src&dst host. 8.Before up network card, recover migration on dst host, will get error l= ike following. 9.Ignore the error of step 8, go on recovering migration on src host: After step 9, qemu on src host will core dump after some seconds: qemu-kvm: ../util/yank.c:107: yank_unregister_instance: Assertion `QLIST_= EMPTY(&entry->yankfns)' failed. 1.sh: line 38: 44662 Aborted (core dumped) Reported-by: Li Xiaohui Signed-off-by: Peter Xu Message-Id: <20210708190653.252961-2-peterx@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index d717cd089a..38ebc6c1ab 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2818,12 +2818,12 @@ out: * Maybe there is something we can do: it looks like a * network down issue, and we pause for a recovery. */ + qemu_fclose(rp); + ms->rp_state.from_dst_file =3D NULL; + rp =3D NULL; if (postcopy_pause_return_path_thread(ms)) { /* Reload rp, reset the rest */ - if (rp !=3D ms->rp_state.from_dst_file) { - qemu_fclose(rp); - rp =3D ms->rp_state.from_dst_file; - } + rp =3D ms->rp_state.from_dst_file; ms->rp_state.error =3D false; goto retry; } --=20 2.31.1 From nobody Sat Feb 7 06:39:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1626189957; cv=none; d=zohomail.com; s=zohoarc; b=BFatpbKqfNA/wmPmw4T+WLcPYnmu6TPMhrJxSSANKTrM5l0c9R0WPnqJsuFUP8F3OubK/Ga3GhqSVMdhgT2v16fbpiI1Xn8YGoFT+CQxRKEnriV6TfuBTVpfAHJ4y9E2RKRSIqAy9vFu+f5G+GpIgo3WhH40f3JuSQ3dxzMPK4E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1626189957; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CYkl2IofeyeQumPRxRyNIkr/pAekVpIOtEyuFrgduXI=; b=nL47V3xW0Vo3g11vzm0s3wk40s8hvc/dSebzHf4CIazangriHa3NXVvTwdtGKX4sBwHVe0saLbdBYJLh1r8MHP1kcfjZRbZIWbrifPvc5+1aDV27wlFxEG606QKAaxqZRkx5rfulaO+zJ+1UszSm/msmAypZIyeR83W4KsDF8JI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626189957129767.1846647034984; Tue, 13 Jul 2021 08:25:57 -0700 (PDT) Received: from localhost ([::1]:57248 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3KIE-0005pl-VH for importer@patchew.org; Tue, 13 Jul 2021 11:25:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53010) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGX-00034R-NR for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49357) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGW-0001vO-7V for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:09 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-435-ldUujXumO5qKE9XG5ZiJQg-1; Tue, 13 Jul 2021 11:24:06 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2B4B9804141; Tue, 13 Jul 2021 15:24:05 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-214.ams2.redhat.com [10.36.114.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id 092D45D9CA; Tue, 13 Jul 2021 15:24:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626189847; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CYkl2IofeyeQumPRxRyNIkr/pAekVpIOtEyuFrgduXI=; b=AqAFoOzcKHru9nSZeOGPiVmGc6AZ5mgSUqpE8OKVIopaMWAmqIaFaQuzHiTPuwQMYTT3Sz fp1cXsgW4lQ1sQBnPmaYoaB4lgs5ei+J9bMFHfL+HCYS0Z3zlcByf3IvOOPsyt0VPCkaUi bxxQwvmJXw7M/zo1ND36rtbgs6HmM9A= X-MC-Unique: ldUujXumO5qKE9XG5ZiJQg-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, lizhijian@cn.fujitsu.com, lvivier@redhat.com, peterx@redhat.com Subject: [PULL 4/6] migration: Don't do migrate cleanup if during postcopy resume Date: Tue, 13 Jul 2021 16:23:22 +0100 Message-Id: <20210713152324.217255-5-dgilbert@redhat.com> In-Reply-To: <20210713152324.217255-1-dgilbert@redhat.com> References: <20210713152324.217255-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1626189958806100001 Content-Type: text/plain; charset="utf-8" From: Peter Xu Below process could crash qemu with postcopy recovery: 1. (hmp) migrate -d .. 2. (hmp) migrate_start_postcopy 3. [network down, postcopy paused] 4. (hmp) migrate -r $WRONG_PORT when try the recover on an invalid $WRONG_PORT, cleanup_bh will be cle= ared 5. (hmp) migrate -r $RIGHT_PORT [qemu crash on assert(cleanup_bh)] The thing is we shouldn't cleanup if it's postcopy resume; the error is set mostly because the channel is wrong, so we return directly waiting for the = user to retry. migrate_fd_cleanup() should only be called when migration is cancelled or completed. Signed-off-by: Peter Xu Message-Id: <20210708190653.252961-3-peterx@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 38ebc6c1ab..20c48cfff1 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3979,7 +3979,18 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) } if (error_in) { migrate_fd_error(s, error_in); - migrate_fd_cleanup(s); + if (resume) { + /* + * Don't do cleanup for resume if channel is invalid, but only= dump + * the error. We wait for another channel connect from the us= er. + * The error_report still gives HMP user a hint on what failed. + * It's normally done in migrate_fd_cleanup(), but call it here + * explicitly. + */ + error_report_err(error_copy(s->error)); + } else { + migrate_fd_cleanup(s); + } return; } =20 --=20 2.31.1 From nobody Sat Feb 7 06:39:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1626189959; cv=none; d=zohomail.com; s=zohoarc; b=hRv/TClkoIF94hT489lSeBAXxi6vqYwKII3cBihm0FBAheWkG7B+WCUiAf8zf00CNv+1p+IBR922yel2CIsbjbY1N4ZmhdfY3v0yBLsqa6kUNBNs62HxqNY1M3WooNRzeQup7Vgx1E/R9YROkXm7zLfGqlmzllfXVZEf1noFv+E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1626189959; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cLicAbQ6BbohQeCm60hkQbJ59Id12TDfMlMmNWs0oxg=; b=nK85UhYos/K0GpX260w7dccXrlv1oJY8X6wiz6jGbFcgmkppuvLdY78lqxZl5MAL50s9naZToW6oyr3RNmyXHk6aq6NjTxNaop/iWAif9Kve25lN52hKwnqbkH/6mnl7J6Yblte2Lq4xc5ZQ53cW4vPPV38HxEKYS+A+wOiRIzE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626189959410324.0258310949183; Tue, 13 Jul 2021 08:25:59 -0700 (PDT) Received: from localhost ([::1]:57464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3KII-0005yU-9t for importer@patchew.org; Tue, 13 Jul 2021 11:25:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53022) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGZ-000398-3T for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39032) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGX-0001wC-IP for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:10 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-312-ldgtMkcJOSedQI7o5WtDcw-1; Tue, 13 Jul 2021 11:24:07 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 995F2800050; Tue, 13 Jul 2021 15:24:06 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-214.ams2.redhat.com [10.36.114.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id 766AA5D9CA; Tue, 13 Jul 2021 15:24:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626189849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cLicAbQ6BbohQeCm60hkQbJ59Id12TDfMlMmNWs0oxg=; b=gDYTBDhST69yeNxRIFGYmnipH1r/iFdHxtZVHoa3kdP3wNDZ5qNZyQxUFSj/LKzSnpB/ZF ee7vsco3USMHZ/iODvipmmfyO43tz4N16hmArWMc4ZV/8Ba19rG4hEhWvRGgop5gDxdkc2 n2DVC7PxamCW67S9866YOp/3LqVQXI8= X-MC-Unique: ldgtMkcJOSedQI7o5WtDcw-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, lizhijian@cn.fujitsu.com, lvivier@redhat.com, peterx@redhat.com Subject: [PULL 5/6] migration: Clear error at entry of migrate_fd_connect() Date: Tue, 13 Jul 2021 16:23:23 +0100 Message-Id: <20210713152324.217255-6-dgilbert@redhat.com> In-Reply-To: <20210713152324.217255-1-dgilbert@redhat.com> References: <20210713152324.217255-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1626189960940100003 Content-Type: text/plain; charset="utf-8" From: Peter Xu For each "migrate" command, remember to clear the s->error before going on. For one reason, when there's a new error it'll be still remembered; see migrate_set_error() who only sets the error if error=3D=3DNULL. Meanwhile = if a failed migration completes (e.g., postcopy recovered and finished), we shouldn't dump an error when calling migrate_fd_cleanup() at last. Signed-off-by: Peter Xu Message-Id: <20210708190653.252961-4-peterx@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 20c48cfff1..2d306582eb 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1855,6 +1855,15 @@ void migrate_set_error(MigrationState *s, const Erro= r *error) } } =20 +static void migrate_error_free(MigrationState *s) +{ + QEMU_LOCK_GUARD(&s->error_mutex); + if (s->error) { + error_free(s->error); + s->error =3D NULL; + } +} + void migrate_fd_error(MigrationState *s, const Error *error) { trace_migrate_fd_error(error_get_pretty(error)); @@ -3970,6 +3979,13 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) int64_t rate_limit; bool resume =3D s->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED; =20 + /* + * If there's a previous error, free it and prepare for another one. + * Meanwhile if migration completes successfully, there won't have an = error + * dumped when calling migrate_fd_cleanup(). + */ + migrate_error_free(s); + s->expected_downtime =3D s->parameters.downtime_limit; if (resume) { assert(s->cleanup_bh); --=20 2.31.1 From nobody Sat Feb 7 06:39:47 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1626190113; cv=none; d=zohomail.com; s=zohoarc; b=hIWEmNGjL3O05C9+GN7qLuGpBK5DYIySkLxVCB5bX73PDrRpu+ExFw2IyTx6gRaRRtjoe5fWQ1bf0f1VcUDB9RygJ30dXXrKl+sI8+SVwQO+C2byv8B3Drf0DACjL/GeIM8ifDoItimNKhMZRsDYqQ6aiWIOWSLhWg+YBJEEF2o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1626190113; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DJ2w3lhbVOHQjZ618XP2D/ex2FdjvwV9xOmFoIVcL7E=; b=k6EgDK7Q8QAxJo9IBxOSg6Z/G6XAAFvPtEnHNxkuKFSbNUw/bZ25XtKoqsFYkp1on0c8yCYvkyRbDvOTRI+TUVWUVrnEGcLhWPvnQ18kVVbE5252xIXEPF7ZSCoPkXnsoOjXRjqspH2Cgf1/vpiF5j45w03dzgrbLdn2n4GNG4U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626190113681716.149008341679; Tue, 13 Jul 2021 08:28:33 -0700 (PDT) Received: from localhost ([::1]:37662 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3KKm-0003QK-NK for importer@patchew.org; Tue, 13 Jul 2021 11:28:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53040) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGa-0003GN-UA for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43054) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3KGZ-0001yj-6L for qemu-devel@nongnu.org; Tue, 13 Jul 2021 11:24:12 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-236-8wlGzym6P72wtePpqKv1Ng-1; Tue, 13 Jul 2021 11:24:09 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 352CF801107; Tue, 13 Jul 2021 15:24:08 +0000 (UTC) Received: from dgilbert-t580.localhost (ovpn-114-214.ams2.redhat.com [10.36.114.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4FA15D9CA; Tue, 13 Jul 2021 15:24:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626189850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DJ2w3lhbVOHQjZ618XP2D/ex2FdjvwV9xOmFoIVcL7E=; b=Mel/dy3ckGdNRdqY4RRlsj6CnGVB/BOvST3T91NmQUx5Z2VQG2MfjwUYJdx7zzPS8DwVe1 TgdL84sdeIT7b6AvvSFNJHVyKEsNsxjbutN1XXe0s6RRx9BcU5JYYgV0R7DW6JYjKuPe6e DIntTTRXqwwGFh0CzSqJtNVZbzQlVjg= X-MC-Unique: 8wlGzym6P72wtePpqKv1Ng-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, lizhijian@cn.fujitsu.com, lvivier@redhat.com, peterx@redhat.com Subject: [PULL 6/6] migration: Move bitmap_mutex out of migration_bitmap_clear_dirty() Date: Tue, 13 Jul 2021 16:23:24 +0100 Message-Id: <20210713152324.217255-7-dgilbert@redhat.com> In-Reply-To: <20210713152324.217255-1-dgilbert@redhat.com> References: <20210713152324.217255-1-dgilbert@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: quintela@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1626190115064100001 Content-Type: text/plain; charset="utf-8" From: Peter Xu Taking the mutex every time for each dirty bit to clear is too slow, especi= ally we'll take/release even if the dirty bit is cleared. So far it's only used= to sync with special cases with qemu_guest_free_page_hint() against migration thread, nothing really that serious yet. Let's move the lock to be upper. There're two callers of migration_bitmap_clear_dirty(). For migration, move it into ram_save_iterate(). With the help of MAX_WAIT logic, we'll only run ram_save_iterate() for no more than 50ms-ish time, so taking the lock once there at the entry. It also means any call sites to qemu_guest_free_page_hint() can be delayed; but it should be very rare, only during migration, and I don't see a problem with it. For COLO, move it up to colo_flush_ram_cache(). I think COLO forgot to take that lock even when calling ramblock_sync_dirty_bitmap(), where another exa= mple is migration_bitmap_sync() who took it right. So let the mutex cover both = the ramblock_sync_dirty_bitmap() and migration_bitmap_clear_dirty() calls. It's even possible to drop the lock so we use atomic operations upon rb->bm= ap and the variable migration_dirty_pages. I didn't do it just to still be sa= fe, also not predictable whether the frequent atomic ops could bring overhead t= oo e.g. on huge vms when it happens very often. When that really comes, we can keep a local counter and periodically call atomic ops. Keep it simple for = now. Cc: Wei Wang Cc: David Hildenbrand Cc: Hailiang Zhang Cc: Dr. David Alan Gilbert Cc: Juan Quintela Cc: Leonardo Bras Soares Passos Signed-off-by: Peter Xu Message-Id: <20210630200805.280905-1-peterx@redhat.com> Reviewed-by: Wei Wang Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 88ff34f574..b5fc454b2f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -795,8 +795,6 @@ static inline bool migration_bitmap_clear_dirty(RAMStat= e *rs, { bool ret; =20 - QEMU_LOCK_GUARD(&rs->bitmap_mutex); - /* * Clear dirty bitmap if needed. This _must_ be called before we * send any of the page in the chunk because we need to make sure @@ -2834,6 +2832,14 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) goto out; } =20 + /* + * We'll take this lock a little bit long, but it's okay for two reaso= ns. + * Firstly, the only possible other thread to take it is who calls + * qemu_guest_free_page_hint(), which should be rare; secondly, see + * MAX_WAIT (if curious, further see commit 4508bd9ed8053ce) below, wh= ich + * guarantees that we'll at least released it in a regular basis. + */ + qemu_mutex_lock(&rs->bitmap_mutex); WITH_RCU_READ_LOCK_GUARD() { if (ram_list.version !=3D rs->last_version) { ram_state_reset(rs); @@ -2893,6 +2899,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) i++; } } + qemu_mutex_unlock(&rs->bitmap_mutex); =20 /* * Must occur before EOS (or any QEMUFile operation) @@ -3682,6 +3689,7 @@ void colo_flush_ram_cache(void) unsigned long offset =3D 0; =20 memory_global_dirty_log_sync(); + qemu_mutex_lock(&ram_state->bitmap_mutex); WITH_RCU_READ_LOCK_GUARD() { RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(ram_state, block); @@ -3710,6 +3718,7 @@ void colo_flush_ram_cache(void) } } trace_colo_flush_ram_cache_end(); + qemu_mutex_unlock(&ram_state->bitmap_mutex); } =20 /** --=20 2.31.1