From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161222; cv=none; d=zohomail.com; s=zohoarc; b=Vqw5zZeQ6PnyyZW1ii3zKODBzrUSiIo5BuMayx6T/edRlGM07nLVz406uaPlSDcA68HLpQv+25VdhNMh2lUeNvk2hSfgLaL921Dq0VO+ARH3sEOEemqRYGhM9tgn1t3hy0kM3CO7Hm4+66XmuUQTZDR5buK41CeG3RPAhjo58Yg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161222; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=dmxRfIQ+sqvYYFl9da01pUfgcatf1F566P8Ys2epb/o=; b=HTxFI47RoGaHagO+aGhfYdNqAbbBDluk3oESyGfxH6uONgJ/YrwR4De9G/8eF1DDJELucFViFhSVqL4abIO6jca6dC5MKcrbdbVkaRXmBSlLXpHcFD1H76GQSImpU/EixhjM6KLwhwr8t4EQTKZAOy5Gb7WJX6HnjEdQTAkSXCk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161222574716.7579697079569; Wed, 22 Oct 2025 12:27:02 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTh-0002tL-Sw; Wed, 22 Oct 2025 15:26:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTc-0002sR-At for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTV-0001QT-Pr for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:27 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-659-vrTuVHpUOButF8jZzEE1ZA-1; Wed, 22 Oct 2025 15:26:18 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e895a6ca73so15858051cf.2 for ; Wed, 22 Oct 2025 12:26:18 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161179; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dmxRfIQ+sqvYYFl9da01pUfgcatf1F566P8Ys2epb/o=; b=Erk8cCFxvIIW/vi1RWNmBmT/6z5eFfXYr3Vn5Xk8C69nlP9+/6c7Vejkkuxwc5ziAOfKU5 v/Z6q15SoItpHW0aGzVbLJGPw9U5Sr4zumxiuPz9PIpqvaW/KCTkKfcW0ADIGlChlmR/EL 8gs3/TXNQ3lUXIqiSJDjFjgJfbMlgEQ= X-MC-Unique: vrTuVHpUOButF8jZzEE1ZA-1 X-Mimecast-MFC-AGG-ID: vrTuVHpUOButF8jZzEE1ZA_1761161178 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161177; x=1761765977; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dmxRfIQ+sqvYYFl9da01pUfgcatf1F566P8Ys2epb/o=; b=dyfdrN1kD346M0Hh5TTtq/yt3Bc1IiUeIrbvGnG/utrDKrT2gjJ+7/4HG7vxyVL/4N lbWn7qjZgal5B9HAI95DlG0a6/JAj6cEXc4Ti/c+pilPoD1cbCc2khlQLnbNE0OVGb/r MVZTVF6tCjwg0NP06/y3A8IU8ZG42YKn5B9ao/jznIKWVXcW1nYpZ8wkdCT63h9KA5sX lVSEAMfNXguZ+ml/LlVGRvxQOKrMp23EEZxU8spd8l/Ob2Uv4F9gIz3xDv1hZIcWHNEe 2npBYLKR31Z0jt50QFJubMuKe50mGqtkA1+Y8y118Byd4kZBSoVAwFk2h1dZHi1wmRLB latg== X-Gm-Message-State: AOJu0YzDJxwYkhei2FUiOTpmRiT03mqUGkUsEFS2g4T6vTjntFxcpnLh OtID5Htzr45ZjSqHIgXb4aNVN3tNZaYUSG6FDowGbX9HZqBNjLdFLywV36cevhuGE9LP69CBEsC 2Y5FnVVQGSISdkMpRy71gXBgkzeTnzw1workdcD9XTSiU7Eo4yVaTE6ZM5GdRALkvk2moigwqGx caqbgTCeLsxFbIevGetTGgYTUkSo+Xm7x+3i04pA== X-Gm-Gg: ASbGncuHd9FAM0rdgn0C2yPMvxuUpRs4o5YItiMcwihe6ZfEOxVJMy+syeo6XMdR5Ga MYVCTvYEiKacb/Vi+ua9HvPUgZqXKEgi3G5AYCn/wqcsc1EXUWgWP2QHypSuiVbjAyGknYQRHAy J7OqBrYAZMoNq1Y0QMh0v+G0ErK29QZK3UK0bkP4SULZVlCvJhJ6HwZXw3C82MZ+zQGJGE4sP+o qxNrDCisl+BR10veW/yTWB3g1Gg/XefPz1+FwBdqvJuuuDCkRwknqxALwAAu8/IpG/yVILjzArc 4LdxuwzSWcc6TOEiZkj4kRYvyYkClnZZfqNxIPtYer+fTgZoNCFA35Z2EGQcik3T X-Received: by 2002:a05:622a:11c4:b0:4e5:8352:9db2 with SMTP id d75a77b69052e-4e89d3f3eb9mr266588191cf.74.1761161177345; Wed, 22 Oct 2025 12:26:17 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFRioOfkw7DIT0mYQwuJiZgOJNMQAWRhYH4cDScwSXfN/lL1J4Ga6snRnj0nxPIwKo2v4AX0w== X-Received: by 2002:a05:622a:11c4:b0:4e5:8352:9db2 with SMTP id d75a77b69052e-4e89d3f3eb9mr266587681cf.74.1761161176698; Wed, 22 Oct 2025 12:26:16 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 01/13] io: Add qio_channel_wait_cond() helper Date: Wed, 22 Oct 2025 15:26:00 -0400 Message-ID: <20251022192612.2737648-2-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161239325158500 Add the helper to wait for QIO channel's IO availability in any context (coroutine, or non-coroutine). Use it tree-wide for three occurences. Cc: Daniel P. Berrang=C3=A9 Signed-off-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/io/channel.h | 15 +++++++++++++++ io/channel.c | 21 +++++++++++---------- migration/qemu-file.c | 6 +----- 3 files changed, 27 insertions(+), 15 deletions(-) diff --git a/include/io/channel.h b/include/io/channel.h index 0f25ae0069..d6d5bf2b5f 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -871,6 +871,21 @@ void qio_channel_wake_read(QIOChannel *ioc); void qio_channel_wait(QIOChannel *ioc, GIOCondition condition); =20 +/** + * qio_channel_wait_cond: + * @ioc: the channel object + * @condition: the I/O condition to wait for + * + * Block execution from the current thread until + * the condition indicated by @condition becomes + * available. + * + * This will work with/without a coroutine context, by automatically select + * the proper API to wait. + */ +void qio_channel_wait_cond(QIOChannel *ioc, + GIOCondition condition); + /** * qio_channel_set_aio_fd_handler: * @ioc: the channel object diff --git a/io/channel.c b/io/channel.c index 852e684938..b18fc346ff 100644 --- a/io/channel.c +++ b/io/channel.c @@ -159,11 +159,7 @@ int coroutine_mixed_fn qio_channel_readv_full_all_eof(= QIOChannel *ioc, len =3D qio_channel_readv_full(ioc, local_iov, nlocal_iov, local_f= ds, local_nfds, flags, errp); if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { - if (qemu_in_coroutine()) { - qio_channel_yield(ioc, G_IO_IN); - } else { - qio_channel_wait(ioc, G_IO_IN); - } + qio_channel_wait_cond(ioc, G_IO_IN); continue; } =20 @@ -268,11 +264,7 @@ int coroutine_mixed_fn qio_channel_writev_full_all(QIO= Channel *ioc, nfds, flags, errp); =20 if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { - if (qemu_in_coroutine()) { - qio_channel_yield(ioc, G_IO_OUT); - } else { - qio_channel_wait(ioc, G_IO_OUT); - } + qio_channel_wait_cond(ioc, G_IO_OUT); continue; } if (len < 0) { @@ -774,6 +766,15 @@ void qio_channel_wait(QIOChannel *ioc, g_main_context_unref(ctxt); } =20 +void qio_channel_wait_cond(QIOChannel *ioc, + GIOCondition condition) +{ + if (qemu_in_coroutine()) { + qio_channel_yield(ioc, condition); + } else { + qio_channel_wait(ioc, condition); + } +} =20 static void qio_channel_finalize(Object *obj) { diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 2d4ce174a5..4b5a409a80 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -343,11 +343,7 @@ static ssize_t coroutine_mixed_fn qemu_fill_buffer(QEM= UFile *f) QIO_CHANNEL_READ_FLAG_FD_PRESERVE_BLO= CKING, &local_error); if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { - if (qemu_in_coroutine()) { - qio_channel_yield(f->ioc, G_IO_IN); - } else { - qio_channel_wait(f->ioc, G_IO_IN); - } + qio_channel_wait_cond(f->ioc, G_IO_IN); } } while (len =3D=3D QIO_CHANNEL_ERR_BLOCK); =20 --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161298; cv=none; d=zohomail.com; s=zohoarc; b=YQ2r40gMdZzvtldBubawzPCmX+Zlt4B8nejdIBk4VS8Ei8P9siy1HdDifQLKFbMjStJEqfRr4QZ2LePcaF6jO7U6z/LxszwPgMzOlAAlcYxT/+aJ29OnP7S2W0W1Ly94JrVvmSnZZ8Xlsi+vLg1jXata9+HZEOL8BBdcIb8Q1MQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161298; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=61OlonQ1mj4eAWEnn8sA5zmXZ1Lvykf4bLCwkotKe7k=; b=RlS7zynaIuAnBs+JQkFgG0cCgwgIJqEfn0ic8Z2h6zpegTe/cL0tixqkT/twAy/no0bU4E76cwocawzQIJqoh/pHvJ/1qR+6s+3gYL1sQeMny77jnGpprNWRyxiQsPCw2lHk3MtdFWQO8ylMJHiybp5RntsUtj7rb/ZPMHGt1rk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161298414564.0251753472237; Wed, 22 Oct 2025 12:28:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTt-0002ue-94; Wed, 22 Oct 2025 15:26:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTc-0002sQ-AQ for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTZ-0001R0-9z for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:27 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-592-BAukl99nPHCukyQQcdU8OQ-1; Wed, 22 Oct 2025 15:26:20 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e8916e8d4aso67858841cf.2 for ; Wed, 22 Oct 2025 12:26:20 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161181; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=61OlonQ1mj4eAWEnn8sA5zmXZ1Lvykf4bLCwkotKe7k=; b=Q22eI8f4SqdTT9MGAXffbAThCM6Tr67epp8ttgGPxVybzaHtUsAD/p3VdYjbNiwbPXDNtc AR/Olr4Hm8PpdcPhMaUe8veGXUF4EYLxuClQZkrDF/8XH/HFgjxei6bLXsnahuOZTBFRWn D0pl45VuIg3YdzDuNcbkpgNabsDAXW0= X-MC-Unique: BAukl99nPHCukyQQcdU8OQ-1 X-Mimecast-MFC-AGG-ID: BAukl99nPHCukyQQcdU8OQ_1761161179 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161179; x=1761765979; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=61OlonQ1mj4eAWEnn8sA5zmXZ1Lvykf4bLCwkotKe7k=; b=IZ6m1L1N6RHNAvg7U+6yRuBRtzMBnvZvyOwIdkK0hPiEWFSe+4FhqY3pw6IKl8Tci3 Vy0inht+LLRHjD+Z6WUW+me0IpfhiObwhhF2nNnajVGh3p7J8Es+nq87WhBlYC2Jv8qJ lmRgsBVo8KKkPze0UhNTv8ozWsBJG27nA5THHzVUJC/uA5sQeHVj9yCkr8SWPMYdIh8z 93YPht9XfHV06dLr98BvB0NncsgRylQ2KRhrdYtBVHhOMv70qtKHbB+kJX653pzDkuub nSeRyEfDoVWWQr0ys5uOSP0YSfVHeRT02AC00WFzNj76VildT8NAcr6OwWQqyjOhk3Jr hTNQ== X-Gm-Message-State: AOJu0YyqsIxHiYV8E8B0t+F+1tRfRwVr28KgwHKuOzLOnQdXfeDQ2Csr jGaQJXst0KsiT7CJD/V1eMgco1hTNFVqRrVqXQ61WSkvDyKliMOsXnrIoeIaN9FjC+NAsZXBHe8 Arso5Ab/lvEk46bMU0o6sWsXvTb69OI42UHgk5xduk9s8pwWE+0IE5RiwxuCPXyYisYrFdWJ3et d7afMp36Q2ax4LLD9aH7LEzc+i4+XiuBviBtw5zQ== X-Gm-Gg: ASbGncsKSYzwwtpoV567v/easr273QNchvuOrw1/nXrM3RpOEPLhnOgFDB8VF1IvrGa bOD3Dl9MN+xFsThJLNHy+F+4q4+fUY1/CdW9QZjAuCTFkCyqXbwqD/+F6PYl1iVsLsP+VpHR++S kosl6pMh1hxulBTTNmaBYbEBVbGWO7F26xMNb2iyVedYIGtZyR4mOQtQykXoN8vh7w6lSSKhCzg +SuKjf+8xA4PDmHzppeVcIDELwZYmnCcPjoSoNob8pF/W+TY85D+rOoyf8zhKut7ljULYu1TJdx 7Qeq7D6vfIS6cfg+ym9DxC4yspMan+aCo8vS4vYybn9z9pUOU5FZxG9l+WT4pGi1 X-Received: by 2002:a05:622a:1a9f:b0:4e8:9809:82b5 with SMTP id d75a77b69052e-4e89d1d994dmr236809231cf.5.1761161179073; Wed, 22 Oct 2025 12:26:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEatqoyacPnKNZo7Jnm+fR9tBJpJFoFqlKY/9GAgG99O+3Zgv/7SpB6pt3hl+BFskr/hQEg5g== X-Received: by 2002:a05:622a:1a9f:b0:4e8:9809:82b5 with SMTP id d75a77b69052e-4e89d1d994dmr236808901cf.5.1761161178590; Wed, 22 Oct 2025 12:26:18 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 02/13] migration: Properly wait on G_IO_IN when peeking messages Date: Wed, 22 Oct 2025 15:26:01 -0400 Message-ID: <20251022192612.2737648-3-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161301662158500 Content-Type: text/plain; charset="utf-8" migration_channel_read_peek() used to do explicit waits of a short period when peeking message needs retry. Replace it with explicit polls on the io channel, exactly like what qemu_fill_buffer() does. Signed-off-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Reviewed-by: Vladimir Sementsov-Ogievskiy --- migration/channel.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/migration/channel.c b/migration/channel.c index a547b1fbfe..462cc183e1 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -135,12 +135,7 @@ int migration_channel_read_peek(QIOChannel *ioc, break; } =20 - /* 1ms sleep. */ - if (qemu_in_coroutine()) { - qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 1000000); - } else { - g_usleep(1000); - } + qio_channel_wait_cond(ioc, G_IO_IN); } =20 return 0; --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161233; cv=none; d=zohomail.com; s=zohoarc; b=AJRJ9EdlM6NttcH9g+i6YVs75xKB+kEzDTmOt9QqcDGq0OTiuoAkdZ70ToIk2q7W5IvBphuTL7R9s/Mnc/d3TIJdA/pPoBPZ2gLef80ibNLdiF7gBGV9VF3RNLli1T8ssh3NVovysmeCyDaOygCZfzFzRFMdUg0opWdHjqUt2Yg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161233; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=o0Y12EUm6tzSrzap8kCRMoEI4B4aZdSYBje8g8u1lmQ=; b=YkM0ROs7OHOTgJoUoYACWAPuGGF328nL5rnoPUoVP8Yo2wbc9xqd3g0zTNShZ3hYqKcHPprlSlSocxKSg1OcA6UIr6mUhIwPw87sTP/T2xAW85Rb+GaJO0A202ESqpVGqRKAtxnTxR3WBySWh+Z0T0TZCGz/5RZPw5TIVyk3fzI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17611612330881002.7752183787553; Wed, 22 Oct 2025 12:27:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTu-0002vQ-GD; Wed, 22 Oct 2025 15:26:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTd-0002se-94 for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTZ-0001SM-HM for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:28 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-580-qDXt4KGaOy-CnY70OB4oiQ-1; Wed, 22 Oct 2025 15:26:21 -0400 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-4e89265668fso42312741cf.3 for ; Wed, 22 Oct 2025 12:26:21 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o0Y12EUm6tzSrzap8kCRMoEI4B4aZdSYBje8g8u1lmQ=; b=a5gfZ0nC3C2J1hukHP553Mv3ExwzTQZROr9rcsfR5Lxy3kwMDNeyYYuvCHwTX5lw6McL6H Rux7gtOxLhqj9WVO3/t3X/J5OzTgDhOlaCmH8wr3ZqS3klaBR2Scy9B8O1P2D0cJQfKS6Q vuwkqUcfEOO7vLqbSzMBW9NLJcOPQAo= X-MC-Unique: qDXt4KGaOy-CnY70OB4oiQ-1 X-Mimecast-MFC-AGG-ID: qDXt4KGaOy-CnY70OB4oiQ_1761161181 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161181; x=1761765981; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o0Y12EUm6tzSrzap8kCRMoEI4B4aZdSYBje8g8u1lmQ=; b=kTzBYodJrx1Tu6MP7rEoKdiv8TKywcRcQ3mglulqUAd61Ywu5l/EeLTBg1Bu0nj1S8 1fvYlWlQKaQ2v3Ty834shBfFhMDLG9v5dsuzT7TK2XLq2cLGd1sEypukQZglW12m00RS U3k5snFyk1E46iSAt7XkXGlFh25B1lSecxPTm0CiDFE1XA9WzfHoSwm6geQPK4bquWcP Cean8DO9UOcozf881w7Y7wvWWv0JkRNa6GAA2b8OW9pStEaOcGATNiua4GxL4yahlBZQ iFF7PcoYo1JGf0n7z2p+Hn+oTDC6WGoh9BNBcsRLtyVkOin91dCGRIGVT/aNJdMruX9K DCoA== X-Gm-Message-State: AOJu0YylDud3IJTSQw6O0zOlk5tLMohD+1cey2EmveA/hdRRKIdRsD0t /74lC9BI+os7qZnJEJnbxMlb05RTA2/2juuCfEDxzN8sh6srk1nC2wVw/uiSrNDCW0q7Yssvht2 crukSNDLLAMhPf/mFxXI3L3frm+ht1/n4qFWuSiNqND4Fl68k5JKWMucgKpuXQfk4cQMqQVbgQ9 X6E1bTSBkwEWBhzZ8bC0OMu5fsiKQRQW2mQN325A== X-Gm-Gg: ASbGncutxePVDDqKeT1pWKwpASEiDF7uqcInCdDvb5hiiIQjEmYRGfdczQuKwaSi7PD QgLVkD5QI1mjjh3HiOtfLA4u+ZuAfOJvdF1M01RHlYgmIRkQR/Foe+b26lL78inbYoq+kcdxrOw a16KP82b6Q78tlKxppuOCUPsSANwyCYWg5OZLwMzlH9KBbHHszSCS+/MsGVxURaAF9w9hc7irG+ U76jCec9CABfgA0QxfWqtaHplV3epY1wfYnA+/VsHsATTnCOulxYKMPc5IrpWKSliA2Wb8BX8fV y7QN96aEpWTUiVTkVIfuYD2hrlYogqm/P8qlPO5RO1wGoh17ZLorKsZmLiqptoyJ X-Received: by 2002:ac8:5d87:0:b0:4d8:372b:e16a with SMTP id d75a77b69052e-4e89d1f48d9mr304194551cf.4.1761161180725; Wed, 22 Oct 2025 12:26:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFDVD5J3AIWUWrtFeF5As8P8/yQlMPmUx0PwmtfSfmkYfW5IVIE/Erd4RPNFMqFTNJdGsvTew== X-Received: by 2002:ac8:5d87:0:b0:4d8:372b:e16a with SMTP id d75a77b69052e-4e89d1f48d9mr304194051cf.4.1761161180209; Wed, 22 Oct 2025 12:26:20 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin , Lidong Chen Subject: [PATCH 03/13] migration/rdma: Fix wrong context in qio_channel_rdma_shutdown() Date: Wed, 22 Oct 2025 15:26:02 -0400 Message-ID: <20251022192612.2737648-4-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161234822154100 Content-Type: text/plain; charset="utf-8" The rdmaout should be a cache of rioc->rdmaout, not rioc->rdmain. Cc: Lidong Chen Fixes: 54db882f07 ("migration: implement the shutdown for RDMA QIOChannel") Reviewed-by: Zhijian Li (Fujitsu) Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/rdma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index 2d839fce6c..e6837184c8 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2986,7 +2986,7 @@ qio_channel_rdma_shutdown(QIOChannel *ioc, RCU_READ_LOCK_GUARD(); =20 rdmain =3D qatomic_rcu_read(&rioc->rdmain); - rdmaout =3D qatomic_rcu_read(&rioc->rdmain); + rdmaout =3D qatomic_rcu_read(&rioc->rdmaout); =20 switch (how) { case QIO_CHANNEL_SHUTDOWN_READ: --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161284; cv=none; d=zohomail.com; s=zohoarc; b=B2tFDHdqqgd4qO+b2TZATH5kLfzo1bqaoV4Qor72Jj27x+WLyTJteI6hwtGQxDpZPcukY8daDy3j96qLM5LCwow2sKB74R0IaEX4CR98z0AZXdb6Hrh8k1hOmgfVLIIR63fXX96pmhMu/rq39bQFPGX6cNLv1oRkhE76jbv9m+A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161284; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kpjAE4slBEhS+3yJmkKxJqSXI1XYW1Z8hdORf1mQ/rc=; b=DGAhsK6T4VTI4ND4uKXEsBfMZc82R1AdAX65WrNXIjR3VyNlhlB5Q7gwa5QSlgx1rbMGo6aj1X72Is/bVZh2+h6GWK5zvSQ+wQCyiiaMxx8alvk7SoVc5VpAoJjY2EwMiQjMCbh9zaA+VWBF0mR4yY0WANZcunNnf2sOAKzI//A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161284213692.6210969749332; Wed, 22 Oct 2025 12:28:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTv-0002vU-9Q; Wed, 22 Oct 2025 15:26:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTd-0002sf-IL for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTZ-0001UZ-MS for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:29 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-347-EcmBHQtTNCKh7FxOi0o6fQ-1; Wed, 22 Oct 2025 15:26:23 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e8984d8833so65872801cf.0 for ; Wed, 22 Oct 2025 12:26:23 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161184; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kpjAE4slBEhS+3yJmkKxJqSXI1XYW1Z8hdORf1mQ/rc=; b=bDMGOwCUeG4l1Kk8QfbkmWscE1P7w72siewj0QxY8ywoNAhRSI4Wsc9hYMUEImRk3Dh+PO hOpmPvwlNT0mCjNdMMnQEqUnvrjTbjj6byTJaQREsyJaYawobLpohgl8hBJiZReew1QElW 9aIFIEu1XcXOYknZf1eIff9tfpShz1k= X-MC-Unique: EcmBHQtTNCKh7FxOi0o6fQ-1 X-Mimecast-MFC-AGG-ID: EcmBHQtTNCKh7FxOi0o6fQ_1761161183 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161182; x=1761765982; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kpjAE4slBEhS+3yJmkKxJqSXI1XYW1Z8hdORf1mQ/rc=; b=kJesnu8IsyKl9RAb1QyyoNp749QPFnipFaI8Tg8o13ZjI2I+1wcUfMbDuX5LmT8D+v qnEyL5BaEvPNS8TXH3oyNxU6kcC5rNstPddXtvr+TivnRqIrkD7dmF5PESUrgMW+/l8u sRZXRj+JfiwhwrnS1u/cZhFU8JKER4JlUzq2YzN6ObVMxuIM4y7jUNCkWMt9SeIMordI QpgQbB2iLkUVWvZ8HfWyglFX1E+n7jo5zRqoG8hfbnQ0m1h3GRgp7viCUlTHXHvPP/KX iiHMFyvihtNENqZQ0kORTpX1Vawy2o5xvQXDw/4YB43+OZu8mzCDW2De7fusgb6IAHYx WSsg== X-Gm-Message-State: AOJu0YyWfQ2ETF6TTXOtlT95pUvF0/LwV7rbDKWXljwL5sPvnHcpPSqI hMC2TaI1Rq2tjx+JrDhJ06LvA1VH0YxrItStbTVdHFzsNQy/uc9z3PRk2aex7YlcI4R0IhiSXbc /d9zvhh/CXJSvFRdqWczKVJR4a1RbJa/oEFGGFdvImWVm34v4VrZ1c5ytqm+Vm0m08AFM/qvrcr 9yBSUr3cRCklRmTeCokV+zquWxtqUjStBtER/mEw== X-Gm-Gg: ASbGncv3Dsag47+GLdpy6z33WKSGOjgCjL4IDj6WKX68XIE/0cksMS75loK9GuajkYr bA+sUJt+BsDMdr6HXYrVSZBJ/V6UkXnOpVToRwV7BC2fv1hbz4LoBi3gGswPx913bgpxtbXwu/M pVaBEieIsPwIbnrbfiB4MNCzUcKVucw+18CtRUMW2Pl2yaCwtY6ayF/o4dconY7yXk9QMG0D2ur iZGvSVQilJftrWZjeQ8XI8N2rj9gE5s8jB1+dB6ke6SmkzHELG5HSBPlwN4weNqJfxs7/+xgeWJ Run0IYcHrXS+3dTyQyy0Qf5JXIeVSpYexqR/CKEKMxl5aKteJeNefwggKazr2sVS X-Received: by 2002:a05:622a:40e:b0:4e8:9a7d:90ec with SMTP id d75a77b69052e-4e89d33641dmr302151711cf.42.1761161182307; Wed, 22 Oct 2025 12:26:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGOl+qm2DDhLeCPWyZmUZYzH2UBjRR/AyJkHtutmjxA6whe24rUS2klBRrSxzQJGTwAYdnsDw== X-Received: by 2002:a05:622a:40e:b0:4e8:9a7d:90ec with SMTP id d75a77b69052e-4e89d33641dmr302151281cf.42.1761161181793; Wed, 22 Oct 2025 12:26:21 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 04/13] migration/rdma: Allow qemu_rdma_wait_comp_channel work with thread Date: Wed, 22 Oct 2025 15:26:03 -0400 Message-ID: <20251022192612.2737648-5-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161287490158500 Content-Type: text/plain; charset="utf-8" It's almost there, except that currently it relies on a global flag showing that it's in incoming migration. Change it to detect coroutine instead. Then we achieved two things in one shot: - Drop migration_started_on_destination, which is not needed anymore, and - It starts to work in a thread when loadvm using RDMA Signed-off-by: Peter Xu Reviewed-by: Li Zhijian Reviewed-by: Vladimir Sementsov-Ogievskiy --- migration/rdma.c | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index e6837184c8..13dd391c14 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -357,13 +357,6 @@ typedef struct RDMAContext { /* Index of the next RAMBlock received during block registration */ unsigned int next_src_index; =20 - /* - * Migration on *destination* started. - * Then use coroutine yield function. - * Source runs in a thread, so we don't care. - */ - int migration_started_on_destination; - int total_registrations; int total_writes; =20 @@ -1352,12 +1345,7 @@ static int qemu_rdma_wait_comp_channel(RDMAContext *= rdma, { struct rdma_cm_event *cm_event; =20 - /* - * Coroutine doesn't start until migration_fd_process_incoming() - * so don't yield unless we know we're running inside of a coroutine. - */ - if (rdma->migration_started_on_destination && - migration_incoming_get_current()->state =3D=3D MIGRATION_STATUS_AC= TIVE) { + if (qemu_in_coroutine()) { yield_until_fd_readable(comp_channel->fd); } else { /* This is the source side, we're in a separate thread @@ -3884,7 +3872,6 @@ static void rdma_accept_incoming_migration(void *opaq= ue) return; } =20 - rdma->migration_started_on_destination =3D 1; migration_fd_process_incoming(f); } =20 --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161278; cv=none; d=zohomail.com; s=zohoarc; b=MKTWtOJjzdH3AeVaBe0+kiRS8kjHFK/t2KRyTE0VlI9gL0E6HMJrnloBDM6JDEVyQb2dA26RLOcz50pTb/tT15To/93NAehE/y4sA9EIl524he6GLcnJphQIWzTwBVfMC9ujK1j+ui0HK7mvIg9j7LDhHv8TWyajojy+vP8RKeY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161278; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SWalDO9UGdGD6EJXyoisbCsd9GWW6yspcoUh/J0vPH0=; b=PAdIq4Jxt7kvb3sId6+rI9Dl3mk9Q+w3Bdz35uIGA80niz5H0W6TT8D7tGsu6f2Cf6dg0NAI9Y6bqcURTkb6jdPx7xtRaq7F8aWXrf7wjcXSxnSJ9XPD6QCpgHPvG/ACzdf76R93ViDkQ7LzYDRsyQZm8RFKYwNCsr9Lj6twnMU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161278571557.1130320044649; Wed, 22 Oct 2025 12:27:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTt-0002ud-H1; Wed, 22 Oct 2025 15:26:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTf-0002su-Ru for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTb-0001WX-Rq for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:29 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-519-7k1KVtluMhGIudvI83ItOg-1; Wed, 22 Oct 2025 15:26:25 -0400 Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-4e8a89c9750so49670071cf.0 for ; Wed, 22 Oct 2025 12:26:25 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161186; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SWalDO9UGdGD6EJXyoisbCsd9GWW6yspcoUh/J0vPH0=; b=U7Lty0a7XWoqhs9b0xnD8DTThYHYSiOEgYZi92s5PfpRclBL9jrh04QHzXZII21UDMpFYB jEk8uaMhabDv+OYg4Y5jetgqnuTWdbM2TBidwXALLwwY/xc3cVKDI3/4fE3zDA4LZxeLdU 8sFHHze9LFpt/bIqOB2AGlzABEBaRI8= X-MC-Unique: 7k1KVtluMhGIudvI83ItOg-1 X-Mimecast-MFC-AGG-ID: 7k1KVtluMhGIudvI83ItOg_1761161185 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161184; x=1761765984; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SWalDO9UGdGD6EJXyoisbCsd9GWW6yspcoUh/J0vPH0=; b=BFE0C+7KNGIa1Np3PWB6brGkgxT8hWDsesheOCmVd6QNJ2g8TBYBDwPUzrwGjTX/Vq HrjTAKUF6zoQ34UwOJ401wg9rsgpex+QfCsD28hHP7DrjvMX7FXpPnt9zCSehktm2XNu 5hiainQKcizb0xIwJp84j571OtqFCkwiabGwRpUW1i9vu/KUcNBvkkg/bERnXfdeWCMU PsEc+P03xL1n0wmZALYD2IZO7v+9+0jN55DsCw6qFCP3NZhDS04kJE9628y6p+/ugOn4 OZKBVo55XofHPJ8AZ+az5fGXt9G08x/atVu7qob9kRv7jUb1uZEiSjHc9KFl6izOfSAd 573g== X-Gm-Message-State: AOJu0Ywc7Fwb//XfO0yxp3B7unH4QAnCk3FeSkOmauTgomEs7zsWXIS/ /OjcVzQcqVRPfFRd4rHNFaKHEZyzU8Tfa8GBsWJmH1j5qcVmgUSofvyJiW8grhE5q3uWpq/6f+q wzsGuLENuqlT4CJae4f5BFYkwAKYWm9IKv810QajoneFmJL6JSAeqJtr/fseaHq949iQAZslJhI Ipra/blaH7ppSFlpfw1SfUNSRV4csiF2Wlq5Ac+w== X-Gm-Gg: ASbGnctMSkcmFK8a5POIUBwjZAYm5VjZr9yt/5r0phx85Xqg+8W0wJdjozQB4EMCV7Y f5BcEkrtH++eASO8fh3drNrfIO2hVHV5zyHiPn02rHgPh3F/RHdBTmW788kwjqHveCaluxt8IiF 4elDNGXKJlBfSe+25o10jaqH8erVCEq8dRkbvql7aLn3nymwdKs8EWOpkVh729YDopdX6zH+SX3 GnCAExorNA+5D+8PicKB5YryGxvScCbtKf85C9hcmmu2ViFdojzfdpq0T/7Ajcbh24nzeNjOk+u fL1enXPntBz4rOTVP/GYXP0gdDI1gv3j0bAoTEeyc3CN3OjmWr1tkUk1no7D+qMW X-Received: by 2002:a05:622a:1191:b0:4b3:104:792c with SMTP id d75a77b69052e-4e89d38e3b9mr278915311cf.57.1761161184155; Wed, 22 Oct 2025 12:26:24 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFhOrheVBqGyXuZA/rrIXuxaqUX5Nv0oDj04wJbzKRE259wtQ9DQInY23Vhj8ZgGOwfFTrhbg== X-Received: by 2002:a05:622a:1191:b0:4b3:104:792c with SMTP id d75a77b69052e-4e89d38e3b9mr278914801cf.57.1761161183608; Wed, 22 Oct 2025 12:26:23 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 05/13] migration/rdma: Change io_create_watch() to return immediately Date: Wed, 22 Oct 2025 15:26:04 -0400 Message-ID: <20251022192612.2737648-6-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161281143154100 Content-Type: text/plain; charset="utf-8" The old RDMA's io_create_watch() isn't really doing much work anyway. For G_IO_OUT, it already does return immediately. For G_IO_IN, it will try to detect some RDMA context length however normally nobody will be able to set it at all. Simplify the code so that RDMA iochannels simply always rely on synchronous reads and writes. It is highly likely what 6ddd2d76ca6f86f was talking about, that the async model isn't really working well. To be eplicit, incoming migration should always have marked the iochannel to be nonblocking. For non-RDMA channels, what happens with current master branch is when we have nothing to read, QEMU yields the coroutine at qemu_fill_buffer(). For RDMA, what I see is it always polls on its own and it yields at qemu_rdma_wait_comp_channel(). A sample stack: #0 qemu_coroutine_yield #1 0x0000562e46e51f77 in yield_until_fd_readable #2 0x0000562e46927823 in qemu_rdma_wait_comp_channel #3 0x0000562e46927b35 in qemu_rdma_block_for_wrid #4 0x0000562e46927e6f in qemu_rdma_post_send_control #5 0x0000562e4692857f in qemu_rdma_exchange_recv #6 0x0000562e4692ab5e in qio_channel_rdma_readv #7 0x0000562e46c1f2d7 in qio_channel_readv_full #8 0x0000562e46c13a6e in qemu_fill_buffer #9 0x0000562e46c14ba8 in qemu_peek_byte #10 0x0000562e46c14c09 in qemu_get_byte #11 0x0000562e46c14e2a in qemu_get_be32 #12 0x0000562e46c14e8a in qemu_get_be64 #13 0x0000562e46913f08 in ram_load_precopy #14 0x0000562e46914448 in ram_load #15 0x0000562e469186e3 in vmstate_load #16 0x0000562e4691ce6d in qemu_loadvm_section_part_end #17 0x0000562e4691d99b in qemu_loadvm_state_main #18 0x0000562e4691db87 in qemu_loadvm_state #19 0x0000562e468f2e87 in process_incoming_migration_co This patch may or may not help in reality, the whole IO watch may or may not be working at all for RDMA iochannels. In all cases, this patch makes sure above will be the only place that RDMA can poll on IOs. Tested-by: Zhijian Li (Fujitsu) Signed-off-by: Peter Xu Acked-by: Li Zhijian --- migration/rdma.c | 69 +++--------------------------------------------- 1 file changed, 3 insertions(+), 66 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 13dd391c14..0e5e02cdca 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2776,56 +2776,14 @@ static gboolean qio_channel_rdma_source_prepare(GSource *source, gint *timeout) { - QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma; - GIOCondition cond =3D 0; *timeout =3D -1; - - RCU_READ_LOCK_GUARD(); - if (rsource->condition =3D=3D G_IO_IN) { - rdma =3D qatomic_rcu_read(&rsource->rioc->rdmain); - } else { - rdma =3D qatomic_rcu_read(&rsource->rioc->rdmaout); - } - - if (!rdma) { - error_report("RDMAContext is NULL when prepare Gsource"); - return FALSE; - } - - if (rdma->wr_data[0].control_len) { - cond |=3D G_IO_IN; - } - cond |=3D G_IO_OUT; - - return cond & rsource->condition; + return TRUE; } =20 static gboolean qio_channel_rdma_source_check(GSource *source) { - QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma; - GIOCondition cond =3D 0; - - RCU_READ_LOCK_GUARD(); - if (rsource->condition =3D=3D G_IO_IN) { - rdma =3D qatomic_rcu_read(&rsource->rioc->rdmain); - } else { - rdma =3D qatomic_rcu_read(&rsource->rioc->rdmaout); - } - - if (!rdma) { - error_report("RDMAContext is NULL when check Gsource"); - return FALSE; - } - - if (rdma->wr_data[0].control_len) { - cond |=3D G_IO_IN; - } - cond |=3D G_IO_OUT; - - return cond & rsource->condition; + return TRUE; } =20 static gboolean @@ -2835,29 +2793,8 @@ qio_channel_rdma_source_dispatch(GSource *source, { QIOChannelFunc func =3D (QIOChannelFunc)callback; QIOChannelRDMASource *rsource =3D (QIOChannelRDMASource *)source; - RDMAContext *rdma; - GIOCondition cond =3D 0; - - RCU_READ_LOCK_GUARD(); - if (rsource->condition =3D=3D G_IO_IN) { - rdma =3D qatomic_rcu_read(&rsource->rioc->rdmain); - } else { - rdma =3D qatomic_rcu_read(&rsource->rioc->rdmaout); - } - - if (!rdma) { - error_report("RDMAContext is NULL when dispatch Gsource"); - return FALSE; - } - - if (rdma->wr_data[0].control_len) { - cond |=3D G_IO_IN; - } - cond |=3D G_IO_OUT; =20 - return (*func)(QIO_CHANNEL(rsource->rioc), - (cond & rsource->condition), - user_data); + return (*func)(QIO_CHANNEL(rsource->rioc), rsource->condition, user_da= ta); } =20 static void --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161233; cv=none; d=zohomail.com; s=zohoarc; b=T+Mvk8uhRQ396HtWMDKQ9OTttXRNJL86dqs1/bRZRaETNxC8NztDqnV4Uo/EFutaIw3BEVxFYSCQZZXFZ1Qw/ueTeGBYbBiLTyUERqWgIw9ZhGO5ryai1nGh2vkRFyNrw0vcnGmhAqfyeUArWBP5Kr8pVFDJu2ryQRWwkEkU408= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161233; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lEZkhyoFmaVmDzC4IYCUW4XvW8OD8L6dAS9sarlyY8A=; b=eR4BT/U3yt+nLY5BRbcWgwlL5GQT0cjJMnU+Wlnlc2FQVgyfDx8shOEHR1XMFTEPTtIcP71pH8qyH+ZkXAqCqbrfnqvvvGSkqHhPzf7lKjEDw5CCZthCPT4MobOc8zQAwGnJzRpeXnc4vsbaVHGv8cI8ZHNwC7OwdviP9/oa44k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161233719947.559692706269; Wed, 22 Oct 2025 12:27:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTn-0002tX-0g; Wed, 22 Oct 2025 15:26:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTf-0002sv-Ry for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTd-0001XO-Cw for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:30 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-612-6KnhmGIHNYme-L0CzaxcVw-1; Wed, 22 Oct 2025 15:26:26 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e8a73256easo47324441cf.0 for ; Wed, 22 Oct 2025 12:26:26 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161188; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lEZkhyoFmaVmDzC4IYCUW4XvW8OD8L6dAS9sarlyY8A=; b=Z+cenydxgVKLPZWGA56LHOo3WgFVXdDzYpXrhDOcEp7T56PNbBxectmQeIM4W7OB//LHGj X10yBt+XHOX5Ajt8Qm1rkLLpH1NAkrCr+skzUu/EfLi5KMc1KtHZk2SzXLVYsk8HUhqW3K Mo2DxaFRsonB5FyRdVfet504PlKdyhY= X-MC-Unique: 6KnhmGIHNYme-L0CzaxcVw-1 X-Mimecast-MFC-AGG-ID: 6KnhmGIHNYme-L0CzaxcVw_1761161186 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161186; x=1761765986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lEZkhyoFmaVmDzC4IYCUW4XvW8OD8L6dAS9sarlyY8A=; b=qyb8guPxHcZJcUxX3mmm5mV6wPxCeo/KPUX/9wN4Cs6/lOyIIr3a4UFz7/MlPLon72 AMn1QV0F0T+BbiZB9AQqxMj+x3TrLt4QjNjI3zG449RmCJenWRBbx5hJHOy8mrpLemr6 lze8C1dtGGGoocMX0p9JQilpJgjM3PtY9lzgQcBUxIRt2woBJYAirJvx2RTxPqiQFpVd B0uw6H6+SzydEYjrLrO8bErqXu639q2BANJxpfVqjaH5TefkEcetyZALYUoBL0NyydzD 60ssFvpQ1zlTR5GjeAC3HkIZB8X07LpUw+16Hf331IVnnisa8NtJ0UVCsYqN3HH4M99e ypPg== X-Gm-Message-State: AOJu0YyIv4kuesKmQM7B6PjEyiFMB/6R5kmoZ5XncSlhxqdr82SUxPzk X62UO5EYaUDz3L3p9VCie6ukUxFyZxWtmYe+8Ca/unWPwT8uPgfpCWAC5yjaiRz1n9+qm9t5tnr mzSQVuC49wfmLUYujujmJDRgOL25JdwxtfRp0Z69F+NJwbLo7A2iLDTKJExzXlOFpKnu0mALwGj +2OpwKOcFRziQL8GEhsAKIlcu+9BBRzGL7ZgVj9Q== X-Gm-Gg: ASbGncvXAZ9VHmiPJuFbwtNdeSTFdHzh6Mhq2Hpk5jSMR5WaHlNtWvm6ZLWYP2fqHTK atIIeQUIowgyn89BwZkP+CTSeQ5rLttRdK6jGnTOuepPsucom3niwPuAcN5E3jvs3RTlvk9sHnI slf52snzs6tX91bOCkNSRow39xcqrT53nJm31XtOF9SQa8qHz5uR5kZK5L+Pzt0dGux6KnrinCw 6NcUbq3cPD6C2wbODyztM5Uyg+/I8V4OqSqyoPFxPS4ArWEVK0LXbZU2a+FhG5H7QJJABNYjYlW IprjwmLs7rAXtCVxlCfRl88Dy2t6RUGsLzwMAewwI/1Nqd9PdhGYDRhB9D1g1plS X-Received: by 2002:ac8:7fc8:0:b0:4b3:4fa9:4cd2 with SMTP id d75a77b69052e-4eb7331fff9mr69350061cf.33.1761161185834; Wed, 22 Oct 2025 12:26:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE8kQu6hhKmMWi+k3MxYkOzl1GpTzwGguJhzriIZxs7mUS/Ndio7/tWDGOt/FIXdcCcVRHyVA== X-Received: by 2002:ac8:7fc8:0:b0:4b3:4fa9:4cd2 with SMTP id d75a77b69052e-4eb7331fff9mr69349511cf.33.1761161185234; Wed, 22 Oct 2025 12:26:25 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 06/13] migration: Introduce WITH_BQL_HELD() / WITH_BQL_RELEASED() Date: Wed, 22 Oct 2025 15:26:05 -0400 Message-ID: <20251022192612.2737648-7-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161236698154100 Content-Type: text/plain; charset="utf-8" Introduce the helpers to conditionally take or release BQL for a process. Signed-off-by: Peter Xu Reviewed-by: Vladimir Sementsov-Ogievskiy --- migration/migration.h | 95 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/migration/migration.h b/migration/migration.h index 01329bf824..e1c5029110 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -20,6 +20,7 @@ #include "qobject/json-writer.h" #include "qemu/thread.h" #include "qemu/coroutine.h" +#include "qemu/main-loop.h" #include "io/channel.h" #include "io/channel-buffer.h" #include "net/announce.h" @@ -42,6 +43,100 @@ #define MIGRATION_THREAD_DST_LISTEN "mig/dst/listen" #define MIGRATION_THREAD_DST_PREEMPT "mig/dst/preempt" =20 +struct WithBqlHeldAuto; +typedef struct WithBqlHeldAuto WithBqlHeldAuto; + +static inline WithBqlHeldAuto * +with_bql_held_auto_lock(bool bql_held, const char *file, int line) +{ + assert(bql_held =3D=3D bql_locked()); + if (!bql_held) { + bql_lock_impl(file, line); + return (WithBqlHeldAuto *)1; + } + return (WithBqlHeldAuto *)2; +} + +static inline void +with_bql_held_auto_unlock(WithBqlHeldAuto *v) +{ + assert(bql_locked()); + if (v =3D=3D (WithBqlHeldAuto *)1) { + bql_unlock(); + } +} + +G_DEFINE_AUTOPTR_CLEANUP_FUNC(WithBqlHeldAuto, with_bql_held_auto_unlock); + +#define _WITH_BQL_HELD(bql_held, var) \ + for (g_autoptr(WithBqlHeldAuto) var =3D \ + with_bql_held_auto_lock(bql_held, __FILE__, __LINE__); \ + var; \ + with_bql_held_auto_unlock(var), var =3D NULL) + +/** + * WITH_BQL_HELD(): Run a block of code, making sure BQL is held + * @bql_held: Whether BQL is already held + * + * Example use case: + * + * WITH_BQL_HELD(bql_held) { + * // BQL is guaranteed to be held within this block. + * // If bql_held=3D=3Dfalse, bql will be released when the block fini= shes. + * } + */ +#define WITH_BQL_HELD(bql_held) \ + _WITH_BQL_HELD(bql_held, glue(with_bql_held_var, __COUNTER__)) + +struct WithBqlReleaseAuto; +typedef struct WithBqlReleaseAuto WithBqlReleaseAuto; + +static inline WithBqlReleaseAuto * +with_bql_release_auto_unlock(bool bql_held) +{ + assert(bql_held =3D=3D bql_locked()); + if (bql_held) { + bql_unlock(); + return (WithBqlReleaseAuto *)1; + } + return (WithBqlReleaseAuto *)2; +} + +static inline void +with_bql_release_auto_lock(WithBqlReleaseAuto *v) +{ + assert(!bql_locked()); + if (v =3D=3D (WithBqlReleaseAuto *)1) { + /* + * NOTE: cleanup function cannot take more than 1 argument. Keep + * it simple here by not passing __FILE__/__LINE__ from the caller. + */ + bql_lock_impl(__FILE__, __LINE__); + } +} + +G_DEFINE_AUTOPTR_CLEANUP_FUNC(WithBqlReleaseAuto, with_bql_release_auto_lo= ck); + +#define _WITH_BQL_RELEASED(bql_held, var) \ + for (g_autoptr(WithBqlReleaseAuto) var =3D \ + with_bql_release_auto_unlock(bql_held); \ + var; \ + with_bql_release_auto_lock(var), var =3D NULL) + +/** + * WITH_BQL_RELEASED(): Run a block of code, making sure BQL is release + * @bql_held: Whether BQL is already held + * + * Example use case: + * + * WITH_BQL_RELEASE(bql_held) { + * // BQL is guaranteed to be release within this block. + * // If bql_held=3D=3Dtrue, bql will be re-taken when the block finis= hes. + * } + */ +#define WITH_BQL_RELEASED(bql_held) \ + _WITH_BQL_RELEASED(bql_held, glue(with_bql_release_var, __COUNTER__)) + struct PostcopyBlocktimeContext; typedef struct ThreadPool ThreadPool; =20 --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161323; cv=none; d=zohomail.com; s=zohoarc; b=MplescUGgkC/8Sq2foOQDshHobsIZ4Km0ZLnuTZIQiJFzYGOq7toKBQB3/MUnKsBKurWAYPtgG2DHSYxAhhQWLwWZfZwZhmm2+lBdiOPWYfqdAThtg17ixCp/IYgVxT67HA8X+hWr4etaecwkAAxuBao7SQau2VK8jhxzVD1Dw0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161323; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LeWIh/aS3jgszbhRw8H9hH7gHBVXRI8hO1i9Z5a4Upo=; b=MjoDhQScliD01avTXv7gHBneCsWLeSFwhJzrQB1HDLD/9q5nN/hBkR1hcDymwnT7CxOsKbfdvILhPh5hvuBkeDr23hvIDPE0mtGFoJsmfLqGxItAgJdFaXi0FzKa5yNeHiTcMKpR/SIDu/W7jv3Gj7Z9iqbT6702FHRyscrVUUE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161323097339.2080117730751; Wed, 22 Oct 2025 12:28:43 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTv-0002vT-3K; Wed, 22 Oct 2025 15:26:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTj-0002tV-EP for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTg-0001YG-Cm for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:35 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-163-6Md2XWVLPbuqse2QXdCmpw-1; Wed, 22 Oct 2025 15:26:28 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e894937010so46490211cf.3 for ; Wed, 22 Oct 2025 12:26:28 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161190; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LeWIh/aS3jgszbhRw8H9hH7gHBVXRI8hO1i9Z5a4Upo=; b=iBk4wRc+ONOAQmbPICVaRxVW+DhlE5UHL79HXdZbQJ2Gw3mNpAUKlqXbSHPpCaZGa+Svez edbNIdP2eyAX/L4jEfQ2CWaz/3obHpJR3GHibHVtifxU7doeY10K03dsEtMIp37/B/dTfH fNNeaqCY+b1WoMziuc+S7UkAZ9pNoDk= X-MC-Unique: 6Md2XWVLPbuqse2QXdCmpw-1 X-Mimecast-MFC-AGG-ID: 6Md2XWVLPbuqse2QXdCmpw_1761161188 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161187; x=1761765987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LeWIh/aS3jgszbhRw8H9hH7gHBVXRI8hO1i9Z5a4Upo=; b=o83AOsy/FVpU8746awozVf/J2D8YNiodufLSdmecYRhAd4T4gNIh9byI6kbctmcSLQ fxAH6CE/22IVQ4QZn5ExAtKmlnHKeAq/nmmJO3VhE/uYK9Tk0UMfGgVOmmnsnbjHNp7R 1xpxOVL1UDGHaVCSTiAzVa+j9Xs8mdgYwsZ+PBaaIpuViHNJMTK3hyG/2uvt21cFUBGG NMKGwNBOpaMU0JD+RM6XtDJUq67Wl+ALY37D/8s4nbgfSYgNV5a6dIV4UIqh+hyIX5wA BErSuzf4jTkuMHzKFGQfCaBMEgclvE9EElR65gbJsggd0QnXOhGGMwIR4Nh64MouG36u 0rZQ== X-Gm-Message-State: AOJu0YwaqtrVcxIQLY0SdWLTnYFjXF+Zh8SynjAYlKiu3opIo5jiYqD0 R8urhd6f7fWF5Kjp+F0xhRpxBQNXBxjGV7EAc6T5SWyYq1DE6exsJfIky3FI8CxSkglgG33x2GG OyZnfSRFXc3+y3/HJtG7+6jmC9qFjT3UrAXASE6B0W7t0WQns9KlN5ZA4u40vlbte8DGWR7XYH8 Oi6wdG0bNt1XENeGAnR8ZxmmAb/tgdEXE1MlJt5g== X-Gm-Gg: ASbGnctFlzqVVVuS6F+UkTb185j4lly91qnLrLGoyQIyrtNWhDtrcvR73j0AHQv0bO6 4Hk1DVeElov67Olhv3uCuwynUVtoN0qByiQJ1M3msnuCumEwqb/SDySGzH4312fGEKD/hZ6eCVv FUHU8v0LFHLTcr4bbo+OIZMKzC8sgV0pUnpePRvnOlIjeMbqAlkvy4yRWPTDwbub4vIrIdH74MU lFxG646wrQqhB9aeFwOLJzfyBGbKliWjaoKoFiNK0C+stQFt9Dop9p1pt3qXcy7VIZizRqZj6Na cSgKvTP2222Sya7eMFKMRKzdZsfXbxDTuC8oEtcgma/7c7WW8veMU1wq2jDDopLm X-Received: by 2002:ac8:5993:0:b0:4e8:a6ca:8cc4 with SMTP id d75a77b69052e-4e8a6ca8ea4mr227194081cf.36.1761161187474; Wed, 22 Oct 2025 12:26:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHwTDHEAZMLO1MTiXCwf6Xt4CUbqzsrdFcrClDRzBgnR4sZ1//qgW9IpLzR+9Mk37q6lAIs1Q== X-Received: by 2002:ac8:5993:0:b0:4e8:a6ca:8cc4 with SMTP id d75a77b69052e-4e8a6ca8ea4mr227193571cf.36.1761161186896; Wed, 22 Oct 2025 12:26:26 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 07/13] migration: Pass in bql_held information from qemu_loadvm_state() Date: Wed, 22 Oct 2025 15:26:06 -0400 Message-ID: <20251022192612.2737648-8-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161334218158500 Content-Type: text/plain; charset="utf-8" Teach qemu_loadvm_state() and some of the internal functions to know whether we're holding BQL or not. So far, all the callers still always pass in TRUE, hence no functional change expected. But it may change in the near future. To reviewers: even if this is not functional change yet, it'll be the major core functional change after we switch to threadified loadvm soon. Please Treat it as one to add explicit code to mark out which part of incoming live migration would need to be executed always with the BQL, or would need to be run always without BQL. Signed-off-by: Peter Xu --- migration/savevm.h | 4 +-- migration/colo.c | 2 +- migration/migration.c | 2 +- migration/savevm.c | 72 ++++++++++++++++++++++++++++++------------- 4 files changed, 55 insertions(+), 25 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index c337e3e3d1..a04dee4166 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -64,10 +64,10 @@ void qemu_savevm_send_colo_enable(QEMUFile *f); void qemu_savevm_live_state(QEMUFile *f); int qemu_save_device_state(QEMUFile *f); =20 -int qemu_loadvm_state(QEMUFile *f, Error **errp); +int qemu_loadvm_state(QEMUFile *f, bool bql_held, Error **errp); void qemu_loadvm_state_cleanup(MigrationIncomingState *mis); int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, - Error **errp); + bool bql_held, Error **errp); int qemu_load_device_state(QEMUFile *f, Error **errp); int qemu_loadvm_approve_switchover(void); int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f, diff --git a/migration/colo.c b/migration/colo.c index db783f6fa7..4fd586951a 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -686,7 +686,7 @@ static void colo_incoming_process_checkpoint(MigrationI= ncomingState *mis, =20 bql_lock(); cpu_synchronize_all_states(); - ret =3D qemu_loadvm_state_main(mis->from_src_file, mis, errp); + ret =3D qemu_loadvm_state_main(mis->from_src_file, mis, true, errp); bql_unlock(); =20 if (ret < 0) { diff --git a/migration/migration.c b/migration/migration.c index 4ed2a2e881..38a584afae 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -878,7 +878,7 @@ process_incoming_migration_co(void *opaque) MIGRATION_STATUS_ACTIVE); =20 mis->loadvm_co =3D qemu_coroutine_self(); - ret =3D qemu_loadvm_state(mis->from_src_file, &local_err); + ret =3D qemu_loadvm_state(mis->from_src_file, true, &local_err); mis->loadvm_co =3D NULL; =20 trace_vmstate_downtime_checkpoint("dst-precopy-loadvm-completed"); diff --git a/migration/savevm.c b/migration/savevm.c index 232cae090b..44aadc2f51 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -154,11 +154,12 @@ static void qemu_loadvm_thread_pool_destroy(Migration= IncomingState *mis) } =20 static bool qemu_loadvm_thread_pool_wait(MigrationState *s, - MigrationIncomingState *mis) + MigrationIncomingState *mis, + bool bql_held) { - bql_unlock(); /* Let load threads do work requiring BQL */ - thread_pool_wait(mis->load_threads); - bql_lock(); + WITH_BQL_RELEASED(bql_held) { + thread_pool_wait(mis->load_threads); + } =20 return !migrate_has_error(s); } @@ -2117,7 +2118,7 @@ static void *postcopy_ram_listen_thread(void *opaque) qemu_file_set_blocking(f, true, &error_fatal); =20 /* TODO: sanity check that only postcopiable data will be loaded here = */ - load_res =3D qemu_loadvm_state_main(f, mis, &local_err); + load_res =3D qemu_loadvm_state_main(f, mis, true, &local_err); =20 /* * This is tricky, but, mis->from_src_file can change after it @@ -2420,7 +2421,8 @@ static void loadvm_postcopy_handle_resume(MigrationIn= comingState *mis) * Returns: Negative values on error * */ -static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis, Error *= *errp) +static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis, + bool bql_held, Error **errp) { int ret; size_t length; @@ -2471,7 +2473,7 @@ static int loadvm_handle_cmd_packaged(MigrationIncomi= ngState *mis, Error **errp) qemu_coroutine_yield(); } while (1); =20 - ret =3D qemu_loadvm_state_main(packf, mis, errp); + ret =3D qemu_loadvm_state_main(packf, mis, bql_held, errp); trace_loadvm_handle_cmd_packaged_main(ret); qemu_fclose(packf); object_unref(OBJECT(bioc)); @@ -2571,7 +2573,7 @@ static int loadvm_postcopy_handle_switchover_start(Er= ror **errp) * LOADVM_QUIT All good, but exit the loop * <0 Error */ -static int loadvm_process_command(QEMUFile *f, Error **errp) +static int loadvm_process_command(QEMUFile *f, bool bql_held, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); uint16_t cmd; @@ -2641,7 +2643,8 @@ static int loadvm_process_command(QEMUFile *f, Error = **errp) break; =20 case MIG_CMD_PACKAGED: - return loadvm_handle_cmd_packaged(mis, errp); + /* PACKAGED may have bql dependency internally */ + return loadvm_handle_cmd_packaged(mis, bql_held, errp); =20 case MIG_CMD_POSTCOPY_ADVISE: return loadvm_postcopy_handle_advise(mis, len, errp); @@ -2666,7 +2669,11 @@ static int loadvm_process_command(QEMUFile *f, Error= **errp) return loadvm_process_enable_colo(mis, errp); =20 case MIG_CMD_SWITCHOVER_START: - return loadvm_postcopy_handle_switchover_start(errp); + WITH_BQL_HELD(bql_held) { + /* TODO: drop the BQL dependency */ + ret =3D loadvm_postcopy_handle_switchover_start(errp); + } + return ret; } =20 return 0; @@ -2882,6 +2889,10 @@ static int qemu_loadvm_state_header(QEMUFile *f, Err= or **errp) return -EINVAL; } =20 + /* + * NOTE: this can be invoked with/without BQL. It's safe because + * vmstate_configuration (and its hooks) doesn't rely on BQL statu= s. + */ ret =3D vmstate_load_state(f, &vmstate_configuration, &savevm_stat= e, 0, errp); if (ret) { @@ -3072,7 +3083,7 @@ static bool postcopy_pause_incoming(MigrationIncoming= State *mis) } =20 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, - Error **errp) + bool bql_held, Error **errp) { ERRP_GUARD(); uint8_t section_type; @@ -3094,20 +3105,33 @@ retry: switch (section_type) { case QEMU_VM_SECTION_START: case QEMU_VM_SECTION_FULL: - ret =3D qemu_loadvm_section_start_full(f, section_type, errp); + /* + * FULL should normally require BQL, e.g. during post_load() + * there can be memory region updates. START may or may not + * require it, but just to keep it simple to always hold BQL + * for now. + */ + WITH_BQL_HELD(bql_held) { + ret =3D qemu_loadvm_section_start_full(f, section_type, er= rp); + } if (ret < 0) { goto out; } break; case QEMU_VM_SECTION_PART: case QEMU_VM_SECTION_END: + /* PART / END may be loaded without BQL */ ret =3D qemu_loadvm_section_part_end(f, section_type, errp); if (ret < 0) { goto out; } break; case QEMU_VM_COMMAND: - ret =3D loadvm_process_command(f, errp); + /* + * Be careful; QEMU_VM_COMMAND can embed FULL sections, so it + * may internally need BQL. + */ + ret =3D loadvm_process_command(f, bql_held, errp); trace_qemu_loadvm_state_section_command(ret); if ((ret < 0) || (ret =3D=3D LOADVM_QUIT)) { goto out; @@ -3152,7 +3176,7 @@ out: return ret; } =20 -int qemu_loadvm_state(QEMUFile *f, Error **errp) +int qemu_loadvm_state(QEMUFile *f, bool bql_held, Error **errp) { MigrationState *s =3D migrate_get_current(); MigrationIncomingState *mis =3D migration_incoming_get_current(); @@ -3177,9 +3201,12 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp) qemu_loadvm_state_switchover_ack_needed(mis); } =20 - cpu_synchronize_all_pre_loadvm(); + /* run_on_cpu() requires BQL */ + WITH_BQL_HELD(bql_held) { + cpu_synchronize_all_pre_loadvm(); + } =20 - ret =3D qemu_loadvm_state_main(f, mis, errp); + ret =3D qemu_loadvm_state_main(f, mis, bql_held, errp); qemu_event_set(&mis->main_thread_load_event); =20 trace_qemu_loadvm_state_post_main(ret); @@ -3195,7 +3222,7 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp) /* When reaching here, it must be precopy */ if (ret =3D=3D 0) { if (migrate_has_error(migrate_get_current()) || - !qemu_loadvm_thread_pool_wait(s, mis)) { + !qemu_loadvm_thread_pool_wait(s, mis, bql_held)) { ret =3D -EINVAL; error_setg(errp, "Error while loading vmstate"); @@ -3249,7 +3276,10 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp) } } =20 - cpu_synchronize_all_post_init(); + /* run_on_cpu() requires BQL */ + WITH_BQL_HELD(bql_held) { + cpu_synchronize_all_post_init(); + } =20 return ret; } @@ -3260,7 +3290,7 @@ int qemu_load_device_state(QEMUFile *f, Error **errp) int ret; =20 /* Load QEMU_VM_SECTION_FULL section */ - ret =3D qemu_loadvm_state_main(f, mis, errp); + ret =3D qemu_loadvm_state_main(f, mis, true, errp); if (ret < 0) { return ret; } @@ -3495,7 +3525,7 @@ void qmp_xen_load_devices_state(const char *filename,= Error **errp) f =3D qemu_file_new_input(QIO_CHANNEL(ioc)); object_unref(OBJECT(ioc)); =20 - ret =3D qemu_loadvm_state(f, errp); + ret =3D qemu_loadvm_state(f, true, errp); qemu_fclose(f); if (ret < 0) { error_prepend(errp, "loading Xen device state failed: "); @@ -3573,7 +3603,7 @@ bool load_snapshot(const char *name, const char *vmst= ate, ret =3D -EINVAL; goto err_drain; } - ret =3D qemu_loadvm_state(f, errp); + ret =3D qemu_loadvm_state(f, true, errp); migration_incoming_state_destroy(); =20 bdrv_drain_all_end(); --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161228; cv=none; d=zohomail.com; s=zohoarc; b=lZW1aFgV5QLC5o5FaKLZsffinHcxqJXFjQFfNrKs5Ba/3OBqSWauy59iwdi+SlJz87bddqejrkxqRigFdBPBDer6rRD8omFag7cH7QBwlK67Cw/fDaGn0eZzxp3i/NOa4+93+slaTQHCtTCZ/rDefIqmOUVY6Zwla7LWNyt3fHg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161228; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=8Jc0kveFbdHXC0rLwFGTNnrLna7jwtbutSCIcOsSkwQ=; b=dfvcIszCTH5J4Wlu3MrQ3Wno6ANwdLJG96YHvLEfGOMu1NEcGOrt1PRwG42XEVIKBAcRC+OAlQZgjneq67MME1bYQcToYl5TX/TKFJBTvwLzQENRNzSynt04jfBdm8AU364pR9Z9KVJv0cuEL/jhEgHlsxwtpM1iZBEgO4fOYk4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161228238218.82942884264241; Wed, 22 Oct 2025 12:27:08 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTu-0002vS-Vt; Wed, 22 Oct 2025 15:26:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTk-0002tW-At for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTh-0001YZ-Eb for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:36 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-112-i7OudJHmO7aZt45YhJhrkg-1; Wed, 22 Oct 2025 15:26:31 -0400 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-4e88947a773so66678241cf.2 for ; Wed, 22 Oct 2025 12:26:31 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161192; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8Jc0kveFbdHXC0rLwFGTNnrLna7jwtbutSCIcOsSkwQ=; b=NPPWqOJDFK4u5TH+UyPuh5ShRZq6xGcU9taKEQ9gfCzKXTumQzKzcVrA9DK1PGVVWJbaA4 +0Vem1eBBcX/l5TiLekHvyE4EbVx8XuqWi7f60kS2Zz9ys9mqpo4my9NVS75eeYxEWMxFQ KktHl5RKNwHprdRHTeaXkevYUydG+Lw= X-MC-Unique: i7OudJHmO7aZt45YhJhrkg-1 X-Mimecast-MFC-AGG-ID: i7OudJHmO7aZt45YhJhrkg_1761161191 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161190; x=1761765990; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8Jc0kveFbdHXC0rLwFGTNnrLna7jwtbutSCIcOsSkwQ=; b=C6wJMdlt3bAg2P5F5YKP5QH68Xsc1VgLYInSXmKkZdjIiOelSkvCys/g6DUKEVKUOj kUXx9lidPU2ZzvblA9aPaP9spZH+KrkHQjO56xnkHpQI3xN8l7/xiHBlU+RbQ2l5mJSy iUXkfvPXOVKG3uAzlfGP0xqmzkY4ycyWaVaLabtJlIdn5OneIdAp6U0m3v4ZzpwZ7ZNS wji4x+EQamH9Nu2OARjhbwgKRj78iugTEsJEtMYlAuYue6ZDzevB9AWWV+qEtKKBwj+z 8FSafWDgFlEFqgnt3RqQv9l32+eLUD880kePKRjo0Ap2bZotzqXS+BqNYhjfIUq45/GH E9TQ== X-Gm-Message-State: AOJu0YzYvW2mX3OQ+mUONWLd13M63o8rMNkK0U/CEP8C8mlG6iK30kn1 GQRrdut/ze5iRZrKiPUGIHVl8DYZYROa7746do6m3NqqjaOqoBTtDClHrwBCsPckAOVhtACpLMh dkXn8qGrdzEta8/jyQG0YaxJNWc/7BJ3GEbm9njvgjSzsINr+TAFRFEBIIMHGet49qgmDXvAvyK 14q3Nduj++w0SZp9HbPRd5ROiZgysAObZDDEOTLQ== X-Gm-Gg: ASbGnctd/EEf4Xmt3UCWaXZODQcaFf6HDeZCNCFF03GzMi3MTTDiIexYQeFxB5gJkxy BjsufbkvwM5IlPG8aji4wPF0XHMh6dFDI5fFjnsTth2FiouOtw6b9bZvdw/mWxLGNPp7/RIVe9D UH674vUaVeQhqaES47ckWAsEyzURiy9yxwDFeCqxg3Tv89r9kCTm1asQDFaqUYudtHhVDiqC4W+ uYa3LZYXqpIW2UCDktgI1mfBUQXK2JWNKN//V/+wSmZqPWCwqgW4zCzRQ2fjaWGdEiE5jRyykI9 bo9oGG4DynwclDEMxDyYRxE5lNLzWFP9tq2GIy92YyPdJGEL/Hx8Zl2i2tY3+RBd X-Received: by 2002:a05:622a:1a90:b0:4e8:b64a:2dff with SMTP id d75a77b69052e-4e8b64a3083mr176110581cf.26.1761161189736; Wed, 22 Oct 2025 12:26:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF9d76WEUbLgPJ+MveugtoA2ZeQzMke6T85FxYx0xHcaktwvgcxPfQsU9h4LRtGM1Pofm1aNw== X-Received: by 2002:a05:622a:1a90:b0:4e8:b64a:2dff with SMTP id d75a77b69052e-4e8b64a3083mr176109871cf.26.1761161188746; Wed, 22 Oct 2025 12:26:28 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 08/13] migration: Thread-ify precopy vmstate load process Date: Wed, 22 Oct 2025 15:26:07 -0400 Message-ID: <20251022192612.2737648-9-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161230962154100 Content-Type: text/plain; charset="utf-8" Migration module was there for 10+ years. Initially, it was in most cases based on coroutines. As more features were added into the framework, like postcopy, multifd, etc.. it became a mixture of threads and coroutines. I'm guessing coroutines just can't fix all issues that migration want to resolve. After all these years, migration is now heavily based on a threaded model. Now there's still a major part of migration framework that is still not thread-based, which is precopy load. We do load in a separate thread in postcopy since the 1st day postcopy was introduced, however that requires a separate state transition from precopy loading all devices first, which still happens in the main thread of a coroutine. This patch tries to move the migration incoming side to be run inside a separate thread (mig/dst/main) just like the src (mig/src/main). The entrance to be migration_incoming_thread(). Quite a few things are needed to make it fly.. One note here is we need to change all these things in one patch to not break anything. The other way to do this is add code to make all paths (that this patch touched) be ready for either coroutine or thread. That may cause confusions in another way. So reviewers, please take my sincere apology on the hardness of reviewing this patch: it covers a few modules at the same time, and with some risky changes. BQL Analysis =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Firstly, when moving it over to the thread, it means the thread cannot take BQL during the whole process of loading anymore, because otherwise it can block main thread from using the BQL for all kinds of other concurrent tasks (for example, processing QMP / HMP commands). Here the first question to ask is: what needs BQL during precopy load, and what doesn't? Most of the load process shouldn't need BQL, especially when it's about RAM. After all, RAM is still the major chunk of data to move for a live migration process. VFIO started to change that, though, but still, VFIO is per-device so that shouldn't need BQL either in most cases. Generic device loads will need BQL, likely not when receiving VMSDs, but when applying them. One example is any post_load() could potentially inject memory regions causing memory transactions to happen. That'll need to update the global address spaces, hence requires BQL. The other one is CPU sync operations, even if the sync alone may not need BQL (which is still to be further justified), run_on_cpu() will need it. For that, qemu_loadvm_state() and qemu_loadvm_state_main() functions need to now take a "bql_held" parameter saying whether bql is held. We could use things like BQL_LOCK_GUARD(), but this patch goes with explicit lockings rather than relying on bql_locked TLS variable. In case of migration, we always know whether BQL is held in different context as long as we can still pass that information downwards. COLO =3D=3D=3D=3D COLO assumed the dest VM load happens in a coroutine. After this patch, it's not anymore. Change that by invoking colo_incoming_co() directly from the migration_incoming_thread(). The name (colo_incoming_co()) isn't proper anymore. Change it to colo_incoming_wait(), removing the coroutine annotation alongside. Remove all the bql_lock() implications in COLO, e.g., colo_incoming_co() used to release the lock for a short period while join(). Now it's not needed. Instead, taking BQL but only when needed (colo_release_ram_cache). At the meantime, there's colo_incoming_co variable that used to store the COLO incoming coroutine, only to be kicked off when a secondary failover happens. To recap, what should happen for such failover should be (taking example of a QMP command x-colo-lost-heartbeat triggering on dest QEMU): - The QMP command will kick off both the coroutine and the COLO thread (colo_process_incoming_thread()), with something like: /* Notify COLO incoming thread that failover work is finished */ qemu_event_set(&mis->colo_incoming_event); qemu_coroutine_enter(mis->colo_incoming_co); - The coroutine, which yielded itself before, now resumes after enter(), then it'll wait for the join(): mis->colo_incoming_co =3D qemu_coroutine_self(); qemu_coroutine_yield(); mis->colo_incoming_co =3D NULL; /* Wait checkpoint incoming thread exit before free resource */ qemu_thread_join(&th); Here, when switching to a thread model, it should be fine removing colo_incoming_co variable completely, because if so, the incoming thread will (instead of yielding the coroutine) wait at qemu_thread_join() until the colo thread completes execution (after receiving colo_incoming_event). RDMA =3D=3D=3D=3D With the prior patch making sure io_watch won't block for RDMA iochannels, RDMA threads should only block at its io_readv/io_writev functions. When a disconnection is detected (as in rdma_cm_poll_handler()), the update to "errored" field will be immediately reflected in the migration incoming thread. Hence the coroutine for RDMA is not needed anymore to kick the thread out. When the thread is available, we also can't have rdma_cm_poll_handler() keep polling the fd and operate on it in the main thread. Drop it completely, and it should be fine because qemu_rdma_wait_comp_channel() should also be monitoring it. This almost reverts commit 923709896b1b01fb982c93492ad01b233e6b6023. We need to do this change in this same patch that we introduce the thread, unfortunately, otherwise we can have a risk of racing. TODO =3D=3D=3D=3D Currently the BQL is taken during loading of a START|FULL section. When the IO hangs (e.g. network issue) during this process, it could potentially block others like the monitor servers. One solution is breaking BQL to smaller granule and leave IOs to be always BQL-free. That'll need more justifications. For example, there are at least four things that need some closer attention: - SaveVMHandlers's load_state(): this likely DO NOT need BQL, but we need to justify all of them (not to mention, some of them look like prone to be rewritten as VMSDs..) - VMSD's pre_load(): in most cases, this DO NOT really need BQL, but sometimes maybe it will! Double checking on this will be needed. - VMSD's post_load(): in many cases, this DO need BQL, for example on address space operations. Likely we should just take it for any post_load(). - VMSD field's get(): this is tricky! It could internally be anything even if it was only a field. E.g. there can be users to use a SINGLE field to load a whole VMSD, which can further introduce more possibilities. In general, QEMUFile IOs should not need BQL, that is when receiving the VMSD data and waiting for e.g. the socket buffer to get refilled. But that's the easy part. Signed-off-by: Peter Xu Reviewed-by: Li Zhijian # COLO and RDMA Tested-by: Li Zhijian # COLO and RDMA --- include/migration/colo.h | 6 ++-- migration/migration.h | 14 +++----- migration/colo-stubs.c | 2 +- migration/colo.c | 24 ++++--------- migration/migration.c | 77 +++++++++++++++++++++++++--------------- migration/rdma.c | 34 +----------------- migration/savevm.c | 8 ++--- migration/trace-events | 4 +-- 8 files changed, 69 insertions(+), 100 deletions(-) diff --git a/include/migration/colo.h b/include/migration/colo.h index d4fe422e4d..5de7d715a7 100644 --- a/include/migration/colo.h +++ b/include/migration/colo.h @@ -44,12 +44,10 @@ void colo_do_failover(void); void colo_checkpoint_delay_set(void); =20 /* - * Starts COLO incoming process. Called from process_incoming_migration_co= () + * Starts COLO incoming process. Called from migration_incoming_thread() * after loading the state. - * - * Called with BQL locked, may temporary release BQL. */ -void coroutine_fn colo_incoming_co(void); +void colo_incoming_wait(void); =20 void colo_shutdown(void); #endif diff --git a/migration/migration.h b/migration/migration.h index e1c5029110..0d22dc8cc2 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -214,6 +214,10 @@ struct MigrationIncomingState { bool have_listen_thread; QemuThread listen_thread; =20 + /* Migration main recv thread */ + bool have_recv_thread; + QemuThread recv_thread; + /* For the kernel to send us notifications */ int userfault_fd; /* To notify the fault_thread to wake, e.g., when need to quit */ @@ -272,15 +276,7 @@ struct MigrationIncomingState { =20 MigrationStatus state; =20 - /* - * The incoming migration coroutine, non-NULL during qemu_loadvm_state= (). - * Used to wake the migration incoming coroutine from rdma code. How m= uch is - * it safe - it's a question. - */ - Coroutine *loadvm_co; - - /* The coroutine we should enter (back) after failover */ - Coroutine *colo_incoming_co; + /* Notify secondary VM to move on */ QemuEvent colo_incoming_event; =20 /* Optional load threads pool and its thread exit request flag */ diff --git a/migration/colo-stubs.c b/migration/colo-stubs.c index e22ce65234..ef77d1ab4b 100644 --- a/migration/colo-stubs.c +++ b/migration/colo-stubs.c @@ -9,7 +9,7 @@ void colo_shutdown(void) { } =20 -void coroutine_fn colo_incoming_co(void) +void colo_incoming_wait(void) { } =20 diff --git a/migration/colo.c b/migration/colo.c index 4fd586951a..81276a3e65 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -147,11 +147,6 @@ static void secondary_vm_do_failover(void) } /* Notify COLO incoming thread that failover work is finished */ qemu_event_set(&mis->colo_incoming_event); - - /* For Secondary VM, jump to incoming co */ - if (mis->colo_incoming_co) { - qemu_coroutine_enter(mis->colo_incoming_co); - } } =20 static void primary_vm_do_failover(void) @@ -848,10 +843,8 @@ static void *colo_process_incoming_thread(void *opaque) =20 mis->to_src_file =3D qemu_file_get_return_path(mis->from_src_file); /* - * Note: the communication between Primary side and Secondary side - * should be sequential, we set the fd to unblocked in migration incom= ing - * coroutine, and here we are in the COLO incoming thread, so it is ok= to - * set the fd back to blocked. + * Here we are in the COLO incoming thread, so it is ok to set the fd + * to blocking. */ if (!qemu_file_set_blocking(mis->from_src_file, true, &local_err)) { error_report_err(local_err); @@ -927,27 +920,22 @@ out: return NULL; } =20 -void coroutine_fn colo_incoming_co(void) +/* Wait for failover */ +void colo_incoming_wait(void) { MigrationIncomingState *mis =3D migration_incoming_get_current(); QemuThread th; =20 - assert(bql_locked()); assert(migration_incoming_colo_enabled()); =20 qemu_thread_create(&th, MIGRATION_THREAD_DST_COLO, colo_process_incoming_thread, mis, QEMU_THREAD_JOINABLE); =20 - mis->colo_incoming_co =3D qemu_coroutine_self(); - qemu_coroutine_yield(); - mis->colo_incoming_co =3D NULL; - - bql_unlock(); /* Wait checkpoint incoming thread exit before free resource */ qemu_thread_join(&th); - bql_lock(); =20 - /* We hold the global BQL, so it is safe here */ + bql_lock(); colo_release_ram_cache(); + bql_unlock(); } diff --git a/migration/migration.c b/migration/migration.c index 38a584afae..728d02dbee 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -491,6 +491,11 @@ void migration_incoming_state_destroy(void) mis->postcopy_qemufile_dst =3D NULL; } =20 + if (mis->have_recv_thread) { + qemu_thread_join(&mis->recv_thread); + mis->have_recv_thread =3D false; + } + cpr_set_incoming_mode(MIG_MODE_NONE); yank_unregister_instance(MIGRATION_YANK_INSTANCE); } @@ -861,30 +866,46 @@ static void process_incoming_migration_bh(void *opaqu= e) migration_incoming_state_destroy(); } =20 -static void coroutine_fn -process_incoming_migration_co(void *opaque) +static void migration_incoming_state_destroy_bh(void *opaque) +{ + struct MigrationIncomingState *mis =3D opaque; + + migration_incoming_state_destroy(); + + if (mis->exit_on_error) { + /* + * NOTE: this exit() should better happen in the main thread, as + * the exit notifier may require BQL which can deadlock. See + * commit e7bc0204e57836 for example. + */ + exit(EXIT_FAILURE); + } +} + +static void *migration_incoming_thread(void *opaque) { MigrationState *s =3D migrate_get_current(); - MigrationIncomingState *mis =3D migration_incoming_get_current(); + MigrationIncomingState *mis =3D opaque; PostcopyState ps; int ret; Error *local_err =3D NULL; =20 + rcu_register_thread(); + assert(mis->from_src_file); + assert(!bql_locked()); =20 mis->largest_page_size =3D qemu_ram_pagesize_largest(); postcopy_state_set(POSTCOPY_INCOMING_NONE); migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_ACTIVE); =20 - mis->loadvm_co =3D qemu_coroutine_self(); - ret =3D qemu_loadvm_state(mis->from_src_file, true, &local_err); - mis->loadvm_co =3D NULL; + ret =3D qemu_loadvm_state(mis->from_src_file, false, &local_err); =20 trace_vmstate_downtime_checkpoint("dst-precopy-loadvm-completed"); =20 ps =3D postcopy_state_get(); - trace_process_incoming_migration_co_end(ret, ps); + trace_process_incoming_migration_end(ret, ps); if (ps !=3D POSTCOPY_INCOMING_NONE) { if (ps =3D=3D POSTCOPY_INCOMING_ADVISE) { /* @@ -898,7 +919,7 @@ process_incoming_migration_co(void *opaque) * Postcopy was started, cleanup should happen at the end of t= he * postcopy thread. */ - trace_process_incoming_migration_co_postcopy_end_main(); + trace_process_incoming_migration_postcopy_end_main(); goto out; } /* Else if something went wrong then just fall out of the normal e= xit */ @@ -911,8 +932,8 @@ process_incoming_migration_co(void *opaque) } =20 if (migration_incoming_colo_enabled()) { - /* yield until COLO exit */ - colo_incoming_co(); + /* wait until COLO exits */ + colo_incoming_wait(); } =20 migration_bh_schedule(process_incoming_migration_bh, mis); @@ -924,28 +945,22 @@ fail: migrate_set_error(s, local_err); error_free(local_err); =20 - migration_incoming_state_destroy(); + WITH_QEMU_LOCK_GUARD(&s->error_mutex) { + error_report_err(s->error); + s->error =3D NULL; + } =20 - if (mis->exit_on_error) { - WITH_QEMU_LOCK_GUARD(&s->error_mutex) { - error_report_err(s->error); - s->error =3D NULL; - } + /* + * There's some step of the destroy process that will need to happen in + * the main thread (e.g. joining this thread itself). Leave to a BH. + */ + migration_bh_schedule(migration_incoming_state_destroy_bh, (void *)mis= ); =20 - exit(EXIT_FAILURE); - } else { - /* - * Report the error here in case that QEMU abruptly exits - * when postcopy is enabled. - */ - WITH_QEMU_LOCK_GUARD(&s->error_mutex) { - error_report_err(s->error); - s->error =3D NULL; - } - } out: /* Pairs with the refcount taken in qmp_migrate_incoming() */ migrate_incoming_unref_outgoing_state(); + rcu_unregister_thread(); + return NULL; } =20 /** @@ -963,8 +978,12 @@ static void migration_incoming_setup(QEMUFile *f) =20 void migration_incoming_process(void) { - Coroutine *co =3D qemu_coroutine_create(process_incoming_migration_co,= NULL); - qemu_coroutine_enter(co); + MigrationIncomingState *mis =3D migration_incoming_get_current(); + + mis->have_recv_thread =3D true; + qemu_thread_create(&mis->recv_thread, "mig/dst/main", + migration_incoming_thread, mis, + QEMU_THREAD_JOINABLE); } =20 /* Returns true if recovered from a paused migration, otherwise false */ diff --git a/migration/rdma.c b/migration/rdma.c index 0e5e02cdca..3389f6448b 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3051,37 +3051,6 @@ int rdma_control_save_page(QEMUFile *f, ram_addr_t b= lock_offset, =20 static void rdma_accept_incoming_migration(void *opaque); =20 -static void rdma_cm_poll_handler(void *opaque) -{ - RDMAContext *rdma =3D opaque; - struct rdma_cm_event *cm_event; - MigrationIncomingState *mis =3D migration_incoming_get_current(); - - if (rdma_get_cm_event(rdma->channel, &cm_event) < 0) { - error_report("get_cm_event failed %d", errno); - return; - } - - if (cm_event->event =3D=3D RDMA_CM_EVENT_DISCONNECTED || - cm_event->event =3D=3D RDMA_CM_EVENT_DEVICE_REMOVAL) { - if (!rdma->errored && - migration_incoming_get_current()->state !=3D - MIGRATION_STATUS_COMPLETED) { - error_report("receive cm event, cm event is %d", cm_event->eve= nt); - rdma->errored =3D true; - if (rdma->return_path) { - rdma->return_path->errored =3D true; - } - } - rdma_ack_cm_event(cm_event); - if (mis->loadvm_co) { - qemu_coroutine_enter(mis->loadvm_co); - } - return; - } - rdma_ack_cm_event(cm_event); -} - static int qemu_rdma_accept(RDMAContext *rdma) { Error *err =3D NULL; @@ -3199,8 +3168,7 @@ static int qemu_rdma_accept(RDMAContext *rdma) NULL, (void *)(intptr_t)rdma->return_path); } else { - qemu_set_fd_handler(rdma->channel->fd, rdma_cm_poll_handler, - NULL, rdma); + qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL); } =20 ret =3D rdma_accept(rdma->cm_id, &conn_param); diff --git a/migration/savevm.c b/migration/savevm.c index 44aadc2f51..991f46593c 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2118,7 +2118,7 @@ static void *postcopy_ram_listen_thread(void *opaque) qemu_file_set_blocking(f, true, &error_fatal); =20 /* TODO: sanity check that only postcopiable data will be loaded here = */ - load_res =3D qemu_loadvm_state_main(f, mis, true, &local_err); + load_res =3D qemu_loadvm_state_main(f, mis, false, &local_err); =20 /* * This is tricky, but, mis->from_src_file can change after it @@ -2415,11 +2415,11 @@ static void loadvm_postcopy_handle_resume(Migration= IncomingState *mis) * Immediately following this command is a blob of data containing an embe= dded * chunk of migration stream; read it and load it. * - * @mis: Incoming state - * @length: Length of packaged data to read + * @mis: Incoming state + * @bql_held: Whether BQL is held already + * @errp: The Error** to set when returning failures. * * Returns: Negative values on error - * */ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis, bool bql_held, Error **errp) diff --git a/migration/trace-events b/migration/trace-events index e8edd1fbba..2b7b522e73 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -193,8 +193,8 @@ source_return_path_thread_resume_ack(uint32_t v) "%"PRI= u32 source_return_path_thread_switchover_acked(void) "" migration_thread_low_pending(uint64_t pending) "%" PRIu64 migrate_transferred(uint64_t transferred, uint64_t time_spent, uint64_t ba= ndwidth, uint64_t avail_bw, uint64_t size) "transferred %" PRIu64 " time_sp= ent %" PRIu64 " bandwidth %" PRIu64 " switchover_bw %" PRIu64 " max_size %"= PRId64 -process_incoming_migration_co_end(int ret, int ps) "ret=3D%d postcopy-stat= e=3D%d" -process_incoming_migration_co_postcopy_end_main(void) "" +process_incoming_migration_end(int ret, int ps) "ret=3D%d postcopy-state= =3D%d" +process_incoming_migration_postcopy_end_main(void) "" postcopy_preempt_enabled(bool value) "%d" migration_precopy_complete(void) "" =20 --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161254; cv=none; d=zohomail.com; s=zohoarc; b=WourvOcxmFMq5rh3SL2RL7Hr0DWuYD+kJvNJJasKuDeaRawU5jYuIktCBrh7PcMriSKWqgEf+JyKLscmb6ylFpjrNep5EK4klvzjfBtw8+XQ0EMzeHYRpuWgPlsBexgOIGBtVyEqBtxx54lvD4dAlncVuWT45OTFhtvzEBZa9ik= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161254; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=3UsHdXQwCwD7WP5y/WUYROrf5LFRHXxP7WcAGO6x+FI=; b=jHh0hXnybyyrRAF6bwoP75Q9vsREnF1ZOyPFjcIJRxxRLVrtVYcBgk0cgwSmr+6XRU/JTA+4csdg/2UDAuLbek+a2GRI91d8LtT6bTecQVmkdNXUzZ4MJ2Auy/dPoL8HKfOJaYjxC3aWCNsjlR+IaESc1tYVXD9eY3zHmJVYkXc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161254781910.073618544913; Wed, 22 Oct 2025 12:27:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTu-0002vP-Fy; Wed, 22 Oct 2025 15:26:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTl-0002tY-3o for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTh-0001Yd-Tv for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:36 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-90-amTWN4HsP5WYxQLqE10pOw-1; Wed, 22 Oct 2025 15:26:31 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e8b8db4a35so40414081cf.1 for ; Wed, 22 Oct 2025 12:26:31 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3UsHdXQwCwD7WP5y/WUYROrf5LFRHXxP7WcAGO6x+FI=; b=Or/ArdDmavQOj/RYY+sh+rjzoe1N2ljX014EBXXjgBgA9dwsPQEVD7aojLrBSldXm4TFJj N5KQa/5N7Nf1scp/UN/Rpmq7b+6QLnO+GYBTETzOpqXdmqmck4OXqyyPRvAguyGdljldvE vyDG8K0qyGNon6NKBXwM4FHFMIWqtG0= X-MC-Unique: amTWN4HsP5WYxQLqE10pOw-1 X-Mimecast-MFC-AGG-ID: amTWN4HsP5WYxQLqE10pOw_1761161191 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161191; x=1761765991; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3UsHdXQwCwD7WP5y/WUYROrf5LFRHXxP7WcAGO6x+FI=; b=NeszA4chKrE+eCuePA+QNu/xGrUYA9gg1dwk6p2x5NH6iLQXbUUz4028B3tcBhx98J RO7kOQWe5Wk74YItQMaig40Ln/7wszGIOmTCNQ8CV7ZrKflHYQd59XU5cXeTskax2jR9 fmlAR1giDrtRrN5QZa6M7mCIsQyF1qAytujBS2U9Ct92FQl8rZdpUXMt25VOCdf+RGpO em7HDT6LtQo2R8PkltLOdXelfo/j+aMf4454gJF0bxolhN4jTQCXm53yYT6a8Jpampf6 LSzcGrORtXULCd8Ma/aNcUFJZbW4G9+dUupkEUkTTJUNCD6lg251mT38YOGmHcA1T2xB f2Gg== X-Gm-Message-State: AOJu0YzZ+yK8HTTaV+4FyLcTFbPFU3VZHg3Impe8lVMZJLmGXYIpBGSW uAd+12dEqSPsfa9QNQm4lY7EbEdug1D6uvZYCmNxbABFg9RmyJ+fToGi3+ZFeIuKbPbR6YTtERm cNWtO63fYTIfcqmbW+h5RR+S/ASmwqB6DIXJZ9yid3ae/Cz5HDJegP4oNldNazK/F6GY8WlIcwq bGRmZ1JM144FSMMT1/I3JK6NuGSGBjsuYxADbVlg== X-Gm-Gg: ASbGncvEie1enMBI+FVQpieRepWnvd8/7Nxcwd8+NxtsWkDfJtZIwCkhACGvIHIzJe0 zJF5OsaOeota+HLJW9stYkmc/8t5ZdYpFy787DpaUi3VXmbP0QdPCumQYIntCUEi+MrmrBPvq4P oZJ7sO5rcS+pMhRLj3yjlHLIXcYoqEAui7SHC9TZFyX7+ZEgVp3HTJTbohaQpnO6uSi1Mfo0pHk 2wx81D39uyLcj1iFBaQSFRmTUz46HiM96fOt6m9cv/kRFdl2EW9tEhnbRfVzh1felk/VgZHv4f5 c7iXlV8Z+3PxxOgORc5x6fzPSQDIRwUuYydyTVLnAknj1ArvEOA/zVUF346ehAoC X-Received: by 2002:a05:622a:1ccd:b0:4e8:a2aa:77c9 with SMTP id d75a77b69052e-4e8a2aa88aamr220367061cf.1.1761161190844; Wed, 22 Oct 2025 12:26:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEOV+yjiKv1OCk6ubza4PxjCziIxPWCkWWJLrLvCUjFUeH/dgGNuskUR34ENQD5xGL1oIlo8A== X-Received: by 2002:a05:622a:1ccd:b0:4e8:a2aa:77c9 with SMTP id d75a77b69052e-4e8a2aa88aamr220366601cf.1.1761161190355; Wed, 22 Oct 2025 12:26:30 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 09/13] migration/rdma: Remove coroutine path in qemu_rdma_wait_comp_channel Date: Wed, 22 Oct 2025 15:26:08 -0400 Message-ID: <20251022192612.2737648-10-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161263523158500 Content-Type: text/plain; charset="utf-8" Now after thread-ified dest VM load during precopy, we will always in a thread context rather than within a coroutine. We can remove this path now. Reviewed-by: Zhijian Li (Fujitsu) Signed-off-by: Peter Xu --- migration/rdma.c | 90 ++++++++++++++++++++++-------------------------- 1 file changed, 41 insertions(+), 49 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 3389f6448b..67119634d7 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -29,7 +29,6 @@ #include "qemu/rcu.h" #include "qemu/sockets.h" #include "qemu/bitmap.h" -#include "qemu/coroutine.h" #include "system/memory.h" #include #include @@ -1345,61 +1344,54 @@ static int qemu_rdma_wait_comp_channel(RDMAContext = *rdma, { struct rdma_cm_event *cm_event; =20 - if (qemu_in_coroutine()) { - yield_until_fd_readable(comp_channel->fd); - } else { - /* This is the source side, we're in a separate thread - * or destination prior to migration_fd_process_incoming() - * after postcopy, the destination also in a separate thread. - * we can't yield; so we have to poll the fd. - * But we need to be able to handle 'cancel' or an error - * without hanging forever. - */ - while (!rdma->errored && !rdma->received_error) { - GPollFD pfds[2]; - pfds[0].fd =3D comp_channel->fd; - pfds[0].events =3D G_IO_IN | G_IO_HUP | G_IO_ERR; - pfds[0].revents =3D 0; - - pfds[1].fd =3D rdma->channel->fd; - pfds[1].events =3D G_IO_IN | G_IO_HUP | G_IO_ERR; - pfds[1].revents =3D 0; - - /* 0.1s timeout, should be fine for a 'cancel' */ - switch (qemu_poll_ns(pfds, 2, 100 * 1000 * 1000)) { - case 2: - case 1: /* fd active */ - if (pfds[0].revents) { - return 0; - } + /* + * This is the source or dest side, either during precopy or + * postcopy. We're always in a separate thread when reaching here. + * Poll the fd. We need to be able to handle 'cancel' or an error + * without hanging forever. + */ + while (!rdma->errored && !rdma->received_error) { + GPollFD pfds[2]; + pfds[0].fd =3D comp_channel->fd; + pfds[0].events =3D G_IO_IN | G_IO_HUP | G_IO_ERR; + pfds[0].revents =3D 0; + + pfds[1].fd =3D rdma->channel->fd; + pfds[1].events =3D G_IO_IN | G_IO_HUP | G_IO_ERR; + pfds[1].revents =3D 0; + + /* 0.1s timeout, should be fine for a 'cancel' */ + switch (qemu_poll_ns(pfds, 2, 100 * 1000 * 1000)) { + case 2: + case 1: /* fd active */ + if (pfds[0].revents) { + return 0; + } =20 - if (pfds[1].revents) { - if (rdma_get_cm_event(rdma->channel, &cm_event) < 0) { - return -1; - } + if (pfds[1].revents) { + if (rdma_get_cm_event(rdma->channel, &cm_event) < 0) { + return -1; + } =20 - if (cm_event->event =3D=3D RDMA_CM_EVENT_DISCONNECTED = || - cm_event->event =3D=3D RDMA_CM_EVENT_DEVICE_REMOVA= L) { - rdma_ack_cm_event(cm_event); - return -1; - } + if (cm_event->event =3D=3D RDMA_CM_EVENT_DISCONNECTED || + cm_event->event =3D=3D RDMA_CM_EVENT_DEVICE_REMOVAL) { rdma_ack_cm_event(cm_event); + return -1; } - break; + rdma_ack_cm_event(cm_event); + } + break; =20 - case 0: /* Timeout, go around again */ - break; + case 0: /* Timeout, go around again */ + break; =20 - default: /* Error of some type - - * I don't trust errno from qemu_poll_ns - */ - return -1; - } + default: /* Error of some type - don't trust errno from qemu_poll_= ns */ + return -1; + } =20 - if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_CANCE= LLING) { - /* Bail out and let the cancellation happen */ - return -1; - } + if (migrate_get_current()->state =3D=3D MIGRATION_STATUS_CANCELLIN= G) { + /* Bail out and let the cancellation happen */ + return -1; } } =20 --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161248; cv=none; d=zohomail.com; s=zohoarc; b=ZyWcNP+I/FoPho1kh+qIqBGOysX2d6F3uO6MNP6/ltKpSj9nINYgCucrzsQiiyi5WmOtrvxHoQevg94tjqkFXM0Jcqg6HZ+XnaLa3jBKZq+0nFukPdtrJERUKm6CNds5tqV08Ss4hj4fYiYpAU2PoyZpF1/uFms9OWVd3YkXH+8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161248; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=JW8aeNeh6Nu+nmUpwT9UAvg9KPt5UcYpUXSC4KDXYdA=; b=crlc18lMAKIJOgw0xa+5fuBexKQYGVHIHg9Q3OzdwTTnzoBLhu4RpsJVMGeDMFFjefqGbpdi6alsdP5CsB0Q0nTFHxUcHi0urr12biUdUtLvdX0W5h5af9EMJkRSzaaEcy8UQm9YRKHGN2ZAJicX236rJJIqpeWJDUC+Ccwzlrw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161248417841.911866535975; Wed, 22 Oct 2025 12:27:28 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeUA-00032W-3W; Wed, 22 Oct 2025 15:27:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTr-0002uH-M5 for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTp-0001al-Ca for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:42 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-YYW8GBPZPOKRjDXDatLV7w-1; Wed, 22 Oct 2025 15:26:33 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e884663b25so50268361cf.0 for ; Wed, 22 Oct 2025 12:26:33 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JW8aeNeh6Nu+nmUpwT9UAvg9KPt5UcYpUXSC4KDXYdA=; b=Yp0/A4dCKMKagzMBElLEkvV/0ttjONmLomM4GBx0deWqeni/KpW0WxBtjH1T6A16KT8ufz NdZhmCpCsCo/NJDbt26mfgGjaaQhxl68Ya0iUWdy2AAES3guZCHwEPXnOhhqF9cChHRxAI 0kNAa1ms8jazsx6JLJ5SacomukGpdBQ= X-MC-Unique: YYW8GBPZPOKRjDXDatLV7w-1 X-Mimecast-MFC-AGG-ID: YYW8GBPZPOKRjDXDatLV7w_1761161193 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161193; x=1761765993; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JW8aeNeh6Nu+nmUpwT9UAvg9KPt5UcYpUXSC4KDXYdA=; b=Sv2SYNPaQ+N+SIT1wuWWPrtkpDkW6pHLrxxf3j+TBmtwBs9Oc+HiZWGLppEDP0pONA P1IvG9nZbbCWbrYKgBUTBksGnmeovo7wAia1RAm59FWNTEUp3oUNrAJvGxddrfM45aFa CZZCQGlNCoWjaC2E5hdFv4H90mXTU6QbyW/dVw02pDc3x7EwV3FfvPXup5e1ZpHlim5u wLxcwPTgXDbln4W1kQp6Wh15BQ3a9dNObWMm1yN5NNDsWGyFuKN+vQ0CtC/d8V1VQxE0 woGpWCHMJwzWvNIvC45ciuGwP4ZL0tYS0roPjUjePE/FTg05lr4ap2gKNMk6cIFsp48b 5uGw== X-Gm-Message-State: AOJu0YzUYAcgBwsMwv8lPOsD8l1Q/6SLz+lr0BcDjtF/7J52wKVbw+uW UpeZ1+/MROJh9mX9nYMTDpM7n0g2v3LICWokuc3WrmcNRNbcQlOwvdAo0aW2V7SFro9uz3bjJR3 VX8lYm4NoJTbl7zPKKP/z9jAqiZSA9F/T0vLHEGhHck06ln406VdqjBJnY1CBJxa8KQoXsTklzK 3ct6UtsgsXFXkyIpjkeBrG5oHgtXAuPLY1n1Bg6A== X-Gm-Gg: ASbGncsKOTXTyUlNiC58oERoYMLSt7dPENWtkczPqmoLayuU+bXzMjPJ4w2X2J/2LY7 D49lvqKDKqBIrGeVvy3Wc3NoxS15QTbTwh79S+0SMwpwbb3lD/j+6aELkQG+sO4eFgqgYQZYleQ u3lEbAH+NhT4iRhj9/DLdASSoRGDvSpkBDIfuWQixJUA9PDodCkxK68mVhsQrw1P9gT6Efp0TtZ 7oGlAtv5gRI2r8feK5ogGLjJU9NIs6Fm8pOjrBKWCQBAKFixaGU6Cn+2iXhfUbvseXDt7yZmFA2 QRmqpWf1DXt6MpCTV/MRhplS3K8M7rScIxU3HRET5IWUP3U8n1K2xYJy/3POww15 X-Received: by 2002:a05:622a:1451:b0:4b3:4457:feca with SMTP id d75a77b69052e-4e89d1ff721mr252775541cf.6.1761161192561; Wed, 22 Oct 2025 12:26:32 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF9t7boqwotXXvEgahMzh123AgygpagLLvET1D2CLM1h99JEh40wGE+UKWXyI4kvz626sCciA== X-Received: by 2002:a05:622a:1451:b0:4b3:4457:feca with SMTP id d75a77b69052e-4e89d1ff721mr252775071cf.6.1761161191927; Wed, 22 Oct 2025 12:26:31 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 10/13] migration/postcopy: Remove workaround on wait preempt channel Date: Wed, 22 Oct 2025 15:26:09 -0400 Message-ID: <20251022192612.2737648-11-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161251065154100 Content-Type: text/plain; charset="utf-8" This reverts commit 7afbdada7effbc2b97281bfbce0c6df351a3cf88. Now after switching to a thread in loadvm process, the main thread should be able to accept() even if loading the package could cause a page fault in userfaultfd path. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/savevm.c | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 991f46593c..16fae635c0 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2452,27 +2452,6 @@ static int loadvm_handle_cmd_packaged(MigrationIncom= ingState *mis, =20 QEMUFile *packf =3D qemu_file_new_input(QIO_CHANNEL(bioc)); =20 - /* - * Before loading the guest states, ensure that the preempt channel has - * been ready to use, as some of the states (e.g. via virtio_load) mig= ht - * trigger page faults that will be handled through the preempt channe= l. - * So yield to the main thread in the case that the channel create eve= nt - * hasn't been dispatched. - * - * TODO: if we can move migration loadvm out of main thread, then we - * won't block main thread from polling the accept() fds. We can drop - * this as a whole when that is done. - */ - do { - if (!migrate_postcopy_preempt() || !qemu_in_coroutine() || - mis->postcopy_qemufile_dst) { - break; - } - - aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_sel= f()); - qemu_coroutine_yield(); - } while (1); - ret =3D qemu_loadvm_state_main(packf, mis, bql_held, errp); trace_loadvm_handle_cmd_packaged_main(ret); qemu_fclose(packf); --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161233; cv=none; d=zohomail.com; s=zohoarc; b=f0BoYq3k/x+a+xR2eyFR0+i+zf8Pi+jzvQdaSdnt5GNXFaHLJdHus+X+V3g24eFVALxQrS+tKO0qhDAfL9mjFq/qvu6QiNg8W+lfjVPZNIPYURvNqy9h++PFLh34swpETKVNHNXg3HM2lInDPyMZbmt9gm5coKy67bj0YE77SOs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161233; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fOhkWGBswSimZhnhemfbMKnjzI+fXszEQjpRE4VPOI8=; b=jU/NzABD3NCSPuy+G43SkHoCzp/4mbUR+qozf81Jj27/IAMcu/rErXdapnt45TjTQAko4JbeovmE6N7nLnfUfiaTLu075rtdYqnvJUKsQcxuNhHGCBxjXSCEacb7zwhtX1dmU0Hk8usfxPUJKDZFdmGaKxtXb9TY5zfxrw0BE70= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 176116123332899.5881436073696; Wed, 22 Oct 2025 12:27:13 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTw-0002wI-Ug; Wed, 22 Oct 2025 15:26:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTn-0002ts-VI for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTl-0001aI-En for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:39 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-453-drhglUVyPgO_8hWNbnXhiA-1; Wed, 22 Oct 2025 15:26:35 -0400 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-4e89265668fso42320391cf.3 for ; Wed, 22 Oct 2025 12:26:34 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161196; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fOhkWGBswSimZhnhemfbMKnjzI+fXszEQjpRE4VPOI8=; b=JuM+ERecdkS0QW+QzwIXV0lOc00K6/x6vKJG510pl3sX3Homa08llm8TCY2OoFBxHgjZHe jyRXHFvolcVtoE/iJQtpDpPoywSr/Ek+WvyUoKdS3alRr4UhfY2BI2t1ziWuoipQI9/6Mx i6NvL3RAYd0WfqdcznQfY/63KNuwDmg= X-MC-Unique: drhglUVyPgO_8hWNbnXhiA-1 X-Mimecast-MFC-AGG-ID: drhglUVyPgO_8hWNbnXhiA_1761161194 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161194; x=1761765994; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fOhkWGBswSimZhnhemfbMKnjzI+fXszEQjpRE4VPOI8=; b=lA3gOllVn2hvdkUqWUUFFpRlMvSU0E1wJdboZaEevhiA1uULr1sEmksC8kjfY4a1f+ Vgub1XvikKM17xyNhcKnX5wJzsvz+4WabHRDJkdnRR6xjh2cnKu2ff/Z6OhzOX+GD64C /E2xqZycpEwTW0z6dTeLZBsSE9bbdv+I1ylnvpC7/Mib4P5Xr3dq0GPcJzQfYf8KEKwy olWsIqkivICNogRiver4HXtHovqc6lzeS/t2PWoNp8j1cxhbEbFaxjMkboI9iPSC8yU9 bLLKXB3ztGbCNVbazytwrxlEXqGgHkWPX45h9CsXl3FzqnuHzAYw5XKaALKYlCUCNooF v+8Q== X-Gm-Message-State: AOJu0YyS6UFXmpk1PQKYJt2Wpao83lmhs1MBLoRWhV+vfzQ4a6/anS0/ Yk48MIDcj0uuNoenF8zI0ZAjb9Uu2t8fy8hDlc7p4EsCoNaiLPBigYla3M0dxOTdNjiIK5wzwwn Q5H7AM0oOrBTMHV4MwdqSH0lcgT3814aZkyDP447UFnkI3MB01ftZu6u5lfrrSaGz9kZeAwKAMq uZAg7iflN7YiCjbcch0AxmvwEy6pSMRjVCb803zg== X-Gm-Gg: ASbGncvV7+YMqejpUJsFqk//4Hcq1X/I2YFZJJ39c7sGyZ1nO5VEPMFllzeDuhZ2sMq iY0BuztrA1DOB28rhDOgRRHBPfOLzSzVa92v5xuNjT5rkXDADqCmmCqO7Ue1lp8TirqcVD1pO54 Ek4rgAwgYXaFW88mNiLw41kOcl0y4Pd/Mm3V/3higKQbXT1qZtmeYgCwYEOKjHvg6Ga5XytOeU2 wSDY9o9sNbT1H+D0GNwamKoy8lmOgTfXJNLmFagPMRpM6AsZsBbPFd51wEAKniZlwgJtHWFjxbh 5RkjUabf0pQ8/BCGkNhbquEzijZM7KZknb56sJ67CA9qBOmg7R2oFa6cahmhjBRf X-Received: by 2002:ac8:5ac6:0:b0:4de:cb90:dbe4 with SMTP id d75a77b69052e-4e89d393b93mr287185051cf.66.1761161194054; Wed, 22 Oct 2025 12:26:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGJ8zZNeARO/JM75anJs6i/kTg4x4XiXSoJyeRTsYAiP17yDsWdwepbnJDPz65oJf4yguXO9g== X-Received: by 2002:ac8:5ac6:0:b0:4de:cb90:dbe4 with SMTP id d75a77b69052e-4e89d393b93mr287184651cf.66.1761161193521; Wed, 22 Oct 2025 12:26:33 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 11/13] migration/ram: Remove workaround on ram yield during load Date: Wed, 22 Oct 2025 15:26:10 -0400 Message-ID: <20251022192612.2737648-12-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161236703154100 Content-Type: text/plain; charset="utf-8" This reverts e65cec5e5d97927d22b39167d3e8edeffc771788. RAM load path had a hack in the past explicitly yield the thread to the main coroutine when RAM load spinning in a tight loop. Not needed now because precopy RAM load now happens without the main thread. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/ram.c | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 29f016cb25..16e81cf3bb 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -4304,7 +4304,7 @@ static int parse_ramblocks(QEMUFile *f, ram_addr_t to= tal_ram_bytes) static int ram_load_precopy(QEMUFile *f) { MigrationIncomingState *mis =3D migration_incoming_get_current(); - int flags =3D 0, ret =3D 0, invalid_flags =3D 0, i =3D 0; + int flags =3D 0, ret =3D 0, invalid_flags =3D 0; =20 if (migrate_mapped_ram()) { invalid_flags |=3D (RAM_SAVE_FLAG_HOOK | RAM_SAVE_FLAG_MULTIFD_FLU= SH | @@ -4317,17 +4317,6 @@ static int ram_load_precopy(QEMUFile *f) void *host =3D NULL, *host_bak =3D NULL; uint8_t ch; =20 - /* - * Yield periodically to let main loop run, but an iteration of - * the main loop is expensive, so do it each some iterations - */ - if ((i & 32767) =3D=3D 0 && qemu_in_coroutine()) { - aio_co_schedule(qemu_get_current_aio_context(), - qemu_coroutine_self()); - qemu_coroutine_yield(); - } - i++; - addr =3D qemu_get_be64(f); ret =3D qemu_file_get_error(f); if (ret) { --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161350; cv=none; d=zohomail.com; s=zohoarc; b=aNIDskF+OFmsacAKRrd1MA3TaLjaFe04I4yIIO/RVJJ/IhoAea8xmtpR0vULyhcUyNIMaBIFqfMdWxk9INZzrigABxRDe7DcrX/tyRl9HdHgLJTfMEeSPOexq6euJw2htGgrNLGCvYKMBhPuaCS6QXhH6Y4zEaUBzw2xCy3vwXE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161350; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CkfFzD5ZM5wWqXJjmTYhmklbUzKo4o7bvR9HregSsu4=; b=hySGmKO57gNznqQ41bLrO65MJNh+g21tcJhaOZfILolRKeufwTEbnFrwpnMjXmcMbZ3IpM1L1D1gyGoSyCHiDrdQU5aoAYHz2OmghR3bDB3HCQt328CnoN23C/G1K136j1rCg7tvk1c4FOqNLt5Qiu6dWcS74SGJe5s586yXvtU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161350050662.8687883212752; Wed, 22 Oct 2025 12:29:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTv-0002vY-KO; Wed, 22 Oct 2025 15:26:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTn-0002tr-VJ for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTm-0001aT-Dk for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:39 -0400 Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-505-YZeCk_AyP5aFkrFqK_0s6Q-1; Wed, 22 Oct 2025 15:26:36 -0400 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-87dfd03fb72so41274426d6.3 for ; Wed, 22 Oct 2025 12:26:36 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161197; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CkfFzD5ZM5wWqXJjmTYhmklbUzKo4o7bvR9HregSsu4=; b=eF0YlI/vT51f3fLn/lmdC2szlh2epYryZCz1fUMwOLlTX3Pc+OeDdxF7OehxYiQoyWUWWy 9S6y1RrNm87qG/bb9Rh0Vwrm60Qjk/YSfK/RfzvTKX/NJD+qCmS7S9OendS/a7lTzIIOtt Kl6VibXIq0WZ2E963nzAmun8apfgoss= X-MC-Unique: YZeCk_AyP5aFkrFqK_0s6Q-1 X-Mimecast-MFC-AGG-ID: YZeCk_AyP5aFkrFqK_0s6Q_1761161196 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161195; x=1761765995; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CkfFzD5ZM5wWqXJjmTYhmklbUzKo4o7bvR9HregSsu4=; b=HCp7aUIFvavxddmc7BB8lh578G5F+rgDPWEsNJx1LRA2Gg0HTg8qjlZf+0NykU9XXH JYloyUDarEksaFEsx81iXgoJqHwv6U4sdY/+V3xUFl9RSPhwFHrwoPktGagQDdjp4dm/ a1q1wuaVNbpYnFA3lYmo/54yd4/01efLkviSPbmocU7d3Qut4CE9pqn5GYSWwzj89D2w bk4V+azAZJ4vjbv8tl76xGzAZKPfiPUNeY951GRKWE18CGZvYd7yuADtGbAYoqe+52V6 mJpZjjEt+CzpuFjNy6wadjs5xDTa8QipdO9Wf3P5pUS5lNXb82mptqtHPp2yVoyiIjEO kHFQ== X-Gm-Message-State: AOJu0YwHYKBHakvFDScIC0VumluIFvDSmcvnfbUMvG8tJqAKzgf//Ap6 I1Ts4yMrGi16B8BEQ+wcgSw2r56pOTGA1zNVCVtCNYNIJ3NNhenANZ0JL9YUU1xK8y2NNC11RMZ qjW8IPW2BkI7gHuTl3VJHKUQ62eEb/2ZR3J/MMvmHtuX9pEj3kN6lXo0Q2fjsLn8MM/9CKh+h2p ObqS2159wTWtxXuUFVPxkQ+yUZ7t4XBDRsLd7itw== X-Gm-Gg: ASbGncsdxbzl6DPuQxZLo2G6DYOxin9/RpgK6KAYnac+JoCEmeDvP6KaonrdNjnTFjI AeuKPV15HhAdTgKRDV+HJuH8HpJbfKdW5dK83rpkuPcPKauM/V456Fm+fdUD3G0uA1kWdhr/Eax 4R87bf0OKBAQKbEHbpWd3GGlFeswaKq550LpIDM7F/CJaBtxIv/bjSaBslImT74FT9+NoxJROOL r2LiJLNDHE0p0aQlWsBMz3Vuw8mWGLLYcLnAFNsY1tXQ8QgVuLys8eovsX7L881FNm9XqXAueIJ Jej3NZN+J4PSsZX3gci+DKgIerX4Fj99z2drv/NxYemKgHt/Qn/S8pMHMUKUrrw4 X-Received: by 2002:a05:622a:1116:b0:4e8:9683:3a35 with SMTP id d75a77b69052e-4e89d065ea6mr264266471cf.0.1761161195462; Wed, 22 Oct 2025 12:26:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGEnuXwuHSwpW8s4w+ENkN1x9+zbXu+ktZfvdqQj+MZNEmrPbf1Lbi+g5Ma+KrRNR4brXu/fw== X-Received: by 2002:a05:622a:1116:b0:4e8:9683:3a35 with SMTP id d75a77b69052e-4e89d065ea6mr264266101cf.0.1761161195018; Wed, 22 Oct 2025 12:26:35 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin Subject: [PATCH 12/13] migration: Allow blocking mode for incoming live migration Date: Wed, 22 Oct 2025 15:26:11 -0400 Message-ID: <20251022192612.2737648-13-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161358315158500 Content-Type: text/plain; charset="utf-8" We used to set nonblocking mode for incoming live migration. It was required because while running inside a coroutine we cannot block in a syscall. Instead of that, when a syscall needs blocking, we need to register to the iochannel's GIO events and proactively yield the coroutine so as to make sure other things can keep running on the QEMU main thread. When the migration channel has pending data to read, QEMU main thread will re-enter the blocked incoming migration coroutine. Now, incoming migration is completely run inside a thread now. We can safely switch to blocking mode for the main incoming channel. It's still slightly more efficient to use blocking mode than nonblocking, because QEMU can avoid creating temporary mainloops and avoid the extra poll() syscall. Now it's safe to directly block in the iochannel's syscall (e.g. io_readv()). Signed-off-by: Peter Xu --- migration/migration.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 728d02dbee..abd76801c2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -973,7 +973,9 @@ static void migration_incoming_setup(QEMUFile *f) =20 assert(!mis->from_src_file); mis->from_src_file =3D f; - qemu_file_set_blocking(f, false, &error_abort); + + /* Incoming migration runs in a thread now, nonblocking is not needed = */ + qemu_file_set_blocking(f, true, &error_abort); } =20 void migration_incoming_process(void) --=20 2.50.1 From nobody Fri Nov 14 19:40:01 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1761161246; cv=none; d=zohomail.com; s=zohoarc; b=QHk0r6xQng7CclI9bqhpUDnxhbwaXnzBIwzwkXrCo3bC5Omkp0cLlSPe2OmNJexDLbYQh/qUlImxqkSur4OZkV0NXXnXCCIsjbl0TTk3/6fNHHposAo/ZeoXYFGT95E2hUMeEmv9i5+T7hJn/hM6qLWeD8Rjg4KAVwPg9mBBedA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1761161246; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RCz1Mt3gTG/0s7R16hMFmFgj60vzRZb4DcUvnfXRMOU=; b=bLnAXqiAjvRB62m2D4fDMTLdJw1NOtV66Gz2GX1LTvjrW4boo3BV+ulsya5QtPQ15mx0FcxRboIQkLBx0eDKyOkdMirIpNK3q+91DLVfDNGAfERXW8kWL4f4/U+X77NFN+y0jq6FsXUlaNyCp8gE7nSz6SDyzLVmS1oTusEPhTk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1761161246949730.7983286440171; Wed, 22 Oct 2025 12:27:26 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vBeTw-0002vl-CM; Wed, 22 Oct 2025 15:26:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTr-0002uI-KM for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vBeTp-0001ad-CR for qemu-devel@nongnu.org; Wed, 22 Oct 2025 15:26:42 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-631-ty4JxtnaPZyFCEu1Q7JMHQ-1; Wed, 22 Oct 2025 15:26:38 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-4e8b33c1d8eso44217061cf.2 for ; Wed, 22 Oct 2025 12:26:38 -0700 (PDT) Received: from x1.com ([142.188.210.50]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4e8aaf34307sm99748561cf.1.2025.10.22.12.26.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Oct 2025 12:26:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761161199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RCz1Mt3gTG/0s7R16hMFmFgj60vzRZb4DcUvnfXRMOU=; b=GzlOipz/eigCBGAr6MHPij+B6Mt2IdEP0wQQsHU0jxxm1PTXhZW8YN8uERnnZWIKJ+Jysy VpTvcyINxRqa3PLC5obNW7miJH4mmNQIx0FJRwZLXaXBiTo1covD1xwhyc2nkRzd0t2fPE 95c7r4GbHBv3Jpiw7IEI/+sretSbWbw= X-MC-Unique: ty4JxtnaPZyFCEu1Q7JMHQ-1 X-Mimecast-MFC-AGG-ID: ty4JxtnaPZyFCEu1Q7JMHQ_1761161198 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761161197; x=1761765997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RCz1Mt3gTG/0s7R16hMFmFgj60vzRZb4DcUvnfXRMOU=; b=DVi+NE918Ik5q5q0VPxzPMi383Md5S/S1FcWeXcmDgEovO1yJDTT10N/blr0G5kcWR YE9cl7V1nUnouSYjbV1PJqy3vsiqpRAkfAvJpz/qDqgETih8I1RnUj42YIZ2KRCwQ4Ym E5It2zaQdkeh9DY+Uic68Av5+i7a9GcoE6eJBrbXi0aHnDf+b2qLP6FNuH+7HWV+PM4i S1cSqmDZcJ/GK2EBFn0KOlDQoNX6VtgvFWJ2v7layDnRPXWyN8l85FUyLplv7Av8KiRn CZkNCIvoBRXErar7kfzTPxRiyGW9OSC+xeXk9ubKfoMe8kEPujRGM3GvZ7zAt6j/urQP oImw== X-Gm-Message-State: AOJu0Ywx85HRGxu754BX7+9vK1qBml7iZ3BEaVQggYyGsDHeECEt4rfx tr4XgUU8LpvbT9nsBEzzDLNT8yFmPpWYY6xS2MQgTlxy+kVsokyOEuiNG5Anxs3/Jc3Rg2IVmE3 2QUIrGZ/EO6mk6lW1hCNbfYL24RWarF9vgiSQdkr1erDiWKMLnm9KvGUyAA8jzJKau1WNEmsH5l JxYkRM8sfng42Mh8wjaBbUJD+WzjX9hEprOr8eqw== X-Gm-Gg: ASbGncujy4dqRtg/D6I5bmq5pA0UFyEnK+k0gciMO24ZMvSRiGhGH4jcZZVYip+C0jg Pr3fYuewis8ivvjKbetacMpkcN/RCKR5Wz2/Y7gUbMJ89X+iQvgJD3tq7t9uv7b9J/6KmX2U9E6 PZWdPFmJt9+vLowBJYkSI0eQU4c3G6LcN8cUnr2J8pBxi3Ofl/F/t3rjTrDECD86o/Sx5xV02M3 QylIXmROKuo+F1KYCxny8bRqU/9IcAz1uCds4vDzBEUAhoI6zmM79Q01bqduAJy0MODMtEtpvTY zG2O/710deYFO4tdAiO53E7Ns+2W3e+H2XOeU9riAcLUDeeGF/4DZfHBtGcfHHiq X-Received: by 2002:ac8:7f50:0:b0:4e8:999b:70a9 with SMTP id d75a77b69052e-4e89d4123dfmr243437021cf.72.1761161197226; Wed, 22 Oct 2025 12:26:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE+7+PkJ8yuFjcIgOIzPsP1vhB0nMTTXKNBqSVlLqqnJXccpPbF8tX8sCpxXNiqLG9IqdlLEA== X-Received: by 2002:ac8:7f50:0:b0:4e8:999b:70a9 with SMTP id d75a77b69052e-4e89d4123dfmr243436591cf.72.1761161196686; Wed, 22 Oct 2025 12:26:36 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Li Zhijian , Hailiang Zhang , Kevin Wolf , Vladimir Sementsov-Ogievskiy , peterx@redhat.com, =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Fabiano Rosas , Zhang Chen , "Dr . David Alan Gilbert" , Prasad Pandit , Paolo Bonzini , Yury Kotov , Juraj Marcin , "Maciej S. Szmigiero" , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH 13/13] migration/vfio: Drop BQL dependency for loadvm SWITCHOVER_START Date: Wed, 22 Oct 2025 15:26:12 -0400 Message-ID: <20251022192612.2737648-14-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251022192612.2737648-1-peterx@redhat.com> References: <20251022192612.2737648-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1761161248910154100 That was there only because we used to do loadvm in a coroutine on the main thread with BQL held, while there was a locking order issue. Now make the API to not depend on BQL anymore. Instead making sure we don't take BQL invoking switchover_start() hooks. Cc: Maciej S. Szmigiero Cc: C=C3=A9dric Le Goater Signed-off-by: Peter Xu --- hw/vfio/migration-multifd.c | 3 --- migration/savevm.c | 3 +-- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/hw/vfio/migration-multifd.c b/hw/vfio/migration-multifd.c index e4785031a7..c824f30fb0 100644 --- a/hw/vfio/migration-multifd.c +++ b/hw/vfio/migration-multifd.c @@ -766,13 +766,10 @@ int vfio_multifd_switchover_start(VFIODevice *vbasede= v) =20 assert(multifd); =20 - /* The lock order is load_bufs_mutex -> BQL so unlock BQL here first */ - bql_unlock(); WITH_QEMU_LOCK_GUARD(&multifd->load_bufs_mutex) { assert(!multifd->load_bufs_thread_running); multifd->load_bufs_thread_running =3D true; } - bql_lock(); =20 qemu_loadvm_start_load_thread(vfio_load_bufs_thread, vbasedev); =20 diff --git a/migration/savevm.c b/migration/savevm.c index 16fae635c0..f67b97c9ab 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2648,8 +2648,7 @@ static int loadvm_process_command(QEMUFile *f, bool b= ql_held, Error **errp) return loadvm_process_enable_colo(mis, errp); =20 case MIG_CMD_SWITCHOVER_START: - WITH_BQL_HELD(bql_held) { - /* TODO: drop the BQL dependency */ + WITH_BQL_RELEASED(bql_held) { ret =3D loadvm_postcopy_handle_switchover_start(errp); } return ret; --=20 2.50.1