From nobody Mon Sep 28 00:34:17 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1786963951; cv=none; d=zohomail.com; s=zohoarc; b=Pox79MPZfDxhUCkKcg3tBP4mKGexLmAhVVMfQxjsMNeUx+qvxKbonTw9dfPR0tZ9dVk0ZlaC1n7A3u9zZ5+s/aTT2cq25vhLAW1CfWOLwRGIi263wLl999RQHlwEjSu2auBrgu9+gZP989NgLaSQmVt2joxkaWqNbhJOEjNOCo4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786963951; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=KeNB/O771Wsc6HXEPN6puL6MICpGNmrAmdDLsnkeGs0=; b=k8ahyWVrW1v51opmgk0OhJCNBYRC5FO6hVaYbtMgQo+WBI3anqyRLGGfmU0ZDh2TfmuPbXnXPsuTdmtHObulv11nEXW6aIc8QUXHqsbAeqten8oHqgdQyJqXea8ZuRfDhkGV8Vo2W7D/6BojJVxkPNZi3mm03+QwKduiDGT0wCg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786963950954524.1965728759006; Mon, 17 Aug 2026 03:52:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wvuwW-0008Dm-4C; Mon, 17 Aug 2026 06:51:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wvuwS-0007tp-1o for qemu-devel@nongnu.org; Mon, 17 Aug 2026 06:51:44 -0400 Received: from va-1-113.ptr.blmpb.com ([209.127.230.113]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wvuwN-0005Ft-Pk for qemu-devel@nongnu.org; Mon, 17 Aug 2026 06:51:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1786963894; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=KeNB/O771Wsc6HXEPN6puL6MICpGNmrAmdDLsnkeGs0=; b=A9G+PP+c/EyfQfV30CF6C4qIpZKGdxJkivoW1bL5W9N1YHJi/tS56kuK5KDGQrDMNM6lUc chNl8qrpR1Ct9eRQC4+Ptbxde/gE2Pc2WEf0nhRMAYUpha17nrdcknaXgRIOXmK1TjP8g3 kw7KxWaootSZZ8Zv5TUNcEgmUlm/V9SXWJE5L8CMSmV12GQqv5YbGfY1mpyqP3/8dYzWCO wy9WnbC6IYpn9TJ5dUEABMhLsgXDB7RucuI1kXwplM/Truc9ro84vxp0AGOoOv9OWpMY/V 5av72ZJj3NwJmiPuqyLiXPDW9VGuiR0Oel0THTAqWSBZg6P6Vlx2sNdu6P5uEA== Content-Transfer-Encoding: quoted-printable X-Original-From: Yanfei Xu To: Cc: , , , "Yanfei Xu" Message-Id: <20260817105117.3072040-2-yanfei.xu@bytedance.com> In-Reply-To: <20260817105117.3072040-1-yanfei.xu@bytedance.com> X-Mailer: git-send-email 2.20.1 X-Lms-Return-Path: Subject: [PATCH 1/3] migration/rdma: Fix write-side shutdown Date: Mon, 17 Aug 2026 18:51:15 +0800 References: <20260817105117.3072040-1-yanfei.xu@bytedance.com> From: "Yanfei Xu" Mime-Version: 1.0 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=209.127.230.113; envelope-from=yanfei.xu@bytedance.com; helo=va-1-113.ptr.blmpb.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1786963952892158500 Content-Type: text/plain; charset="utf-8" qio_channel_rdma_shutdown() reads rdmain for both the input and output contexts. As a result, a write-side shutdown marks the input context as errored while leaving the output context active. A shutdown of both directions also fails to stop the output context. Read rdmaout from the output context so each shutdown direction marks the corresponding RDMA context as errored. Fixes: 54db882f073a ("migration: implement the shutdown for RDMA QIOChannel= ") Signed-off-by: Yanfei Xu Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu --- migration/rdma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index 438419d189..e17ace62b0 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2982,7 +2982,7 @@ qio_channel_rdma_shutdown(QIOChannel *ioc, RCU_READ_LOCK_GUARD(); =20 rdmain =3D qatomic_rcu_read(&rioc->rdmain); - rdmaout =3D qatomic_rcu_read(&rioc->rdmain); + rdmaout =3D qatomic_rcu_read(&rioc->rdmaout); =20 switch (how) { case QIO_CHANNEL_SHUTDOWN_READ: --=20 2.20.1 From nobody Mon Sep 28 00:34:17 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1786963985; cv=none; d=zohomail.com; s=zohoarc; b=B8YhugkPvmZnmrG5aDGwHJvLBS2rBd+qnCiecvdUmIByQeyTjDvE4i2Qryy1MSjG6TUbv6i4qJoeOnG+fabGFA7r3bscyrf7rNHkL1lDZKmHSpSLa1bW5/9qFFdUjtm4oKGJqPoJulWIdUgc5sOXNR6MLoNfwR8jNaEzTPMtbJ0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786963985; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2FsRJH0EtbgoAnU9VEA61B6WUwpkCLdDGB2Fom+vdiU=; b=PN1M5it3lT8bhv7TAB37u9HzMp7cYO6pt6iv+IMlsZpMtNDu9+1F5mP+kLVWSlwPacVPArrUqjjhV5ctesrLcTh+ZjAMXqcpBp2F5hZiWybha42ZQxEEmQ1ftPIHbyP2DLNpoUV8aJMmw41vh1OMhgLI0oSMnMN2kQRg60vIcR0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786963985780632.4377408093414; Mon, 17 Aug 2026 03:53:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wvuwd-0008U2-Gn; Mon, 17 Aug 2026 06:51:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wvuwb-0008Rl-LQ for qemu-devel@nongnu.org; Mon, 17 Aug 2026 06:51:53 -0400 Received: from va-1-113.ptr.blmpb.com ([209.127.230.113]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wvuwa-0005Kh-5B for qemu-devel@nongnu.org; Mon, 17 Aug 2026 06:51:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1786963901; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=2FsRJH0EtbgoAnU9VEA61B6WUwpkCLdDGB2Fom+vdiU=; b=ZEDO+XCpIvEpGD8JevtnvUdVWjOq5oz58CzxTJNgzJaYpixlo/6adjZYBlPz/bnZ0Gbutf fUDWR9Dc3gFLrGEem8Yw/B51x9/Qg2dXTJn0rbesZtD8l+gA689kadMOJK6XNZakNsK7/T YpKZ1mJc/482QLWwpEzVc06q5BVU5sKUOyYr767sGqzbjnahgUY88EhjLgOcGlvVmj8Kix d112YS6aooYLYGH+v3e6ZLYm7rEnzYKqSRhvOKrA1jwxSi9f+GYHzrBseu1ff1yPd02Y20 +7WKMHfCJjB83K0M1akBlruRKgi2UuPCRN3PrRWzJY3L7GQHqq6lXa7yKzUAgw== Date: Mon, 17 Aug 2026 18:51:16 +0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.20.1 To: Subject: [PATCH 2/3] migration/rdma: Post initial receive before accepting In-Reply-To: <20260817105117.3072040-1-yanfei.xu@bytedance.com> From: "Yanfei Xu" Message-Id: <20260817105117.3072040-3-yanfei.xu@bytedance.com> Content-Transfer-Encoding: quoted-printable Cc: , , , "Yanfei Xu" X-Original-From: Yanfei Xu X-Lms-Return-Path: References: <20260817105117.3072040-1-yanfei.xu@bytedance.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=209.127.230.113; envelope-from=yanfei.xu@bytedance.com; helo=va-1-113.ptr.blmpb.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1786963987142158500 Content-Type: text/plain; charset="utf-8" The source return-path thread can send READY before the destination has posted its initial receive. Since RNR retries default to zero, the send can fail and abort migration. Post the initial receive before rdma_accept() to close this window. Fixes: 55cc1b5937a8 ("migration: create a dedicated connection for rdma ret= urn path") Signed-off-by: Yanfei Xu Reviewed-by: Jack Wang Reviewed-by: Peter Xu --- migration/rdma.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index e17ace62b0..6e8436ccc1 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3263,6 +3263,12 @@ static int qemu_rdma_accept(RDMAContext *rdma) } } =20 + ret =3D qemu_rdma_post_recv_control(rdma, RDMA_WRID_READY, &err); + if (ret < 0) { + error_report_err(err); + goto err_rdma_dest_wait; + } + /* Accept the second connection request for return path */ if ((migrate_postcopy() || migrate_return_path()) && !rdma->is_return_path) { @@ -3295,12 +3301,6 @@ static int qemu_rdma_accept(RDMAContext *rdma) rdma_ack_cm_event(cm_event); rdma->connected =3D true; =20 - ret =3D qemu_rdma_post_recv_control(rdma, RDMA_WRID_READY, &err); - if (ret < 0) { - error_report_err(err); - goto err_rdma_dest_wait; - } - qemu_rdma_dump_gid("dest_connect", rdma->cm_id); =20 return 0; --=20 2.20.1 From nobody Mon Sep 28 00:34:17 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1786963989; cv=none; d=zohomail.com; s=zohoarc; b=cNWgn5jD0ntyMYU1LruIgcGyfZg6KBx37ZTnr6TiEKXGGYL9LwwfJBE96Yz3sGCs8WA63/esXpYwApo+OSVdsKlCjeKjAMOG1QYBhq7GYLEDrxDi6PilZbY3EQMT8iItK7GXO0Mr+q2Nnqc9dq514g55FHpSpXwGc5KLDXgoFRY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1786963989; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Bns1KSbePA20dhJKI9rmE1kfQHZ8i2phCZvZRWjPPAw=; b=A7JyUZbsrUon6tm38czYJ8wPspmpS6KRjkCu7ax65ZiAiGnrPSZbtALSl9hznZj500xr9Pg5kO8gxpO/5yG3wIYMPpjF8cyvhvMXOPdZ5eSguAutKfOoMN4c/Bo8+wvIArDc7A9cWe2VOLzywVAzbO4bKScnZGYCEwx/sVwG92Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1786963989881739.6432563415202; Mon, 17 Aug 2026 03:53:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wvuwg-00007U-15; Mon, 17 Aug 2026 06:51:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wvuwd-0008Ui-Jv for qemu-devel@nongnu.org; Mon, 17 Aug 2026 06:51:55 -0400 Received: from va-1-113.ptr.blmpb.com ([209.127.230.113]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wvuwb-0005Ft-Vx for qemu-devel@nongnu.org; Mon, 17 Aug 2026 06:51:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1786963910; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=Bns1KSbePA20dhJKI9rmE1kfQHZ8i2phCZvZRWjPPAw=; b=oZU64RfvlvTL7NjnVjJRwtLOPvQ6PgXTY46goFERVtY5oHy6nH7IYuG3f/YKK0HFgFNw0E Tm4TTEfnRbbcR5upIqAhTSYnDQbgZ+U/2I/YRy5SrQ/sSaNWzlsiQEI7DoreXC3fOL36Kt Pmuii4EvtwV/rl8I9Pi+IgyhJ56W1Gv5PxG/RNBuMiTG/wns7sfqHlxmP9eaLhGstnfIXg OYzALfGIZ4vNrvH8hRWpq6WsLZh09eO6zDjFDS0pvjBKpuZ8VppQXNyoER5d0NlDsgiQ4H 7uKodqXG26f3j9/Nyzsus4XFyCM5vCClxIzTnGA6+cFqCIf3VJ2xOrmg3+hFEA== Mime-Version: 1.0 X-Lms-Return-Path: To: Date: Mon, 17 Aug 2026 18:51:17 +0800 In-Reply-To: <20260817105117.3072040-1-yanfei.xu@bytedance.com> Subject: [PATCH 3/3] migration/rdma: Retry control sends on full queue Message-Id: <20260817105117.3072040-4-yanfei.xu@bytedance.com> Content-Transfer-Encoding: quoted-printable Cc: , , , "Yanfei Xu" References: <20260817105117.3072040-1-yanfei.xu@bytedance.com> X-Mailer: git-send-email 2.20.1 From: "Yanfei Xu" X-Original-From: Yanfei Xu Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=209.127.230.113; envelope-from=yanfei.xu@bytedance.com; helo=va-1-113.ptr.blmpb.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1786963991051158500 Content-Type: text/plain; charset="utf-8" RAM writes and control messages share the send queue. If outstanding writes fill it, RDMA writes drain a completion and retry, but control sends fail the migration. Drain one outstanding write and retry the control send on ENOMEM. Signed-off-by: Yanfei Xu --- migration/rdma.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index 6e8436ccc1..d1f44a5f55 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1572,9 +1572,19 @@ static int qemu_rdma_post_send_control(RDMAContext *= rdma, uint8_t *buf, memcpy(wr->control + sizeof(RDMAControlHeader), buf, head->len); } =20 - +retry: ret =3D ibv_post_send(rdma->qp, &send_wr, &bad_wr); =20 + if (ret =3D=3D ENOMEM && rdma->nb_sent) { + ret =3D qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, NULL); + if (ret < 0) { + error_setg(errp, "rdma migration: failed to make room for " + "control send"); + return -1; + } + goto retry; + } + if (ret > 0) { error_setg(errp, "Failed to use post IB SEND for control"); return -1; --=20 2.20.1