From nobody Fri Aug 21 21:27:07 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1787230776; cv=none; d=zohomail.com; s=zohoarc; b=SZgkyXekiHYw/eNImfuUEwYMe00xZK64qptIQmEFW33SC7KNXsVziMpCKvg46SNbz0HNmjGMsBn6UNignZ2Lornr81NCeLgI/aEs/FUiAXO+zQ/s7f79G/Sii9cr7cWZ+gBN+K4IZNRYp1Y2Cocu4rx8g04zc4hwPnevjq20kFw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787230776; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=bNmPWQY9+XYhV/JRx6TQAUQSp0U3XZW6In1fSxl7oR0=; b=BsHwOPiI7CmDnqts2ttgcMf612VZHlmysnDY3hj2O5r099R/4L0M09/Uz0V6iHOY9iOppNeyUF+mtK6WMqAo6YheiR3jf4kTvYE0OVOY9fkQHJocKqJ4jte8Wtqeyyf3cg8qyxred5MjCA5a19P/js9S2rL2ax3Rz3W4/6NG6fc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787230776066466.9795354559052; Thu, 20 Aug 2026 05:59:36 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wx2MP-0005NL-58; Thu, 20 Aug 2026 08:59:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wx2MM-0005N2-8H for qemu-devel@nongnu.org; Thu, 20 Aug 2026 08:59:06 -0400 Received: from va-1-112.ptr.blmpb.com ([209.127.230.112]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wx2MK-0005TK-3s for qemu-devel@nongnu.org; Thu, 20 Aug 2026 08:59:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1787230738; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=bNmPWQY9+XYhV/JRx6TQAUQSp0U3XZW6In1fSxl7oR0=; b=fviMtCXvnll8tQ2RI06uT99Oqu1euW1z+20WQYLZixdUpXBwE6jC6Ey8p1g0FFW+eZlfI4 mqbdqLgT0i5/gRQ+wUjB06f3vifVKCe5B8c6ZHR7s0Yi911Krg1z1n5oXoYJNDN5hWujzX wiE6O81AdDQiggXRYQtDbx/81MbMdcXKMJe6/Jh5qjdMlWlvJHiJrZCNBj6jrVjxqIQl+4 96KH5MJalROv4RNlB+PT9TcNV+Hnmt/rdSnTNt10scsdVbzOY2kwMN8itTuIH0BNAgSqey LR/72Yk7Ah8eGpquJhdrOPh3NxoOvu/Kbi31He5l2YzwoKjP5e7Bxu/7/cJaBA== In-Reply-To: <20260820125833.1541756-1-yanfei.xu@bytedance.com> Cc: , , , , , "Yanfei Xu" Message-Id: <20260820125833.1541756-2-yanfei.xu@bytedance.com> Content-Transfer-Encoding: quoted-printable X-Lms-Return-Path: References: <20260820125833.1541756-1-yanfei.xu@bytedance.com> To: X-Original-From: Yanfei Xu X-Mailer: git-send-email 2.20.1 From: "Yanfei Xu" Date: Thu, 20 Aug 2026 20:58:32 +0800 Subject: [RFC PATCH v1 1/2] migration/rdma: Track in-flight writes with refcounts Mime-Version: 1.0 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=209.127.230.112; envelope-from=yanfei.xu@bytedance.com; helo=va-1-112.ptr.blmpb.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1787230778875158500 Content-Type: text/plain; charset="utf-8" The transit bitmap can only represent one outstanding write per registration chunk. Replace it with a per-chunk reference count and assert the count on each update. Signed-off-by: Yanfei Xu --- migration/rdma.c | 48 ++++++++++++++++++++++++++++++------------ migration/trace-events | 4 ++-- 2 files changed, 37 insertions(+), 15 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index d1f44a5f55..973a7a745a 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -199,7 +199,7 @@ typedef struct RDMALocalBlock { unsigned int src_index; /* (Only used on dest) */ bool is_ram_block; int nb_chunks; - unsigned long *transit_bitmap; + unsigned int *transit_refcnt; unsigned long *unregister_bitmap; } RDMALocalBlock; =20 @@ -552,6 +552,29 @@ static inline uint8_t *ram_chunk_end(const RDMALocalBl= ock *rdma_ram_block, return result; } =20 +static inline bool qemu_rdma_chunk_in_transit(const RDMALocalBlock *block, + uint64_t chunk) +{ + assert(chunk < block->nb_chunks); + return block->transit_refcnt[chunk] > 0; +} + +static inline void qemu_rdma_chunk_transit_inc(RDMALocalBlock *block, + uint64_t chunk) +{ + assert(chunk < block->nb_chunks); + assert(block->transit_refcnt[chunk] < UINT_MAX); + block->transit_refcnt[chunk]++; +} + +static inline void qemu_rdma_chunk_transit_dec(RDMALocalBlock *block, + uint64_t chunk) +{ + assert(chunk < block->nb_chunks); + assert(block->transit_refcnt[chunk] > 0); + block->transit_refcnt[chunk]--; +} + static void rdma_add_block(RDMAContext *rdma, const char *block_name, void *host_addr, ram_addr_t block_offset, uint64_t length) @@ -585,8 +608,7 @@ static void rdma_add_block(RDMAContext *rdma, const cha= r *block_name, block->index =3D local->nb_blocks; block->src_index =3D ~0U; /* Filled in by the receipt of the block lis= t */ block->nb_chunks =3D ram_chunk_index(host_addr, host_addr + length) + = 1UL; - block->transit_bitmap =3D bitmap_new(block->nb_chunks); - bitmap_clear(block->transit_bitmap, 0, block->nb_chunks); + block->transit_refcnt =3D g_new0(unsigned int, block->nb_chunks); block->unregister_bitmap =3D bitmap_new(block->nb_chunks); bitmap_clear(block->unregister_bitmap, 0, block->nb_chunks); block->remote_keys =3D g_new0(uint32_t, block->nb_chunks); @@ -601,8 +623,7 @@ static void rdma_add_block(RDMAContext *rdma, const cha= r *block_name, (uintptr_t) block->local_host_addr, block->offset, block->length, (uintptr_t) (block->local_host_addr + block->leng= th), - BITS_TO_LONGS(block->nb_chunks) * - sizeof(unsigned long) * 8, + block->nb_chunks * sizeof(*block->transit_refcnt), block->nb_chunks); =20 local->nb_blocks++; @@ -673,8 +694,8 @@ static void rdma_delete_block(RDMAContext *rdma, RDMALo= calBlock *block) block->mr =3D NULL; } =20 - g_free(block->transit_bitmap); - block->transit_bitmap =3D NULL; + g_free(block->transit_refcnt); + block->transit_refcnt =3D NULL; =20 g_free(block->unregister_bitmap); block->unregister_bitmap =3D NULL; @@ -716,8 +737,9 @@ static void rdma_delete_block(RDMAContext *rdma, RDMALo= calBlock *block) trace_rdma_delete_block(block, (uintptr_t)block->local_host_addr, block->offset, block->length, (uintptr_t)(block->local_host_addr + block->le= ngth), - BITS_TO_LONGS(block->nb_chunks) * - sizeof(unsigned long) * 8, block->nb_chunks= ); + block->nb_chunks * + sizeof(*block->transit_refcnt), + block->nb_chunks); =20 g_free(old); =20 @@ -1237,7 +1259,7 @@ static int qemu_rdma_unregister_waiting(RDMAContext *= rdma) */ clear_bit(chunk, block->unregister_bitmap); =20 - if (test_bit(chunk, block->transit_bitmap)) { + if (qemu_rdma_chunk_in_transit(block, chunk)) { trace_qemu_rdma_unregister_waiting_inflight(chunk); continue; } @@ -1328,7 +1350,7 @@ static int qemu_rdma_poll(RDMAContext *rdma, struct i= bv_cq *cq, index, chunk, block->local_host_addr, (void *)(uintptr_t)block->remote_host_a= ddr); =20 - clear_bit(chunk, block->transit_bitmap); + qemu_rdma_chunk_transit_dec(block, chunk); =20 if (rdma->nb_sent > 0) { rdma->nb_sent--; @@ -1889,7 +1911,7 @@ retry: chunk_end =3D ram_chunk_end(block, chunk + chunks); =20 =20 - while (test_bit(chunk, block->transit_bitmap)) { + while (qemu_rdma_chunk_in_transit(block, chunk)) { (void)count; trace_qemu_rdma_write_one_block(count++, current_index, chunk, sge.addr, length, rdma->nb_sent, block->nb_chunks); @@ -2053,7 +2075,7 @@ retry: return -1; } =20 - set_bit(chunk, block->transit_bitmap); + qemu_rdma_chunk_transit_inc(block, chunk); qatomic_add(&mig_stats.normal_pages, sge.length / qemu_target_page_siz= e()); /* * We are adding to transferred the amount of data written, but no diff --git a/migration/trace-events b/migration/trace-events index af0e784535..172761be78 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -256,9 +256,9 @@ qemu_rdma_write_one_recvregres(int mykey, int theirkey,= uint64_t chunk) "Receive qemu_rdma_write_one_sendreg(uint64_t chunk, int len, int index, int64_t of= fset) "Sending registration request chunk %" PRIu64 " for %d bytes, index: = %d, offset: %" PRId64 qemu_rdma_write_one_top(uint64_t chunks, uint64_t size) "Writing %" PRIu64= " chunks, (%" PRIu64 " MB)" qemu_rdma_write_one_zero(uint64_t chunk, int len, int index, int64_t offse= t) "Entire chunk is zero, sending compress: %" PRIu64 " for %d bytes, index= : %d, offset: %" PRId64 -rdma_add_block(const char *block_name, int block, uint64_t addr, uint64_t = offset, uint64_t len, uint64_t end, uint64_t bits, int chunks) "Added Block= : '%s':%d, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: = %" PRIu64 " bits %" PRIu64 " chunks %d" +rdma_add_block(const char *block_name, int block, uint64_t addr, uint64_t = offset, uint64_t len, uint64_t end, uint64_t refcnt_bytes, int chunks) "Add= ed Block: '%s':%d, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64= " end: %" PRIu64 " refcnt-bytes %" PRIu64 " chunks %d" rdma_block_notification_handle(const char *name, int index) "%s at %d" -rdma_delete_block(void *block, uint64_t addr, uint64_t offset, uint64_t le= n, uint64_t end, uint64_t bits, int chunks) "Deleted Block: %p, addr: %" PR= Iu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " bits %" PR= Iu64 " chunks %d" +rdma_delete_block(void *block, uint64_t addr, uint64_t offset, uint64_t le= n, uint64_t end, uint64_t refcnt_bytes, int chunks) "Deleted Block: %p, add= r: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " re= fcnt-bytes %" PRIu64 " chunks %d" rdma_registration_handle_compress(int64_t length, int index, int64_t offse= t) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64 rdma_registration_handle_finished(void) "" rdma_registration_handle_ram_blocks(void) "" --=20 2.20.1 From nobody Fri Aug 21 21:27:07 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1787230790; cv=none; d=zohomail.com; s=zohoarc; b=A/qy8GItVQ22TddBlGt4G2m2tMHzD/MOpIQTXyof0s1yOiWmeQ8SW/VXQZzLKWbcv9ur4VU96/gkWqEGtKaIAVM/yDBTpNC4fJ65STnTffBWdK4UjL8nI5xmoA1UMdGeqxHsngzCoil4l2HleJHoQaojKT41HouA1lJ+v7d1/Ic= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787230790; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=c2OFBpaMKAYU75s6016ad4m6ba8dR4V1MhOGQxkaS7M=; b=Rt/K10e1mVnbmVgCStEoLVrC3fN67LZ9ZNHPMDIaQEeSi0dKmKRFylfCiv86kISgG3MEELrpHE0IcA9gruTqK1wAJHePreu3iRp8YzR6bh17lJCvFu5s8kjS5UaeE8jhxQXeng2IFUFanjJO7RMN+lat9GGG5vFquQt/Go4iS2I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787230790118226.67186888536673; Thu, 20 Aug 2026 05:59:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wx2Ma-0005Oz-TG; Thu, 20 Aug 2026 08:59:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wx2MZ-0005Oo-Fl for qemu-devel@nongnu.org; Thu, 20 Aug 2026 08:59:19 -0400 Received: from va-1-113.ptr.blmpb.com ([209.127.230.113]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wx2MX-0005V4-Gc for qemu-devel@nongnu.org; Thu, 20 Aug 2026 08:59:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1787230747; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=c2OFBpaMKAYU75s6016ad4m6ba8dR4V1MhOGQxkaS7M=; b=e0ffiUFDpoxx3i74DSr9GTr/D8VIna8Ba5WJU1VleTcUWTsFaasFCaIIjs+UTYyhEoTVFY Sr7JTyzNvELS1aG98NlTgT+rd6oRSDDmgBO12Tp/JA44XJnNZML4RYTAO1h7HERc2EXMo7 HT+xJsBeflR0TpT0WH01qsGBUG9DmTWC7GASjMz9FZrwyWzrQvmbFTCbT+VO7XezNbsUsH 38CqvX5r2967o/xecgq3eTDXac/7Tx0kxeLWjvw1/wOK0h1h/2MXCRpQaKBxSpmZlh2RV6 0rw5kfQm164uPTpAfJ1mfLw/jYhk78dyTcYuu20iXAfMkjMZ0up6NrNOxPoMMQ== Content-Transfer-Encoding: quoted-printable X-Lms-Return-Path: X-Mailer: git-send-email 2.20.1 X-Original-From: Yanfei Xu To: Cc: , , , , , "Yanfei Xu" Date: Thu, 20 Aug 2026 20:58:33 +0800 From: "Yanfei Xu" Message-Id: <20260820125833.1541756-3-yanfei.xu@bytedance.com> Mime-Version: 1.0 Subject: [RFC PATCH v1 2/2] migration/rdma: Allow multiple in-flight writes per chunk In-Reply-To: <20260820125833.1541756-1-yanfei.xu@bytedance.com> References: <20260820125833.1541756-1-yanfei.xu@bytedance.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=209.127.230.113; envelope-from=yanfei.xu@bytedance.com; helo=va-1-113.ptr.blmpb.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1787230792414158500 Content-Type: text/plain; charset="utf-8" qemu_rdma_write_one() waits for an earlier write to the same registration chunk to complete. This serializes disjoint dirty ranges in a chunk and leaves send queue capacity unused. Remove the per-chunk wait and use the reference counts to track all outstanding writes. The existing per-iteration drain remains the completion barrier. Signed-off-by: Yanfei Xu --- migration/rdma.c | 19 +------------------ migration/trace-events | 1 - 2 files changed, 1 insertion(+), 19 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 973a7a745a..63bc357657 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1869,7 +1869,7 @@ static int qemu_rdma_write_one(RDMAContext *rdma, struct ibv_sge sge; struct ibv_send_wr send_wr =3D { 0 }; struct ibv_send_wr *bad_wr; - int reg_result_idx, ret, count =3D 0; + int reg_result_idx, ret; uint64_t chunk, chunks; uint64_t chunk_size =3D migrate_rdma_chunk_size(); uint8_t *chunk_start, *chunk_end; @@ -1910,23 +1910,6 @@ retry: =20 chunk_end =3D ram_chunk_end(block, chunk + chunks); =20 - - while (qemu_rdma_chunk_in_transit(block, chunk)) { - (void)count; - trace_qemu_rdma_write_one_block(count++, current_index, chunk, - sge.addr, length, rdma->nb_sent, block->nb_chunks); - - ret =3D qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, NULL); - - if (ret < 0) { - error_setg(errp, "Failed to Wait for previous write to complet= e " - "block %d chunk %" PRIu64 - " current %" PRIu64 " len %" PRIu64 " %d", - current_index, chunk, sge.addr, length, rdma->nb_sent); - return -1; - } - } - if (!rdma->pin_all || !block->is_ram_block) { if (!block->remote_keys[chunk]) { /* diff --git a/migration/trace-events b/migration/trace-events index 172761be78..253ff71891 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -249,7 +249,6 @@ qemu_rdma_unregister_waiting_proc(uint64_t chunk, int p= os) "Processing unregiste qemu_rdma_unregister_waiting_send(uint64_t chunk) "Sending unregister for = chunk: %" PRIu64 qemu_rdma_unregister_waiting_complete(uint64_t chunk) "Unregister for chun= k: %" PRIu64 " complete." qemu_rdma_write_flush(int sent) "sent total: %d" -qemu_rdma_write_one_block(int count, int block, uint64_t chunk, uint64_t c= urrent, uint64_t len, int nb_sent, int nb_chunks) "(%d) Not clobbering: blo= ck: %d chunk %" PRIu64 " current %" PRIu64 " len %" PRIu64 " %d %d" qemu_rdma_write_one_post(uint64_t chunk, long addr, long remote, uint32_t = len) "Posting chunk: %" PRIu64 ", addr: 0x%lx remote: 0x%lx, bytes %" PRIu32 qemu_rdma_write_one_queue_full(void) "" qemu_rdma_write_one_recvregres(int mykey, int theirkey, uint64_t chunk) "R= eceived registration result: my key: 0x%x their key 0x%x, chunk %" PRIu64 --=20 2.20.1