From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352503; cv=none; d=zohomail.com; s=zohoarc; b=ai1Ptu7ITx5NTVVGQWVwCbhI048CCGmbEx8RUtndVpT8e/TWuSiexLS9zYFR/BmTmmGRYIkjo/WAZDAWQTj5qU80uz598dmAYo47sFpC3MHVfSmPB5mqF0Vw1WyjHxqpuoJ/OV4ePaV0GE3Agq+/hocAuzIkI46MB+NNNsp6Vxc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352503; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=BQTziWc8i9CyJZknsQwFL2OKVJrU1qwkxlEOdcl+8cU=; b=iENyH4fW70zwIDGYHPTtgPlTB+QFMo1UVSYqf+MefSrMws1VAsbYzJYZeQAGBHWPRMv53+JpHGDLMSdEVGEL9cRY4LKyH4mVBN+koQyANwaUR4/VllFnWYVMRSOclZCXBOvpOvpNtVjYff8sFiI1FC6UumN5KF/R/UjLVKIzz58= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352503891431.1774594463669; Fri, 25 Sep 2026 09:08:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8S6-0007AE-00; Fri, 25 Sep 2026 12:07:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8DQ-0007pU-PV for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:02 -0400 Received: from mail-wm2-x11.google.com ([2a00:1450:4864:31::11]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8DO-0005wF-7D for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:00 -0400 Received: by mail-wm2-x11.google.com with SMTP id 5b1f17b1804b1-49cd38e0f79so6339935e9.3 for ; Fri, 25 Sep 2026 08:51:57 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.51.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:51:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351517; x=1790956317; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=BQTziWc8i9CyJZknsQwFL2OKVJrU1qwkxlEOdcl+8cU=; b=EGfr/On3A52ZGPgAAGoII/3WbCP4TPpZa4oEYgFvfPQFdP1UhzMX4XpUlmZTjRC+eW oyOaAinCE7+JJ+yWhpCM1FgvtKPWVuws+ZWrXtunQ7VC8YpvDBo/5bdl5zwBUCCr5lyb Za4T5teUr4Gy5LEdU0oI7bSJo0aZJe3o9A/tFdo8ycWOp6Su8rXK3UrItft5CV9cklsu vSp1FwonuAUnf4Bnxc2XDko5P2JzGFn3XwsVgC1fjWGk0VUf4OMZp0ic5lxVtY3ErStA 281oYeAzDFx2/PcDNehYh68yr2+EaOTr9QLk7J3OUr2a+xVQW6lmW8ESGrPL6d6hQbFH WeUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351517; x=1790956317; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=BQTziWc8i9CyJZknsQwFL2OKVJrU1qwkxlEOdcl+8cU=; b=LJLI0D9IBsqXquShVVoGw2eSc1O9BDxR/HNIQeVBzoPQM7X9t/NbpH2a3nBU9S5Vo2 /4nnigXrT51JWr1mYOMQlHB9PMGHjjdo9vL5PIqcjy7DeWmzMuTmdGpqG1UWmLrLu4zg JEqpdT1syv5kW4w9aP0rGcC8oqyCp2jQ2NL/4Y3ROHxqvmIQPzZ8oxWyiVmWKTJ80GRq I/c7W6BlLIiRH0Kg57lC+TFAhPSUiLZNqlMPZFNaIxnrW4JEaZ7h+1n6Fl/6DtMyszKA HbHXsmxVHOxXIoV0r01/mDmQxO0qLyDrnAQvTRg32CBCKcpWcDSWWkSI00BYFxTwFPXT Tm6A== X-Gm-Message-State: AFuF++lZv1DaayoK977G0rwV7xyeDBLgVQ9qpD5FlhyJBB8/xaoSHy1b uK2tfbPh2xuljnvMmBeOySKGbR7kDmr3GtFct9yyIDNl9RM9n4NRQC2CGDhkLTHs0HOAFuz38AK bXlbN8WLU6g== X-Gm-Gg: AYBFou3NjzPTgS2C4X2WMGe48n2yUBL2zIdGq7mTbFNZ8S9XO7EA3BT0j8tInh7yGV0 UyQkODy9rUyPRWGnejaGbLcxAm9LOT8CN40FSMry4iZGr/PFrw3ZwMKvmwhg1YlsOTHcXtWUV2c sIkvprAlVCb57OQgYZ0yw5i05HZVwLLKBFzb8WNMqNMkVLlMzF1RbD2PeqlvWvsGvFoDY1RUn1X G1L1+wo4xzczatTc6TNqbFDGK4nrY6DEIOgNkhG282qINf6CLfVE6w/P2uyWCk45fZXAjcu/fQO RebxfCDi0fjk48faCN+UafHoqvdWTj2wzqq4LTkpKcIXDv5IrsMzm/Kc5Z11gjqQfhHfPyDzJt6 hVye4SfVvB/082a3HzuQ56cdbuvY3CyaPnI8rvyCz7j3HXoUQLCF4+nJ7hAP3ikNY66eAV+NyZT KhXH948ezFqqhkCfO3lr47B/xPxFMjUG3igxgjj/Rq74fKxIDX+Q5dP0aXYwuJG+rzhGl29KPLx 81S2Xkr4Tss5xBxNbXpHR+BKSfWfjgVxZCtg6kmhQ== X-Received: by 2002:a05:600c:4f10:b0:49c:dca4:94c with SMTP id 5b1f17b1804b1-49fe66d09c5mr125779165e9.13.1790351516791; Fri, 25 Sep 2026 08:51:56 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 01/13] migration/rdma: Add nb_sent send-queue-depth stats Date: Fri, 25 Sep 2026 17:51:36 +0200 Message-ID: <20260925155149.843870-2-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::11; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x11.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:04:40 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352505466158500 Content-Type: text/plain; charset="utf-8" Track whole-migration counters for the RDMA send queue: peak nb_sent (max outstanding writes), the number of posts, how many posts hit a full send queue, and how many times the queue drained to zero. Reset at RAM_CONTROL_SETUP and dumped at RAM_CONTROL_FINISH; migration-thread only, no locking. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 60 ++++++++++++++++++++++++++++++++++++++++++ migration/trace-events | 1 + 2 files changed, 61 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index 444123cbca..50f5d7b839 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -302,6 +302,12 @@ typedef struct RDMAContext { /* number of outstanding writes */ int nb_sent; =20 + /* nb_sent statistics */ + uint64_t nbsent_max; /* peak outstanding writes */ + uint64_t nbsent_posts; /* posts (nb_sent++) */ + uint64_t nbsent_full_hits; /* posts that hit a full SQ */ + uint64_t nbsent_zero_hits; /* drains to zero */ + /* store info about current buffer so that we can merge it with future sends */ uint64_t current_addr; @@ -1164,6 +1170,11 @@ static uint64_t qemu_rdma_make_wrid(uint64_t wr_id, = uint64_t index, return result; } =20 +/* nb_sent statistics counters */ +static void rdma_nbsent_inc(RDMAContext *rdma, uint64_t nb_sent); +static void rdma_nbsent_full(RDMAContext *rdma); +static void rdma_nbsent_zero(RDMAContext *rdma); + /* * Consult the connection manager to see a work request * (of any kind) has completed. @@ -1213,6 +1224,9 @@ static int qemu_rdma_poll(RDMAContext *rdma, struct i= bv_cq *cq, =20 if (rdma->nb_sent > 0) { rdma->nb_sent--; + if (rdma->nb_sent =3D=3D 0) { + rdma_nbsent_zero(rdma); + } } } else { trace_rdma_poll_other(wr_id, rdma->nb_sent); @@ -1438,6 +1452,7 @@ static bool qemu_rdma_post_send(RDMAContext *rdma, ret =3D ibv_post_send(rdma->qp, send_wr, &bad_wr); if (ret =3D=3D ENOMEM && rdma->nb_sent) { trace_rdma_post_send_queue_full(send_wr->wr_id, rdma->nb_sent); + rdma_nbsent_full(rdma); ret =3D qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, NULL); if (ret < 0) { error_setg(errp, "rdma migration: failed to make room for %s", @@ -1953,6 +1968,7 @@ static int qemu_rdma_write_flush(RDMAContext *rdma, E= rror **errp) if (ret =3D=3D 0) { rdma->nb_sent++; trace_rdma_write_flush(rdma->nb_sent); + rdma_nbsent_inc(rdma, rdma->nb_sent); } =20 rdma->current_length =3D 0; @@ -2998,6 +3014,41 @@ err: return -1; } =20 +static void rdma_nbsent_inc(RDMAContext *rdma, uint64_t nb_sent) +{ + rdma->nbsent_posts++; + if (nb_sent > rdma->nbsent_max) { + rdma->nbsent_max =3D nb_sent; + } +} + +static void rdma_nbsent_full(RDMAContext *rdma) +{ + rdma->nbsent_full_hits++; +} + +static void rdma_nbsent_zero(RDMAContext *rdma) +{ + rdma->nbsent_zero_hits++; +} + +static void rdma_stats_reset(RDMAContext *rdma) +{ + rdma->nbsent_max =3D 0; + rdma->nbsent_posts =3D 0; + rdma->nbsent_full_hits =3D 0; + rdma->nbsent_zero_hits =3D 0; +} + +static void rdma_nbsent_stats_dump(RDMAContext *rdma) +{ + if (rdma->nbsent_posts =3D=3D 0) { + return; + } + trace_rdma_nbsent_stats(rdma->nbsent_max, rdma->nbsent_posts, + rdma->nbsent_full_hits, rdma->nbsent_zero_hits= ); +} + int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { @@ -3547,6 +3598,10 @@ int rdma_registration_start(QEMUFile *f, uint64_t fl= ags) return -1; } =20 + if (flags =3D=3D RAM_CONTROL_SETUP) { + rdma_stats_reset(rdma); + } + trace_rdma_registration_start(flags); qemu_put_be64(f, RAM_SAVE_FLAG_HOOK); return qemu_fflush(f); @@ -3581,6 +3636,11 @@ int rdma_registration_stop(QEMUFile *f, uint64_t fla= gs) } =20 qemu_fflush(f); + + if (flags =3D=3D RAM_CONTROL_FINISH) { + rdma_nbsent_stats_dump(rdma); + } + ret =3D qemu_rdma_write_flush(rdma, &err); =20 if (ret < 0) { diff --git a/migration/trace-events b/migration/trace-events index db911fe6f2..f8ed6a7250 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -245,6 +245,7 @@ rdma_resolve_host_trying(const char *host, const char *= ip) "Trying %s =3D> %s" rdma_write_flush(int sent) "sent total: %d" rdma_write_one_post(uint64_t chunk, long addr, long remote, uint32_t len) = "Posting chunk: %" PRIu64 ", addr: 0x%lx remote: 0x%lx, bytes %" PRIu32 rdma_post_send_queue_full(uint64_t wr_id, int sent) "send queue full, wr_i= d=3D%" PRIu64 ", outstanding writes=3D%d" +rdma_nbsent_stats(uint64_t max, uint64_t posts, uint64_t full_hits, uint64= _t zero_hits) "nb_sent max %" PRIu64 " posts %" PRIu64 " full-queue hits %"= PRIu64 " drains to zero %" PRIu64 rdma_write_one_recvregres(int mykey, int theirkey, uint64_t chunk) "Receiv= ed registration result: my key: 0x%x their key 0x%x, chunk %" PRIu64 rdma_write_one_sendreg(uint64_t chunk, int len, int index, int64_t offset)= "Sending registration request chunk %" PRIu64 " for %d bytes, index: %d, o= ffset: %" PRId64 rdma_write_one_top(uint64_t chunks, uint64_t size) "Writing %" PRIu64 " ch= unks, (%" PRIu64 " MB)" --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352354; cv=none; d=zohomail.com; s=zohoarc; b=Jt8TMkH9/u8q3W8VjcsEFxrRYKAIKNITxmQCrPfrwUx8b+O37qdL7S4YHUvbRHSflgqngVmX8OvOfYxuwOK+gulEKb5ezwzjbp7R1BJv2SYoqtPagRa6qoXyknWiDZer3YC0ytZqogmfkugy8FbuKigNFkXKMILfAHLLdYEd8JI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352354; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=aSftoR6E6Ri0aJ0ChW9kw3c0X9HiQbgax0Jl26xDQyY=; b=J4dBNpozOHZRGkmisclabX0rAa0cWfmHVQUJS8PL+kCoFM/GRaJ4btz81Kc5bmFgXFRxxQo876uDTZqbKlTbo5vTosHD+J+nh/7HGihxKUtxsI08F61mwrbuLm1g3IlwWETKYsl0MwTHr4lSp4aNp62W630QgiOqH7keknvsE0w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352354066825.6673394676052; Fri, 25 Sep 2026 09:05:54 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8QJ-0005Ze-GQ; Fri, 25 Sep 2026 12:05:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8DT-0007pv-Jm for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:03 -0400 Received: from mail-wm2-x10.google.com ([2a00:1450:4864:31::10]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8DP-0005wS-UM for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:03 -0400 Received: by mail-wm2-x10.google.com with SMTP id 5b1f17b1804b1-49cd4ba9f68so11798225e9.1 for ; Fri, 25 Sep 2026 08:51:59 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.51.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:51:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351518; x=1790956318; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=aSftoR6E6Ri0aJ0ChW9kw3c0X9HiQbgax0Jl26xDQyY=; b=Y5LCLpApQmewYhw93KGmvZwgei7JmIKqVjqual3Z7qeM82PPDRpt5qv4Mlp3UHCrGu aZAh5dSyqZRDYkQb2uQ3wtENDYUvOQM9vinsBFZz61GYtbl46PcMZZqU39OT4pbd/Q3o KPcUND/t8l5AJjSi18c949muRw3PF+T4oWds3b6auNilhLo21JvqoROzY2MxmsaQk3lF D6ktY/KFTY4lnzLTCEUdlISpN4ciHsNcYDlgK7ExGJfD1oN/NCMqFKI/d5F41y3iKqoL Mp3iaczJ7YCWymIBzwl1eiuEyCEzEezN4Z6LjbhFftT7WOzC7ETMQmzTSJJBdpCb+oER c+7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351518; x=1790956318; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=aSftoR6E6Ri0aJ0ChW9kw3c0X9HiQbgax0Jl26xDQyY=; b=jMHz17qHotjWrVyhY4lU4ZvIPt3GhjnQFnOqpJ/4PmGWFkpBJFdPhJQ8eltjpnQ7x4 7mmPolZz7EI4Ik/T4AJ+vtcwfzZPXyFAJH8xtfamGoBQoATtAwKjV0S26ngL5cLww4GQ JEH1tW0/6b09+1ZGgcTVMo0JLIaPFzNMpATn7o/p08ZN6IKej79o5XIavhZT1NmES4OW 3vRycEpF5E7OytOKl/0w0lyR9XQ7dFB4N/tHhcMlRYalyoR40aoz18M1B/+UHlR8npqx aA2WUBhAxMhDADDaapYLzYfSSVgpoQfzie0ZuYTw6LHCAHcdTC4J3l4DoGDo9xjw9QHi fVqg== X-Gm-Message-State: AFuF++mx1F97MBHZ8uCUjEIAt2MoPGitVi2ag745olbhAHeX1HyFPMgm 6if7dInFZl/X15q8LnkeIwCS1sVV9aG15kwoYiam8vBhHZBiIdh+nuUew82dTYYV6CwsdMGjOmZ Yc1Fptl+NWA== X-Gm-Gg: AYBFou1NbHnhTBar0rnPWFr+e+cY2krqeDmM3KoNGuFljfRaMgjxouqubXbGU4qbZNQ wfpzIpGkytPbZ47H/+dDWMKOr3HrSeHGhfe/Wifac0WlgyROMnrPz0gfzzOCTv2EajD8gdOTl8U nwpYXl/Uot98oivG+sDApmYTcY5OrjRqS9Nn5YlsAzBkXTVMfOw92/aWi1y1ghVe83SX+ezFWcj C87gux3kplIl/E+KjaNV1D/+auqXlcy0q5HR8c7zesaW4bxZfgp2zygamj680hSR4G/xLdLwkiR /0QCmt0kOfiIq5XVJPpnwb4LNIHk74Ewj8DwmBSpu+WKiXWIf9X1TMJCraBEzvEtud624p5AQuU P43Kbn4o1tFFgtPrB072fKVEYRrVv8k1QfEoL5bQ5nacdbHPr7FYXQo6wnnd+TpNVcompEAz7em 1e5ZfLHRO7UxToealZgxZlKVyzEXHxAjh/w/uFHln5uYblxqE/ztkrtkjt9fLGb7rFHf56JHvNm iS6H17VVi+C74TWD3T0ikUXehRY2awHOA8RcqF5vw== X-Received: by 2002:a05:600c:4e53:b0:49f:d58f:1f8b with SMTP id 5b1f17b1804b1-49fe66d807bmr105364325e9.18.1790351518569; Fri, 25 Sep 2026 08:51:58 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 02/13] migration/rdma: Add producer/consumer send-ring types Date: Fri, 25 Sep 2026 17:51:37 +0200 Message-ID: <20260925155149.843870-3-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::10; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x10.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:04:43 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352356341158500 Content-Type: text/plain; charset="utf-8" Define the SPSC send-ring descriptor (RDMACmd), the ring itself (RDMASendRing) and the producer/consumer fields in RDMAContext. No functional change. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index 50f5d7b839..e4ede950de 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -29,6 +29,7 @@ #include "qemu/main-loop.h" #include "qemu/module.h" #include "qemu/rcu.h" +#include "qemu/thread.h" #include "qemu/sockets.h" #include "qemu/coroutine.h" #include "system/memory.h" @@ -278,6 +279,33 @@ typedef struct RDMALocalBlocks { RDMALocalBlock *block; } RDMALocalBlocks; =20 +/* + * Producer/consumer split (source side): the migration thread (producer) + * scans dirty pages and enqueues chunk-write and barrier commands into an + * SPSC ring; a dedicated consumer thread owns the QP/CQ and does all RDMA. + */ +typedef enum { + RDMA_CMD_WRITE, /* write one dirty span: block/chunk + offset/leng= th */ + RDMA_CMD_FLUSH, /* drain nb_sent -> 0 (round boundary) */ + RDMA_CMD_REG_STOP, /* send REGISTER_FINISHED; flags =3D RAM_CONTROL_*= */ + RDMA_CMD_FILE, /* QEMUFile control bytes: iov/niov */ + RDMA_CMD_FINISH, /* drain then stop the consumer */ +} RDMACmdType; + +typedef struct RDMACmd { + RDMACmdType type; + uint32_t block_index; + uint32_t chunk_index; + uint64_t offset; /* WRITE: dirty span start, block-relati= ve */ + uint32_t length; /* WRITE: dirty span length (within a chu= nk) */ + uint64_t flags; /* REG_STOP */ + const struct iovec *iov; /* FILE (producer blocks -> stays valid)= */ + size_t niov; /* FILE */ + unsigned seq; /* barrier completion sequence (0 =3D no= ne) */ +} RDMACmd; + +typedef struct RDMASendRing RDMASendRing; + /* * Main data structure for RDMA state. * While there is only one copy of this structure being allocated right no= w, @@ -317,6 +345,17 @@ typedef struct RDMAContext { /* index of the chunk in the current ram block */ int current_chunk; =20 + /* producer/consumer split (source side) */ + RDMASendRing *send_ring; /* SPSC ring; allocated at consumer star= t */ + QemuThread consumer_thread; + bool consumer_started; /* producer-only: route via ring once se= t */ + + /* producer-only: pending contiguous WRITE span, not yet enqueued */ + int pend_block; /* block index, -1 =3D none */ + uint32_t pend_chunk; /* chunk index of the span */ + uint64_t pend_offset; /* span start, block-relative bytes */ + uint64_t pend_length; /* span length in bytes (within one chun= k) */ + bool pin_all; =20 /* @@ -2671,6 +2710,49 @@ static NotifierWithReturn rdma_ram_round_notifier = =3D { .notify =3D rdma_ram_round_notify, }; =20 +/* + * -----------------------------------------------------------------------= ---- + * Producer/consumer send ring (source side). + * + * Lock-free SPSC ring: the producer (migration thread) owns head, the + * consumer (RDMA thread) owns tail. A mutex+condvars are used only to + * sleep/wake at the empty/full/abort/barrier edges (guarded by the *_wait= ing + * flags so the steady state stays lock-free). Barriers ride the ring as + * ordered descriptors and publish completion via barrier_done_seq. + * + * head/tail are free-running unsigned counters, never reset. Wrap is + * implicit: occupancy is the modular difference head - tail (always in + * [0, N] since the producer stops at full), and the slot index is + * counter & MASK. Because N is a power of two and 2^32 is an exact multi= ple + * of N, masking stays continuous across the 2^32 counter wrap. Barrier s= eqs + * use the same modular scheme with signed distance; seq 0 is reserved as = the + * "no barrier" sentinel and skipped on wrap. + * -----------------------------------------------------------------------= ---- + */ +#define RDMA_SEND_RING_SIZE 2048 /* power of two; > SQ depth (5= 12) */ +#define RDMA_SEND_RING_MASK (RDMA_SEND_RING_SIZE - 1) + +struct RDMASendRing { + RDMACmd ring[RDMA_SEND_RING_SIZE]; + unsigned head; /* producer owns */ + unsigned tail; /* consumer owns */ + unsigned barrier_done_seq; /* consumer owns; producer rea= ds */ + unsigned barrier_next_seq; /* producer owns */ + bool abort; /* sticky */ + QemuMutex wait_lock; + QemuCond not_full; /* producer sleeps */ + QemuCond not_empty; /* consumer sleeps */ + QemuCond barrier_cv; /* producer sleeps for ack */ + bool producer_waiting; + bool consumer_waiting; + + /* statistics, dumped at FINISH */ + uint64_t pushwait_max_ns; /* longest producer wait on a full ring */ + uint64_t pushwait_total_ns; /* total producer wait on a full ring */ + uint64_t occupancy_max; /* peak queued commands */ + uint64_t idle_max_ns; /* longest consumer wait on an empty ring= */ +}; + static int qio_channel_rdma_set_blocking(QIOChannel *ioc, bool blocking, Error **errp) --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352515; cv=none; d=zohomail.com; s=zohoarc; b=hyr5hOdImwUTMIc693dPG/CX/Dh1H7thI5rumxomWYt8lDbecoM6oElBQifFA7OMYXLWQ4cF8BGbZyPU+O5kfQduhtcPezFCi4Un0wE/Br9TRiUNacGhoGU4Am0Xsy07612msIV4puSXhoHizDgeqt0+VAJcZn3/CMWUIWk3iBo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352515; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=wz5J7BtRm96GQteyQTuXbDxbX79LxGun+611Yug+ddc=; b=jsKfLB2W9qObuRvCKPx6CISopMGJRn0jBKYjB7uXU7mZEQANSJ8zceIGTr3HdM/aKgLb6BEYZXwm5fXI7O8Xdc4JMtXMPPrJlu9ND2S8oJLRVPA43VSSATYb/xvv41mm4Q6fCyw8sdZka5zpBxxfm0xowCuh3HthEI0Kfy//23I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352515393612.2085637158208; Fri, 25 Sep 2026 09:08:35 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8SM-0007ld-5X; Fri, 25 Sep 2026 12:07:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8DX-0007r3-2i for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:07 -0400 Received: from mail-wm2-x11.google.com ([2a00:1450:4864:31::11]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8DV-0005xP-8l for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:06 -0400 Received: by mail-wm2-x11.google.com with SMTP id 5b1f17b1804b1-49ccead2aecso5203835e9.0 for ; Fri, 25 Sep 2026 08:52:04 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351524; x=1790956324; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=wz5J7BtRm96GQteyQTuXbDxbX79LxGun+611Yug+ddc=; b=PkfO7I6tfwUpTxeW3n2piAaWS9E78WbmL3PB+59bNV9m6LaLDMd5f7eZ/68lvNt1d4 5zkcG59JebyEGMFGr16Y31zlKXUOehbmWFu4/tvZpi5//L8MxOQSSqwlZgkjBrkHfshD RV0QUXCsKybT1eSQkuuQWoUfJKsNP/38sR7ftGPG/iCrPXfY4zTE1cF4XdqKstEiUWTh IiGLPS/LsD9My037FO0s0IRJJzBqJ7eC+OTl/HqDSX4kQxMh6X5+qXHqCkJTMfCgpcVl b7gPsAT97vP0dIrL9PWERw+t86KnZ5cyRt9kBf1Ia50irxcrImcRWIUdceJpSCAnevbD 8I9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351524; x=1790956324; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=wz5J7BtRm96GQteyQTuXbDxbX79LxGun+611Yug+ddc=; b=bgDB+J1svLecYHwEeslbyLswrzQhWTPe5R+kY3UWfqTKWXii75GSm3EjD1TEeZA5Zo udu8yOVsIwkbqZ+PZ4Go6nXwTwdYqOY3sHHOw21umzZvHJY7yWPPOPWsZ/KgPntcpQGA gVJjOqewaDq3UfGTKPONGL3MgW0BjWMsW7in6ZjMz54q6wYNjBpOgmGwbm9a8q00bxeo 9tJ9BPR71LKOMYmAdrrkxPpew36qxcqBNptfa9x7zFeu2hunjcJVy0hUJFxO2JNh+6AJ kYEq6RjIHr65Z1A7NQXWWGh6O9WrZMZvdCkqKGVY2pzBTQOVxKcamlluV0jhADs6egJn yqhQ== X-Gm-Message-State: AFuF++kyRmFtiCrmFHmEjEau4xvuzBCH7igkNZvekPiVcrgDp8Q64Vt1 3YO/cUoDoTbLJI1XMZGJBAe8v/C0PB9SGNVkTFjlp6/RXByPfwyVjq7iVylEg5TzMUc8Qz+IqmO QPzRBKvk3Aw== X-Gm-Gg: AYBFou0UfXZmh9Z6Row0VYYY4B4hiY0UrH/dtVL2WT9J/FeTnh9NAamPfdsiQDvo4js bbNbwOJy6dvW4DpZaNAdMV3XkFYZX7yP45G/LofFAoh9vsQnUzp63192GHmYC4wVW5u6VlZncYg uj9/gPP805LTpP/cPhDCBE+pD4CGEk9CDBTF8oQ+O4WoNB8ZfuCnJpM8HWkr57WM6q2IxpI+fiw thQJYoRlgFlJ8aOfq6ojNpgQChlkO12HnaVwHckmuWPrg5Nf1JM7RUibE9SbErwtUFhiZLakFO0 ToR1M2gR1GpLJ/6zlX6xr4sTMrV9Eq8i2DSIjPN+M3AshhjOw4hHDb+akHRByFLY+2Y+g0BKRzp c+IgyOt6alokgCEl/wfVCv+BaayXJLnpTzAAX9OLFpEwf/Cg4vScnJMwcnaQ9NQwnyk0jrfm0eg Yl8BnxpTKGeueSKzaqw6B9NtQgRRUm4+0jCJXmjuQSBxM8i6q6Mklqh2EsMX60yQQEtzCdELShE 9tuUS4GR6qd4THpvvFzRdGLoecrvN4u9eHubHPzuA== X-Received: by 2002:a05:600c:1d1c:b0:49f:bc43:9e96 with SMTP id 5b1f17b1804b1-49ff06bcd5fmr50845065e9.8.1790351523927; Fri, 25 Sep 2026 08:52:03 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 03/13] migration/rdma: Add send-ring lifecycle (no consumer thread yet) Date: Fri, 25 Sep 2026 17:51:38 +0200 Message-ID: <20260925155149.843870-4-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::11; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x11.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:04:57 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352517431158500 Content-Type: text/plain; charset="utf-8" Introduce the producer side of the SPSC send ring: rdma_consumer_start allocates the ring at registration setup, rdma_consumer_stop frees it at cleanup, plus ring_push/ring_wake_consumer. No thread runs yet and the existing send path is unchanged. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index e4ede950de..2d4807c6b8 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1214,6 +1214,9 @@ static void rdma_nbsent_inc(RDMAContext *rdma, uint64= _t nb_sent); static void rdma_nbsent_full(RDMAContext *rdma); static void rdma_nbsent_zero(RDMAContext *rdma); =20 +static void rdma_consumer_start(RDMAContext *rdma); +static void rdma_consumer_stop(RDMAContext *rdma); + /* * Consult the connection manager to see a work request * (of any kind) has completed. @@ -2108,6 +2111,9 @@ static void qemu_rdma_cleanup(RDMAContext *rdma) { Error *err =3D NULL; =20 + /* Free the send ring before touching the QP. */ + rdma_consumer_stop(rdma); + if (rdma->cm_id && rdma->connected) { if ((rdma->errored || migrate_get_current()->state =3D=3D MIGRATION_STATUS_CANCELLI= NG) && @@ -2753,6 +2759,91 @@ struct RDMASendRing { uint64_t idle_max_ns; /* longest consumer wait on an empty ring= */ }; =20 +static bool ring_full(RDMASendRing *r) +{ + return (r->head - qatomic_load_acquire(&r->tail)) >=3D RDMA_SEND_RING_= SIZE; +} + +/* Producer -> wake the consumer if it is idle on an empty ring. */ +static void ring_wake_consumer(RDMASendRing *r) +{ + smp_mb(); /* order the *_waiting load below vs the sleeper's store */ + if (qatomic_read(&r->consumer_waiting)) { + qemu_mutex_lock(&r->wait_lock); + qemu_cond_signal(&r->not_empty); + qemu_mutex_unlock(&r->wait_lock); + } +} + +static int ring_push(RDMAContext *rdma, const RDMACmd *cmd) +{ + RDMASendRing *r =3D rdma->send_ring; + + assert(r); + + if (ring_full(r)) { + qemu_mutex_lock(&r->wait_lock); + r->producer_waiting =3D true; + smp_mb(); /* pair with ring_wake_producer() = */ + while (ring_full(r) && !r->abort) { + qemu_cond_wait(&r->not_full, &r->wait_lock); + } + r->producer_waiting =3D false; + qemu_mutex_unlock(&r->wait_lock); + } + if (qatomic_read(&r->abort)) { + return -1; + } + r->ring[r->head & RDMA_SEND_RING_MASK] =3D *cmd; + qatomic_store_release(&r->head, r->head + 1); /* publish payload+slo= t */ + ring_wake_consumer(r); + return 0; +} + +/* + * Producer: allocate the send ring once (idempotent). The consumer + * thread that drains it is created in a later change. + */ +static void rdma_consumer_start(RDMAContext *rdma) +{ + RDMASendRing *r; + + if (rdma->consumer_started) { + return; + } + r =3D g_new0(RDMASendRing, 1); + qemu_mutex_init(&r->wait_lock); + qemu_cond_init(&r->not_full); + qemu_cond_init(&r->not_empty); + qemu_cond_init(&r->barrier_cv); + rdma->send_ring =3D r; + rdma->pend_block =3D -1; /* no pending WRITE span yet */ + /* Producer-only flag (migration thread + cleanup, never concurrent). = */ + rdma->consumer_started =3D true; +} + +/* Producer: tear down the send ring. */ +static void rdma_consumer_stop(RDMAContext *rdma) +{ + RDMASendRing *r =3D rdma->send_ring; + RDMACmd cmd =3D { .type =3D RDMA_CMD_FINISH }; + + if (!rdma->consumer_started) { + return; + } + /* Ask the consumer to drain and exit; if it already aborted, just joi= n. */ + if (!qatomic_read(&r->abort)) { + ring_push(rdma, &cmd); + } + qemu_mutex_destroy(&r->wait_lock); + qemu_cond_destroy(&r->not_full); + qemu_cond_destroy(&r->not_empty); + qemu_cond_destroy(&r->barrier_cv); + g_free(r); + rdma->send_ring =3D NULL; + rdma->consumer_started =3D false; +} + static int qio_channel_rdma_set_blocking(QIOChannel *ioc, bool blocking, Error **errp) @@ -3810,6 +3901,11 @@ int rdma_registration_stop(QEMUFile *f, uint64_t fla= gs) goto err; } =20 + if (flags =3D=3D RAM_CONTROL_SETUP) { + /* Setup complete: allocate the send ring (consumer added later). = */ + rdma_consumer_start(rdma); + } + return 0; err: rdma->errored =3D true; --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352460; cv=none; d=zohomail.com; s=zohoarc; b=VPIGBZaaLFr6QCLshMRfuztjOAVejj8CQzOIBIoOaeer6qBkpKp4lndH1TU57DRR0EkPWlTA/9/bqRkMKX3Sgxv4Ba6kAcDrsNDo3i/KXrmSqD4H1y/oPUwsdxsRX3jORB8adNeZ8Y/yz+Hbq5yexYBotpBV6Iy7QKl1koF+5M4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352460; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=vF6kFcDMQ1VuXLdbyXfYLNVRnEpNUNDUbAvytuAdsMo=; b=KOvlgFDFq8Z8rrQXOSOVXFiLxxQtVDA9xMnWRJqXGDfYsXHFhjbMFR+zA507rdepJ5Y0TC+tvoDnMsBzAIbFolG6x2S5xlE78fnjbvftD6VrcHm2yBz3kzNoiqLorgmue8jRbCzzGG84KTq4SvxT7kkpzk8oTgedeyWx5bfOqUc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352460520898.5481243666054; Fri, 25 Sep 2026 09:07:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8SP-00083i-3K; Fri, 25 Sep 2026 12:07:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8DZ-0007s3-Um for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:10 -0400 Received: from mail-wr2-x13.google.com ([2a00:1450:4864:30::13]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8DX-0005xX-TU for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:09 -0400 Received: by mail-wr2-x13.google.com with SMTP id ffacd0b85a97d-482f6350f89so674184f8f.3 for ; Fri, 25 Sep 2026 08:52:07 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351526; x=1790956326; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=vF6kFcDMQ1VuXLdbyXfYLNVRnEpNUNDUbAvytuAdsMo=; b=BmG4kewkHlK0+9Uq6dqchujzQO2qWUtwWFDDXd91VrdsqHCRk22BC9mGm/nwDkFjZb 11RAoBx1e3+4aTpgTSy5ruOVMVNxyoCSYjNC6Ui+of9KTiN2XJK0TAyKwak4PDRl+rVO wPfMvnFDIfb33peIm+XVzwZDh6QJYXZO9JVqtBWT80iGN1Ukvnu1zduoU6maE2WwVzRb al37e+IdgbOQ9QVJzEfIhhaQsJlqL3AUO/kx+7B3B5hwgdkSh97nmiguTyYbRxH4o+7S 7YA5iN1qgh/w3Df0hEFgW8jP1+WLpvfjmUtlki7xhbgC4yxb3BZEzSueL7CXTWQZ8e3x qJvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351526; x=1790956326; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=vF6kFcDMQ1VuXLdbyXfYLNVRnEpNUNDUbAvytuAdsMo=; b=ZwnirxRx1mW1Omsd1cf7I7sX2VolR7Rb71VO48QjMhHPNbsT+xfwVUBAx2dp+hcEpD 8yhsi7Rsq/515nCpsSJQVCme0jdhk78qMDBruluHI7OzuBnEaLaQQxme2nQ8W5WGa7rU dFs/w0GEkCtf7TeUqhrdZvhtqLF7cAHLb3/39Rv0tVZCAXNqq+SWogH3cZxbbkgYPAMD iZsgNRsWfahX1hq8RN35fc2z9if8dDv4Uwn0rKXoU8d1g/0wsXMvfEU3v0PS2zwCMPXT LnSdllUJqUICPjyg7YwylfA8U4o8r5vCwsetXfxjJ+D6TqAL0Mmd8yZuzzAGXiCkHf6w nVTg== X-Gm-Message-State: AFuF++kjyNAF2FNQKJ+HRruhYx+TGH62av/qQLiZtNmkDi7yQPgqEKCO rlQ9ci+kX2DrC93fO9Cc+aIn4xQCdff9zywsYq+R6K2cNizI2SNZeK0VRCG+9aGapdcOsOfZUq4 +Zwnuw60WPw== X-Gm-Gg: AYBFou040ymqPl8udkw0NyNKD7/Pl1bKCCRR++2Sir+P/xzTFMo3vA8fGNxMhVZxOo0 xd/dP89TorXCEG67R7HUk003pjByAGsU1pCs5bsSwvImTZ3r+9QCLcLlDCOfbhjZ0976yFkgjhV DVqyW2AXCxfswTmgwGzsW9mY2B305B/v2MRYWI1GPEadZdxsYFw+dYtf4ff8SBtldRBC/wz0WZW y5SGlu3qvc2/rYPsbidESRNGJIGKd+qG2gjo6AAieIiQu7tBi3VzBXiLsgOOYxv/P5oSAUJt68w tg+rekgN6VCAfeq7TdW+PGMoLUOk0DB0AP8wDw8lGuD4gkfyHqEijuui7/+7A/O0vCqnLiGK6E9 rkkDeLTtzEJi7IogyPx+SmSMU8Dlr7zw1SywzYmh5xBPC1WS3v1jdsYseecBok/7osWaBKV1fuy LN/IDrVM+KUGaQP1Zme4UUHhlJAG6vvWG32Wvq1nhNae66EqdUYCWckjszN0D7N8CqW2RTdYBx0 jlFk47YYTDr3eYhvRLETHTRcuRyA05povNH+WPJEg== X-Received: by 2002:a05:600d:486:20b0:49f:e941:e26 with SMTP id 5b1f17b1804b1-49ff06b2f01mr36998955e9.4.1790351526029; Fri, 25 Sep 2026 08:52:06 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 04/13] migration/rdma: Add the send consumer thread (drain only) Date: Fri, 25 Sep 2026 17:51:39 +0200 Message-ID: <20260925155149.843870-5-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:30::13; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wr2-x13.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:04:59 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352461142158500 Content-Type: text/plain; charset="utf-8" Create and join the consumer thread in rdma_consumer_start/stop and add the ring operations it needs to drain and shut down: ring_pop, ring_advance_tail, ring_idle_wait and ring_set_abort. rdma_consumer_dispatch handles only RDMA_CMD_FINISH (drain the CQ and exit). The old send path is still in charge, so the ring carries only the FINISH pushed at teardown. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 120 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 114 insertions(+), 6 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 2d4807c6b8..c3070b7a92 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2111,7 +2111,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma) { Error *err =3D NULL; =20 - /* Free the send ring before touching the QP. */ + /* Stop and join the consumer before touching the QP it owns. */ rdma_consumer_stop(rdma); =20 if (rdma->cm_id && rdma->connected) { @@ -2764,6 +2764,37 @@ static bool ring_full(RDMASendRing *r) return (r->head - qatomic_load_acquire(&r->tail)) >=3D RDMA_SEND_RING_= SIZE; } =20 +static bool ring_empty(RDMASendRing *r) +{ + return (qatomic_load_acquire(&r->head) - r->tail) =3D=3D 0; +} + +static void ring_set_abort(RDMAContext *rdma) +{ + RDMASendRing *r =3D rdma->send_ring; + + if (!r) { + return; + } + qemu_mutex_lock(&r->wait_lock); + qatomic_set(&r->abort, true); + qemu_cond_broadcast(&r->not_full); + qemu_cond_broadcast(&r->not_empty); + qemu_cond_broadcast(&r->barrier_cv); + qemu_mutex_unlock(&r->wait_lock); +} + +static void ring_wake_producer(RDMASendRing *r) +{ + smp_mb(); /* order the *_waiting load below vs the sleeper's store */ + if (qatomic_read(&r->producer_waiting)) { + qemu_mutex_lock(&r->wait_lock); + qemu_cond_broadcast(&r->not_full); + qemu_cond_broadcast(&r->barrier_cv); + qemu_mutex_unlock(&r->wait_lock); + } +} + /* Producer -> wake the consumer if it is idle on an empty ring. */ static void ring_wake_consumer(RDMASendRing *r) { @@ -2800,10 +2831,84 @@ static int ring_push(RDMAContext *rdma, const RDMAC= md *cmd) return 0; } =20 -/* - * Producer: allocate the send ring once (idempotent). The consumer - * thread that drains it is created in a later change. - */ +static bool ring_pop(RDMASendRing *r, RDMACmd *cmd) +{ + if (ring_empty(r)) { + return false; + } + *cmd =3D r->ring[r->tail & RDMA_SEND_RING_MASK]; /* after load_acqui= re */ + return true; +} + +static void ring_advance_tail(RDMASendRing *r) +{ + qatomic_store_release(&r->tail, r->tail + 1); + ring_wake_producer(r); +} + +/* Consumer: sleep until the ring is non-empty or aborted. */ +static void ring_idle_wait(RDMASendRing *r) +{ + qemu_mutex_lock(&r->wait_lock); + r->consumer_waiting =3D true; + smp_mb(); /* pair with ring_wake_consumer() = */ + while (ring_empty(r) && !r->abort) { + qemu_cond_wait(&r->not_empty, &r->wait_lock); + } + r->consumer_waiting =3D false; + qemu_mutex_unlock(&r->wait_lock); +} + +static int rdma_consumer_dispatch(RDMAContext *rdma, const RDMACmd *cmd, + bool *stop) +{ + Error *err =3D NULL; + int ret =3D 0; + + switch (cmd->type) { + case RDMA_CMD_FINISH: + ret =3D qemu_rdma_drain_cq(rdma, &err); + *stop =3D true; + break; + default: + break; + } + if (ret < 0 && err) { + error_report_err(err); + } + return ret; +} + +static void *rdma_consumer_thread(void *opaque) +{ + RDMAContext *rdma =3D opaque; + RDMASendRing *r =3D rdma->send_ring; + bool stop =3D false; + + rcu_register_thread(); + + while (!qatomic_read(&r->abort) && !stop) { + RDMACmd cmd; + + if (ring_pop(r, &cmd)) { + if (rdma_consumer_dispatch(rdma, &cmd, &stop) < 0) { + ring_set_abort(rdma); + break; + } + ring_advance_tail(r); + continue; + } + ring_idle_wait(r); + } + + if (qatomic_read(&r->abort)) { + /* make the failure visible to the producer's rdma_errored() check= s */ + rdma->errored =3D true; + } + rcu_unregister_thread(); + return NULL; +} + static void rdma_consumer_start(RDMAContext *rdma) { RDMASendRing *r; @@ -2820,9 +2925,10 @@ static void rdma_consumer_start(RDMAContext *rdma) rdma->pend_block =3D -1; /* no pending WRITE span yet */ /* Producer-only flag (migration thread + cleanup, never concurrent). = */ rdma->consumer_started =3D true; + qemu_thread_create(&rdma->consumer_thread, "rdma-send", + rdma_consumer_thread, rdma, QEMU_THREAD_JOINABLE); } =20 -/* Producer: tear down the send ring. */ static void rdma_consumer_stop(RDMAContext *rdma) { RDMASendRing *r =3D rdma->send_ring; @@ -2835,6 +2941,8 @@ static void rdma_consumer_stop(RDMAContext *rdma) if (!qatomic_read(&r->abort)) { ring_push(rdma, &cmd); } + qemu_thread_join(&rdma->consumer_thread); + qemu_mutex_destroy(&r->wait_lock); qemu_cond_destroy(&r->not_full); qemu_cond_destroy(&r->not_empty); --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352377; cv=none; d=zohomail.com; s=zohoarc; b=Az/He5UJg57FO0qkes6kcD+P9cvARg7ke2Er0hGKVZP6Ir0fxBFBGk/7F0P+0DF64iEF12XeCtEflYoM0/m+vKRLURpc1Nc7LqatB9DHtsejQuah//QKFH3dhOyZKhrV4ozSk+5K8YOiZmaVdcIBZYvpTVNTz6la3CKdyzRDImE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352377; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RpbJfFtFXtW5DVtUKxFYbq48SJgv8dFmBAbxHxL31/s=; b=d1dA9QnwkYQdze+tmBuzzFeW/dbWVUHjxgsO34YC1TSwqCciPrZe/8jpmEdbhvn5w7k+NtdUhD55CmXcqye0EgTo49G8alio6pSOa5FwyllmT0rdIEyH5itEBrQtD04Py1ncrb5ReoH54Tx8Y6ObzPMT6uMNfR9dWfv4QBEtLH4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352376796344.80449857197993; Fri, 25 Sep 2026 09:06:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8R0-000600-0C; Fri, 25 Sep 2026 12:06:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Da-0007sW-TS for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:10 -0400 Received: from mail-wr2-x0f.google.com ([2a00:1450:4864:30::f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8DZ-0005xq-6I for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:10 -0400 Received: by mail-wr2-x0f.google.com with SMTP id ffacd0b85a97d-48882c1f261so204211f8f.0 for ; Fri, 25 Sep 2026 08:52:08 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351528; x=1790956328; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=RpbJfFtFXtW5DVtUKxFYbq48SJgv8dFmBAbxHxL31/s=; b=NO43u7dqx5+MMu+oUdCpWXTFnrnql4KZLXB2Ns6gqlvO9afixA+i2j0rMmQqXHuzM0 jGMhHLPcNlAILF1RwvENN2gX7ZFVZD1sD0Kz7WKs5KFgU/8hyh4WWQSCSVPxrtnRBBC7 dzqmX2PONfifZrOJKUniIQUKA+1Y6Op7ukOptOlSMIz6l6daDnwqBeTtb5rK0LGWw+Up /Sh56rtImiSHhbhX/TNo6u94qe2ItikQ8/JNYgWFQru/yAVoSqqLh3yJY9Vi9yOC4llF +seOHKzTj6896TEfjs6Fj89qGrqOLxz+eQdak/783//E8xn6REqMhFKOZTfGQO2aTj0V X/yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351528; x=1790956328; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=RpbJfFtFXtW5DVtUKxFYbq48SJgv8dFmBAbxHxL31/s=; b=t9HwT3v/isr26bf8haTbS+yPUI9sOgI2Q9zFim+Xmpf2PN+IM3J054MmPql2CF7gpY eBujblVD4aEUnivjFJiUUUAT4L6D4sKmuNMBkzQOXqTYs51KQVDbR8zxMr8008DVcy63 GaaGQ5W26jMKdSzNzGAOm/bS8wxVGNhh5A7eCqY/sBSYB74ilKAVhCstq1aFW/ZiLSJ2 aZTXpZhRrXHxhg4hxPSGhYViF123d/qlayHZc1mpT29RBDLm4PEN9hR34XIAsN1mFEvc GPy0YdxGl1tg6JJod39JsVRI5r9cn7hDqK/SVr+IgOy3vezbimDnEWLuUtxwEUg3cxOp 4x+g== X-Gm-Message-State: AFuF++nIe7Q8JweDzsBz3FG+b7IutGw5L8hcaBtp2mE7P3jaKXNIbGDc pGYOAeathY1EFDBtsjC5RpQq+wO1cc+h4Ub+danYQuL8mCY1p1BxIyFA5YpDpCrL4nCMDNiUvsP 74CUMZBC5kQ== X-Gm-Gg: AYBFou3kF9s6jepNyvzJQVRXnXupVmjS73hAlCpA9EaIGunfwgijG3afBhMTfQ4ZUxy 0j9j4p7DB1aJxeLqUPc7cfHMIt4BhQ1JTK3Kse4/er9MUq54k8EQI1kebyHuixvkBjxYpuBaVQE GDoHqomCpvjrjAfiTh4NR8Sb+84sbmCYXRtR0hW5Ba8ZYQueBm7aVnOMozoM1gMURk+4NMQSj3/ JjD56JPz17kmvqvqhcs00MpaumahVOxf75cG7OhWTQ5QonpHbzPRjExI5PwVevli3Mj975hHuFx wc489RQWo57ABXB2NjOtAlBvRdWwhifh4MjKA0DNP9Gv21wwdFgmXORAarfHghiukdP3Eki3P/E NHkt0AQQLDM4O6yai7NeaAQHsm57jxohfpVoMwpcIbU6k6p+SNzyqwSS3bHnRDK9wK5k+P3QxtD gSKpDbcrsDKdDH+uCHhLuI/6XO9UAZo6Z//6vHFV6nNSjJlckRQWRm3a4zZ+aCgpMyRJMDF8gl3 4ZYfRJkA3Fjy549DvmU6SCi5fcM6S7+zVyvJDjvQGpUqGjNRjm9 X-Received: by 2002:a05:600c:4e8f:b0:49e:7c8c:bc77 with SMTP id 5b1f17b1804b1-49fe66d062bmr110866185e9.7.1790351527744; Fri, 25 Sep 2026 08:52:07 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 05/13] migration/rdma: Handle write/flush/reg-stop on the consumer Date: Fri, 25 Sep 2026 17:51:40 +0200 Message-ID: <20260925155149.843870-6-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:30::f; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wr2-x0f.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:04:59 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352380457158500 Content-Type: text/plain; charset="utf-8" Extend rdma_consumer_dispatch to run the real send work off the ring: RDMA_CMD_WRITE posts one chunk via qemu_rdma_write_chunk, RDMA_CMD_FLUSH drains the CQ, and RDMA_CMD_REG_STOP sends REGISTER_FINISHED (draining and dumping stats first on FINISH). The thread acks barrier commands via ring_barrier_complete. The producer still drives the old inline path, so the ring still carries only FINISH. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index c3070b7a92..d11f6e0da8 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1216,6 +1216,7 @@ static void rdma_nbsent_zero(RDMAContext *rdma); =20 static void rdma_consumer_start(RDMAContext *rdma); static void rdma_consumer_stop(RDMAContext *rdma); +static void rdma_nbsent_stats_dump(RDMAContext *rdma); =20 /* * Consult the connection manager to see a work request @@ -1986,6 +1987,33 @@ static int qemu_rdma_write_one(RDMAContext *rdma, return 0; } =20 +/* Consumer: write one whole chunk */ +static int qemu_rdma_write_chunk(RDMAContext *rdma, uint32_t block_index, + uint32_t chunk_index, Error **errp) +{ + RDMALocalBlock *block =3D &rdma->local_ram_blocks.block[block_index]; + uint64_t chunk_off =3D (uint64_t)chunk_index * migrate_rdma_chunk_size= (); + uint64_t len =3D block->length - chunk_off; + int ret; + + if (len > migrate_rdma_chunk_size()) { + len =3D migrate_rdma_chunk_size(); + } + + /* write_one wants current_addr in ram_addr_t space (block->offset bas= ed) */ + ret =3D qemu_rdma_write_one(rdma, block_index, block->offset + chunk_o= ff, + len, errp); + if (ret < 0) { + return -1; + } + if (ret =3D=3D 0) { + rdma->nb_sent++; + trace_rdma_write_flush(rdma->nb_sent); + rdma_nbsent_inc(rdma, rdma->nb_sent); + } + return 0; +} + /* * Push out any unwritten RDMA operations. * @@ -2846,6 +2874,13 @@ static void ring_advance_tail(RDMASendRing *r) ring_wake_producer(r); } =20 +/* Consumer: publish barrier completion. */ +static void ring_barrier_complete(RDMASendRing *r, unsigned seq) +{ + qatomic_store_release(&r->barrier_done_seq, seq); + ring_wake_producer(r); +} + /* Consumer: sleep until the ring is non-empty or aborted. */ static void ring_idle_wait(RDMASendRing *r) { @@ -2859,6 +2894,23 @@ static void ring_idle_wait(RDMASendRing *r) qemu_mutex_unlock(&r->wait_lock); } =20 +/* Consumer: send REGISTER_FINISHED; for FINISH drain and dump stats first= . */ +static int rdma_consumer_reg_stop(RDMAContext *rdma, uint64_t flags, + Error **errp) +{ + RDMAControlHeader head =3D { .len =3D 0, + .type =3D RDMA_CONTROL_REGISTER_FINISHED, + .repeat =3D 1 }; + + if (flags =3D=3D RAM_CONTROL_FINISH) { + if (qemu_rdma_drain_cq(rdma, errp) < 0) { + return -1; + } + rdma_nbsent_stats_dump(rdma); + } + return qemu_rdma_exchange_send(rdma, &head, NULL, NULL, NULL, NULL, er= rp); +} + static int rdma_consumer_dispatch(RDMAContext *rdma, const RDMACmd *cmd, bool *stop) { @@ -2866,6 +2918,16 @@ static int rdma_consumer_dispatch(RDMAContext *rdma,= const RDMACmd *cmd, int ret =3D 0; =20 switch (cmd->type) { + case RDMA_CMD_WRITE: + ret =3D qemu_rdma_write_chunk(rdma, cmd->block_index, + cmd->chunk_index, &err); + break; + case RDMA_CMD_FLUSH: + ret =3D qemu_rdma_drain_cq(rdma, &err); + break; + case RDMA_CMD_REG_STOP: + ret =3D rdma_consumer_reg_stop(rdma, cmd->flags, &err); + break; case RDMA_CMD_FINISH: ret =3D qemu_rdma_drain_cq(rdma, &err); *stop =3D true; @@ -2895,6 +2957,9 @@ static void *rdma_consumer_thread(void *opaque) ring_set_abort(rdma); break; } + if (cmd.seq) { + ring_barrier_complete(r, cmd.seq); + } ring_advance_tail(r); continue; } --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352439; cv=none; d=zohomail.com; s=zohoarc; b=ZxTlO9x4WFADgoQu4gBG+KyQY4XFajtYxiq2w9xxldHhYAvfPijhJD7vBLWyBiz9Tks6RbnJhsqXzO+vj3EP7Wdx+7FCbJxDrxJ1ebN/eWnPsUpl4/gVWg+u8EwOS5kkSDCTXrSKiFu3A8sav1KaUmZYgXJVoza/kHX3CwBMyK4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352439; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xyCVIEiWAWEWFWWLsQfxm3BQ7wFhpZ59Cs/kPlajGTY=; b=j4hXoXrRJXJpgRBs8hS6TXaWZRMBqXzBwzLmjlV81TsDa0D0LCN6RJdwDzAvmfnTYmoRmlQBo3RnAQ/GAcNsRmRU1R+64g+G4TFY4tK9a4DxebdvUWepwK8JvxE37x+kIKPVdJdEFWmrQZ49qOysQWrWa4CDG6W6QOYWkU0sbrE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352439801647.8182967997028; Fri, 25 Sep 2026 09:07:19 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8R0-00060r-2V; Fri, 25 Sep 2026 12:06:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Dc-0007sy-Q2 for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:12 -0400 Received: from mail-wm2-x11.google.com ([2a00:1450:4864:31::11]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Da-0005yI-Vv for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:12 -0400 Received: by mail-wm2-x11.google.com with SMTP id 5b1f17b1804b1-49d1fb0cf5eso9245785e9.3 for ; Fri, 25 Sep 2026 08:52:10 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351529; x=1790956329; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xyCVIEiWAWEWFWWLsQfxm3BQ7wFhpZ59Cs/kPlajGTY=; b=GqFM6Ma1fV3l2w8xpMoELN/n5to9c/1IFknjUIm6rfWD1rxvHeBFQBTpTTMXqXfkGs FuBUu31Ya/kSTxBaHcKNCWkhL6ObrApxZ10Pn6R0MA74zNsVFVT82Zz+UW8WKTO1r4m+ e+6gKlwfsgyzOTs9nub3WdRzfk1J7JppBk/bXJqr/AB3AJIbqZYS+CJLqF5xSe0/e6OP gk5e76OODqULoLRxxM7BHKYtgrlc79VIIsU9k7zYSFtwdF3ptwNKQsWgSd0q0UBBaOtq 8ue1yFqPG+ahEFcFXXP6hrvVEW1vL3kfZSbALPu6sNb+spwNRgj8tqd6H6A5hsXLkuOl K8VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351529; x=1790956329; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xyCVIEiWAWEWFWWLsQfxm3BQ7wFhpZ59Cs/kPlajGTY=; b=oq3ogn2ADZ8O1wyijRaUQ7SEEwqrmuoDFsiI95J9U2ZPSVI3KpNBWMzKduKUwp6JmG Gm4sHHrRHyOQGIcEuS1KwRnq2H37T2liTvAEVISOAF2mz4H9TbLaqbkTgBl5gYDQrASg gY3a9UtX8VxWyQjAgJKtddFPst8Lfyq8aiNKSk7IZMT4W/PKdzd0z66ytsPTtyNHOloy qjz4q6N/6jYCBPW/b28kdUGz4AlMRpaySCT+hpbqovveBn68z8ZHJXtGlaG76G40iK5B vF1vB+W7j8wKx8P+xKH1TSDQRyCCA9h1BMP7jV8hPRX4EwUA+KrjWQJp3yqPwgrug/fL G8ug== X-Gm-Message-State: AFuF++k1GOhUSBsx7NETJJqQYv5FOxtRi2HqLu+NTnnKbI7nhHtU2EzG LwNPy+D2I+IzBtuchxIpe834s63XLHxJO8EqwTT+GLE5CnNGpzgaStGdc/SzU260UDXpIbu+MsH UPIJqiXP6bQ== X-Gm-Gg: AYBFou18KeK/fm1UbXvfG+xfiVfRprfLvxEUCNUgPZLUtHAsx1GmiIrjaJ6n9S9B33b H2fbbRQCjVkRnuDNuJgfDqdehOobbpG6iPdcEVbBD84NM+DkIea/Ro2qmuz1H5I0/26Qdla0wMY 9J/PStmiTSfjzXItVYxaPCRPlSisN8Ynz+dk9q6ycRzP4qUXGFvvBQSkDV7ala6RvDoiLZ56cIB q963Amh7e3T2iTwbUFfXAxpR6SpDu6unbx6YHMMr/zhwZj4CljG3cR1z3qlYSWFciUVXFc0/jgg kGt4OtQe4AqJoTTxUnri5aQTq+/qji3wM8k5iqgTAe6Wizx/CBG1eyZJOls1+xnUNrZ76ubKIa1 5wI30QkBY50BbLYK07Uh9dxB/UNldBqCvoMvJ/tARZrhFNsZALr4dxVi1nzchkixLD6FebvQmii I6J9X1m2obKPZBbmsRuDE6mpN1VnY3XfCjSeh1NyZccSegQzjhsFNTnPyM89dgoqR8DMhhs1eGf r5sW9HxGtcolTEobko8LT5goliZp3B/p7rFxvtQkA== X-Received: by 2002:a05:600c:630f:b0:49f:bd3c:bc1f with SMTP id 5b1f17b1804b1-49fe66fd51dmr109898975e9.26.1790351529377; Fri, 25 Sep 2026 08:52:09 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 06/13] migration/rdma: Extract qemu_rdma_write_file() and dispatch FILE Date: Fri, 25 Sep 2026 17:51:41 +0200 Message-ID: <20260925155149.843870-7-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::11; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x11.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:00 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352442860158500 Content-Type: text/plain; charset="utf-8" Move the per-iov QEMUFile control-byte send loop out of qio_channel_rdma_writev() into a reusable qemu_rdma_write_file() helper, and route the consumer's RDMA_CMD_FILE command through the same helper. No functional change to writev, which still sends inline. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 62 ++++++++++++++++++++++++++++-------------------- 1 file changed, 36 insertions(+), 26 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index d11f6e0da8..5ec6a87819 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2523,6 +2523,34 @@ static RDMAContext *qemu_rdma_data_init(InetSocketAd= dress *saddr, Error **errp) * SEND messages for control only. * VM's ram is handled with regular RDMA messages. */ +static ssize_t qemu_rdma_write_file(RDMAContext *rdma, const struct iovec = *iov, + size_t niov, Error **errp) +{ + ssize_t done =3D 0; + + for (size_t i =3D 0; i < niov; i++) { + size_t remaining =3D iov[i].iov_len; + uint8_t *data =3D iov[i].iov_base; + + while (remaining) { + RDMAControlHeader head =3D {}; + size_t len =3D MIN(remaining, RDMA_SEND_INCREMENT); + + remaining -=3D len; + head.len =3D len; + head.type =3D RDMA_CONTROL_QEMU_FILE; + + if (qemu_rdma_exchange_send(rdma, &head, data, + NULL, NULL, NULL, errp) < 0) { + return -1; + } + data +=3D len; + done +=3D len; + } + } + return done; +} + static ssize_t qio_channel_rdma_writev(QIOChannel *ioc, const struct iovec *iov, size_t niov, @@ -2535,7 +2563,6 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, RDMAContext *rdma; int ret; ssize_t done =3D 0; - size_t len; =20 RCU_READ_LOCK_GUARD(); rdma =3D qatomic_rcu_read(&rioc->rdmaout); @@ -2561,29 +2588,10 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *= ioc, return -1; } =20 - for (int i =3D 0; i < niov; i++) { - size_t remaining =3D iov[i].iov_len; - uint8_t * data =3D (void *)iov[i].iov_base; - while (remaining) { - RDMAControlHeader head =3D {}; - - len =3D MIN(remaining, RDMA_SEND_INCREMENT); - remaining -=3D len; - - head.len =3D len; - head.type =3D RDMA_CONTROL_QEMU_FILE; - - ret =3D qemu_rdma_exchange_send(rdma, &head, - data, NULL, NULL, NULL, errp); - - if (ret < 0) { - rdma->errored =3D true; - return -1; - } - - data +=3D len; - done +=3D len; - } + done =3D qemu_rdma_write_file(rdma, iov, niov, errp); + if (done < 0) { + rdma->errored =3D true; + return -1; } =20 return done; @@ -2928,12 +2936,14 @@ static int rdma_consumer_dispatch(RDMAContext *rdma= , const RDMACmd *cmd, case RDMA_CMD_REG_STOP: ret =3D rdma_consumer_reg_stop(rdma, cmd->flags, &err); break; + case RDMA_CMD_FILE: + ret =3D qemu_rdma_write_file(rdma, cmd->iov, cmd->niov, &err) < 0 ? + -1 : 0; + break; case RDMA_CMD_FINISH: ret =3D qemu_rdma_drain_cq(rdma, &err); *stop =3D true; break; - default: - break; } if (ret < 0 && err) { error_report_err(err); --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352513; cv=none; d=zohomail.com; s=zohoarc; b=AqglSLHdoT2qOsJ5Q7HXCrno4aBAgrH8sqxsLG29vwN0ipCIVu38IIRUwanDQOhndTRfXnluLOnIkTpGiePIABgy1vDkCvwKTLoSOuRRDmjqL5LjwvoVmvPiWPI9H67sdqaTku0zr75YoIUZVEYR+eUOJQDsQAqWFhQyZkjOLJI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352513; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=pf2ATrAliPD5ewLoMgWuEE/Tzn/xf6Ocf52nYwtMr4A=; b=T9Ef0pk5+W094BHGGdlBay7EvHgqEgSXX4FI36u0YBPoqn2FGRgf/FIeSjYS95TFWBzR/rHBoIiS2i56Y+w27pTDc6P/rl8DNUSy2opQ0nTdviLYTN4Tp4cNmDn12yOCNd2Z+3XkDENRZKcdHIQWAL+5RwcgdEJL+uwaZwxWalI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352513385158.61048592713416; Fri, 25 Sep 2026 09:08:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8SV-0008Vp-Mo; Fri, 25 Sep 2026 12:07:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Dg-0007uD-Am for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:16 -0400 Received: from mail-wm2-x11.google.com ([2a00:1450:4864:31::11]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Dd-0005yy-9E for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:16 -0400 Received: by mail-wm2-x11.google.com with SMTP id 5b1f17b1804b1-49cd38e0e5dso13765705e9.2 for ; Fri, 25 Sep 2026 08:52:12 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351532; x=1790956332; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=pf2ATrAliPD5ewLoMgWuEE/Tzn/xf6Ocf52nYwtMr4A=; b=jBGU/fsZ9flxushMFVBlqJygjXpFGn4SJtNCpEJOorscQ0VH/p/w4BYfaLMEpmjhL7 a5EM8Anng8YMVjvZ/BQW5ElNwGdTcOdOk9U2lj5bm9JZiyIa/WUBIg6Y9l6KazOzQaUO pwUipJFv2dFQ8FwPeeea4kp5t/QtVOH+f4YWorhh34EQcfd/lGFbREnSNDAN3s2IE1LR AZDW9+XJd9u5kbgS1ZghX1plTa1+sxACGcADHcN1W6ybiduo10gWI6smVYvopb4K1EZg HTbweoDPDUsvpt/6eTJ2+LVkgJmFjd5fRrtyOUs+0fXce3HbP8oyLj5LD9qByr0S+zWX HjgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351532; x=1790956332; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=pf2ATrAliPD5ewLoMgWuEE/Tzn/xf6Ocf52nYwtMr4A=; b=FlAg2jAARtF52goepppcsQtMaJnWgWghypP6T0B2NTFf2Z1aRQKmIXkfQA1GtzHXcm vK039TDDyaw0knt9q8+oj7ecIyZXHvV908BJphGwVde9Qpiw4VKsiLVVLZ8vpWgDtorB ashNnKnOAHcJHUz1MuPYQIuEnnrjmJT5MZh22zSy9BpJt/3exSs7H7WMLX9+GyMCmwYo DXztrY1m9i31ae2P0co6/sqbPyeEzPGBwknK1NrKeJMWRQLbGjgpv4wVz6gEDZ0YmMKv aoXdKAUAexmBpQCh7+T5Dsu21lULDyxR4Mr93F3xBzIE+rjKs0sUS3P+N1k9vOC4xbQc RQCw== X-Gm-Message-State: AFuF++m2RV5+Oos7Scy0awgMb5KeYxc54iMdKhHnLTqZazyMt4oh+5OU O+8WR2MIOxP0W3JGuqDDG5yq4bPtAkoKuBrRcYPoH/hcWDyP9dZwWDQkCAt4TzJNoTXzr5IsT9U 8QrY5m48Jbg== X-Gm-Gg: AYBFou1XBe0uLN3xbUJVpyxDkv9UINVuOJDayI2FSjvQ33xUdvfaz1aNe9cms7wA1lG ldqnWQhakUYmSDZ+Y1qkqB7M6cx2SANhBKGX5gioQmdYtVvWfihDRt2T5DJUoMjPiNOklbYpEB8 ivrJVIed1z7/ehof8nbsl9BM7WltTShuEdgFkQoAwTcpcK5sYrsKn0Q2sw9hQeDRLT/5WZ+zADZ YriI1G91tzr/j1y6y77XYSdPhZUhreQ+wQSozq2BjPN/qUCamUu2immhOl8uumS+lBtNgomv41F e8VxIQC7pniawaze1G4vzR4g9AGFnL8W/ZmV6JIhcNFqNiS5OjQo4YtRj4v6G4ZiT6z40kNcu8h /w04g9PSh/KH0663WsddOaIY3RGrfwmYNolsMcxBaPF5UyUAWm9VZDVaVa576coXAmr9w6X4OZn VDo7FBaMAoEkHJM2rTQLXIg5x7UcNycHXit6LTbIwKG/BseS20p2suq+rMkmdRKNsH25/xQUcYS R3q3hotaiP/4kNyMB/p47T4AfZ30LUKzmlnII0hbQ== X-Received: by 2002:a05:600c:4eca:b0:49d:2450:68ac with SMTP id 5b1f17b1804b1-49fe66caac9mr111280715e9.5.1790351531803; Fri, 25 Sep 2026 08:52:11 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 07/13] migration/rdma: Switch the producer onto the send ring Date: Fri, 25 Sep 2026 17:51:42 +0200 Message-ID: <20260925155149.843870-8-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::11; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x11.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:01 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352515524158501 Content-Type: text/plain; charset="utf-8" Move the migration thread off the QP: qemu_rdma_save_page enqueues chunk writes (ring_push_write), qio_channel_rdma_writev and the round/finish notifiers post FILE/FLUSH/REG_STOP barriers, and the consumer reaps completions. Drop the per-page merge machinery (qemu_rdma_write / buffer_mergeable / write_flush / current_*) now that the consumer owns the transfer, and make errors set the sticky ring abort so both sides unwind. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 417 +++++++++++++++++++++-------------------------- 1 file changed, 189 insertions(+), 228 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 5ec6a87819..a0ff73e313 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -327,7 +327,7 @@ typedef struct RDMAContext { */ int control_ready_expected; =20 - /* number of outstanding writes */ + /* number of outstanding writes (consumer thread only) */ int nb_sent; =20 /* nb_sent statistics */ @@ -336,15 +336,6 @@ typedef struct RDMAContext { uint64_t nbsent_full_hits; /* posts that hit a full SQ */ uint64_t nbsent_zero_hits; /* drains to zero */ =20 - /* store info about current buffer so that we can - merge it with future sends */ - uint64_t current_addr; - uint64_t current_length; - /* index of ram block the current buffer belongs to */ - int current_index; - /* index of the chunk in the current ram block */ - int current_chunk; - /* producer/consumer split (source side) */ RDMASendRing *send_ring; /* SPSC ring; allocated at consumer star= t */ QemuThread consumer_thread; @@ -1216,6 +1207,12 @@ static void rdma_nbsent_zero(RDMAContext *rdma); =20 static void rdma_consumer_start(RDMAContext *rdma); static void rdma_consumer_stop(RDMAContext *rdma); +static int ring_push_write(RDMAContext *rdma, uint32_t bidx, uint32_t cidx, + uint64_t offset, uint64_t length); +static int producer_flush_pending(RDMAContext *rdma); +static int ring_barrier(RDMAContext *rdma, RDMACmdType type, uint64_t flag= s, + const struct iovec *iov, size_t niov); +static void ring_set_abort(RDMAContext *rdma); static void rdma_nbsent_stats_dump(RDMAContext *rdma); =20 /* @@ -1987,151 +1984,24 @@ static int qemu_rdma_write_one(RDMAContext *rdma, return 0; } =20 -/* Consumer: write one whole chunk */ +/* Consumer: write one dirty span (offset/length are block-relative bytes)= */ static int qemu_rdma_write_chunk(RDMAContext *rdma, uint32_t block_index, - uint32_t chunk_index, Error **errp) + uint64_t offset, uint64_t length, Error *= *errp) { RDMALocalBlock *block =3D &rdma->local_ram_blocks.block[block_index]; - uint64_t chunk_off =3D (uint64_t)chunk_index * migrate_rdma_chunk_size= (); - uint64_t len =3D block->length - chunk_off; int ret; =20 - if (len > migrate_rdma_chunk_size()) { - len =3D migrate_rdma_chunk_size(); - } - /* write_one wants current_addr in ram_addr_t space (block->offset bas= ed) */ - ret =3D qemu_rdma_write_one(rdma, block_index, block->offset + chunk_o= ff, - len, errp); - if (ret < 0) { - return -1; - } - if (ret =3D=3D 0) { - rdma->nb_sent++; - trace_rdma_write_flush(rdma->nb_sent); - rdma_nbsent_inc(rdma, rdma->nb_sent); - } - return 0; -} - -/* - * Push out any unwritten RDMA operations. - * - * We support sending out multiple chunks at the same time. - * Not all of them need to get signaled in the completion queue. - */ -static int qemu_rdma_write_flush(RDMAContext *rdma, Error **errp) -{ - int ret; - - if (!rdma->current_length) { - return 0; - } - - ret =3D qemu_rdma_write_one(rdma, rdma->current_index, rdma->current_a= ddr, - rdma->current_length, errp); - + ret =3D qemu_rdma_write_one(rdma, block_index, block->offset + offset, + length, errp); if (ret < 0) { return -1; } - if (ret =3D=3D 0) { rdma->nb_sent++; trace_rdma_write_flush(rdma->nb_sent); rdma_nbsent_inc(rdma, rdma->nb_sent); } - - rdma->current_length =3D 0; - rdma->current_addr =3D 0; - - return 0; -} - -static inline bool qemu_rdma_buffer_mergeable(RDMAContext *rdma, - uint64_t offset, uint64_t len) -{ - RDMALocalBlock *block; - uint8_t *host_addr; - uint8_t *chunk_end; - - if (rdma->current_index < 0) { - return false; - } - - if (rdma->current_chunk < 0) { - return false; - } - - block =3D &(rdma->local_ram_blocks.block[rdma->current_index]); - host_addr =3D block->local_host_addr + (offset - block->offset); - chunk_end =3D ram_chunk_end(block, rdma->current_chunk); - - if (rdma->current_length =3D=3D 0) { - return false; - } - - /* - * Only merge into chunk sequentially. - */ - if (offset !=3D (rdma->current_addr + rdma->current_length)) { - return false; - } - - if (offset < block->offset) { - return false; - } - - if ((offset + len) > (block->offset + block->length)) { - return false; - } - - if ((host_addr + len) > chunk_end) { - return false; - } - - return true; -} - -/* - * We're not actually writing here, but doing three things: - * - * 1. Identify the chunk the buffer belongs to. - * 2. If the chunk is full or the buffer doesn't belong to the current - * chunk, then start a new chunk and flush() the old chunk. - * 3. To keep the hardware busy, we also group chunks into batches - * and only require that a batch gets acknowledged in the completion - * queue instead of each individual chunk. - */ -static int qemu_rdma_write(RDMAContext *rdma, - uint64_t block_offset, uint64_t offset, - uint64_t len, Error **errp) -{ - uint64_t current_addr =3D block_offset + offset; - uint64_t index =3D rdma->current_index; - uint64_t chunk =3D rdma->current_chunk; - - /* If we cannot merge it, we flush the current buffer first. */ - if (!qemu_rdma_buffer_mergeable(rdma, current_addr, len)) { - if (qemu_rdma_write_flush(rdma, errp) < 0) { - return -1; - } - rdma->current_length =3D 0; - rdma->current_addr =3D current_addr; - - qemu_rdma_search_ram_block(rdma, block_offset, - offset, len, &index, &chunk); - rdma->current_index =3D index; - rdma->current_chunk =3D chunk; - } - - /* merge it */ - rdma->current_length +=3D len; - - /* flush it if buffer is too large */ - if (rdma->current_length >=3D rdma_merge_max()) { - return qemu_rdma_write_flush(rdma, errp); - } - return 0; } =20 @@ -2510,8 +2380,7 @@ static RDMAContext *qemu_rdma_data_init(InetSocketAdd= ress *saddr, Error **errp) RDMAContext *rdma =3D NULL; =20 rdma =3D g_new0(RDMAContext, 1); - rdma->current_index =3D -1; - rdma->current_chunk =3D -1; + rdma->pend_block =3D -1; /* no pending WRITE span yet */ =20 rdma->host =3D g_strdup(saddr->host); rdma->port =3D atoi(saddr->port); @@ -2561,7 +2430,6 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); RDMAContext *rdma; - int ret; ssize_t done =3D 0; =20 RCU_READ_LOCK_GUARD(); @@ -2578,22 +2446,28 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *= ioc, return -1; } =20 + if (!rdma->consumer_started) { + /* Setup phase: no consumer yet, no queued RAM writes -> send inli= ne. */ + done =3D qemu_rdma_write_file(rdma, iov, niov, errp); + if (done < 0) { + rdma->errored =3D true; + return -1; + } + return done; + } + /* - * Push out any writes that - * we're queued up for VM's ram. + * Consumer owns the QP. Route through the ring so this control messa= ge + * is ordered after all chunk writes queued so far; block until it has + * been sent (iov stays valid for the duration). */ - ret =3D qemu_rdma_write_flush(rdma, errp); - if (ret < 0) { - rdma->errored =3D true; + if (ring_barrier(rdma, RDMA_CMD_FILE, 0, iov, niov) < 0) { + error_setg(errp, "RDMA consumer aborted during control write"); return -1; } - - done =3D qemu_rdma_write_file(rdma, iov, niov, errp); - if (done < 0) { - rdma->errored =3D true; - return -1; + for (size_t i =3D 0; i < niov; i++) { + done +=3D iov[i].iov_len; } - return done; } =20 @@ -2705,10 +2579,6 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *io= c, /* Block until all outstanding writes have been delivered by the hardware.= */ static int qemu_rdma_drain_cq(RDMAContext *rdma, Error **errp) { - if (qemu_rdma_write_flush(rdma, errp) < 0) { - return -1; - } - while (rdma->nb_sent) { if (qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, NULL) < 0= ) { error_setg(errp, "rdma migration: completion polling error"); @@ -2745,7 +2615,16 @@ static int rdma_ram_round_notify(NotifierWithReturn = *n G_GNUC_UNUSED, return -1; } =20 - return qemu_rdma_drain_cq(rdma, errp); + /* Complete all chunks in the ring before the next round. */ + if (ring_barrier(rdma, RDMA_CMD_FLUSH, 0, NULL, 0) < 0) { + error_setg(errp, "RDMA consumer aborted during round drain"); + return -1; + } + /* + * The FLUSH barrier already flushed any pending span; start the next + * round with a clean accumulator (no cross-round merging). + */ + return 0; } =20 static NotifierWithReturn rdma_ram_round_notifier =3D { @@ -2846,6 +2725,7 @@ static int ring_push(RDMAContext *rdma, const RDMACmd= *cmd) { RDMASendRing *r =3D rdma->send_ring; =20 + /* SETUP starts the consumer before any WRITE/FLUSH/REG_STOP/FILE. */ assert(r); =20 if (ring_full(r)) { @@ -2867,6 +2747,87 @@ static int ring_push(RDMAContext *rdma, const RDMACm= d *cmd) return 0; } =20 +static int ring_push_write(RDMAContext *rdma, uint32_t bidx, uint32_t cidx, + uint64_t offset, uint64_t length) +{ + RDMACmd cmd =3D { + .type =3D RDMA_CMD_WRITE, + .block_index =3D bidx, + .chunk_index =3D cidx, + .offset =3D offset, + .length =3D length, + }; + return ring_push(rdma, &cmd); +} + +/* Producer: enqueue the accumulated contiguous span (if any) as one WRITE= . */ +static int producer_flush_pending(RDMAContext *rdma) +{ + int ret; + + if (rdma->pend_block < 0) { + return 0; + } + ret =3D ring_push_write(rdma, rdma->pend_block, rdma->pend_chunk, + rdma->pend_offset, rdma->pend_length); + rdma->pend_block =3D -1; + return ret; +} + +/* Producer: block until the consumer has completed barrier seq (or abort)= . */ +static int ring_wait_barrier(RDMASendRing *r, unsigned seq) +{ + if ((int)(qatomic_load_acquire(&r->barrier_done_seq) - seq) < 0) { + qemu_mutex_lock(&r->wait_lock); + r->producer_waiting =3D true; + smp_mb(); /* pair with ring_wake_producer() = */ + while ((int)(qatomic_load_acquire(&r->barrier_done_seq) - seq) < 0= && + !r->abort) { + qemu_cond_wait(&r->barrier_cv, &r->wait_lock); + } + r->producer_waiting =3D false; + qemu_mutex_unlock(&r->wait_lock); + } + /* A completed barrier is success even after a late abort. */ + return (int)(qatomic_load_acquire(&r->barrier_done_seq) - seq) >=3D 0 ? + 0 : -1; +} + +/* Producer: enqueue a barrier command and wait for its completion. */ +static int ring_barrier(RDMAContext *rdma, RDMACmdType type, uint64_t flag= s, + const struct iovec *iov, size_t niov) +{ + RDMASendRing *r =3D rdma->send_ring; + unsigned seq; + RDMACmd cmd; + + assert(r); /* consumer started at SETUP; see ring_push() */ + + /* Order any accumulated writes ahead of this barrier. */ + if (producer_flush_pending(rdma) < 0) { + return -1; + } + + seq =3D ++r->barrier_next_seq; + + if (seq =3D=3D 0) { /* seq 0 is the "no barrier" senti= nel */ + seq =3D ++r->barrier_next_seq; + } + cmd =3D (RDMACmd){ + .type =3D type, + .flags =3D flags, + .iov =3D iov, + .niov =3D niov, + .seq =3D seq, + }; + + if (ring_push(rdma, &cmd) < 0) { + return -1; + } + + return ring_wait_barrier(r, seq); +} + static bool ring_pop(RDMASendRing *r, RDMACmd *cmd) { if (ring_empty(r)) { @@ -2928,7 +2889,7 @@ static int rdma_consumer_dispatch(RDMAContext *rdma, = const RDMACmd *cmd, switch (cmd->type) { case RDMA_CMD_WRITE: ret =3D qemu_rdma_write_chunk(rdma, cmd->block_index, - cmd->chunk_index, &err); + cmd->offset, cmd->length, &err); break; case RDMA_CMD_FLUSH: ret =3D qemu_rdma_drain_cq(rdma, &err); @@ -2973,6 +2934,15 @@ static void *rdma_consumer_thread(void *opaque) ring_advance_tail(r); continue; } + if (rdma->nb_sent > 0) { + /* Ring empty but writes in flight: reap (bounded, error-aware= ). */ + if (qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, + NULL) < 0) { + ring_set_abort(rdma); + break; + } + continue; + } ring_idle_wait(r); } =20 @@ -3014,6 +2984,7 @@ static void rdma_consumer_stop(RDMAContext *rdma) } /* Ask the consumer to drain and exit; if it already aborted, just joi= n. */ if (!qatomic_read(&r->abort)) { + producer_flush_pending(rdma); /* order the last span before FINI= SH */ ring_push(rdma, &cmd); } qemu_thread_join(&rdma->consumer_thread); @@ -3298,7 +3269,8 @@ static int qemu_rdma_save_page(QEMUFile *f, ram_addr_= t block_offset, QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(qemu_file_get_ioc(f)); Error *err =3D NULL; RDMAContext *rdma; - int ret; + RDMALocalBlock *block; + uint64_t bidx, first_chunk, chunk_size, pos, end; =20 RCU_READ_LOCK_GUARD(); rdma =3D qatomic_rcu_read(&rioc->rdmaout); @@ -3313,60 +3285,46 @@ static int qemu_rdma_save_page(QEMUFile *f, ram_add= r_t block_offset, =20 qemu_fflush(f); =20 - /* - * Add this page to the current 'chunk'. If the chunk - * is full, or the page doesn't belong to the current chunk, - * an actual RDMA write will occur and a new chunk will be formed. - */ - ret =3D qemu_rdma_write(rdma, block_offset, offset, size, &err); - if (ret < 0) { - error_report_err(err); - goto err; - } + qemu_rdma_search_ram_block(rdma, block_offset, offset, size, + &bidx, &first_chunk); + block =3D &rdma->local_ram_blocks.block[bidx]; + chunk_size =3D migrate_rdma_chunk_size(); =20 /* - * Drain the Completion Queue if possible, but do not block, - * just poll. - * - * If nothing to poll, the end of the iteration will do this - * again to make sure we don't overflow the request queue. + * Split the dirty range into per-chunk segments (a WRITE never crosse= s a + * chunk boundary) and coalesce with the pending span: pages are scann= ed + * in increasing order, so contiguous dirty pages extend one span, and= we + * enqueue only the actually-dirty bytes instead of whole chunks. */ - while (1) { - uint64_t wr_id, wr_id_in; - ret =3D qemu_rdma_poll(rdma, rdma->recv_cq, &wr_id_in, NULL); - - if (ret < 0) { - error_report("rdma migration: polling error"); - goto err; - } - - wr_id =3D wr_id_in & RDMA_WRID_TYPE_MASK; - - if (wr_id =3D=3D RDMA_WRID_NONE) { - break; - } - } - - while (1) { - uint64_t wr_id, wr_id_in; - ret =3D qemu_rdma_poll(rdma, rdma->send_cq, &wr_id_in, NULL); - - if (ret < 0) { - error_report("rdma migration: polling error"); - goto err; - } - - wr_id =3D wr_id_in & RDMA_WRID_TYPE_MASK; - - if (wr_id =3D=3D RDMA_WRID_NONE) { - break; + pos =3D (block_offset + offset) - block->offset; + end =3D pos + size; + while (pos < end) { + uint64_t c =3D pos / chunk_size; + uint64_t seg_end =3D MIN(end, (c + 1) * chunk_size); + uint64_t seg_len =3D seg_end - pos; + + if (rdma->pend_block =3D=3D (int)bidx && rdma->pend_chunk =3D=3D (= uint32_t)c && + rdma->pend_offset + rdma->pend_length =3D=3D pos) { + rdma->pend_length +=3D seg_len; /* extend the current sp= an */ + } else { + if (producer_flush_pending(rdma) < 0) { + error_setg(&err, "rdma migration: consumer aborted"); + goto err; + } + rdma->pend_block =3D bidx; + rdma->pend_chunk =3D c; + rdma->pend_offset =3D pos; + rdma->pend_length =3D seg_len; } + pos =3D seg_end; } =20 return RAM_SAVE_CONTROL_DELAYED; =20 err: + error_report_err(err); rdma->errored =3D true; + ring_set_abort(rdma); return -1; } =20 @@ -3991,20 +3949,25 @@ int rdma_registration_stop(QEMUFile *f, uint64_t fl= ags) return -1; } =20 - qemu_fflush(f); - - if (flags =3D=3D RAM_CONTROL_FINISH) { - rdma_nbsent_stats_dump(rdma); - } + trace_rdma_registration_stop(flags); =20 - ret =3D qemu_rdma_write_flush(rdma, &err); + qemu_fflush(f); =20 - if (ret < 0) { - error_report_err(err); - goto err; - } + if (flags !=3D RAM_CONTROL_SETUP) { + /* + * ROUND / FINISH: the consumer sends REGISTER_FINISHED, ordered + * after all chunk writes queued so far. For FINISH it also drains + * outstanding writes and dumps the stats first. + */ + if (ring_barrier(rdma, RDMA_CMD_REG_STOP, flags, NULL, 0) < 0) { + error_report("rdma migration: consumer aborted at registration" + " stop"); + goto err; + } + return 0; =20 - if (flags =3D=3D RAM_CONTROL_SETUP) { + } else { + /* SETUP runs on the producer, before the consumer exists. */ RDMAControlHeader resp =3D {.type =3D RDMA_CONTROL_RAM_BLOCKS_RESU= LT }; RDMALocalBlocks *local =3D &rdma->local_ram_blocks; int reg_result_idx, nb_dest_blocks; @@ -4072,26 +4035,24 @@ int rdma_registration_stop(QEMUFile *f, uint64_t fl= ags) rdma->dest_blocks[i].remote_host_addr; local->block[i].remote_rkey =3D rdma->dest_blocks[i].remote_rk= ey; } - } - - trace_rdma_registration_stop(flags); =20 - head.type =3D RDMA_CONTROL_REGISTER_FINISHED; - ret =3D qemu_rdma_exchange_send(rdma, &head, NULL, NULL, NULL, NULL, &= err); + head.type =3D RDMA_CONTROL_REGISTER_FINISHED; + ret =3D qemu_rdma_exchange_send(rdma, &head, NULL, NULL, NULL, + NULL, &err); =20 - if (ret < 0) { - error_report_err(err); - goto err; - } + if (ret < 0) { + error_report_err(err); + goto err; + } =20 - if (flags =3D=3D RAM_CONTROL_SETUP) { - /* Setup complete: allocate the send ring (consumer added later). = */ + /* Setup complete: start the consumer; it now owns all QP work. */ rdma_consumer_start(rdma); } =20 return 0; err: rdma->errored =3D true; + ring_set_abort(rdma); return -1; } =20 --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352332; cv=none; d=zohomail.com; s=zohoarc; b=ajxVMAhU8e7iO3UQaoiX18SVB38T0m4c9snClgO9oONPWOR7rgZ20DqVoBWwlQPlWnHsR2XZvB9xdUzjZ4cfKcoZlkMF1wZG+OTMwgmqzU8z1KqpdPlUXRSsv2dAS9yp7gcYhxTNmAhhWJQ+4pKroM1Hl7qIi+ZAlPdhKLFq15k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352332; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9NETAQlR/XFiQLgadgf42XnVl/ITfeWvA2y/lQhR7CA=; b=Th4OBCixgQ2BKgUUrODixWCqd/cAruwDXGkdgbuwhcRdOqoZlxJ2n5k4kWDnbFdU3x7xwFS35gQlyd7yPLLflIRLuPm/4UKQr2LzoDbVGWfLilcErTHofkEho0DmZwErj3z1mfnafNyiRUfQpoovuLmSa5mU1n3Lbx67fYNApxk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352332231399.2066662715688; Fri, 25 Sep 2026 09:05:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8QG-000599-7K; Fri, 25 Sep 2026 12:05:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Di-0007um-7K for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:18 -0400 Received: from mail-wm2-x10.google.com ([2a00:1450:4864:31::10]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Df-0005zP-0F for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:17 -0400 Received: by mail-wm2-x10.google.com with SMTP id 5b1f17b1804b1-49ccf3ca626so6410155e9.0 for ; Fri, 25 Sep 2026 08:52:14 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351533; x=1790956333; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=9NETAQlR/XFiQLgadgf42XnVl/ITfeWvA2y/lQhR7CA=; b=MNikOVIuV59tip0GYqh+IC4htvZX/T27taREF2paIFt8Tl0Tq5YwSqUZMgbVsE2ooV 0XRlg8q9eEv8QzVDsrhwH9nrqoXanJQtgB6BJYDzsnl3nmCUOLp09Cfqli9SDbULt30L qa1nFuxmSz7KYEQ4ixvQeYIwqKQIlt1UVJI2aOw7ORuXHKd2QJWkksPr1ROZ1NT1riJ1 M6BzV42JrsjIMPgdKFfE5XSljxrH8TjxUw9Firb+MPy6lSZK+4pY9G52DsGR6Pi+aMdv BW7YRniBy+exv/olZOjWzbmp5IguN8RP+c7ZYSIyjIsLRNC6R3FcNVO3CnRGS+wWKzd2 jiIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351533; x=1790956333; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=9NETAQlR/XFiQLgadgf42XnVl/ITfeWvA2y/lQhR7CA=; b=jwoxSXrMXJrcJ3UZ2bBRZ/nX+FnhL9P9QqWQlnKHAuviY7AwKu9TOmTvEfQ6c79DI3 wLSxYeLHDn0Sg6386cd2XgTCHI2yQ1KUZkP+v9UGipZaQ3SLfDbQJDXQgHrU5UhOlQtm ktWkHp/qaeWB0n1s0hHS70JmbA02WqcHTMKtYoEK8IKE4/CYM3H0rEtzoaBJpZIgh26x 3JKi20rOshH1DF9Mm6Wt2SCHgjD6Rv+5ttYnXVpuVPY/D+SzwacE98FeqxmcigO/I/E8 XxwEmAPN/qmceJI9so+Pbl1OcpncrBoAxJWGF3DJYT3OCyfITxYFQBdn3Nm4Sbn1SAbp lOgQ== X-Gm-Message-State: AFuF++nqlh5lfd5LexQ0Nb7FHYhA0BB/daN/b1t7dEgM2yU3JAkLtgw5 k7h0Ea8mmtKw/pEJ83yl6Ca6BPPbVL9feDyws1BmDucS5leHsi+Wb4YYEsr8lWLMSvC6m+2RcVH E+P7/VI0VAA== X-Gm-Gg: AYBFou1zkjdT3cu2qcKsyPwpgipjryl2m0X91JfVjyTOdaYngzEcaOU5AvWScvp1AgE MgnutUE1aHFvKZNLFY1DMwRspNIWRiBRT+rflskTdZK26kGwH8beoSKvxiRsqL51VbvzAqTxB6Z +CxEtpXeRbpOPU+sbElRZgEHfaHLyp6yqt1Qzep2bGLJ/t00kR60psTUaXBqf8crxQlZ7HEZYtk vP7cE/Do8iAWzJUR6YU8ACnSKNfxwgSdOGYqLgj76/GCVUc9jKcb7khbfDYxa4Lhm29h47VvIgZ vU/xDsrtL63qYxllacKkxHOAXVmz6FtQK/t8Wux4Hlf+z30Vfx6w8yx3wZvbUWXs3pbRCsm/jfr 57uD1dgG6TOnZItqtL717XGsqiifW2tWfjcP5j9uxSjZ4XULiK/wwwyDgAD/1kmUOY5qUY+vxIr yxvHjgtzJY/5HEKgFLSiiiFNMWgVJBwxdhE7exJERcZLJpcAmTi6QDfvRSoNBn1oT31oTgHznLV ZgYGvCM1Ctm4CknZQi6f8cfTEPFVrxJe/BCby42/w== X-Received: by 2002:a05:600c:19d3:b0:49e:6777:da79 with SMTP id 5b1f17b1804b1-49fe66fb4a7mr101046365e9.26.1790351533636; Fri, 25 Sep 2026 08:52:13 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 08/13] migration/rdma: Add producer/consumer tuning instrumentation Date: Fri, 25 Sep 2026 17:51:43 +0200 Message-ID: <20260925155149.843870-9-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::10; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x10.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:02 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352334355158500 Content-Type: text/plain; charset="utf-8" Add producer full-wait, ring occupancy and consumer idle counters, the per-migration prodcons stats dump, and trace events for ring push, barrier, wake, dispatch and consumer lifecycle. Observability only; the transfer path is unchanged. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 75 +++++++++++++++++++++++++++++++++++++++++- migration/trace-events | 13 ++++++++ 2 files changed, 87 insertions(+), 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index a0ff73e313..628afaccaf 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -32,6 +32,7 @@ #include "qemu/thread.h" #include "qemu/sockets.h" #include "qemu/coroutine.h" +#include "qemu/timer.h" #include "system/memory.h" #include #include @@ -1213,7 +1214,9 @@ static int producer_flush_pending(RDMAContext *rdma); static int ring_barrier(RDMAContext *rdma, RDMACmdType type, uint64_t flag= s, const struct iovec *iov, size_t niov); static void ring_set_abort(RDMAContext *rdma); +static unsigned ring_occupancy(RDMAContext *rdma); static void rdma_nbsent_stats_dump(RDMAContext *rdma); +static void rdma_prodcons_stats_dump(RDMASendRing *r); =20 /* * Consult the connection manager to see a work request @@ -2616,6 +2619,7 @@ static int rdma_ram_round_notify(NotifierWithReturn *= n G_GNUC_UNUSED, } =20 /* Complete all chunks in the ring before the next round. */ + trace_rdma_ram_round_notify(ring_occupancy(rdma)); if (ring_barrier(rdma, RDMA_CMD_FLUSH, 0, NULL, 0) < 0) { error_setg(errp, "RDMA consumer aborted during round drain"); return -1; @@ -2684,6 +2688,14 @@ static bool ring_empty(RDMASendRing *r) return (qatomic_load_acquire(&r->head) - r->tail) =3D=3D 0; } =20 +/* Producer-side snapshot of queued-but-unprocessed commands in the ring. = */ +static unsigned ring_occupancy(RDMAContext *rdma) +{ + RDMASendRing *r =3D rdma->send_ring; + + return r->head - qatomic_read(&r->tail); +} + static void ring_set_abort(RDMAContext *rdma) { RDMASendRing *r =3D rdma->send_ring; @@ -2703,6 +2715,7 @@ static void ring_wake_producer(RDMASendRing *r) { smp_mb(); /* order the *_waiting load below vs the sleeper's store */ if (qatomic_read(&r->producer_waiting)) { + trace_rdma_ring_wake_producer(); qemu_mutex_lock(&r->wait_lock); qemu_cond_broadcast(&r->not_full); qemu_cond_broadcast(&r->barrier_cv); @@ -2715,20 +2728,38 @@ static void ring_wake_consumer(RDMASendRing *r) { smp_mb(); /* order the *_waiting load below vs the sleeper's store */ if (qatomic_read(&r->consumer_waiting)) { + trace_rdma_ring_wake_consumer(); qemu_mutex_lock(&r->wait_lock); qemu_cond_signal(&r->not_empty); qemu_mutex_unlock(&r->wait_lock); } } =20 +static const char *rdma_cmd_type_str(RDMACmdType type) +{ + switch (type) { + case RDMA_CMD_WRITE: return "WRITE"; + case RDMA_CMD_FLUSH: return "FLUSH"; + case RDMA_CMD_REG_STOP: return "REG_STOP"; + case RDMA_CMD_FILE: return "FILE"; + case RDMA_CMD_FINISH: return "FINISH"; + default: return "UNKNOWN"; + } +} + static int ring_push(RDMAContext *rdma, const RDMACmd *cmd) { RDMASendRing *r =3D rdma->send_ring; + unsigned occupancy; =20 /* SETUP starts the consumer before any WRITE/FLUSH/REG_STOP/FILE. */ assert(r); + trace_rdma_ring_push_begin(rdma_cmd_type_str(cmd->type), + cmd->chunk_index); =20 if (ring_full(r)) { + int64_t wait_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + qemu_mutex_lock(&r->wait_lock); r->producer_waiting =3D true; smp_mb(); /* pair with ring_wake_producer() = */ @@ -2737,12 +2768,23 @@ static int ring_push(RDMAContext *rdma, const RDMAC= md *cmd) } r->producer_waiting =3D false; qemu_mutex_unlock(&r->wait_lock); + + wait_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - wait_ns; + r->pushwait_total_ns +=3D wait_ns; + if ((uint64_t)wait_ns > r->pushwait_max_ns) { + r->pushwait_max_ns =3D wait_ns; + } + trace_rdma_ring_push_full_wait(cmd->type, wait_ns); } if (qatomic_read(&r->abort)) { return -1; } r->ring[r->head & RDMA_SEND_RING_MASK] =3D *cmd; qatomic_store_release(&r->head, r->head + 1); /* publish payload+slo= t */ + occupancy =3D r->head - qatomic_read(&r->tail); + if (occupancy > r->occupancy_max) { + r->occupancy_max =3D occupancy; + } ring_wake_consumer(r); return 0; } @@ -2798,8 +2840,10 @@ static int ring_barrier(RDMAContext *rdma, RDMACmdTy= pe type, uint64_t flags, const struct iovec *iov, size_t niov) { RDMASendRing *r =3D rdma->send_ring; + int64_t wait_ns; unsigned seq; RDMACmd cmd; + int ret; =20 assert(r); /* consumer started at SETUP; see ring_push() */ =20 @@ -2813,6 +2857,7 @@ static int ring_barrier(RDMAContext *rdma, RDMACmdTyp= e type, uint64_t flags, if (seq =3D=3D 0) { /* seq 0 is the "no barrier" senti= nel */ seq =3D ++r->barrier_next_seq; } + trace_rdma_ring_barrier_begin(type, seq); cmd =3D (RDMACmd){ .type =3D type, .flags =3D flags, @@ -2822,10 +2867,16 @@ static int ring_barrier(RDMAContext *rdma, RDMACmdT= ype type, uint64_t flags, }; =20 if (ring_push(rdma, &cmd) < 0) { + trace_rdma_ring_barrier_end(type, seq, 0, -1); return -1; } =20 - return ring_wait_barrier(r, seq); + /* time spent blocked waiting for the consumer to complete the barrier= */ + wait_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + ret =3D ring_wait_barrier(r, seq); + wait_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - wait_ns; + trace_rdma_ring_barrier_end(type, seq, wait_ns, ret); + return ret; } =20 static bool ring_pop(RDMASendRing *r, RDMACmd *cmd) @@ -2853,12 +2904,21 @@ static void ring_barrier_complete(RDMASendRing *r, = unsigned seq) /* Consumer: sleep until the ring is non-empty or aborted. */ static void ring_idle_wait(RDMASendRing *r) { + int64_t start_ns, idle_ns; + qemu_mutex_lock(&r->wait_lock); r->consumer_waiting =3D true; smp_mb(); /* pair with ring_wake_consumer() = */ + start_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + trace_rdma_consumer_idle_begin(); while (ring_empty(r) && !r->abort) { qemu_cond_wait(&r->not_empty, &r->wait_lock); } + idle_ns =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start_ns; + trace_rdma_consumer_idle_end(idle_ns); + if ((uint64_t)idle_ns > r->idle_max_ns) { + r->idle_max_ns =3D idle_ns; + } r->consumer_waiting =3D false; qemu_mutex_unlock(&r->wait_lock); } @@ -2876,6 +2936,7 @@ static int rdma_consumer_reg_stop(RDMAContext *rdma, = uint64_t flags, return -1; } rdma_nbsent_stats_dump(rdma); + rdma_prodcons_stats_dump(rdma->send_ring); } return qemu_rdma_exchange_send(rdma, &head, NULL, NULL, NULL, NULL, er= rp); } @@ -2886,6 +2947,9 @@ static int rdma_consumer_dispatch(RDMAContext *rdma, = const RDMACmd *cmd, Error *err =3D NULL; int ret =3D 0; =20 + trace_rdma_consumer_dispatch(cmd->type, cmd->block_index, + cmd->chunk_index, cmd->seq); + switch (cmd->type) { case RDMA_CMD_WRITE: ret =3D qemu_rdma_write_chunk(rdma, cmd->block_index, @@ -2919,6 +2983,7 @@ static void *rdma_consumer_thread(void *opaque) bool stop =3D false; =20 rcu_register_thread(); + trace_rdma_consumer_thread_start(); =20 while (!qatomic_read(&r->abort) && !stop) { RDMACmd cmd; @@ -2950,6 +3015,7 @@ static void *rdma_consumer_thread(void *opaque) /* make the failure visible to the producer's rdma_errored() check= s */ rdma->errored =3D true; } + trace_rdma_consumer_thread_stop(qatomic_read(&r->abort)); rcu_unregister_thread(); return NULL; } @@ -3363,6 +3429,13 @@ static void rdma_nbsent_stats_dump(RDMAContext *rdma) rdma->nbsent_full_hits, rdma->nbsent_zero_hits= ); } =20 +/* Producer/consumer ring stats for the whole migration (consumer at FINIS= H). */ +static void rdma_prodcons_stats_dump(RDMASendRing *r) +{ + trace_rdma_prodcons_stats(r->idle_max_ns, r->pushwait_max_ns, + r->pushwait_total_ns, r->occupancy_max); +} + int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { diff --git a/migration/trace-events b/migration/trace-events index f8ed6a7250..833351a03f 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -246,6 +246,19 @@ rdma_write_flush(int sent) "sent total: %d" rdma_write_one_post(uint64_t chunk, long addr, long remote, uint32_t len) = "Posting chunk: %" PRIu64 ", addr: 0x%lx remote: 0x%lx, bytes %" PRIu32 rdma_post_send_queue_full(uint64_t wr_id, int sent) "send queue full, wr_i= d=3D%" PRIu64 ", outstanding writes=3D%d" rdma_nbsent_stats(uint64_t max, uint64_t posts, uint64_t full_hits, uint64= _t zero_hits) "nb_sent max %" PRIu64 " posts %" PRIu64 " full-queue hits %"= PRIu64 " drains to zero %" PRIu64 +rdma_consumer_dispatch(int type, uint32_t bidx, uint32_t cidx, uint32_t se= q) "cmd type=3D%d block=3D%" PRIu32 " chunk=3D%" PRIu32 " seq=3D%" PRIu32 +rdma_ring_wake_producer(void) "waking blocked producer" +rdma_ring_wake_consumer(void) "waking idle consumer" +rdma_ring_push_begin(const char *type, uint32_t chunk_index) "push begin t= ype=3D%s chunk_index=3D%" PRIu32 +rdma_ring_push_full_wait(int type, int64_t wait_ns) "push full-wait type= =3D%d wait=3D%" PRId64 " ns" +rdma_ring_barrier_begin(int type, uint32_t seq) "barrier begin type=3D%d s= eq=3D%" PRIu32 +rdma_ring_barrier_end(int type, uint32_t seq, int64_t wait_ns, int ret) "b= arrier end type=3D%d seq=3D%" PRIu32 " wait=3D%" PRId64 " ns ret=3D%d" +rdma_consumer_thread_start(void) "consumer thread started" +rdma_consumer_thread_stop(bool aborted) "consumer thread stopped, aborted= =3D%d" +rdma_consumer_idle_begin(void) "consumer idle: waiting for commands" +rdma_consumer_idle_end(int64_t idle_ns) "consumer idle: woke after %" PRId= 64 " ns" +rdma_ram_round_notify(uint32_t occupancy) "ram round notify: ring occupanc= y=3D%" PRIu32 +rdma_prodcons_stats(uint64_t idle_max, uint64_t pushwait_max, uint64_t pus= hwait_total, uint64_t occupancy_max) "idle max %" PRIu64 " ns, push full-wa= it max %" PRIu64 " ns total %" PRIu64 " ns, ring occupancy max %" PRIu64 rdma_write_one_recvregres(int mykey, int theirkey, uint64_t chunk) "Receiv= ed registration result: my key: 0x%x their key 0x%x, chunk %" PRIu64 rdma_write_one_sendreg(uint64_t chunk, int len, int index, int64_t offset)= "Sending registration request chunk %" PRIu64 " for %d bytes, index: %d, o= ffset: %" PRId64 rdma_write_one_top(uint64_t chunks, uint64_t size) "Writing %" PRIu64 " ch= unks, (%" PRIu64 " MB)" --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352704; cv=none; d=zohomail.com; s=zohoarc; b=bEm4u+DyD/OBL0Mnv9tZJMdks+COwZagTmteDLtq6DwaZXij3D8Rif7hg8sBqmtFAm/aKnSVh8/R0+jm0NghreTfdZGjCAtTd+9TbniDYR6LLSKTL4GT1mZQqt1DM/KsW9dmM/jT54LvosAfvco6leEc29wroXR15Hu0mCrNctE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352704; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=PEbTPI/qyKxqRmY+I5qxg6KvizpRcLX6eTefYNr+pp4=; b=Ev9i2kM94Cri9gboX4DuWnh7KgiZfh4yfk0wUk9oQsx6p+P8rO27G6BoOb9BR86vzFfxB6CNTCS030e0u6G+xKCScf8WklpsuLyqTbH2QlZYovKpqj4E6DsVozH1knGqIY3OdbT9ajeXzgqNuoSAVa6/Zl8kK6AM68F8QPpovpI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352704129314.8571964503749; Fri, 25 Sep 2026 09:11:44 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8Vz-0002Pf-LW; Fri, 25 Sep 2026 12:11:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Dj-0007vG-6Q for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:19 -0400 Received: from mail-wm2-x11.google.com ([2a00:1450:4864:31::11]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Dh-00060B-6c for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:18 -0400 Received: by mail-wm2-x11.google.com with SMTP id 5b1f17b1804b1-49e6b885ef8so6548965e9.1 for ; Fri, 25 Sep 2026 08:52:16 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351536; x=1790956336; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=PEbTPI/qyKxqRmY+I5qxg6KvizpRcLX6eTefYNr+pp4=; b=QbRcjVxoULheVtHPe94enQ2c1U+2OFJpv5g+BytqSwL4vQuj9yHVsajhM4wDO0Jwp8 cExFBBjTrxYzh1QcPfVF4dLz6tZqCK+npWkqWqXLiKxtCrNWtg0SrukGiwMQ27YPWnNV jl/7HIFEB8oaI6HuHX/L0cWCsNx/ZZZRmRoAYZWQjpR5Ko1nciIfG/km9DVvsXsDnoPp ppj7zf0wPC4wrslwsYAZaX3odBfGcZ2YM/BVxJIlK/b4KaAVUPYLYj/8xjM4/niBKeKg oNzdkGLAXg1QoCvzBMBeGn07W7OJPzTitEOLK0clCPThWkH3OCife/vZw2KvRsTc8Lt0 yq3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351536; x=1790956336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=PEbTPI/qyKxqRmY+I5qxg6KvizpRcLX6eTefYNr+pp4=; b=WW3nyM6UKa0bfqmL4IRlHItr8UpqXpb/wL2CMYL1xAhaApw1agMdPGLsesiknt11Ni We1SZCAJiTsOwxVdg+ZM6V+d+MAAgMEJ3MSceb7LQ+xqbcGEXEdSkFlwEmYAMCF5ehXs v5+4dxAq6vXW+S309DhujJR1WFp59NZsu9IxiODGYj6V8zhNJEO/7xZOT0V6EE9ecwVN MXtzGB0RMOE0cLHRmxCsNIuLJqUxB0O1Jjm8I5hdV0hZfgFYZCPBwKNuF3Ld6qqFyqDS p0zeXIlfAnW/zwljvGYHQW4EhSFellP2+P21q54iSIx2bSM1Il0fiFjF2mfjC9nWAYvm +eFA== X-Gm-Message-State: AFuF++lhelBHuxYJNp/sa03zPAFwAAUwJlYEGMDXsj8Jv+g7Tgi/EOXS SRsFyBuLv2zN7FRxRpUTl3MwFchUyGb9vBneZjC/wVvmKybW3NnT40NOzHWg1Y3g7JWW6gyyd5+ TtuWe/r7LQQ== X-Gm-Gg: AYBFou3nI3rVu9ZPtPCKDROBjQKhJpdth/QYyJj4UoH0XyjHao48Ul2lTdmLGTTAOhf ntz0eyGIt1xxDdn0tmV5ZtF6/itxKBICnX+9Tac2N3iHm0hD3v+rAZEaJz7pxSTgmz+SpCVHRGs 5ibOkxDz+tGEMMUHZhaNcKI9DRkMQYUv50zCnC5kcWkvwc6ZZZnDVXVKyS9v7ztSOv5Bw5DwiIG D7fK/TjzwudcL0na/j2Scl+/GPUNXDttXm8QBk6jgslH11OBCZ1lICq8tukyvi8WEincPq2NOZt f/L0wI3SjMz4+eUaw/nu5vTZTi1wGiPQL9DzbB6Y1K8sukKhYB++v+gOQirBc1/FvWsoXD4X/ZO f/8Lyc/RK4dSMrk07bxoIqyhHIfNllbd4jlDUXZbX+T7cdNUSOO/DPmMZ4SjA9Vtr7A6lA5nwJg WQSg8FLLewFfzuah1q479gDfqLtAb5SCuH7G3WIfi8UtXWL2vipOGKekWbz6nns04Q+GEEjkYA1 IWuapAls5twAvSe0mFT1uryXK4VQD5X+uHTdaKBJg== X-Received: by 2002:a05:600c:6986:b0:49c:fa21:e73c with SMTP id 5b1f17b1804b1-49fe66f147bmr108555135e9.18.1790351535869; Fri, 25 Sep 2026 08:52:15 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 09/13] migration/rdma: drain the send ring in batches Date: Fri, 25 Sep 2026 17:51:44 +0200 Message-ID: <20260925155149.843870-10-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::11; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x11.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:02 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352704842158500 Content-Type: text/plain; charset="utf-8" The consumer snapshots ring occupancy, processes up to RDMA_CONSUMER_BATCH_MAX commands, and advances the tail once per batch instead of once per command. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 81 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 58 insertions(+), 23 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 628afaccaf..db1f1e3fc6 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -67,6 +67,9 @@ static inline uint64_t rdma_merge_max(void) #define RDMA_CONTROL_MAX_BUFFER (512 * 1024) #define RDMA_CONTROL_MAX_COMMANDS_PER_MESSAGE 4096 =20 +/* Max WRITE commands the consumer drains per pass. */ +#define RDMA_CONSUMER_BATCH_MAX 128 + #define RDMA_CONTROL_VERSION_CURRENT 1 /* * Capabilities for negotiation. @@ -2688,6 +2691,12 @@ static bool ring_empty(RDMASendRing *r) return (qatomic_load_acquire(&r->head) - r->tail) =3D=3D 0; } =20 +/* Consumer-side count of published, unconsumed commands. */ +static unsigned ring_avail(RDMASendRing *r) +{ + return qatomic_load_acquire(&r->head) - r->tail; +} + /* Producer-side snapshot of queued-but-unprocessed commands in the ring. = */ static unsigned ring_occupancy(RDMAContext *rdma) { @@ -2879,18 +2888,10 @@ static int ring_barrier(RDMAContext *rdma, RDMACmdT= ype type, uint64_t flags, return ret; } =20 -static bool ring_pop(RDMASendRing *r, RDMACmd *cmd) +/* Consumer: release n drained slots at once, waking a blocked producer on= ce. */ +static void ring_advance_tail_by(RDMASendRing *r, unsigned n) { - if (ring_empty(r)) { - return false; - } - *cmd =3D r->ring[r->tail & RDMA_SEND_RING_MASK]; /* after load_acqui= re */ - return true; -} - -static void ring_advance_tail(RDMASendRing *r) -{ - qatomic_store_release(&r->tail, r->tail + 1); + qatomic_store_release(&r->tail, r->tail + n); ring_wake_producer(r); } =20 @@ -2986,29 +2987,63 @@ static void *rdma_consumer_thread(void *opaque) trace_rdma_consumer_thread_start(); =20 while (!qatomic_read(&r->abort) && !stop) { - RDMACmd cmd; + unsigned avail =3D ring_avail(r); + unsigned batch, processed =3D 0; + + if (avail =3D=3D 0) { + /* + * Ring empty: reap ready completions without blocking, then + * sleep until the next push so a new command wakes us at once. + * Round draining is handled by RDMA_CMD_FLUSH. + */ + while (rdma->nb_sent > 0) { + uint64_t wr_id; + + if (qemu_rdma_poll(rdma, rdma->send_cq, &wr_id, NULL) < 0)= { + ring_set_abort(rdma); + break; + } + if (wr_id =3D=3D RDMA_WRID_NONE) { + break; /* nothing more ready right now */ + } + } + if (qatomic_read(&r->abort)) { + break; + } + ring_idle_wait(r); + continue; + } + + /* + * Drain up to RDMA_CONSUMER_BATCH_MAX commands, or fewer if that = is + * all that is queued, then publish tail once for the whole batch. + */ + batch =3D avail < RDMA_CONSUMER_BATCH_MAX ? avail + : RDMA_CONSUMER_BATCH_MAX; + + while (processed < batch) { + unsigned idx =3D r->tail + processed; + RDMACmd cmd =3D r->ring[idx & RDMA_SEND_RING_MASK]; =20 - if (ring_pop(r, &cmd)) { if (rdma_consumer_dispatch(rdma, &cmd, &stop) < 0) { ring_set_abort(rdma); break; } + processed++; if (cmd.seq) { ring_barrier_complete(r, cmd.seq); } - ring_advance_tail(r); - continue; - } - if (rdma->nb_sent > 0) { - /* Ring empty but writes in flight: reap (bounded, error-aware= ). */ - if (qemu_rdma_block_for_wrid(rdma, RDMA_WRID_RDMA_WRITE, - NULL) < 0) { - ring_set_abort(rdma); + if (stop) { break; } - continue; } - ring_idle_wait(r); + + if (processed) { + ring_advance_tail_by(r, processed); + } + if (qatomic_read(&r->abort)) { + break; + } } =20 if (qatomic_read(&r->abort)) { --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352446; cv=none; d=zohomail.com; s=zohoarc; b=AahLlA4ZtegIyMkIh/wVOI5VhCwixzaO5dC1/5nULT7/xtwaYeb0V8Sc70fdIXE+bWw88rcQz32Bz8IOlCwKKRRxKxnY1AVkHWA6aSrKqniR/YfkwprbuaQOz71CT7ihUYHTfNs3pISlOfBrFeidgGwl1BkQsU45jrgtP/oKJu8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352446; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=NTTOBszhae6b65tYdYeG+YHSC5v2PqdRKV+Vycg+4ck=; b=XpBmDIKbbPtxcqNXlev8E5OG0pt8NppteOZQcY+TNL9CSBchFu+RrHo7wP/naHNCMraR4eMNQwE2IZz9B4cAggQZIusCpwIOIqIlBD7wuAUo5eltHoPGOL4vYsGj4FCCQpY+lhYnGvL6OieVy9yJ51/xcYD9P233Ouw133OOG6I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17903524465581015.0368209509797; Fri, 25 Sep 2026 09:07:26 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8Rl-0006rV-Rj; Fri, 25 Sep 2026 12:06:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Dk-0007vw-Qq for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:21 -0400 Received: from mail-wm2-x11.google.com ([2a00:1450:4864:31::11]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Dj-00060a-6c for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:20 -0400 Received: by mail-wm2-x11.google.com with SMTP id 5b1f17b1804b1-49b912d37b5so6542755e9.2 for ; Fri, 25 Sep 2026 08:52:18 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351537; x=1790956337; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NTTOBszhae6b65tYdYeG+YHSC5v2PqdRKV+Vycg+4ck=; b=M0Ko1r3XnVO6cGVVkrTP9IO1rF7t2ruT04vWIneWpWfvPVsoeJsAyae/JSXXjowm4j qd21Ki4XKqFb7e1VRJ5k9UhMtY5sWhhiloCqO440nNPqnDX+KSs0VeykEa39qmf/NFJI s448aDs4WxrBWLEVcK9FES2jLVc6xIWwbIFcUx5ZEC7mHqPB+kf4gWJnHH/wLQyvFY8H 0aR52MflHDLRjFZxDV3iJeYMIJvguj3W86vakEIo6v46H2Zy/lyijJtr/Y6vV4yfH4Gg mhVfTVvyZmtIHjf2i6kVhIe3dikwxjR3ieQSkBnfz1EPPRHhFgJPLJ70Cne//OD9zHhj 6XKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351537; x=1790956337; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NTTOBszhae6b65tYdYeG+YHSC5v2PqdRKV+Vycg+4ck=; b=e0/sBuGGUr6o/WzqE80gPH2fijVHXZcVM4n0mebbxH1vF1Yu7K4s8bgkXR3I47jdrc yyIlzWOTrz3HzAwBNUlbBNIfn/lNEOpMWIJwSJfQZD0WSV4mCHTjHLSiwbKGIMZnCceE ndyeUK8tbLq/NxXU5/3V9YYyZWA3CdGEoqhwJLX8QKvny66vFHWcomtoEoA53i4VBlR/ dXHSPxs6TJ5LRl+IM+bI/L9vNo1gPiYPPznAt38c+kVyCvm2VcLV2NW6rzWOMIT/7TM3 WbNE3KUGOEtsuhxvbNZlQDGaiwLRS0E02UwMhRPUwn5oGHA3wN3l26iAN4p4O1dL6dnQ ZYYA== X-Gm-Message-State: AFuF++nvB4GzxsDV7wmmZa+a+jJ+2Pd7tD3G+XXKll3G8/GQkQHgW//A t9MNEBW+uq0sZIICu0hxR51AkIlvDRW/dCF7Mv5Ik39Txrcy7YlYLYkV1Sayk0wAyh6ES35B8KT VfRuce2K19w== X-Gm-Gg: AYBFou1bMw/GYVW6jzW876CHb6EekJPZDTbjl1T0sVXZeaJeukQs2dT06Vg31KPIz1w g6pA3HGPuiEqm9QGVeOmpZ+UcLedjhB1QOnpF7zJrrThF3IRrGEk+nQThrJoodpNQVlB9GZONl8 NQekXBNG8SppYYOhmB07gT73p/nv3kxu6qwkDhsm6I5tdeQfBed/20wVmElHfLtXtDP3oGvMlQd ur/PpkYPfB17gRnKFdXeCwNo4+QmR0a6yArQQMDZqW4hjrnVqxmttKDdMcZThWorqhMGnRUNp7+ GJr8QgUg+FwsFctLnaxBdbITUiT6f/CMWevA1p5k5E3VOcpqWcFBXHagafxdZTB9/g94c0g8FWQ xDPopVyzeG6ehGCL6N2cByuJG+hzfUUN54t+PPZudXS7SrDaWSl3o+LNYCHNC6kNuLL4dhFgpjk qsKEV4M5nOfnCz6LjJRrJtfWYvJvruE4cGVNjTKOY8fYjXW03VvaDJ6UeswxoxclWUijex91yfo l8dYmI06V9q8LMwuSTXGH91od/VOKAdCi53uYlTHoE= X-Received: by 2002:a05:600c:1c25:b0:49f:ce78:3561 with SMTP id 5b1f17b1804b1-49fe66f19demr121022405e9.18.1790351537568; Fri, 25 Sep 2026 08:52:17 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 10/13] migration/rdma: size reg_resp.len by the register request count Date: Fri, 25 Sep 2026 17:51:45 +0200 Message-ID: <20260925155149.843870-11-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::11; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x11.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:02 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352451208158500 Content-Type: text/plain; charset="utf-8" The destination assumed repeat=3D1; set reg_resp.len to one RDMARegisterResult per requested chunk so a batched REGISTER_REQUEST is answered in full. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index db1f1e3fc6..bffc7d33ef 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3870,6 +3870,8 @@ int rdma_registration_handle(QEMUFile *f) trace_rdma_registration_handle_register(head.repeat); =20 reg_resp.repeat =3D head.repeat; + /* one result per requested chunk (source may batch registers)= */ + reg_resp.len =3D head.repeat * sizeof(RDMARegisterResult); registers =3D (RDMARegister *) rdma->wr_data[idx].control_curr; =20 /* Making sure the register buffers to read are valid */ --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352690; cv=none; d=zohomail.com; s=zohoarc; b=gPiZV/sYu80k8CGvwOoGabX0hV0rXFETnSXrHi0n2X8uARjQ4QZq/0f/UdK91ZdV/X2BrAtuxWM+m8/xkxA+fRibWyXNDOmtzrnhX9JJrpCsgK9iqecMqDpzcgGZu5jRm5EkWa0+6poFRpBRSWMDc2/WCIOkdiK0Uyj7lMqmnKY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352690; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=sDHXhYLoaGfkzyQh3NRvDgSSgSWl+Zi7fDwtmLD0Jl0=; b=XSjvDXuTxXhIdm7WBiEB1TqbNGhDCRD+isCbDbPYL0Zvl0yxJjzflMifqD4cB7jJpnVruNNsNhT8VH980s3a9UCKeBRf3ekPGijY+5QZ+ewKJFMau3YUyOSHDiCd3IDP5ldGvwX8d55AsJDY7zT/4lO8fNX2i5qnZMS9lz9QuyQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352690067337.6057880148194; Fri, 25 Sep 2026 09:11:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8TA-0000Zn-Lq; Fri, 25 Sep 2026 12:08:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Do-0007xH-19 for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:24 -0400 Received: from mail-wm2-x08.google.com ([2a00:1450:4864:31::8]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Dl-000612-65 for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:23 -0400 Received: by mail-wm2-x08.google.com with SMTP id 5b1f17b1804b1-49e8361492fso4359585e9.0 for ; Fri, 25 Sep 2026 08:52:20 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351540; x=1790956340; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=sDHXhYLoaGfkzyQh3NRvDgSSgSWl+Zi7fDwtmLD0Jl0=; b=UCkjeX9ATb4YJp3NqITu9ie4+KdOPhVvCV0gQr6pxehTy7ug+9rCSSbs6TfN80gRjE 5RQspJiulXBLQSkfEwkuFKRWAw2ySGngxQl8HdV9nVStoxzR7q/Z8RFMIQKuATgi+O4w kxi4m8uMnk33rO9/UxwpWXsdAp9l/5usZ1H2RDXRFSmw4iL5yov+8vaTzzErYox5qvU/ kzaJwsILRuoGliG8Hqh7UTeW3OXm4g4iZNUNj6X0rMk1lF8cBXMs1NRtHQZsyEu3iYyi CXNqODXf9kwhni38qpwZWwP5BWRq81xQzFEQp99UYCOS2KW+4uBee2op92Iv2be5QMtN VDkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351540; x=1790956340; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=sDHXhYLoaGfkzyQh3NRvDgSSgSWl+Zi7fDwtmLD0Jl0=; b=14j+J/v79KbaJV+7wQf/do2yK6Ymjkew2+rsNO9rwglX/UYFIKMLE6A3YU2Txb1pL9 OxkfmmsDDUPMwokNoth88lIMrDuvs5ujWmwTmtrBLcloN7R25wW85FMA9JKrHPw29bhN WJLlERAXK2jldaRu/kEz3Wa/briwqIM7nqMxnwKmc5/Qmqm3wI/zIPFozQNJQc76YTwL nZiSrwQPul5Q8rq8aSPY7PwDUe8xopXZSDLlLt6VpHnWePdAPvJieFGQWK3ZdBJ8Aj+4 Q2IHnHs8XnONeWHu8cJf1pKYwGJs7NMPeXk1pJQK4s/95ojnbD8vbY14Db/r1PQ1Yi25 HlRw== X-Gm-Message-State: AFuF++nKPUM4lbwDgV0sXlMb51A1SkDfjluiVq3vluXET4JvsBE2OQ+r 0fDH5XaN/eoXncFMKgfb0Axou3sia6PYwmjOTCqeqMQED39f6rDLWVZm0TukGLiO2JbYDoK/sJM fqK09RKoRXlzJ X-Gm-Gg: AYBFou06+b+IL1eKACn+mHkfNyjW+IVuWkbdwubvPo7I4eQLqrTdOG+Vz6RnXK9CYQQ 67O4GFpMNPfNJnp8/5xpJYZPfm4zwTIkNsToiVTzm9p44Dfzbgsd8Es+qWnpcFBilnOSiFz0KhM lHxSt65tmre1RBQAZL95paPyXKZ5yPlqTxd8aVo3ND8oYBDpDjC3q55ddURLccpZ2kcwTB9cAdu zzM3S8LeVIS6fb9P6Hv3HvK003lmjkXJ5uAP00cjtOj/L0gvYc/pcOAeg0kvbe/aZ5RfUE8/KBw y6GMzVaevSS0MOh8kWmkZesyGwLEFmTGdMrZASx7vCpEQJNYQt6KKVMkredhFcc7ZtFaZdw4X/b j+BZKXBOX9FVeBKCGAlQGLJplDwBe60uwiWcuQFzKG+bYXBmiXagqj4HhRY5HRcqPqgD9grbu+J E2+hFMbkfglmR9B0E9aR+pO98bfTml5bOdYtpftImbv7mxJs+IRWO581y8103x2ppJAhpFne+P9 2g+DrruRUp+WmmoHVfOAjQAgDX2fHopEr1pRIvUWg== X-Received: by 2002:a05:600c:3e06:b0:49f:f099:6f9b with SMTP id 5b1f17b1804b1-49ff0997088mr46675435e9.13.1790351539513; Fri, 25 Sep 2026 08:52:19 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 11/13] migration/rdma: batch chunk registration in one round-trip Date: Fri, 25 Sep 2026 17:51:46 +0200 Message-ID: <20260925155149.843870-12-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:31::8; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wm2-x08.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:02 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352690787158500 Content-Type: text/plain; charset="utf-8" Split qemu_rdma_write_one() into three helpers: write_one posts one dirty span of an already-registered chunk (both pin modes), write_compress handles an all-zero span, and write_register folds several never-seen chunks into one REGISTER_REQUEST. The consumer scans a run of WRITE commands, posting ready and zero spans inline and registering the never-seen chunks in a single round-trip before posting them. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 372 ++++++++++++++++++++++++++++------------------- 1 file changed, 223 insertions(+), 149 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index bffc7d33ef..aa1a81619f 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1208,6 +1208,7 @@ static uint64_t qemu_rdma_make_wrid(uint64_t wr_id, u= int64_t index, static void rdma_nbsent_inc(RDMAContext *rdma, uint64_t nb_sent); static void rdma_nbsent_full(RDMAContext *rdma); static void rdma_nbsent_zero(RDMAContext *rdma); +static void rdma_reg_batch_record(RDMASendRing *r, unsigned count); =20 static void rdma_consumer_start(RDMAContext *rdma); static void rdma_consumer_stop(RDMAContext *rdma); @@ -1806,149 +1807,41 @@ static int qemu_rdma_exchange_recv(RDMAContext *rd= ma, RDMAControlHeader *head, return 0; } =20 -/* - * Write an actual chunk of memory using RDMA. - * - * If we're using dynamic registration on the dest-side, we have to - * send a registration command first. - */ -static int qemu_rdma_write_one(RDMAContext *rdma, - int current_index, uint64_t current_addr, - uint64_t length, Error **errp) +/* Post one dirty span of an already-registered chunk (both pin modes). */ +static int qemu_rdma_write_one(RDMAContext *rdma, int current_index, + uint64_t current_addr, uint64_t length, + Error **errp) { struct ibv_sge sge; struct ibv_send_wr send_wr =3D { 0 }; - int reg_result_idx, ret; - uint64_t chunk, chunks; - uint64_t chunk_size =3D migrate_rdma_chunk_size(); + RDMALocalBlock *block =3D &rdma->local_ram_blocks.block[current_index]; + uint64_t chunk; uint8_t *chunk_start, *chunk_end; - RDMALocalBlock *block =3D &(rdma->local_ram_blocks.block[current_index= ]); - RDMARegister reg; - RDMARegisterResult *reg_result; - RDMAControlHeader resp =3D { .type =3D RDMA_CONTROL_REGISTER_RESULT }; - RDMAControlHeader head =3D { .len =3D sizeof(RDMARegister), - .type =3D RDMA_CONTROL_REGISTER_REQUEST, - .repeat =3D 1, - }; =20 sge.addr =3D (uintptr_t)(block->local_host_addr + - (current_addr - block->offset)); + (current_addr - block->offset)); sge.length =3D length; =20 - chunk =3D ram_chunk_index(block->local_host_addr, - (uint8_t *)(uintptr_t)sge.addr); + chunk =3D ram_chunk_index(block->local_host_addr, (uint8_t *)sge.addr); chunk_start =3D ram_chunk_start(block, chunk); - chunks =3D length / chunk_size; - - if (chunks && ((length % chunk_size) =3D=3D 0)) { - chunks--; - } - - trace_rdma_write_one_top(chunks + 1, - (chunks + 1) * chunk_size / 1024 / 1024); + chunk_end =3D ram_chunk_end(block, chunk); =20 - chunk_end =3D ram_chunk_end(block, chunk + chunks); - - if (!rdma->pin_all) { + if (rdma->pin_all) { + send_wr.wr.rdma.rkey =3D block->remote_rkey; + } else { if (!block->remote_keys[chunk]) { - /* - * This chunk has not yet been registered, so first check to s= ee - * if the entire chunk is zero. If so, tell the other size to - * memset() + madvise() the entire chunk without RDMA. - */ - - if (buffer_is_zero((void *)(uintptr_t)sge.addr, length)) { - RDMACompress comp =3D { - .offset =3D current_addr, - .value =3D 0, - .block_idx =3D current_index, - .length =3D length, - }; - - head.len =3D sizeof(comp); - head.type =3D RDMA_CONTROL_COMPRESS; - - trace_rdma_write_one_zero(chunk, sge.length, - current_index, current_addr= ); - - compress_to_network(rdma, &comp); - ret =3D qemu_rdma_exchange_send(rdma, &head, - (uint8_t *) &comp, NULL, NULL, NULL, errp); - - if (ret < 0) { - return -1; - } - - /* - * TODO: Here we are sending something, but we are not - * accounting for anything transferred. The following is = wrong: - * - * stat64_add(&mig_stats.rdma_bytes, sge.length); - * - * because we are using some kind of compression. I - * would think that head.len would be the more similar - * thing to a correct value. - */ - qatomic_add(&mig_stats.zero_pages, - sge.length / qemu_target_page_size()); - return 1; - } - - /* - * Otherwise, tell other side to register. - */ - reg.current_index =3D current_index; - reg.current_addr =3D current_addr; - reg.chunks =3D chunks; - - trace_rdma_write_one_sendreg(chunk, sge.length, current_index, - current_addr); - - register_to_network(rdma, ®); - ret =3D qemu_rdma_exchange_send(rdma, &head, (uint8_t *) ®, - &resp, ®_result_idx, NULL, errp); - if (ret < 0) { - return -1; - } - - /* try to overlap this single registration with the one we sen= t. */ - if (qemu_rdma_register_and_get_keys(rdma, block, sge.addr, - &sge.lkey, NULL, chunk, - chunk_start, chunk_end)) { - error_setg(errp, "cannot get lkey"); - return -1; - } - - reg_result =3D (RDMARegisterResult *) - rdma->wr_data[reg_result_idx].control_curr; - - network_to_result(reg_result); - - trace_rdma_write_one_recvregres(block->remote_keys[chunk], - reg_result->rkey, chunk); - - block->remote_keys[chunk] =3D reg_result->rkey; - block->remote_host_addr =3D reg_result->host_addr; - } else { - /* already registered before */ - if (qemu_rdma_register_and_get_keys(rdma, block, sge.addr, - &sge.lkey, NULL, chunk, - chunk_start, chunk_end)) { - error_setg(errp, "cannot get lkey!"); - return -1; - } + error_setg(errp, "rdma: write_one on unregistered chunk %" PRI= u64, + chunk); + return -1; } - send_wr.wr.rdma.rkey =3D block->remote_keys[chunk]; - } else { - send_wr.wr.rdma.rkey =3D block->remote_rkey; + } =20 - if (qemu_rdma_register_and_get_keys(rdma, block, sge.addr, - &sge.lkey, NULL, chun= k, - chunk_start, chunk_en= d)) { - error_setg(errp, "cannot get lkey!"); - return -1; - } + /* Local MR (idempotent): pins our own memory so the NIC can read it. = */ + if (qemu_rdma_register_and_get_keys(rdma, block, sge.addr, &sge.lkey, + NULL, chunk, chunk_start, chunk_en= d)) { + error_setg(errp, "cannot get lkey"); + return -1; } =20 /* @@ -1956,18 +1849,17 @@ static int qemu_rdma_write_one(RDMAContext *rdma, * We will use this information at the time of completion * to identify the completed write in trace output. */ - send_wr.wr_id =3D qemu_rdma_make_wrid(RDMA_WRID_RDMA_WRITE, - current_index, chunk); - + send_wr.wr_id =3D qemu_rdma_make_wrid(RDMA_WRID_RDMA_WRITE, current_in= dex, + chunk); send_wr.opcode =3D IBV_WR_RDMA_WRITE; send_wr.send_flags =3D IBV_SEND_SIGNALED; send_wr.sg_list =3D &sge; send_wr.num_sge =3D 1; send_wr.wr.rdma.remote_addr =3D block->remote_host_addr + - (current_addr - block->offset); + (current_addr - block->offset); =20 trace_rdma_write_one_post(chunk, sge.addr, send_wr.wr.rdma.remote_addr, - sge.length); + sge.length); =20 if (!qemu_rdma_post_send(rdma, &send_wr, errp)) { return -1; @@ -1986,27 +1878,86 @@ static int qemu_rdma_write_one(RDMAContext *rdma, qatomic_add(&mig_stats.rdma_bytes, sge.length); ram_transferred_add(sge.length); rdma->total_writes++; + rdma->nb_sent++; + trace_rdma_write_flush(rdma->nb_sent); + rdma_nbsent_inc(rdma, rdma->nb_sent); + return 0; +} =20 +/* Zero an all-zero span on the destination without RDMA (pin_all=3Doff). = */ +static int qemu_rdma_write_compress(RDMAContext *rdma, int current_index, + uint64_t current_addr, uint64_t length, + Error **errp) +{ + RDMALocalBlock *block =3D &rdma->local_ram_blocks.block[current_index]; + uint64_t chunk =3D (current_addr - block->offset) / migrate_rdma_chunk= _size(); + RDMACompress comp =3D { + .offset =3D current_addr, + .value =3D 0, + .block_idx =3D current_index, + .length =3D length, + }; + RDMAControlHeader head =3D { .len =3D sizeof(comp), + .type =3D RDMA_CONTROL_COMPRESS, + .repeat =3D 1 }; + + trace_rdma_write_one_zero(chunk, length, current_index, current_addr); + + compress_to_network(rdma, &comp); + if (qemu_rdma_exchange_send(rdma, &head, (uint8_t *)&comp, + NULL, NULL, NULL, errp) < 0) { + return -1; + } + qatomic_add(&mig_stats.zero_pages, length / qemu_target_page_size()); return 0; } =20 -/* Consumer: write one dirty span (offset/length are block-relative bytes)= */ -static int qemu_rdma_write_chunk(RDMAContext *rdma, uint32_t block_index, - uint64_t offset, uint64_t length, Error *= *errp) +/* + * Register a batch of never-seen non-zero chunks in one REGISTER_REQUEST + * (pin_all=3Doff); regs[] carry current_index/current_addr in host order.= One + * control round-trip, then cache each chunk's remote rkey/host_addr so the + * following write_one() calls can post. + */ +static int qemu_rdma_write_register(RDMAContext *rdma, RDMARegister *regs, + unsigned count, Error **errp) { - RDMALocalBlock *block =3D &rdma->local_ram_blocks.block[block_index]; - int ret; + RDMARegister net[RDMA_CONSUMER_BATCH_MAX]; + RDMARegisterResult *results; + RDMAControlHeader resp =3D { .type =3D RDMA_CONTROL_REGISTER_RESULT }; + RDMAControlHeader head =3D { .type =3D RDMA_CONTROL_REGISTER_REQUEST }; + int reg_result_idx; + unsigned i; =20 - /* write_one wants current_addr in ram_addr_t space (block->offset bas= ed) */ - ret =3D qemu_rdma_write_one(rdma, block_index, block->offset + offset, - length, errp); - if (ret < 0) { + assert(count >=3D 1 && count <=3D RDMA_CONSUMER_BATCH_MAX); + + /* network-order copy to send; regs[] stays host-order for caching bel= ow */ + for (i =3D 0; i < count; i++) { + net[i] =3D regs[i]; + net[i].chunks =3D 0; /* exactly one chunk per entry */ + register_to_network(rdma, &net[i]); + } + head.len =3D count * sizeof(RDMARegister); + head.repeat =3D count; + + rdma_reg_batch_record(rdma->send_ring, count); + + if (qemu_rdma_exchange_send(rdma, &head, (uint8_t *)net, + &resp, ®_result_idx, NULL, errp) < 0) { return -1; } - if (ret =3D=3D 0) { - rdma->nb_sent++; - trace_rdma_write_flush(rdma->nb_sent); - rdma_nbsent_inc(rdma, rdma->nb_sent); + + results =3D (RDMARegisterResult *)rdma->wr_data[reg_result_idx].contro= l_curr; + for (i =3D 0; i < count; i++) { + RDMALocalBlock *block =3D + &rdma->local_ram_blocks.block[regs[i].current_index]; + uint64_t chunk =3D + (regs[i].current_addr - block->offset) / migrate_rdma_chunk_si= ze(); + + network_to_result(&results[i]); + trace_rdma_write_one_recvregres(block->remote_keys[chunk], + results[i].rkey, chunk); + block->remote_keys[chunk] =3D results[i].rkey; + block->remote_host_addr =3D results[i].host_addr; } return 0; } @@ -2679,6 +2630,7 @@ struct RDMASendRing { uint64_t pushwait_total_ns; /* total producer wait on a full ring */ uint64_t occupancy_max; /* peak queued commands */ uint64_t idle_max_ns; /* longest consumer wait on an empty ring= */ + uint64_t reg_batch_max; /* largest batched REGISTER_REQUEST */ }; =20 static bool ring_full(RDMASendRing *r) @@ -2942,6 +2894,105 @@ static int rdma_consumer_reg_stop(RDMAContext *rdma= , uint64_t flags, return qemu_rdma_exchange_send(rdma, &head, NULL, NULL, NULL, NULL, er= rp); } =20 +/* + * Dirty-span parameters for a ring WRITE command: block, block-relative + * offset, migration address and length. The span is the actually-dirtied + * byte range (within one chunk), not the whole chunk. + */ +static void rdma_get_chunk_args(RDMAContext *rdma, const RDMACmd *cmd, + RDMALocalBlock **block, uint64_t *off, + uint64_t *addr, uint64_t *len) +{ + RDMALocalBlock *b =3D &rdma->local_ram_blocks.block[cmd->block_index]; + + *block =3D b; + *off =3D cmd->offset; + *addr =3D b->offset + cmd->offset; + *len =3D cmd->length; +} + +/* + * Consumer: process the maximal run of consecutive RDMA_CMD_WRITE command= s at + * ring position 'pos' (at most 'max'), stopping at the first barrier. Ea= ch + * command carries one dirty span. Post ready (already registered / pin_a= ll) + * and zero spans as they are met; defer spans whose chunk needs registeri= ng, + * register those chunks in one batched REGISTER_REQUEST, then post the sp= ans. + * Returns the WRITE commands consumed, or -1. + */ +static int qemu_rdma_write_run(RDMAContext *rdma, RDMASendRing *r, + unsigned pos, unsigned max, Error **errp) +{ + unsigned reg_ring_idx[RDMA_CONSUMER_BATCH_MAX]; /* deferred span write= s */ + RDMARegister regs[RDMA_CONSUMER_BATCH_MAX]; /* distinct chunks to = reg */ + unsigned n =3D 0, regn =3D 0, nreg =3D 0, i; + int last_bidx =3D -1; /* last chunk queued into regs[] */ + int64_t last_cidx =3D -1; + + /* Phase 1: scan the run; post ready/zero now, defer never-registered.= */ + while (n < max) { + RDMACmd *cmd =3D &r->ring[(pos + n) & RDMA_SEND_RING_MASK]; + RDMALocalBlock *block; + uint64_t off, addr, len; + + if (cmd->type !=3D RDMA_CMD_WRITE) { + break; + } + rdma_get_chunk_args(rdma, cmd, &block, &off, &addr, &len); + + if (rdma->pin_all || block->remote_keys[cmd->chunk_index]) { + if (qemu_rdma_write_one(rdma, cmd->block_index, addr, len, + errp) < 0) { + return -1; + } + } else if (buffer_is_zero((void *)(uintptr_t) + (block->local_host_addr + off), len)) { + if (qemu_rdma_write_compress(rdma, cmd->block_index, addr, len, + errp) < 0) { + return -1; + } + } else { + /* Defer: record every span for a post-registration write. */ + reg_ring_idx[regn++] =3D pos + n; + /* + * Registration is per whole chunk. Chunk indices arrive + * non-decreasing within a run, so a repeat chunk is the last = one + * queued; add each chunk to the REGISTER batch only once. + */ + if ((int)cmd->block_index !=3D last_bidx || + (int64_t)cmd->chunk_index !=3D last_cidx) { + regs[nreg].current_index =3D cmd->block_index; + regs[nreg].current_addr =3D addr; + nreg++; + last_bidx =3D cmd->block_index; + last_cidx =3D cmd->chunk_index; + } + } + n++; + } + + if (regn =3D=3D 0) { + return (int)n; + } + + /* Phase 2: one REGISTER_REQUEST for the deduped deferred chunks. */ + if (qemu_rdma_write_register(rdma, regs, nreg, errp) < 0) { + return -1; + } + + /* Phase 3: post every deferred span (chunks are now registered). */ + for (i =3D 0; i < regn; i++) { + RDMACmd *cmd =3D &r->ring[reg_ring_idx[i] & RDMA_SEND_RING_MASK]; + RDMALocalBlock *block; + uint64_t off, addr, len; + + rdma_get_chunk_args(rdma, cmd, &block, &off, &addr, &len); + if (qemu_rdma_write_one(rdma, cmd->block_index, addr, len, errp) <= 0) { + return -1; + } + } + return (int)n; +} + static int rdma_consumer_dispatch(RDMAContext *rdma, const RDMACmd *cmd, bool *stop) { @@ -2953,8 +3004,8 @@ static int rdma_consumer_dispatch(RDMAContext *rdma, = const RDMACmd *cmd, =20 switch (cmd->type) { case RDMA_CMD_WRITE: - ret =3D qemu_rdma_write_chunk(rdma, cmd->block_index, - cmd->offset, cmd->length, &err); + /* WRITE runs are handled in bulk by qemu_rdma_write_run(). */ + g_assert_not_reached(); break; case RDMA_CMD_FLUSH: ret =3D qemu_rdma_drain_cq(rdma, &err); @@ -2981,6 +3032,7 @@ static void *rdma_consumer_thread(void *opaque) { RDMAContext *rdma =3D opaque; RDMASendRing *r =3D rdma->send_ring; + Error *err =3D NULL; bool stop =3D false; =20 rcu_register_thread(); @@ -3025,6 +3077,21 @@ static void *rdma_consumer_thread(void *opaque) unsigned idx =3D r->tail + processed; RDMACmd cmd =3D r->ring[idx & RDMA_SEND_RING_MASK]; =20 + /* WRITE runs are drained in bulk with batched registration. */ + if (cmd.type =3D=3D RDMA_CMD_WRITE) { + int n =3D qemu_rdma_write_run(rdma, r, idx, + batch - processed, &err); + if (n < 0) { + if (err) { + error_report_err(err); + } + ring_set_abort(rdma); + break; + } + processed +=3D (unsigned)n; + continue; + } + if (rdma_consumer_dispatch(rdma, &cmd, &stop) < 0) { ring_set_abort(rdma); break; @@ -3447,6 +3514,13 @@ static void rdma_nbsent_zero(RDMAContext *rdma) rdma->nbsent_zero_hits++; } =20 +static void rdma_reg_batch_record(RDMASendRing *r, unsigned count) +{ + if (count > r->reg_batch_max) { + r->reg_batch_max =3D count; + } +} + static void rdma_stats_reset(RDMAContext *rdma) { rdma->nbsent_max =3D 0; --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352705; cv=none; d=zohomail.com; s=zohoarc; b=lW55doig3WWTUAV75cTZUlu6899J0tks+72B+7gLX/qHrRJPEDoPAog/4OvIv7Xl65w32ML5b3/IBhfRLemCE76OxHOBU1+CwBDajEWIhTd9YvpWvbEGnU25ZUGE9sGstgN4Y6SuiO25mmAe5ZrTRui6CD2d4CIHV3/3afbxn8M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352705; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lNWrB/oV8rNLv19MecIm7xnc9TkBjo7V49u7kMzEQRM=; b=Bj64xbLmCd7gq0NuaPzmXBQ8jiYDd/lhu+nQFY+AiWI14SD66Vrr0ZnPKWfZnPhv8uOdAAc2Pu7OWfmhlZkXAch1AmLjqvskVJbuhq5Uj/ZSz2NqCPnzvyynX8wl+yjIOE06UADsA3IFrGWCBfd22v7CV7j7TJIWfBGZjpCKfrQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1790352705673466.8255844290578; Fri, 25 Sep 2026 09:11:45 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8Tx-00015r-Sm; Fri, 25 Sep 2026 12:09:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Do-0007xi-UH for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:24 -0400 Received: from mail-wr2-x0f.google.com ([2a00:1450:4864:30::f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Dn-00061e-6A for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:24 -0400 Received: by mail-wr2-x0f.google.com with SMTP id ffacd0b85a97d-48882c1f261so204307f8f.0 for ; Fri, 25 Sep 2026 08:52:22 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351542; x=1790956342; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=lNWrB/oV8rNLv19MecIm7xnc9TkBjo7V49u7kMzEQRM=; b=CV63rIYIKR7u+zG6tpmKkUC7OTT8uSRqfR0XVitHGve6b4124H801bjkq1rtscw4pQ HYhikNWxW6hBSgVhlCTxMxmJxm2/unof3ZNQ8oylueb4y8oUCujxWlWJ0/1X5gpsWmFz IbUWJyuVHicW2m+A4guoDfBgEqX9atDEBOIMuPmSDmwaAdBMQI3VOeJjicfXfxXdg9uu qUhTN/AF9KX2YWvOAG6pj7CM0/Mk5eMqLvfvJSp96EZtkCK0GOwWRRxKwJr+xTFgvS8A h2Xe1Rn/dqGzVyPYtBOplGJBdz5BOYcEGhBbpTvXiA6CKOyE4Gw/o5v+wS8tGHjk/wiN wXmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351542; x=1790956342; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=lNWrB/oV8rNLv19MecIm7xnc9TkBjo7V49u7kMzEQRM=; b=cv26bzMpvsLG9H/oCtv86ranfxpAgvFasPFIYwSQSpdiuEhB8c4LOo0VzZ0NQMtMWR 69UaqTHU71I9NxHYerPp/XRJJtg3o3hGY5lRn3k9gkMNd1V9TSE8zs1Y7+38k6hDMslX 56c9h8gxxXqOXz+1I2K1P+K5N78oNHsqiOFAWhqrqxEL9nLSdLVGsmeg76xyEOCp0XvB 6/CY/9SjFXCUJ5CNobCxkuEA/tUCNTgcIWuOHhPpLy7vJslpTi8OwwPh2qzpkWqaaS4l zP1JIaOptnUkQ9PL12ri/uUPbWbPmxoNOU/j7KPp0qPoQGCaSaOHQGoX9a5+VJWI5LyH FOUQ== X-Gm-Message-State: AFuF++mk8aPLnhcx4Xz35dcuSq74j3cIU9dA7c1Jo8SIaIeBkVKJE9xe ituLOHfb0hQ/THUvYSJs1OChhqQYHA44lMtnTckbD6oXqw/gEghciR7oD+sTFbAgxv6BlenFUVs SqFVe1hGMiw== X-Gm-Gg: AYBFou3ZRyKRqUov8gIi223NmhDwQkxVuBNGtxOkdFOSpbc8Ec7IgndLLlDkW/VpNYl YhTcYf+HsisJsON9O3IEL1e0gKeEfaUACAtLbg7EapBHl8bF6nvABTZ/jFPe3/ev+c8W8EmZAMf oQrO7mTkhY15s8I9lZTTug3+X/MLwD6RaDCuAZzMQFwevzmv8wLQuXZB9yRFwkgemTzBn8YYDp9 qP42EsgITFSPOban0b7QPGaLupp7lsSlh+F6whS1rQeP8SDqp45Qaw1HjeiZalZnjloRjyGTkdx 9iBRKb8K8E69Es/1J4AvTXo2vUNIYKzf80XICk5x13K+cdQheJUIAgeO/YSgyXLVOAOk/OEOJ+5 U+kxsjyVPKVl9A54u/MiHUYE0jz8maUIM6fPZVxqv0vTdspinstzcAwro4hMQdKymwICjrcGANl yt3dudFSIo1MPRScbTvlove79dFEnb4vGjJXVBRz1Fa/FPuD5gWfWGYGMSdzrryq9Bt5G1LgvLT bHgaEqdYIo6MSu/OVEmkaWhHPsa2AIp/mPcLduO4g== X-Received: by 2002:a05:600c:8b5b:b0:49c:f13e:e4d with SMTP id 5b1f17b1804b1-49fe66f17damr107180165e9.10.1790351541928; Fri, 25 Sep 2026 08:52:21 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 12/13] migration/rdma: trace batching and register runs Date: Fri, 25 Sep 2026 17:51:47 +0200 Message-ID: <20260925155149.843870-13-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:30::f; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wr2-x0f.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:02 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352706721158500 Content-Type: text/plain; charset="utf-8" Add trace points for the consumer batch drain, the WRITE run, and the batched REGISTER_REQUEST, and log the peak register batch size at migration end. Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 6 +++++- migration/trace-events | 5 ++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index aa1a81619f..863165de26 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1940,6 +1940,7 @@ static int qemu_rdma_write_register(RDMAContext *rdma= , RDMARegister *regs, head.repeat =3D count; =20 rdma_reg_batch_record(rdma->send_ring, count); + trace_rdma_write_register_batch(count); =20 if (qemu_rdma_exchange_send(rdma, &head, (uint8_t *)net, &resp, ®_result_idx, NULL, errp) < 0) { @@ -2970,6 +2971,7 @@ static int qemu_rdma_write_run(RDMAContext *rdma, RDM= ASendRing *r, n++; } =20 + trace_rdma_write_run(n); if (regn =3D=3D 0) { return (int)n; } @@ -3072,6 +3074,7 @@ static void *rdma_consumer_thread(void *opaque) */ batch =3D avail < RDMA_CONSUMER_BATCH_MAX ? avail : RDMA_CONSUMER_BATCH_MAX; + trace_rdma_consumer_batch(batch, avail); =20 while (processed < batch) { unsigned idx =3D r->tail + processed; @@ -3542,7 +3545,8 @@ static void rdma_nbsent_stats_dump(RDMAContext *rdma) static void rdma_prodcons_stats_dump(RDMASendRing *r) { trace_rdma_prodcons_stats(r->idle_max_ns, r->pushwait_max_ns, - r->pushwait_total_ns, r->occupancy_max); + r->pushwait_total_ns, r->occupancy_max, + r->reg_batch_max); } =20 int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, diff --git a/migration/trace-events b/migration/trace-events index 833351a03f..e05ca23f32 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -243,10 +243,13 @@ rdma_register_odp_mr(const char *name) "Try to regist= er On-Demand Paging memory rdma_advise_mr(const char *name, uint32_t len, uint64_t addr, const char *= res) "Try to advise block %s prefetch at %" PRIu32 "@0x%" PRIx64 ": %s" rdma_resolve_host_trying(const char *host, const char *ip) "Trying %s =3D>= %s" rdma_write_flush(int sent) "sent total: %d" +rdma_write_run(uint32_t n) "consumer write run: %" PRIu32 " chunks" +rdma_write_register_batch(uint32_t count) "batch register: %" PRIu32 " chu= nks" rdma_write_one_post(uint64_t chunk, long addr, long remote, uint32_t len) = "Posting chunk: %" PRIu64 ", addr: 0x%lx remote: 0x%lx, bytes %" PRIu32 rdma_post_send_queue_full(uint64_t wr_id, int sent) "send queue full, wr_i= d=3D%" PRIu64 ", outstanding writes=3D%d" rdma_nbsent_stats(uint64_t max, uint64_t posts, uint64_t full_hits, uint64= _t zero_hits) "nb_sent max %" PRIu64 " posts %" PRIu64 " full-queue hits %"= PRIu64 " drains to zero %" PRIu64 rdma_consumer_dispatch(int type, uint32_t bidx, uint32_t cidx, uint32_t se= q) "cmd type=3D%d block=3D%" PRIu32 " chunk=3D%" PRIu32 " seq=3D%" PRIu32 +rdma_consumer_batch(uint32_t batch, uint32_t occupancy) "consumer batch: d= raining %" PRIu32 " of %" PRIu32 " queued" rdma_ring_wake_producer(void) "waking blocked producer" rdma_ring_wake_consumer(void) "waking idle consumer" rdma_ring_push_begin(const char *type, uint32_t chunk_index) "push begin t= ype=3D%s chunk_index=3D%" PRIu32 @@ -258,7 +261,7 @@ rdma_consumer_thread_stop(bool aborted) "consumer threa= d stopped, aborted=3D%d" rdma_consumer_idle_begin(void) "consumer idle: waiting for commands" rdma_consumer_idle_end(int64_t idle_ns) "consumer idle: woke after %" PRId= 64 " ns" rdma_ram_round_notify(uint32_t occupancy) "ram round notify: ring occupanc= y=3D%" PRIu32 -rdma_prodcons_stats(uint64_t idle_max, uint64_t pushwait_max, uint64_t pus= hwait_total, uint64_t occupancy_max) "idle max %" PRIu64 " ns, push full-wa= it max %" PRIu64 " ns total %" PRIu64 " ns, ring occupancy max %" PRIu64 +rdma_prodcons_stats(uint64_t idle_max, uint64_t pushwait_max, uint64_t pus= hwait_total, uint64_t occupancy_max, uint64_t reg_batch_max) "idle max %" P= RIu64 " ns, push full-wait max %" PRIu64 " ns total %" PRIu64 " ns, ring oc= cupancy max %" PRIu64 ", register batch max %" PRIu64 rdma_write_one_recvregres(int mykey, int theirkey, uint64_t chunk) "Receiv= ed registration result: my key: 0x%x their key 0x%x, chunk %" PRIu64 rdma_write_one_sendreg(uint64_t chunk, int len, int index, int64_t offset)= "Sending registration request chunk %" PRIu64 " for %d bytes, index: %d, o= ffset: %" PRId64 rdma_write_one_top(uint64_t chunks, uint64_t size) "Writing %" PRIu64 " ch= unks, (%" PRIu64 " MB)" --=20 2.39.5 From nobody Sat Sep 26 19:12:24 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=cloud.ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1790352751; cv=none; d=zohomail.com; s=zohoarc; b=TnluXWYuF76/ALKM9nzNc/Gr3IAuwBOCmCiS3V/D+lBhOwc9ePztnh8rPzcr2jLqv0fUyj7PvSofUgZ2w6IGPKN7aPKDIqWmyvY2zG8yLnTl/lNzqh+ZDXIvtwhByNgAt23BXfKXpHJUA6W1XSZ25OBABJ5qEJTVi+y1cvfRHD4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1790352751; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GIOmBx1IGb/7oU+3HeMgEeumH7X5vzideD5xlFaVCN8=; b=Fxup22dVtvTWm6f/JEfD1LT8VMXFAXURCNyI+fyF9NRdR/LEghG50Q1Po3/tXqBwh/fzmPnIQGgPMlqofwiHZn7S3fsa00rWcZ8hna15aieXeCmqcmaFIF6A7uUrhb9H+wcIGijfoV3dRflK+SO4LvY6GkyeXUU33qsvqv1Ph+M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17903527510236.104698958702329; Fri, 25 Sep 2026 09:12:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1xA8Uq-0001WH-69; Fri, 25 Sep 2026 12:10:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1xA8Dr-0007yp-Ju for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:27 -0400 Received: from mail-wr2-x10.google.com ([2a00:1450:4864:30::10]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1xA8Dp-00062B-PZ for qemu-devel@nongnu.org; Fri, 25 Sep 2026 11:52:27 -0400 Received: by mail-wr2-x10.google.com with SMTP id ffacd0b85a97d-482f6350f91so636252f8f.1 for ; Fri, 25 Sep 2026 08:52:25 -0700 (PDT) Received: from PF-5Z7P7F.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49ff0f1f2e3sm112435875e9.0.2026.09.25.08.52.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2026 08:52:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; t=1790351544; x=1790956344; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=GIOmBx1IGb/7oU+3HeMgEeumH7X5vzideD5xlFaVCN8=; b=acERh8Wotn7bHAjqkVvALTHm9kXJu5kGg9DSLmozzmhYUI874FCpzZCUDJ/O4l6ZCK oz96eG/LFAZSwh3hcLStx5ugCuOl5xFz29BA+Qq8ihEvPOq4yQivXxAEP8AreFNHoYrx oXErWE8oXJoclsV+B2pqQncIbzr9t6kdFAIoIfSdubfTPQtKw92gkK/4givJX/cr0xNJ Cq+09mz5f9ar6TZAapJRtokbX7EJKjFXdaombQkT0uXnxxNTv4PZVAmelUQOwqSNa/UQ PLOUuUvFpQ1b4BBRjMGjIpWfu9sFfgqb0KwB9Ogc7bNlbFNnCpa8ciQ8xTJ/cNnSZG3a SQ0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790351544; x=1790956344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=GIOmBx1IGb/7oU+3HeMgEeumH7X5vzideD5xlFaVCN8=; b=UwZhmuSaLnpNAFdkmgb2W/7hQlF0bdTIWXhultGHsRk3GG9kebKIJFKBpoAGWK4KjM UL5Bbj1PiZSNNx1oZhMYPAJIVqESgV9PvKOiR80AGtlsohRnUqZNqa86z5IhqjWNFukO GhLSdPWIk45Jurd6a0zSKNsXLnGHgAJX+b19c1h12iMzPd6zO7jNmboHdpJ830EOdNMp 3znuK+nCMovB7vSMRVMoCvayGPtID4eRkHHSQYs9qTXQM7R4u0XwNck5Qwl2cV+o7J/f vVzXTaQMpH36a0g2UwX/FGfwtziPuChUciFuOh4xS6ODkSE20i8hxHeU4LRxoV3Ob9Mg clpQ== X-Gm-Message-State: AFuF++kV+HrSGteCWFjXRabtvaeUTyJNukpfkyRXx4MpBgqeyfCwZxx9 FKVm/VnVU5KQB0jVjGfUPAfPhWO6mtK29BiZOOXkmcKBFRlnOyODDWsdl0l+FwyCcEVI7CVHivx lkHTMm5AnDw== X-Gm-Gg: AYBFou1dJqrmBptZpPiPIl/+AAKUzCL3sDPiN55O3ijG51pSdVcZEakjw3reoNM3VNo +8KTQtBBIMOgBMdO+O8aYBxTqS79G3F8nfmWhw04l0cWufqV8TMtcnnb5idfBq7ndMxg31g1DeK eSh8/Zdq7er3ZNc+IQzMddXPoxKAbzN+8aUD3M1N1P7CoMUNrTC717naJdy/1IKHtlJoAZrSYxJ wzuew8OacPxbozKJCUxaVVvlI6KGUmB36LehqwdDRX3FxLbDaJw+DdaLrDF60/EH9FB4S23Rkoh 8m5DxVhR/TOWI6lu2/5EVr7Tnx7ZgPJxowDt6GR5thBmVG0Kz4WgJX9DGoW25Wt21zcwQ3B5EF2 WmRmkB6a0MlTktMpdtLnE4AhKLK3ow7iQpgHsS8ljCUue478aZc8XeNXzkLudtui8k1Bo7/0LeO 4YHl4baAGSErE8c0S2mh2FANUJsU7ZEORbCzCUMhU2kLfxFy/AqelrLCTYPf4MkqZktEwLI7MRZ OLaGVWKM98zvB6hmlz1yoe/om/mAraBHWOHH4l5TnflAd7kIJke X-Received: by 2002:a05:600c:348e:b0:49e:8191:e5cb with SMTP id 5b1f17b1804b1-49ff06b2ef6mr49202885e9.5.1790351544332; Fri, 25 Sep 2026 08:52:24 -0700 (PDT) From: Dmitrii Omelchenko To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , Yanfei Xu , Jinpu Wang , Dmitrii Omelchenko Subject: [RFC PATCH 13/13] migration/rdma: negotiate batched chunk registration Date: Fri, 25 Sep 2026 17:51:48 +0200 Message-ID: <20260925155149.843870-14-dmitrii.omelchenko@cloud.ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> References: <20260925155149.843870-1-dmitrii.omelchenko@cloud.ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:30::10; envelope-from=dmitrii.omelchenko@ionos.com; helo=mail-wr2-x10.google.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 25 Sep 2026 12:05:02 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @cloud.ionos.com) X-ZM-MESSAGEID: 1790352753415158500 Content-Type: text/plain; charset="utf-8" From: Jack Wang An old peer echoes REGISTER_RESULT.repeat unchanged but only sends one result, so a batched source would read stale rkeys for the rest. Negotiate RDMA_CAPABILITY_REG_BATCH like PIN_ALL and cap requests at one chunk when unsupported; check resp.len instead of resp.repeat. Signed-off-by: Jack Wang Signed-off-by: Dmitrii Omelchenko --- migration/rdma.c | 37 ++++++++++++++++++++++++++++++++++--- migration/trace-events | 1 + 2 files changed, 35 insertions(+), 3 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 863165de26..29a2480448 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -75,12 +75,15 @@ static inline uint64_t rdma_merge_max(void) * Capabilities for negotiation. */ #define RDMA_CAPABILITY_PIN_ALL 0x01 +/* Peer replies to a batched REGISTER_REQUEST with one result per chunk. */ +#define RDMA_CAPABILITY_REG_BATCH 0x02 =20 /* * Add the other flags above to this list of known capabilities * as they are introduced. */ -static uint32_t known_capabilities =3D RDMA_CAPABILITY_PIN_ALL; +static uint32_t known_capabilities =3D RDMA_CAPABILITY_PIN_ALL | + RDMA_CAPABILITY_REG_BATCH; =20 /* * A work request ID is 64-bits and we split up these bits @@ -352,6 +355,7 @@ typedef struct RDMAContext { uint64_t pend_length; /* span length in bytes (within one chun= k) */ =20 bool pin_all; + bool reg_batch; /* peer negotiated RDMA_CAPABILITY_REG_BATCH */ =20 /* * infiniband-specific variables for opening the device @@ -1947,6 +1951,14 @@ static int qemu_rdma_write_register(RDMAContext *rdm= a, RDMARegister *regs, return -1; } =20 + /* resp.repeat is echoed back unchanged by old peers; resp.len is not.= */ + if (resp.len !=3D count * sizeof(RDMARegisterResult)) { + error_setg(errp, "rdma: REGISTER_RESULT size mismatch: " + "requested %u chunks (%zu bytes), peer returned %u byte= s", + count, count * sizeof(RDMARegisterResult), resp.len); + return -1; + } + results =3D (RDMARegisterResult *)rdma->wr_data[reg_result_idx].contro= l_curr; for (i =3D 0; i < count; i++) { RDMALocalBlock *block =3D @@ -2172,6 +2184,8 @@ static int qemu_rdma_connect(RDMAContext *rdma, bool = return_path, cap.flags |=3D RDMA_CAPABILITY_PIN_ALL; } =20 + cap.flags |=3D RDMA_CAPABILITY_REG_BATCH; + caps_to_network(&cap); =20 ret =3D qemu_rdma_post_recv_control(rdma, RDMA_WRID_READY, errp); @@ -2219,6 +2233,13 @@ static int qemu_rdma_connect(RDMAContext *rdma, bool= return_path, rdma->pin_all =3D false; } =20 + rdma->reg_batch =3D !!(cap.flags & RDMA_CAPABILITY_REG_BATCH); + if (!rdma->reg_batch) { + warn_report("RDMA: Server does not support batched chunk " + "registration. Will register one chunk per request."); + } + trace_rdma_connect_reg_batch_outcome(rdma->reg_batch); + trace_rdma_connect_pin_all_outcome(rdma->pin_all); =20 rdma_ack_cm_event(cm_event); @@ -2952,6 +2973,14 @@ static int qemu_rdma_write_run(RDMAContext *rdma, RD= MASendRing *r, return -1; } } else { + bool new_chunk =3D (int)cmd->block_index !=3D last_bidx || + (int64_t)cmd->chunk_index !=3D last_cidx; + + /* Cap at one never-seen chunk per request without REG_BATCH. = */ + if (new_chunk && nreg =3D=3D 1 && !rdma->reg_batch) { + break; + } + /* Defer: record every span for a post-registration write. */ reg_ring_idx[regn++] =3D pos + n; /* @@ -2959,8 +2988,7 @@ static int qemu_rdma_write_run(RDMAContext *rdma, RDM= ASendRing *r, * non-decreasing within a run, so a repeat chunk is the last = one * queued; add each chunk to the REGISTER batch only once. */ - if ((int)cmd->block_index !=3D last_bidx || - (int64_t)cmd->chunk_index !=3D last_cidx) { + if (new_chunk) { regs[nreg].current_index =3D cmd->block_index; regs[nreg].current_addr =3D addr; nreg++; @@ -3663,6 +3691,9 @@ static int qemu_rdma_accept(RDMAContext *rdma) if (cap.flags & RDMA_CAPABILITY_PIN_ALL) { rdma->pin_all =3D true; } + if (cap.flags & RDMA_CAPABILITY_REG_BATCH) { + rdma->reg_batch =3D true; + } =20 rdma->cm_id =3D cm_event->id; verbs =3D cm_event->id->verbs; diff --git a/migration/trace-events b/migration/trace-events index e05ca23f32..3c5da0fd14 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -223,6 +223,7 @@ rdma_cleanup_disconnect(void) "" rdma_close(void) "" rdma_connect_pin_all_requested(void) "" rdma_connect_pin_all_outcome(bool pin) "%d" +rdma_connect_reg_batch_outcome(bool reg_batch) "%d" rdma_dest_init_trying(const char *host, const char *ip) "%s =3D> %s" rdma_dump_id_failed(const char *who) "%s RDMA Device opened, but can't que= ry port information" rdma_dump_id(const char *who, const char *name, const char *dev_name, cons= t char *dev_path, const char *ibdev_path, int transport, const char *transp= ort_name) "%s RDMA Device opened: kernel name %s uverbs device name %s, inf= iniband_verbs class device path %s, infiniband class device path %s, transp= ort: (%d) %s" --=20 2.39.5