From nobody Sat Sep 26 20:51:04 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1788278628; cv=none; d=zohomail.com; s=zohoarc; b=M1P0aL2Oq0iWVGXvIwNozKvNIJpfGzOmGsj2tL394PBcXqBr0Y96dP2ALkxjOpgGOZ9cRtHSCNtGO7FjRjYUbjH6sNlzZ9nEynkq0fsDHgwaHixfAHZoH2ZNpE7TdDUzCqdbwxHOO17KFeftdmHebFN8YGhOQikxS5OeJp5Gg6Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1788278628; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=+SG4rf34LIFkTsX94P+qqW5zlbLqFuIfbZtiTmVQLoQ=; b=c4eTfuYt5xSUMaePXHSXXbXXP9UufX2TEdmt/L09O7lt+QrXqVqmryXsv0i6VEHIW7ULS6cZgxvvdOsFevbLJ0BeD9qO7sUldZEM0O+mRoivoIDoYez+bwkn/nV4oYP8EfrJRGIAmr9bx4ETaRdfTMtLUYkjS34g0WT4DfGrPO4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1788278628171318.10492853310825; Tue, 1 Sep 2026 09:03:48 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1x1Qxb-0008IL-0K; Tue, 01 Sep 2026 12:03:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1x1QxY-0008Ha-Am for qemu-devel@nongnu.org; Tue, 01 Sep 2026 12:03:40 -0400 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1x1QxW-0001Uq-C6 for qemu-devel@nongnu.org; Tue, 01 Sep 2026 12:03:39 -0400 Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-482e06fda73so8919f8f.1 for ; Tue, 01 Sep 2026 09:03:38 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([2001:9e8:1471:a200:e095:5b15:7c12:5f73]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49cdce0b456sm87367435e9.2.2026.09.01.09.03.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Sep 2026 09:03:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1788278617; x=1788883417; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=+SG4rf34LIFkTsX94P+qqW5zlbLqFuIfbZtiTmVQLoQ=; b=IdUgFICwc1zekkA4efKqHHmR11RUHh1YILgrFnc4FCKuIVGsnPOw2uomHLHi13pUEk M1ac+drghNcqtn5gxi9LYM2n/Sc1ihEIHYGSuBNrrjkz3nhSs5LrV2omXCFTTNykjaSL rYnnW9ZxDsLLCXHuRMXAuM8zvAy4t0qVDnM4zEltakt4TyguGfZvtvAocT8iGAOr6eB0 KG50og1gQINA/kmHGJDhWh2ep35egl6d6ZQDtRWsIKlp79jjbAPeEf9tsH6QKMJKrCiE OIqnlXUGluOMvkk1TftSlZ4crrp3uuXIPg72qJhKL1cUJdjoWfT9xFNadc4SNNFTyr6c brDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788278617; x=1788883417; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=+SG4rf34LIFkTsX94P+qqW5zlbLqFuIfbZtiTmVQLoQ=; b=hY59Pg0pMR23ngBNPwSyt/wz4ikh3ssEcYopDv6EqeaC8Yt2EfyFi9O0wbBZE//Olh lGKKvypn8zB+CMB07MLGuDrjZkSzZkVHKW7Pwhd+qwpjzKPwk8iFsGVVKyCMRee4HArn FOQzzwaGCqQJRBvnq2pEjWPpOBg2JS+AnE/WFdSQjgDNbyWMjrnMqOFNutHWI5dsjQbG p89fZQt3MRWatvMRIlQ96U/V2mCB5ZoM4RH+160M1Fx/IKEOkIylN1dKPDGq/bN6ZApb QB7jJ17BaWtXsAOhZYC2/gFyAaWU+dCGwBcJ4fLatx+M2UGpk6YMCbLIYLUulpw29/K5 7DxA== X-Gm-Message-State: AFuF++lX2JioOFZDZXCHPOtJXLTY1JMnGDK/p20D2DnakMQDh5xOXSEj KQWF0qYelU28AGWvr+bJRCdeqlvTPU9+iv2Eq87wZHj7v8vXeWMvARg6YHRhV6sQYO6ubtBmSAE sXay2mx0= X-Gm-Gg: AR+sD11jtyvLfkm3cxhKRD+SEwNm0fqKa0+4irAIi3/tEklpxxOsHbuh+bquzYk96lj fkGQ8o621RBwY7CV9JIyU12h1RwmtyseNZEgbMak9dNFKRU1Omrr39XgeLhoNF9FhoonKEb+GRD gZdgm9lhTBdpCTNWploDQsQTBPnkksCh1z4c4XFdWcrRkV3F9i+vaVmuWmV4WKbDa6Y9USog3h3 JVDnyT4n3ZtlopXHbKb1izSG8jy6GEKn/djcDhIrUOHW8M7wsMZAec86UfEs448cw2lF2oZKXhH 8XJvg0wiT50BtnX9Ex45EFv7G8QItWt1BqjyJM149J7j24cizfyEGMHNe1Sm8mD4JwfSah7AChM Vq+ozJT5EzEJZnVMiygzejsGeUeB0l90f2BXGvQ9p4os5GMU7F9l+/kTMWrZ6/y7+fmvrNIFNxF QEzBJ+6ghi57z8LXGN5z3Dt3noE9SZCymfiElBtyRQ7qSczpd8kzBaCj69SKEX96INS21xDc6FN 61dJzSpfbGezuJNapphWMisyQt/8y6hzJrL X-Received: by 2002:a05:600c:80c2:b0:49c:ca33:9553 with SMTP id 5b1f17b1804b1-49cca339575mr138484185e9.2.1788278616658; Tue, 01 Sep 2026 09:03:36 -0700 (PDT) From: Jack Wang To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , yanfei.xu@bytedance.com, Jack Wang Subject: [PATCH 1/2] migration/rdma: send small control messages inline Date: Tue, 1 Sep 2026 17:51:20 +0200 Message-ID: <20260901160333.29859-2-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901160333.29859-1-jinpu.wang@ionos.com> References: <20260901160333.29859-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:20::42a; envelope-from=jinpu.wang@ionos.com; helo=mail-wr1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ionos.com) X-ZM-MESSAGEID: 1788278630903158500 Content-Type: text/plain; charset="utf-8" From: Jack Wang Control-channel sends (registration requests, ram block replies, etc.) are tiny, but the HCA still does a separate memory read to fetch them before sending. Ask the QP for a little inline send space at creation time, and use it whenever a message is small enough to fit, so the HCA can just copy the bytes straight out of the work request instead. Falls back to the old path if the provider grants less inline room than we asked for, or none at all. No wire protocol change. Signed-off-by: Jack Wang --- migration/rdma.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index e976739fad3c..08f3b901be4a 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -66,6 +66,14 @@ static inline uint64_t rdma_merge_max(void) #define RDMA_CONTROL_MAX_BUFFER (512 * 1024) #define RDMA_CONTROL_MAX_COMMANDS_PER_MESSAGE 4096 =20 +/* + * Requested max_inline_data for the QP: enough for a control header + * plus the largest fixed-size control payload, so small control + * messages can be sent inline instead of via a separate HCA-side + * memory read. + */ +#define RDMA_CONTROL_MAX_INLINE_DATA 512 + #define RDMA_CONTROL_VERSION_CURRENT 1 /* * Capabilities for negotiation. @@ -329,6 +337,7 @@ typedef struct RDMAContext { struct ibv_context *verbs; struct rdma_event_channel *channel; struct ibv_qp *qp; /* queue pair */ + uint32_t max_inline_data; /* max size for inline sends */ struct ibv_comp_channel *recv_comp_channel; /* recv completion channe= l */ struct ibv_comp_channel *send_comp_channel; /* send completion channe= l */ struct ibv_pd *pd; /* protection domain */ @@ -936,6 +945,14 @@ static int qemu_rdma_alloc_qp(RDMAContext *rdma) attr.cap.max_recv_wr =3D 3; attr.cap.max_send_sge =3D 1; attr.cap.max_recv_sge =3D 1; + /* + * Ask for enough inline data to cover a control header plus the + * largest fixed-size control payload (RDMARegister/RDMACompress), + * so those sends can skip a local memory read on the HCA. The + * provider may grant less (or none); qemu_rdma_post_send_control() + * checks the actual granted size before using IBV_SEND_INLINE. + */ + attr.cap.max_inline_data =3D RDMA_CONTROL_MAX_INLINE_DATA; attr.send_cq =3D rdma->send_cq; attr.recv_cq =3D rdma->recv_cq; attr.qp_type =3D IBV_QPT_RC; @@ -945,6 +962,7 @@ static int qemu_rdma_alloc_qp(RDMAContext *rdma) } =20 rdma->qp =3D rdma->cm_id->qp; + rdma->max_inline_data =3D attr.cap.max_inline_data; return 0; } =20 @@ -1447,6 +1465,10 @@ static int qemu_rdma_post_send_control(RDMAContext *= rdma, uint8_t *buf, .num_sge =3D 1, }; =20 + if (sge.length <=3D rdma->max_inline_data) { + send_wr.send_flags |=3D IBV_SEND_INLINE; + } + trace_rdma_post_send_control(control_desc(head->type)); =20 /* --=20 2.43.0 From nobody Sat Sep 26 20:51:04 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=ionos.com ARC-Seal: i=1; a=rsa-sha256; t=1788278638; cv=none; d=zohomail.com; s=zohoarc; b=crbcGVVFymQBVNJujHvZ5PpCm6S+uQB4zsyqU7Jg6UzBqhRD0N4PQCc8hAWCCnyb6/DcbYgOW2wR0oOWFIcmB+OTrjVW7leROLGFXdt+73MwF8HZi85I1Jq3Zu6j6ZMcZjYBdmNfV2az26ni4k/7sj1IdiyS3gYu8fm+OjiRbj0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1788278638; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MtXOjocUSBmy5iGrbplfxQD72F9xwg0KbRg8o9SR3zk=; b=XvuP9M6Pn/t50L5y7/2JaJN9YjNPX3spK9mzfivNRrs25iwuyPxofTP9DtxPX5RC8y7zjV1wZszsFkjZhxI6Pi0ug6kSQmY4BAfAtjUyvgdZLwfudWbrn3QhStlusUkD5Es9ssnfGt9UxIZGyskaQAY6ogRhSC5QxlEZjfgmzdE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1788278638170485.22919651170923; Tue, 1 Sep 2026 09:03:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1x1Qxe-0008Im-2X; Tue, 01 Sep 2026 12:03:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1x1Qxc-0008Ie-JW for qemu-devel@nongnu.org; Tue, 01 Sep 2026 12:03:44 -0400 Received: from mail-wm1-x32e.google.com ([2a00:1450:4864:20::32e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1x1Qxa-0001Vj-NF for qemu-devel@nongnu.org; Tue, 01 Sep 2026 12:03:44 -0400 Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-49b2e029912so1988135e9.3 for ; Tue, 01 Sep 2026 09:03:42 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([2001:9e8:1471:a200:e095:5b15:7c12:5f73]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49cdce0b456sm87367435e9.2.2026.09.01.09.03.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Sep 2026 09:03:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1788278621; x=1788883421; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=MtXOjocUSBmy5iGrbplfxQD72F9xwg0KbRg8o9SR3zk=; b=d82LhicieyDWnzQRj0sAzqlWZsrI8wnf5rEKsitiXCDSsdMVTzgokCErekiExab51v kpqyfqKz/JYMip6iJgMB0UHeVbzwan/QZtj7G2VoEGDPi2LFxIIyWI5BXbj1QGcszM/K Px6D7y6KGB4YrLDOPcL73qhyVjLPkd1/GqAJv5KBDLZXXFguvasVJxSG734KL3/m5KH1 sqNHoIdGOcjxD0zSsMPzYSC1if3N695IpaCbbWwFYeiMshiMNLD487fIT2ubwMSn3j/U W5G/ekGnjiKMs9F9DFExL0pv62wOysqNYIOi62ubw90yaopSuEEvTmry+zEyGCybKjC2 d64w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788278621; x=1788883421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=MtXOjocUSBmy5iGrbplfxQD72F9xwg0KbRg8o9SR3zk=; b=nHi6QayTgroCBxPnCV7wn2QC2wNNYsa3ZvxezTupZUD9uMvjVsBbMip9/WGJvsINag uG6Q2hDWpDujP4uUYTnpyqlV3OU8Aes7P/k8pPgG7GqJGCYw6Q0718q7FzPwPgnjRhUk LDPNJHB4LDD1xA4aFdbJley8c4LHJY1JDgLn+8pQpvU7RQ1EKWh1WHkZb1CsaJW/k62B Ff8lNgmiawtSrlJqkkHLqIZtsqB3XLdKaqInXys+uEOuu2KzIEm+0ccFDFQRshaScn/o jCy5y1ew+rkFgv4qyXuuN+1HNyv6LOsG6D8vsXdrXYDxRgkhQyqXtb9DC0YkQzzps8E8 tIZg== X-Gm-Message-State: AFuF++nFi9V9odnocn7/QZ0CqjNIOBQlzFKV7hBL3Oloq3idxIOkaME+ YetbAicZ633JLWDWNTE6h7BeYw7wmiDDdPnKQzAKO3pVP60A2dp9k1Fq+ckeZGN3QWbSzzOmesw oUR3GMrs= X-Gm-Gg: AR+sD12hw3mork5NemFKFtmD8a7yWYOF9gyIsCebQk61Pbnl/ROM57KmDsns1QZNnKa zqsHHd2QAiDg4nuz9IPtLk24NGXyv8R/wwLDmeD/E0uflG7Y2UGyMSKcJmbyIO/Cb3BhtQxs9tP eBXH2jCyVzSlQppgRy6ybgqxGpqIJEdSrWg/z5snAeAPjcK0Cu5zPZoqrkRvc8OITbgfqfhI3E0 d8kW8OPpnYSaPGDBkjGlKOahGFt+HbQ6GEkkEhdE/F67vVg/bscv86jnRZc+iyVh2kEMawwCEi2 4dHBToGijr4t9UNCmB4bQYzCFv8bTQ8XPe4K7jHimfxQJ54Hz1Ubk1Z2YtJMU5nrb3iQugoNt0H ugbivaI3sMPwIxpOs4yUMBSeFek1BdsDjqRdfRgOtEEUo6qM+XGVMnq+9ej4ZWzuM5xhExaIrnQ WaQTxLTbA1zByr3QynQkuM8n5yBaIB2honhOV2IxmXP0ylKg6Df1ghCXZSn0kAykibGxGHBERNj YDTc+YJfFapq9uCb14CBT075wbBE++4bqkD X-Received: by 2002:a05:600c:c0c2:b0:49c:cbf4:572b with SMTP id 5b1f17b1804b1-49ccbf4575cmr176591655e9.2.1788278617996; Tue, 01 Sep 2026 09:03:37 -0700 (PDT) From: Jack Wang To: qemu-devel@nongnu.org Cc: Peter Xu , Fabiano Rosas , Li Zhijian , yanfei.xu@bytedance.com, Jack Wang Subject: [PATCH 2/2] migration/rdma: avoid memcpy for inline control sends Date: Tue, 1 Sep 2026 17:51:21 +0200 Message-ID: <20260901160333.29859-3-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260901160333.29859-1-jinpu.wang@ionos.com> References: <20260901160333.29859-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: permerror client-ip=2a00:1450:4864:20::32e; envelope-from=jinpu.wang@ionos.com; helo=mail-wm1-x32e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ionos.com) X-ZM-MESSAGEID: 1788278644567154100 Content-Type: text/plain; charset="utf-8" From: Jack Wang qemu_rdma_post_send_control() always copied the header and payload into a pre-registered scratch buffer before sending, even though inline sends don't need a registered region at all -- the HCA copies straight out of the given SGEs at post_send() time. When a message fits inline, point two SGEs directly at a stack-local header and the caller's payload instead of copying either into the scratch buffer. Confirmed against the mlx4/mlx5 driver source (providers/mlx5/qp.c:set_data_inl_seg(), providers/mlx4/qp.c) that inline SGEs never dereference lkey and are copied in a plain loop over num_sge, so this works for any SGE count the QP was created with. Falls back to the old copy-into-registered-buffer path when a message is too big to inline. This needs the QP to actually support 2 SGEs on a send, which the previous commit's QP creation didn't request (max_send_sge was still 1) -- bump it to 2. RDMA WRITEs are unaffected; they still always post exactly 1 SGE. Signed-off-by: Jack Wang --- migration/rdma.c | 68 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 45 insertions(+), 23 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 08f3b901be4a..88c4981804c3 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -943,7 +943,13 @@ static int qemu_rdma_alloc_qp(RDMAContext *rdma) =20 attr.cap.max_send_wr =3D RDMA_SIGNALED_SEND_MAX; attr.cap.max_recv_wr =3D 3; - attr.cap.max_send_sge =3D 1; + /* + * RDMA WRITEs only ever use 1 SGE. Control sends use up to 2 when + * inlined (see qemu_rdma_post_send_control()): one for the header, + * one for the caller's payload, both pointing at unregistered + * memory that only inline sends can reference directly. + */ + attr.cap.max_send_sge =3D 2; attr.cap.max_recv_sge =3D 1; /* * Ask for enough inline data to cover a control header plus the @@ -1452,39 +1458,55 @@ static int qemu_rdma_post_send_control(RDMAContext = *rdma, uint8_t *buf, int ret; RDMAWorkRequestData *wr =3D &rdma->wr_data[RDMA_WRID_CONTROL]; struct ibv_send_wr *bad_wr; - struct ibv_sge sge =3D { - .addr =3D (uintptr_t)(wr->control), - .length =3D head->len + sizeof(RDMAControlHeade= r), - .lkey =3D wr->control_mr->lkey, - }; + RDMAControlHeader net_head =3D *head; + uint32_t total_len =3D head->len + sizeof(RDMAControlHeader); + struct ibv_sge sge[2]; struct ibv_send_wr send_wr =3D { .wr_id =3D RDMA_WRID_SEND_CONTROL, .opcode =3D IBV_WR_SEND, .send_flags =3D IBV_SEND_SIGNALED, - .sg_list =3D &sge, + .sg_list =3D sge, .num_sge =3D 1, }; =20 - if (sge.length <=3D rdma->max_inline_data) { - send_wr.send_flags |=3D IBV_SEND_INLINE; - } - trace_rdma_post_send_control(control_desc(head->type)); =20 - /* - * We don't actually need to do a memcpy() in here if we used - * the "sge" properly, but since we're only sending control messages - * (not RAM in a performance-critical path), then its OK for now. - * - * The copy makes the RDMAControlHeader simpler to manipulate - * for the time being. - */ assert(head->len <=3D RDMA_CONTROL_MAX_BUFFER - sizeof(*head)); - memcpy(wr->control, head, sizeof(RDMAControlHeader)); - control_to_network((void *) wr->control); + control_to_network(&net_head); =20 - if (buf) { - memcpy(wr->control + sizeof(RDMAControlHeader), buf, head->len); + if (total_len <=3D rdma->max_inline_data) { + /* + * Inline data is copied out of these SGEs by the HCA itself at + * post_send() time, so no registration (and no local copy into + * the pre-registered "control" buffer below) is needed -- point + * straight at the header on our stack and the caller's payload. + */ + sge[0].addr =3D (uintptr_t)&net_head; + sge[0].length =3D sizeof(net_head); + sge[0].lkey =3D 0; + + if (buf && head->len) { + sge[1].addr =3D (uintptr_t)buf; + sge[1].length =3D head->len; + sge[1].lkey =3D 0; + send_wr.num_sge =3D 2; + } + + send_wr.send_flags |=3D IBV_SEND_INLINE; + } else { + /* + * Too big to inline: the HCA will DMA-read this directly, which + * requires a registered region, so fall back to copying into + * the pre-registered "control" buffer. + */ + sge[0].addr =3D (uintptr_t)(wr->control); + sge[0].length =3D total_len; + sge[0].lkey =3D wr->control_mr->lkey; + + memcpy(wr->control, &net_head, sizeof(net_head)); + if (buf) { + memcpy(wr->control + sizeof(RDMAControlHeader), buf, head->len= ); + } } =20 =20 --=20 2.43.0