From nobody Sun Jun 14 21:17:26 2026 Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [91.218.175.184]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72BC7372B59 for ; Mon, 6 Apr 2026 13:29:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.184 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482141; cv=none; b=TCPD+yKnWuI81xt6GKA6wDcD0v1wqpawZPYBTkQ65wD0U46h/8LWZSbsMA8c5tE9N8YnvEmWs/uQBAfEj5YyeJ64FZZtDXVvCPkFjIN28vHJckua6zEFTyEAcnfp7RV8HsfoCDuRTvZ7VinBB4lJt8wlGSc1N5nxEGYqBAhX8y0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482141; c=relaxed/simple; bh=Ha8MhQXEzBEMq7mU9gwl6pNkOyFU4MXYfWIiPZxkUZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BW+IPVFFtK2N8CZf3wfClpf1nbDjx/f5UV/It2fXIGzwOM76vWhQPnMFjI2ErrxmDTVQLew9LO9nquJgfGiKeEMOc2iYMiKOxUrAxe5FMUcVHtm1jIky0ZD9Sr4iIsQGkcLIEhacpD8NZdrCFWq4DVQg6nycV1r1/pCAo2Nxg4o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=HaWbwc+J; arc=none smtp.client-ip=91.218.175.184 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="HaWbwc+J" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1775482138; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YNtIUnZjrPQLnnsG3IaHRIZXfItDLqjxegoUagyMnos=; b=HaWbwc+JYBMksDSJvimj302IRTdp9frDJgPpRboIACt46VqgM+qeP2XAzVy1RhOkTSseEk o83DoHA9K7CRPglnN5rVQ5vdlwDUltmDF7DBAFRf4FLWgszmEfdNrlDqnt01dJR/zKXyyf RmVvNqr/W2QuOEeIk/dQ8tkijWG3lhs= From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, zhenwei pi Subject: [PATCH v4 1/4] RDMA/core: Fix memory free for GID table Date: Mon, 6 Apr 2026 21:28:26 +0800 Message-ID: <20260406132830.435381-2-zhenwei.pi@linux.dev> In-Reply-To: <20260406132830.435381-1-zhenwei.pi@linux.dev> References: <20260406132830.435381-1-zhenwei.pi@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Remove RXE device, kernel shows: RIP: 0010:free_large_kmalloc+0xf6/0x140 Code: 75 28 0f 0b 44 0f b6 2d a5 d6 d1 01 41 80 fd 01 0f 87 7c d1 ad ff 41 = 83 e5 01 74 3d 41 bc 00 f0 ff ff 45 31 ed e9 61 ff ff ff <0f> 0b 48 c7 c6 a= f b1 70 83 48 89 df e8 79 0a fa ff 5b 41 5c 41 5d RSP: 0018:ffffd038c18074d8 EFLAGS: 00010293 RAX: 0017ffffc0000000 RBX: fffff86984219d00 RCX: 0000000000000000 RDX: 00000000000000f0 RSI: ffff899b88674000 RDI: fffff86984219d00 RBP: ffffd038c18074f0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff899b88674000 R13: 0000000000000001 R14: ffff899b88674000 R15: ffff899b86180000 FS: 00007b163c71c740(0000) GS:ffff899c378bf000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007b163c730200 CR3: 0000000106a1d000 CR4: 0000000000350ef0 Call Trace: kfree+0x163/0x3a0 gid_table_release_one+0xaf/0xf0 [ib_core] ib_cache_release_one+0x66/0x80 [ib_core] ib_device_release+0x48/0xb0 [ib_core] device_release+0x44/0xa0 kobject_put+0x9b/0x250 put_device+0x13/0x30 ib_unregister_device_and_put+0x40/0x60 [ib_core] nldev_dellink+0xd3/0x140 [ib_core] rdma_nl_rcv_msg+0x11d/0x300 [ib_core] ? netlink_bind+0x141/0x3a0 rdma_nl_rcv_skb.constprop.0.isra.0+0xba/0x110 [ib_core] rdma_nl_rcv+0xe/0x20 [ib_core] netlink_unicast+0x28d/0x3e0 netlink_sendmsg+0x214/0x470 __sys_sendto+0x21f/0x230 __x64_sys_sendto+0x24/0x40 x64_sys_call+0x1888/0x26e0 do_syscall_64+0xcb/0x14d0 ? _copy_from_user+0x27/0x70 ? do_sock_setsockopt+0xbd/0x190 ? __sys_setsockopt+0x72/0xd0 ? __x64_sys_setsockopt+0x1f/0x40 ? x64_sys_call+0x221b/0x26e0 ? do_syscall_64+0x109/0x14d0 ? exc_page_fault+0x92/0x1c0 entry_SYSCALL_64_after_hwframe+0x76/0x7e GID table is allocated by *kzalloc_flex* instead of raw *kzalloc_obj*, it also should be released in new style. Fixes: 74e2711bb2af ("RDMA/core: Use kzalloc_flex for GID table") Signed-off-by: zhenwei pi --- drivers/infiniband/core/cache.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cach= e.c index 896486fa6185..647a547e2d7f 100644 --- a/drivers/infiniband/core/cache.c +++ b/drivers/infiniband/core/cache.c @@ -801,7 +801,6 @@ static void release_gid_table(struct ib_device *device, } =20 mutex_destroy(&table->lock); - kfree(table->data_vec); kfree(table); } =20 --=20 2.43.0 From nobody Sun Jun 14 21:17:26 2026 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6B133603E9 for ; Mon, 6 Apr 2026 13:29:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482145; cv=none; b=PJA6mSGDaUl2MaSNHCX6QyZ6w4OMM+Qv411tslyzdlvVoLR/lQvGWV471rHjFjv35SiXvNUCGbzoc50xN51aAOs/elUy7Wwk8D1BI/LGUOgeLxcSn8L0DH26Ni71vb91Xckv9oh8Pke1Kh6ySQ5J1TWYMw6r4mo+Nurndf4qBh0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482145; c=relaxed/simple; bh=fOZ7/E3NaHng7RjFBr+jAe5NK9eFXqIiPuCj86C6oLg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O8bWtUmpKAyuCnSRxlUclaAVrXwHGXJ9+0sTVbjotruhEreQERhaGed1AE/EnL67l2n02Fl5vTIQiDQIPerf08fQvKZI9ZzqoQIDILC07WjXR06TPmwpc2D3rnNpZhRgIK1phiVvbgNwhor0ldsa98N7TPyT/STJtRD5Zol63fs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=D4yEhafc; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="D4yEhafc" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1775482142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m5z7OLDqIXiYQp9eRoCufuvK2CrJP9euDFUggwGrBLw=; b=D4yEhafclbOJIBC0gRX0ZaWI+nyT3BhWHeFRYCdz46j+aulQo9xdt70/VoqMpyDQSWwZdQ JAjgoNDYCz8OkUhEg0id6J2ziuhtcuQWZeNpWOq6dd1pTK+nnbdpR+AtMJ9wQ1QeVgBXYJ IXl3ORA/ci67e72iqCrdXp55Zbw1BQ8= From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, zhenwei pi Subject: [PATCH v4 2/4] RDMA/rxe: remove rxe_ib_device_get_netdev() and RXE_PORT Date: Mon, 6 Apr 2026 21:28:27 +0800 Message-ID: <20260406132830.435381-3-zhenwei.pi@linux.dev> In-Reply-To: <20260406132830.435381-1-zhenwei.pi@linux.dev> References: <20260406132830.435381-1-zhenwei.pi@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Suggested by Leon, remove the rxe_ib_device_get_netdev() wrapper and the RXE_PORT definition. These additions do not improve readability, and RXE has always had only a single port. Signed-off-by: zhenwei pi --- drivers/infiniband/sw/rxe/rxe_mcast.c | 4 ++-- drivers/infiniband/sw/rxe/rxe_net.c | 7 +++---- drivers/infiniband/sw/rxe/rxe_verbs.c | 4 ++-- drivers/infiniband/sw/rxe/rxe_verbs.h | 6 ------ 4 files changed, 7 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/= rxe/rxe_mcast.c index 5cad72073eca..acd03bd87794 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -34,7 +34,7 @@ static int rxe_mcast_add(struct rxe_dev *rxe, union ib_gi= d *mgid) struct net_device *ndev; int ret; =20 - ndev =3D rxe_ib_device_get_netdev(&rxe->ib_dev); + ndev =3D ib_device_get_netdev(&rxe->ib_dev, 1); if (!ndev) return -ENODEV; =20 @@ -59,7 +59,7 @@ static int rxe_mcast_del(struct rxe_dev *rxe, union ib_gi= d *mgid) struct net_device *ndev; int ret; =20 - ndev =3D rxe_ib_device_get_netdev(&rxe->ib_dev); + ndev =3D ib_device_get_netdev(&rxe->ib_dev, 1); if (!ndev) return -ENODEV; =20 diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rx= e/rxe_net.c index 211bd3000acc..6621d01ac32d 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -602,7 +602,7 @@ const char *rxe_parent_name(struct rxe_dev *rxe, unsign= ed int port_num) struct net_device *ndev; char *ndev_name; =20 - ndev =3D rxe_ib_device_get_netdev(&rxe->ib_dev); + ndev =3D ib_device_get_netdev(&rxe->ib_dev, 1); if (!ndev) return NULL; ndev_name =3D ndev->name; @@ -646,12 +646,11 @@ static void rxe_sock_put(struct sock *sk, =20 void rxe_net_del(struct ib_device *dev) { - struct rxe_dev *rxe =3D container_of(dev, struct rxe_dev, ib_dev); struct net_device *ndev; struct sock *sk; struct net *net; =20 - ndev =3D rxe_ib_device_get_netdev(&rxe->ib_dev); + ndev =3D ib_device_get_netdev(dev, 1); if (!ndev) return; =20 @@ -699,7 +698,7 @@ void rxe_set_port_state(struct rxe_dev *rxe) { struct net_device *ndev; =20 - ndev =3D rxe_ib_device_get_netdev(&rxe->ib_dev); + ndev =3D ib_device_get_netdev(&rxe->ib_dev, 1); if (!ndev) return; =20 diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/= rxe/rxe_verbs.c index 4e5c429aea37..d3b2d610ca37 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -50,7 +50,7 @@ static int rxe_query_port(struct ib_device *ibdev, goto err_out; } =20 - ndev =3D rxe_ib_device_get_netdev(ibdev); + ndev =3D ib_device_get_netdev(ibdev, 1); if (!ndev) { err =3D -ENODEV; goto err_out; @@ -1450,7 +1450,7 @@ static int rxe_enable_driver(struct ib_device *ib_dev) struct rxe_dev *rxe =3D container_of(ib_dev, struct rxe_dev, ib_dev); struct net_device *ndev; =20 - ndev =3D rxe_ib_device_get_netdev(ib_dev); + ndev =3D ib_device_get_netdev(ib_dev, 1); if (!ndev) return -ENODEV; =20 diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/= rxe/rxe_verbs.h index d92f80d16f78..e800545d1046 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -415,7 +415,6 @@ struct rxe_port { u32 qp_gsi_index; }; =20 -#define RXE_PORT 1 struct rxe_dev { struct ib_device ib_dev; struct ib_device_attr attr; @@ -451,11 +450,6 @@ struct rxe_dev { struct rxe_port port; }; =20 -static inline struct net_device *rxe_ib_device_get_netdev(struct ib_device= *dev) -{ - return ib_device_get_netdev(dev, RXE_PORT); -} - static inline void rxe_counter_inc(struct rxe_dev *rxe, enum rxe_counters = index) { atomic64_inc(&rxe->stats_counters[index]); --=20 2.43.0 From nobody Sun Jun 14 21:17:26 2026 Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [91.218.175.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24DAA372EF0 for ; Mon, 6 Apr 2026 13:29:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482148; cv=none; b=nbk4VhRhAWPjqxArMeLW1Lpf8W62Fdsp3jXsyXdkILjal5/lWLG6+ZdSzUKvbhN00Nwp6T2D0S16vYrA4vsuIQeCs2tk9+IfsXw/s2UQjKktYc3V7KZ6eCE8camvG6VuoKNl42xBv6ZukA/+xv9cLV+YAnl3bQF8dL7RkH/NRAY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482148; c=relaxed/simple; bh=Ufl15mbQo+7C3CIxVRsGI7LULJ2Z/nqj2OO4kNnY4Lc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=g+FunjTJ7h1Ek9L+PhuziJg3dxFmC36eW/WSPuHLfMrZOCYRfLnSFcRlJ+Bv75sxnQuVktYcEi0OJRL5Gc/cAAdFAxG5G9rPmnw6wA3A4w0FacVLBIYcCEvEn4c0MAqLxv5rgUfJfhX8Tc4Fz36AJ13mgNNMMQsDh1PS8Ux8i3U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=EN8GrFne; arc=none smtp.client-ip=91.218.175.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="EN8GrFne" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1775482145; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P/OnNQjJT/T9qW97tpprZs0Xz1E0ZXmQRu9U/Jly9xA=; b=EN8GrFneshN9ShF38pwluNW0pKqJEXUyBDlDZa4Tozap7GSgpJd/1/qh7lztCNQymBPmmS RpNG4fxmZJKF5vpwNqdITMn0eyQMZ8BlxEZH7pbL9jQDwqG8w0gqkw6pUhGpMN0GvEAXiJ rAB4G6osvsdvv2tXDonN7bDJt5r+yYE= From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, zhenwei pi Subject: [PATCH v4 3/4] RDMA/rxe: add SENT/RCVD bytes Date: Mon, 6 Apr 2026 21:28:28 +0800 Message-ID: <20260406132830.435381-4-zhenwei.pi@linux.dev> In-Reply-To: <20260406132830.435381-1-zhenwei.pi@linux.dev> References: <20260406132830.435381-1-zhenwei.pi@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" There is a lack of sent/received counter in bytes. Signed-off-by: zhenwei pi --- drivers/infiniband/sw/rxe/rxe_hw_counters.c | 2 ++ drivers/infiniband/sw/rxe/rxe_hw_counters.h | 2 ++ drivers/infiniband/sw/rxe/rxe_net.c | 2 ++ drivers/infiniband/sw/rxe/rxe_recv.c | 6 ++++++ drivers/infiniband/sw/rxe/rxe_verbs.h | 6 ++++++ 5 files changed, 18 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_hw_counters.c b/drivers/infiniba= nd/sw/rxe/rxe_hw_counters.c index 437917a7d8f2..17edaa9a9b9b 100644 --- a/drivers/infiniband/sw/rxe/rxe_hw_counters.c +++ b/drivers/infiniband/sw/rxe/rxe_hw_counters.c @@ -22,6 +22,8 @@ static const struct rdma_stat_desc rxe_counter_descs[] = =3D { [RXE_CNT_LINK_DOWNED].name =3D "link_downed", [RXE_CNT_RDMA_SEND].name =3D "rdma_sends", [RXE_CNT_RDMA_RECV].name =3D "rdma_recvs", + [RXE_CNT_SENT_BYTES].name =3D "sent_bytes", + [RXE_CNT_RCVD_BYTES].name =3D "rcvd_bytes", }; =20 int rxe_ib_get_hw_stats(struct ib_device *ibdev, diff --git a/drivers/infiniband/sw/rxe/rxe_hw_counters.h b/drivers/infiniba= nd/sw/rxe/rxe_hw_counters.h index 051f9e1c3852..01b355103cbc 100644 --- a/drivers/infiniband/sw/rxe/rxe_hw_counters.h +++ b/drivers/infiniband/sw/rxe/rxe_hw_counters.h @@ -26,6 +26,8 @@ enum rxe_counters { RXE_CNT_LINK_DOWNED, RXE_CNT_RDMA_SEND, RXE_CNT_RDMA_RECV, + RXE_CNT_SENT_BYTES, + RXE_CNT_RCVD_BYTES, RXE_NUM_OF_COUNTERS }; =20 diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rx= e/rxe_net.c index 6621d01ac32d..86660031ffa2 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -503,6 +503,7 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_i= nfo *pkt, int err; int is_request =3D pkt->mask & RXE_REQ_MASK; struct rxe_dev *rxe =3D to_rdev(qp->ibqp.device); + unsigned int skblen =3D skb->len; unsigned long flags; =20 spin_lock_irqsave(&qp->state_lock, flags); @@ -526,6 +527,7 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_i= nfo *pkt, } =20 rxe_counter_inc(rxe, RXE_CNT_SENT_PKTS); + rxe_counter_add(rxe, RXE_CNT_SENT_BYTES, skblen); goto done; =20 drop: diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/r= xe/rxe_recv.c index 5861e4244049..0d9112e95eae 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -318,6 +318,7 @@ void rxe_rcv(struct sk_buff *skb) int err; struct rxe_pkt_info *pkt =3D SKB_TO_PKT(skb); struct rxe_dev *rxe =3D pkt->rxe; + unsigned int skblen =3D skb->len + sizeof(struct udphdr); =20 if (unlikely(skb->len < RXE_BTH_BYTES)) goto drop; @@ -341,6 +342,11 @@ void rxe_rcv(struct sk_buff *skb) if (unlikely(err)) goto drop; =20 + if (skb->protocol =3D=3D htons(ETH_P_IP)) + skblen +=3D sizeof(struct iphdr); + else if (skb->protocol =3D=3D htons(ETH_P_IPV6)) + skblen +=3D sizeof(struct ipv6hdr); + rxe_counter_add(rxe, RXE_CNT_RCVD_BYTES, skblen); rxe_counter_inc(rxe, RXE_CNT_RCVD_PKTS); =20 if (unlikely(bth_qpn(pkt) =3D=3D IB_MULTICAST_QPN)) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/= rxe/rxe_verbs.h index e800545d1046..0f5ffd94643f 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -455,6 +455,12 @@ static inline void rxe_counter_inc(struct rxe_dev *rxe= , enum rxe_counters index) atomic64_inc(&rxe->stats_counters[index]); } =20 +static inline void rxe_counter_add(struct rxe_dev *rxe, enum rxe_counters = index, + s64 val) +{ + atomic64_add(val, &rxe->stats_counters[index]); +} + static inline struct rxe_dev *to_rdev(struct ib_device *dev) { return dev ? container_of(dev, struct rxe_dev, ib_dev) : NULL; --=20 2.43.0 From nobody Sun Jun 14 21:17:26 2026 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B43A37472A for ; Mon, 6 Apr 2026 13:29:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482152; cv=none; b=XLwyS18e6o1pZnTko8XslSkAZur1KavwxTyT/NqfsA19gQghtL6vYRAE7m+AQ/9bnhpoQHA79N6bjIfdLoOcanicowzXjpQjrpK5FLbXx9mGmvg+cARRmSbViGrS00/NMpDPdMhdjmGTX72pgghHN7TENEQVc3YpNEKHec5Ttts= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775482152; c=relaxed/simple; bh=Pb0Ht32CUEfKTpgdfS5Sb8QSBESokRrXsw/1ceFr1FA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j25wYC/5PIhEMp00CHxIP5gI/EwtAhH8oq/cxWUQnTIKG4haCHw+pm/c52dsC8OP2BdAX0T98xI+W0+dMr8J9YueoKfHyzVwSyXp/iu69Mu6A2JaQAOwMAdLnbdgZ57DMFDrnJ4IECaPvdWsNQ23a0ecXR2ooj4CK0MfzBRllZk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Fs8UYGUx; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Fs8UYGUx" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1775482148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f7kCt14hnsEavZYgYShaEjTpC641cL3XlLdxhEoW0x8=; b=Fs8UYGUx5QkhHu92keW3T3GuvcQ4QtYzg01TJuQYtK2SkMPMEMb60ORJ4LAGIYojM7uhOT DEctGvWRLd595QAUg9cLC1iTg9si+MOqDzqyDOnoROpw9dvkxnoSoRMyNy8OjUN4g3QwpG EXc/LIHQI85Rp/Wto87Y49Kay+B/aGY= From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, zhenwei pi Subject: [PATCH v4 4/4] RDMA/rxe: support perf mgmt GET method Date: Mon, 6 Apr 2026 21:28:29 +0800 Message-ID: <20260406132830.435381-5-zhenwei.pi@linux.dev> In-Reply-To: <20260406132830.435381-1-zhenwei.pi@linux.dev> References: <20260406132830.435381-1-zhenwei.pi@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" In RXE, hardware counters are already supported, but not in a standardized manner. For instance, user-space monitoring tools like atop only read from the *counters* directory. Therefore, it is necessary to add perf management support to RXE. Also use rxe_counter_get instead of raw atomic64_read in hw-counters. Signed-off-by: zhenwei pi --- drivers/infiniband/sw/rxe/Makefile | 1 + drivers/infiniband/sw/rxe/rxe_loc.h | 6 ++ drivers/infiniband/sw/rxe/rxe_mad.c | 101 ++++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_verbs.c | 1 + 4 files changed, 109 insertions(+) create mode 100644 drivers/infiniband/sw/rxe/rxe_mad.c diff --git a/drivers/infiniband/sw/rxe/Makefile b/drivers/infiniband/sw/rxe= /Makefile index 3977f4f13258..e097c1ca1874 100644 --- a/drivers/infiniband/sw/rxe/Makefile +++ b/drivers/infiniband/sw/rxe/Makefile @@ -23,6 +23,7 @@ rdma_rxe-y :=3D \ rxe_task.o \ rxe_net.o \ rxe_hw_counters.o \ + rxe_mad.o \ rxe_ns.o =20 rdma_rxe-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) +=3D rxe_odp.o diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rx= e/rxe_loc.h index e095c12699cb..64d636bf80fd 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -242,4 +242,10 @@ static inline int rxe_ib_advise_mr(struct ib_pd *pd, =20 #endif /* CONFIG_INFINIBAND_ON_DEMAND_PAGING */ =20 +/* rxe-mad.c */ +int rxe_process_mad(struct ib_device *ibdev, int mad_flags, u32 port_num, + const struct ib_wc *in_wc, const struct ib_grh *in_grh, + const struct ib_mad *in, struct ib_mad *out, + size_t *out_mad_size, u16 *out_mad_pkey_index); + #endif /* RXE_LOC_H */ diff --git a/drivers/infiniband/sw/rxe/rxe_mad.c b/drivers/infiniband/sw/rx= e/rxe_mad.c new file mode 100644 index 000000000000..7cf6d94e636e --- /dev/null +++ b/drivers/infiniband/sw/rxe/rxe_mad.c @@ -0,0 +1,101 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * Copyright (c) 2026 zhenwei pi + */ + +#include +#include "rxe.h" +#include "rxe_hw_counters.h" + +static int rxe_get_pma_info(struct ib_mad *out) +{ + struct ib_class_port_info cpi =3D {}; + + cpi.capability_mask =3D IB_PMA_CLASS_CAP_EXT_WIDTH; + memcpy((out->data + 40), &cpi, sizeof(cpi)); + + return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY; +} + +static int rxe_get_pma_counters(struct rxe_dev *rxe, struct ib_mad *out) +{ + struct ib_pma_portcounters *pma_cnt =3D (struct ib_pma_portcounters *)(ou= t->data + 40); + s64 val; + + /* IBA release 1.8, 16.1.3.5: During operation, instead of overflowing, t= hey shall stop + * at all ones. + */ + val =3D atomic64_read(&rxe->stats_counters[RXE_CNT_LINK_DOWNED]); + if (val > U8_MAX) + pma_cnt->link_downed_counter =3D U8_MAX; + else + pma_cnt->link_downed_counter =3D (u8)val; + + return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY; +} + +static int rxe_get_pma_counters_ext(struct rxe_dev *rxe, struct ib_mad *ou= t) +{ + struct ib_pma_portcounters_ext *pma_cnt_ext =3D + (struct ib_pma_portcounters_ext *)(out->data + 40); + s64 val; + + val =3D atomic64_read(&rxe->stats_counters[RXE_CNT_SENT_BYTES]); + pma_cnt_ext->port_xmit_data =3D cpu_to_be64(val >> 2); + + val =3D atomic64_read(&rxe->stats_counters[RXE_CNT_RCVD_BYTES]); + pma_cnt_ext->port_rcv_data =3D cpu_to_be64(val >> 2); + + val =3D atomic64_read(&rxe->stats_counters[RXE_CNT_SENT_PKTS]); + pma_cnt_ext->port_xmit_packets =3D cpu_to_be64(val); + + val =3D atomic64_read(&rxe->stats_counters[RXE_CNT_RCVD_PKTS]); + pma_cnt_ext->port_rcv_packets =3D cpu_to_be64(val); + + return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY; +} + +static int rxe_get_perf_mgmt(struct rxe_dev *rxe, const struct ib_mad *in,= struct ib_mad *out) +{ + switch (in->mad_hdr.attr_id) { + case IB_PMA_CLASS_PORT_INFO: + return rxe_get_pma_info(out); + + case IB_PMA_PORT_COUNTERS: + return rxe_get_pma_counters(rxe, out); + + case IB_PMA_PORT_COUNTERS_EXT: + return rxe_get_pma_counters_ext(rxe, out); + + default: + out->mad_hdr.status =3D cpu_to_be16(IB_MGMT_MAD_STATUS_UNSUPPORTED_METHO= D_ATTRIB); + return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY; + } +} + +int rxe_process_mad(struct ib_device *ibdev, int mad_flags, u32 port_num, + const struct ib_wc *in_wc, const struct ib_grh *in_grh, + const struct ib_mad *in, struct ib_mad *out, + size_t *out_mad_size, u16 *out_mad_pkey_index) +{ + struct rxe_dev *rxe =3D to_rdev(ibdev); + u8 mgmt_class =3D in->mad_hdr.mgmt_class; + u8 method =3D in->mad_hdr.method; + + if (port_num !=3D 1) + return IB_MAD_RESULT_FAILURE; + + memset(out, 0, sizeof(*out)); + switch (mgmt_class) { + case IB_MGMT_CLASS_PERF_MGMT: + if (method =3D=3D IB_MGMT_METHOD_GET) + return rxe_get_perf_mgmt(rxe, in, out); + break; + + default: + break; + } + + out->mad_hdr.status =3D cpu_to_be16(IB_MGMT_MAD_STATUS_UNSUPPORTED_METHOD= ); + return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY; +} diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/= rxe/rxe_verbs.c index d3b2d610ca37..1ef5cddf620a 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -1505,6 +1505,7 @@ static const struct ib_device_ops rxe_dev_ops =3D { .post_recv =3D rxe_post_recv, .post_send =3D rxe_post_send, .post_srq_recv =3D rxe_post_srq_recv, + .process_mad =3D rxe_process_mad, .query_ah =3D rxe_query_ah, .query_device =3D rxe_query_device, .query_pkey =3D rxe_query_pkey, --=20 2.43.0