From nobody Tue Sep 29 07:00:19 2026 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [160.30.148.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C57E432E60; Tue, 11 Aug 2026 09:43:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=160.30.148.35 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441388; cv=none; b=H4VckGxz9IzdHwxSw6b9MDmypFN6cUoMtt42XBsLbhm1XIKBBBIk2hgSX0K4A9oxfFxi9nSutDc1dEYU0LfQ1RoTMb8nBmtCAl1B1vvVZlAYIHnb03zaEr8DA3flN1BKrn+XEPWolb0EjgnVrvS39cKlyGSFzoUBvjrXain5H3Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441388; c=relaxed/simple; bh=5UwnjW2vL/sLtY92gtKUHFLMY4LA3TUrkup4dyxqPHc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B2y9fulFprZqHKkf1+2ZUA6wUAy4Nhu7LkFjAV/k9qmta3j1/d9B5PObOsXUOuHFGTvEt8MPSqGHqr37jQw4r/EPG+fM/5GnyqYYtfoy0QTt3AVO42yEaoAhUvm1tggjhJi4qy+edbm6IJaywM9JHasET08Lp+Qyf68awaoBRn4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn; spf=pass smtp.mailfrom=zte.com.cn; arc=none smtp.client-ip=160.30.148.35 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zte.com.cn Received: from mxct.zte.com.cn (unknown [192.168.251.13]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4hK69t0BHSz8Xrr9; Tue, 11 Aug 2026 17:42:58 +0800 (CST) Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxct.zte.com.cn (FangMail) with ESMTPS id 4hK69g2g8bz58fdb; Tue, 11 Aug 2026 17:42:47 +0800 (CST) Received: from szxlzmapp03.zte.com.cn ([10.5.231.207]) by mse-fl2.zte.com.cn with SMTP id 67B9gaj8073846; Tue, 11 Aug 2026 17:42:36 +0800 (+08) (envelope-from zhang.yanze@zte.com.cn) Received: from localhost.localdomain (unknown [192.168.6.15]) by smtp (Zmail) with SMTP; Tue, 11 Aug 2026 17:42:38 +0800 X-Zmail-TransId: 3e816a7aee8e013-44463 X-Zmail-LocalSMTP: 1 X-Zmail-RealSender: zhang.yanze@zte.com.cn From: Yanze Zhang To: jgg@ziepe.ca, leon@kernel.org, julianbraha@gmail.com, huangjunxian6@hisilicon.com, yanjun.zhu@linux.dev, lkp@intel.com Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, wei.quan@zte.com.cn, han.junyang@zte.com.cn, ran.ming@zte.com.cn, han.chengfei@zte.com.cn, zhang.yanze@zte.com.cn Subject: [PATCH rdma v5 1/7] RDMA/zrdma: Add build infrastructure and module skeleton Date: Tue, 11 Aug 2026 17:37:34 +0800 Message-ID: <20260811093740.2607476-2-zhang.yanze@zte.com.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> References: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ZMAIL-USEORIGINALEMLTOOUTBOUND: 1 Content-Transfer-Encoding: quoted-printable X-MAIL: mse-fl2.zte.com.cn 67B9gaj8073846 X-TLS: YES X-ENVELOPE-SENDER: zhang.yanze@zte.com.cn X-SOURCE-IP: 192.168.251.13 unknown Tue, 11 Aug 2026 17:42:58 +0800 X-CLEAN: YES X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 6A7AEEA1.000/4hK69t0BHSz8Xrr9 Content-Type: text/plain; charset="utf-8" Add MAINTAINERS entry, Kconfig, and Makefile entries for the ZTE DingHai RDMA driver. Include a minimal module skeleton with auxiliary driver registration structure to ensure the build system works and the module can be loaded. Signed-off-by: Yanze Zhang --- MAINTAINERS | 6 +++ drivers/infiniband/Kconfig | 1 + drivers/infiniband/hw/Makefile | 1 + drivers/infiniband/hw/zrdma/Kconfig | 12 +++++ drivers/infiniband/hw/zrdma/Makefile | 4 ++ drivers/infiniband/hw/zrdma/zrdma_main.c | 59 ++++++++++++++++++++++++ include/uapi/rdma/ib_user_ioctl_verbs.h | 1 + 7 files changed, 84 insertions(+) create mode 100644 drivers/infiniband/hw/zrdma/Kconfig create mode 100644 drivers/infiniband/hw/zrdma/Makefile create mode 100644 drivers/infiniband/hw/zrdma/zrdma_main.c diff --git a/MAINTAINERS b/MAINTAINERS index 806bd2d80d15..e1369b20a1ef 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -29831,6 +29831,12 @@ S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound.git F: sound/hda/codecs/senarytech.c =20 +ZTE DINGHAI ZRDMA DRIVER +M: Yanze Zhang +L: linux-rdma@vger.kernel.org +S: Supported +F: drivers/infiniband/hw/zrdma/ + THE REST M: Linus Torvalds L: linux-kernel@vger.kernel.org diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig index 086195758a8a..419dbf742f38 100644 --- a/drivers/infiniband/Kconfig +++ b/drivers/infiniband/Kconfig @@ -100,6 +100,7 @@ source "drivers/infiniband/hw/ocrdma/Kconfig" source "drivers/infiniband/hw/qedr/Kconfig" source "drivers/infiniband/hw/usnic/Kconfig" source "drivers/infiniband/hw/vmw_pvrdma/Kconfig" +source "drivers/infiniband/hw/zrdma/Kconfig" source "drivers/infiniband/sw/rdmavt/Kconfig" endif # !UML source "drivers/infiniband/sw/rxe/Kconfig" diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile index c42b22ac3303..a86b867c56d4 100644 --- a/drivers/infiniband/hw/Makefile +++ b/drivers/infiniband/hw/Makefile @@ -16,3 +16,4 @@ obj-$(CONFIG_INFINIBAND_BNXT_RE) +=3D bnxt_re/ obj-$(CONFIG_INFINIBAND_BNG_RE) +=3D bng_re/ obj-$(CONFIG_INFINIBAND_ERDMA) +=3D erdma/ obj-$(CONFIG_INFINIBAND_IONIC) +=3D ionic/ +obj-$(CONFIG_INFINIBAND_ZRDMA) +=3D zrdma/ diff --git a/drivers/infiniband/hw/zrdma/Kconfig b/drivers/infiniband/hw/zr= dma/Kconfig new file mode 100644 index 000000000000..bfe21c123217 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/Kconfig @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +config INFINIBAND_ZRDMA + tristate "ZTE Ethernet Protocol Driver for RDMA" + depends on PCI + depends on NETDEVICES + select AUXILIARY_BUS + help + Say Y or M here to enable support for the ZTE DingHai (ZXDH) Ethernet + Protocol Driver for RDMA. This driver provides RDMA over Converged + Ethernet (RoCE) functionality for ZTE DingHai network adapters. + If you choose to build this driver as a module, it will be built as + a module named zrdma. diff --git a/drivers/infiniband/hw/zrdma/Makefile b/drivers/infiniband/hw/z= rdma/Makefile new file mode 100644 index 000000000000..2ddf50d1fd20 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + +obj-$(CONFIG_INFINIBAND_ZRDMA) +=3D zrdma.o +zrdma-y :=3D zrdma_main.o diff --git a/drivers/infiniband/hw/zrdma/zrdma_main.c b/drivers/infiniband/= hw/zrdma/zrdma_main.c new file mode 100644 index 000000000000..7954d86573e2 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_main.c @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#include +#include + +#define ZXDH_PF_NAME "dinghai10e" +#define ZXDH_RDMA_DEV_NAME "rdma_aux" + +MODULE_LICENSE("Dual BSD/GPL"); +MODULE_AUTHOR("Yanze Zhang "); +MODULE_DESCRIPTION("ZTE Ethernet Protocol Driver for RDMA"); + +static const struct auxiliary_device_id zxdh_auxiliary_id_table[] =3D { + { + .name =3D ZXDH_PF_NAME "." ZXDH_RDMA_DEV_NAME, + }, + {}, +}; + +MODULE_DEVICE_TABLE(auxiliary, zxdh_auxiliary_id_table); + +static int zxdh_probe(struct auxiliary_device *aux_dev, + const struct auxiliary_device_id *id) +{ + /* Placeholder: Real probe logic will be added in subsequent patches */ + dev_info(&aux_dev->dev, "ZTE DingHai RDMA device detected (stub)\n"); + return 0; +} + +static void zxdh_remove(struct auxiliary_device *aux_dev) +{ + dev_info(&aux_dev->dev, "ZTE DingHai RDMA device removed (stub)\n"); +} + +static struct auxiliary_driver zxdh_auxiliary_drv =3D { + .driver =3D { + .name =3D ZXDH_RDMA_DEV_NAME, + }, + .id_table =3D zxdh_auxiliary_id_table, + .probe =3D zxdh_probe, + .remove =3D zxdh_remove, +}; + +static int __init zrdma_module_init(void) +{ + return auxiliary_driver_register(&zxdh_auxiliary_drv); +} + +static void __exit zrdma_module_exit(void) +{ + auxiliary_driver_unregister(&zxdh_auxiliary_drv); +} + +module_init(zrdma_module_init); +module_exit(zrdma_module_exit); diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib= _user_ioctl_verbs.h index 21f86cc7bb1f..98dcea207357 100644 --- a/include/uapi/rdma/ib_user_ioctl_verbs.h +++ b/include/uapi/rdma/ib_user_ioctl_verbs.h @@ -257,6 +257,7 @@ enum rdma_driver_id { RDMA_DRIVER_ERDMA, RDMA_DRIVER_MANA, RDMA_DRIVER_IONIC, + RDMA_DRIVER_ZRDMA, }; =20 enum ib_uverbs_gid_type { --=20 2.27.0 From nobody Tue Sep 29 07:00:19 2026 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [160.30.148.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE506433E89; Tue, 11 Aug 2026 09:43:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=160.30.148.35 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441389; cv=none; b=rZaAgXO1u1v9bGCXV8oCD8kZ+Lw/kAu+Ro5iuraKF7xv7I6TuQuGBkJIfO64plR+faVtImEt/1KdTt7YWXnO19cKX9xF1RDT1ot8N3T4D2lwsw0GqwkJQuB7jLcwc5eBCPG6N2MzYG4WFVkpJiwLQAs16AN6QYUiYaIrwVlJLf0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441389; c=relaxed/simple; bh=7hqjGkskU8oJgvoBPOboEuZKZYjwQIaoOSRA226Y+jQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n44Ug2tN6KwXTjWSkEadkjTcSegT46ZEVhlbfarq15ivZYM7SNiHCyXIZ8KBB1bla4lL5E62wh9nUsfi3FGz4bzOcrDIL8lSXDVeK+9+vGa/bnwo1LnpYdpctW5bfOIRd4Jz0hwB11nSqxnH9Ew8Mnwp/LpWeLsmK0Zk5z1/Hik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn; spf=pass smtp.mailfrom=zte.com.cn; arc=none smtp.client-ip=160.30.148.35 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zte.com.cn Received: from mxct.zte.com.cn (unknown [192.168.251.13]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4hK69w0RwVz8XrrZ; Tue, 11 Aug 2026 17:43:00 +0800 (CST) Received: from mse-fl1.zte.com.cn (unknown [10.5.228.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxct.zte.com.cn (FangMail) with ESMTPS id 4hK69h4nG2z58fdg; Tue, 11 Aug 2026 17:42:48 +0800 (CST) Received: from szxl2zmapp05.zte.com.cn ([10.1.32.37]) by mse-fl1.zte.com.cn with SMTP id 67B9gcpi089530; Tue, 11 Aug 2026 17:42:38 +0800 (+08) (envelope-from zhang.yanze@zte.com.cn) Received: from localhost.localdomain (unknown [192.168.6.15]) by smtp (Zmail) with SMTP; Tue, 11 Aug 2026 17:42:40 +0800 X-Zmail-TransId: 3e816a7aee90013-44472 X-Zmail-LocalSMTP: 1 X-Zmail-RealSender: zhang.yanze@zte.com.cn From: Yanze Zhang To: jgg@ziepe.ca, leon@kernel.org, julianbraha@gmail.com, huangjunxian6@hisilicon.com, yanjun.zhu@linux.dev, lkp@intel.com Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, wei.quan@zte.com.cn, han.junyang@zte.com.cn, ran.ming@zte.com.cn, han.chengfei@zte.com.cn, zhang.yanze@zte.com.cn Subject: [PATCH rdma v5 2/7] RDMA/zrdma: Add core data structures and ABI definitions Date: Tue, 11 Aug 2026 17:37:35 +0800 Message-ID: <20260811093740.2607476-3-zhang.yanze@zte.com.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> References: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ZMAIL-USEORIGINALEMLTOOUTBOUND: 1 Content-Transfer-Encoding: quoted-printable X-MAIL: mse-fl1.zte.com.cn 67B9gcpi089530 X-TLS: YES X-ENVELOPE-SENDER: zhang.yanze@zte.com.cn X-SOURCE-IP: 192.168.251.13 unknown Tue, 11 Aug 2026 17:43:00 +0800 X-CLEAN: YES X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 6A7AEEA4.000/4hK69w0RwVz8XrrZ Content-Type: text/plain; charset="utf-8" Add fundamental data structures, hardware control structures, and user-space ABI definitions required by the driver. These headers provide the type definitions for the subsequent implementation patches. Signed-off-by: Yanze Zhang --- drivers/infiniband/hw/zrdma/zrdma_abi.h | 78 ++++++++++++++++++++++ drivers/infiniband/hw/zrdma/zrdma_ctrl.h | 22 ++++++ drivers/infiniband/hw/zrdma/zrdma_defs.h | 19 ++++++ drivers/infiniband/hw/zrdma/zrdma_type.h | 85 ++++++++++++++++++++++++ 4 files changed, 204 insertions(+) create mode 100644 drivers/infiniband/hw/zrdma/zrdma_abi.h create mode 100644 drivers/infiniband/hw/zrdma/zrdma_ctrl.h create mode 100644 drivers/infiniband/hw/zrdma/zrdma_defs.h create mode 100644 drivers/infiniband/hw/zrdma/zrdma_type.h diff --git a/drivers/infiniband/hw/zrdma/zrdma_abi.h b/drivers/infiniband/h= w/zrdma/zrdma_abi.h new file mode 100644 index 000000000000..d0b7f2f4ef5a --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_abi.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + * + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#ifndef ZRDMA_ABI_H +#define ZRDMA_ABI_H + +/* user-space whose last ABI ver is 5 */ +#define ZXDH_ABI_VER 5 +#define ZXDH_CONTEXT_VER_V1 5 +#define ZXDH_CONTEXT_VER_V2 6 +#define ZXDH_CONTEXT_VER_V3 7 + +enum zxdh_rdma_tool_flags { + ZXDH_QP_EXTEND_OP =3D 1 << 0, + ZXDH_CAPTURE =3D 1 << 1, + ZXDH_GET_HW_DATA =3D 1 << 2, + ZXDH_GET_HW_OBJECT_DATA =3D 1 << 3, + ZXDH_CHECK_HW_HEALTH =3D 1 << 4, + ZXDH_RDMA_TOOL_CFG_DEV_PARAM =3D 1 << 5, + ZXDH_RDMA_TOOL_SHOW_RES_MAP =3D 1 << 5, + ZXDH_RDMA_TOOL_READ_RAM =3D 1 << 6, + ZXDH_RDMA_TOOL_DEVX_MODIFY_CQ =3D 1 << 7, + ZXDH_RDMA_SET_CREDIT_CAP =3D 1 << 8, + ZXDH_RDMA_SWITCH =3D 1 << 9, + ZXDH_RDMA_TOOL_DEVX_EXT_MEM =3D 1 << 10, + ZXDH_RDMA_TOOL_PRIV_EXT =3D 1 << 11, +}; + +enum zxdh_db_addr_type { + ZXDH_DB_ADDR_PHY =3D 0, + ZXDH_DB_ADDR_BAR =3D 1, +}; + +struct zxdh_alloc_pd_resp { + __u32 pd_id; + __u8 rsvd[4]; +}; + +struct zxdh_alloc_ucontext_req { + __u32 rsvd32; + __u8 userspace_ver; + __u8 rsvd8[3]; +}; + +struct zxdh_alloc_ucontext_resp { + __u32 max_pds; + __u32 max_qps; + __u32 wq_size; /* size of the WQs (SQ+RQ) in the mmaped area */ + __u8 kernel_ver; + __u8 db_addr_type; + __u16 rdma_tool_flags; + __aligned_u64 feature_flags; + __aligned_u64 sq_db_mmap_key; + __aligned_u64 cq_db_mmap_key; + __u32 max_hw_wq_frags; + __u32 max_hw_read_sges; + __u32 max_hw_inline; + __u32 max_hw_rq_quanta; + __u32 max_hw_srq_quanta; + __u32 max_hw_wq_quanta; + __u32 max_hw_srq_wr; + __u32 min_hw_cq_size; + __u32 max_hw_cq_size; + __u16 max_hw_sq_chunk; + __u8 hw_rev; + __u8 chip_rev; + __aligned_u64 srq_db_mmap_key; + __aligned_u64 sq_db_bar_off; + __aligned_u64 cq_db_bar_off; + __aligned_u64 srq_db_bar_off; + __u32 srq_db_mmap_size; + __u16 rsv[6]; +}; + +#endif /* ZRDMA_ABI_H */ diff --git a/drivers/infiniband/hw/zrdma/zrdma_ctrl.h b/drivers/infiniband/= hw/zrdma/zrdma_ctrl.h new file mode 100644 index 000000000000..d25570683b61 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_ctrl.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + * + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#ifndef ZRDMA_CTRL_H +#define ZRDMA_CTRL_H + +struct zxdh_hw { + u8 __iomem *hw_addr; + struct device *device; +}; + +struct zxdh_sc_dev { + struct zxdh_hw *hw; + u32 max_ceqs; + u8 ep_id; + u8 driver_load; +}; + +#endif diff --git a/drivers/infiniband/hw/zrdma/zrdma_defs.h b/drivers/infiniband/= hw/zrdma/zrdma_defs.h new file mode 100644 index 000000000000..208f1f60bbae --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_defs.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + * + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#ifndef ZRDMA_DEFS_H +#define ZRDMA_DEFS_H + +#include +#include + +#define ZXDH_FEATURE_RTS_AE 1ULL +#define ZXDH_FEATURE_CQ_RESIZE 2ULL +#define ZXDH_FEATURE_64_BYTE_CQE 128ULL + +#define ZXDH_MAX_USER_PRIORITY 8 + +#endif /* ZRDMA_DEFS_H */ diff --git a/drivers/infiniband/hw/zrdma/zrdma_type.h b/drivers/infiniband/= hw/zrdma/zrdma_type.h new file mode 100644 index 000000000000..5ee9e17f35f7 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_type.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + * + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#ifndef ZRDMA_TYPE_H +#define ZRDMA_TYPE_H + +#include + +enum zxdh_rdma_reset_type { + ZXDH_RDMA_PFR, + ZXDH_RDMA_CORER, + ZXDH_RDMA_GLOBR, +}; + +enum zxdh_function_type { + ZXDH_FUNCTION_TYPE_PF, + ZXDH_FUNCTION_TYPE_VF, +}; + +struct zxdh_rdma_core_dev; + +struct zxdh_rdma_core_ops { + int (*request_reset)(struct zxdh_rdma_core_dev *cdev, + enum zxdh_rdma_reset_type reset_type); + int (*zxdh_common_func)(void *in_param, void *out_param, u32 opcode); +}; + +struct zxdh_rdma_ver_info { + u16 major; + u16 minor; + u64 support; /* 0~7:net_major,8~15:net_minor,16~23:rdma_minor,24~63:rscv = */ +}; + +enum zxdh_rdma_protocol { + ZXDH_RDMA_PROTOCOL_IWARP =3D BIT(0), + ZXDH_RDMA_PROTOCOL_ROCEV2 =3D BIT(1), +}; + +struct zxdh_qos_params { + u8 num_tc; +}; + +struct zxdh_rdma_core_dev { + struct pci_dev *pdev; + struct auxiliary_device *adev; + u8 __iomem *hw_addr; + int zdev_info_id; + struct zxdh_rdma_ver_info ver; + void *auxiliary_priv; + enum zxdh_function_type ftype; + u16 vport_id; + u16 slot_id; + enum zxdh_rdma_protocol rdma_protocol; + + struct zxdh_qos_params qos_info; + struct msix_entry *msix_entries; + u16 msix_count; + const struct zxdh_rdma_core_ops *ops; + void *dh_dev; +}; + +struct zxdh_rdma_if { + void *(*get_rdma_netdev)(void *dh_dev); +}; + +struct zxdh_auxiliary_dev { + struct auxiliary_device adev; + struct zxdh_rdma_core_dev *zxdh_info; + struct zxdh_rdma_if *rdma_ops; + void *ops; + void *parent; + s32 aux_id; + void *auxiliary_ops[18]; +}; + +struct zxdh_sc_pd { + struct zxdh_sc_dev *dev; + u32 pd_id; + int abi_ver; +}; + +#endif /* ZRDMA_TYPE_H */ --=20 2.27.0 From nobody Tue Sep 29 07:00:19 2026 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [160.30.148.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84F4F42B318; Tue, 11 Aug 2026 09:42:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=160.30.148.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441382; cv=none; b=DQ395evtZemm3tbJOlvTCuGtzVoUoqvDpiC7siS9JAfqG7CTB4BKHaGKaiqmTDgOzFNeY2ZidW2wO3OwXxYssVwfbUjAkLJF6BqxII3U/FGYjpl7LAlgZNLZuGIzqMvyMce2pLU0kqBYXqEh3OtsOwowWxk88hws+ZTT/SsJRkM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441382; c=relaxed/simple; bh=j+iZVfyAgHYZEeMLFHb0f1DbkJsZiyI5eVx1tobNjI0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jNdkk2iG1MYYPcUh/iOsXXEu63wP5BmvbkERi5xlYRw9UDlXndLv0HxiTYv2cW55QyM6vxAlAfQojnDWXaR8v97ynwxhYCKoA4NoHO0vWekqWX/0M4WdVoOPazGEpBZ1mP2++bcYyAhPWwUMWV8ZdDqTdMeBq8KE4PR1vAS/+40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn; spf=pass smtp.mailfrom=zte.com.cn; arc=none smtp.client-ip=160.30.148.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zte.com.cn Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4hK69k5nhxz4yjG1; Tue, 11 Aug 2026 17:42:50 +0800 (CST) Received: from szxlzmapp02.zte.com.cn ([10.5.231.79]) by mse-fl2.zte.com.cn with SMTP id 67B9gfua073901; Tue, 11 Aug 2026 17:42:41 +0800 (+08) (envelope-from zhang.yanze@zte.com.cn) Received: from localhost.localdomain (unknown [192.168.6.15]) by smtp (Zmail) with SMTP; Tue, 11 Aug 2026 17:42:44 +0800 X-Zmail-TransId: 3e816a7aee93013-44489 X-Zmail-LocalSMTP: 1 X-Zmail-RealSender: zhang.yanze@zte.com.cn From: Yanze Zhang To: jgg@ziepe.ca, leon@kernel.org, julianbraha@gmail.com, huangjunxian6@hisilicon.com, yanjun.zhu@linux.dev, lkp@intel.com Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, wei.quan@zte.com.cn, han.junyang@zte.com.cn, ran.ming@zte.com.cn, han.chengfei@zte.com.cn, zhang.yanze@zte.com.cn Subject: [PATCH rdma v5 3/7] RDMA/zrdma: Add internal headers and hardware register definitions Date: Tue, 11 Aug 2026 17:37:36 +0800 Message-ID: <20260811093740.2607476-4-zhang.yanze@zte.com.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> References: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ZMAIL-USEORIGINALEMLTOOUTBOUND: 1 Content-Transfer-Encoding: quoted-printable X-MAIL: mse-fl2.zte.com.cn 67B9gfua073901 X-TLS: YES X-ENVELOPE-SENDER: zhang.yanze@zte.com.cn X-SOURCE-IP: 10.5.228.133 unknown Tue, 11 Aug 2026 17:42:50 +0800 X-CLEAN: YES X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 6A7AEE9A.001/4hK69k5nhxz4yjG1 Content-Type: text/plain; charset="utf-8" Add internal header files for main driver logic and hardware registers. These headers define the structures and constants used by the subsequent implementation patches. Signed-off-by: Yanze Zhang --- drivers/infiniband/hw/zrdma/zrdma_hw.h | 21 +++ drivers/infiniband/hw/zrdma/zrdma_main.h | 214 +++++++++++++++++++++++ 2 files changed, 235 insertions(+) create mode 100644 drivers/infiniband/hw/zrdma/zrdma_hw.h create mode 100644 drivers/infiniband/hw/zrdma/zrdma_main.h diff --git a/drivers/infiniband/hw/zrdma/zrdma_hw.h b/drivers/infiniband/hw= /zrdma/zrdma_hw.h new file mode 100644 index 000000000000..16c5041ac075 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_hw.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + * + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#ifndef ZRDMA_HW_H +#define ZRDMA_HW_H +#include "zrdma_main.h" + +#define ZXDH_MAX_CQE 0x200000 +#define ZXDH_MAX_QP 0x4000 +#define ZXDH_MAX_MR 0x10000 +#define ZXDH_MAX_CQ 0x4000 +#define ZXDH_MAX_SRQ 0x2000 +#define ZXDH_MAX_PD 0x100000 +#define ZXDH_MAX_AH 0x10000 +#define ZXDH_MAX_MCG 0x4000 +#define ZXDH_MAX_8K_IDX 0x1c0 + +#endif /* ZRDMA_HW_H */ diff --git a/drivers/infiniband/hw/zrdma/zrdma_main.h b/drivers/infiniband/= hw/zrdma/zrdma_main.h new file mode 100644 index 000000000000..4333df1d7755 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_main.h @@ -0,0 +1,214 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + * + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#ifndef ZRDMA_MAIN_H +#define ZRDMA_MAIN_H + +#include +#include +#include +#include +#include +#include +#include + +#include "zrdma_type.h" +#include "zrdma_ctrl.h" +#include "zrdma_abi.h" +#include "zrdma_defs.h" + +#define ZXDH_PF_NAME "dinghai10e" +#define ZXDH_RDMA_DEV_NAME "rdma_aux" + +#define ZXDH_FUNC_TYPE(vport_id) (((vport_id) >> 11) & 0x1) +#define ZXDH_PF_ID(vport_id) (((vport_id) >> 8) & 0x7) +#define ZXDH_EP_ID(vport_id) (((vport_id) >> 12) & 0x7) + +#define PCI_SBDF_SAFE_LEN 64 +#define ZXDH_RST_TIMEOUT_HZ 4 + +#define ZXDH_RDMA_COMMON_FUNC_CAP 1 +#define ZXDH_RDMA_COMM_FUNC BIT_ULL(24) + +extern struct list_head zxdh_handlers; +extern spinlock_t zxdh_handler_lock; + +struct zxdh_pci_f; + +struct zxdh_rdma_node { + char name[IB_DEVICE_NAME_MAX]; + char sbdf[PCI_SBDF_SAFE_LEN]; + struct list_head list; +}; + +enum zxdh_func_type { + ZXDH_FUNC_NUM_REQUIRE =3D 0, + ZXDH_FUNC_NP_MAC =3D 1, + ZXDH_FUNC_IRQ_REQUEST =3D 2, + ZXDH_FUNC_IRQ_FREE =3D 3, + ZXDH_FUNC_PMTU_INFO_ADD =3D 4, + ZXDH_FUNC_PMTU_INFO_DEL =3D 5, + ZXDH_FUNC_QP_INFO_DEL_WX =3D 6, + ZXDH_FUNC_NETDEV_SPEED_GET =3D 7, + ZXDH_FUNC_SEARCH_MAC_FROM_FW =3D 8, + ZXDH_COMM_FUNC_GET_TRUST_TYPE =3D 9, + ZXDH_FUNC_VF_VLAN_GET =3D 10, + ZXDH_FUNC_NUM_MAX, +}; + +enum init_completion_state { + INVALID_STATE =3D 0, + INITIAL_STATE, + CQP_CREATED, + SMMU_PAGETABLE_INITIALIZED, + DATA_CAP_CREATED, + HMC_OBJS_CREATED, + HW_RSRC_INITIALIZED, + CQP_QP_CREATED, + AEQ_CREATED, + CCQ_CREATED, + CEQ0_CREATED, + CEQS_CREATED, + PBLE_CHUNK_MEM, +}; + +enum zxdh_rdma_vers { + ZXDH_GEN_RSVD, + ZXDH_GEN_1, + ZXDH_GEN_2, +}; + +struct zxdh_handler { + struct list_head list; + struct zxdh_device *zdev; +}; + +struct zxdh_gen_ops { + void (*request_reset)(struct zxdh_pci_f *rf); + int (*zxdh_common_func)(void *in_param, void *out_param, u32 opcode); +}; + +struct zxdh_pci_f { + u8 ftype : 1; + u8 *mem_rsrc; + u8 rdma_ver; + u8 rst_to; + u8 pf_id; + u8 ep_id; + u32 msix_count; + u32 max_mr; + u32 max_qp; + u32 max_cq; + u32 max_ah; + u32 max_mcg; + u32 max_pd; + u32 next_pd; + u32 max_cqe; + u32 max_srq; + u32 max_pri; + u16 max_8k_idx; + u32 *qp_cnt_8k_idxs; + u32 ceqs_count; + + unsigned long *allocated_qps; + unsigned long *allocated_cqs; + unsigned long *allocated_mrs; + unsigned long *allocated_pds; + unsigned long *allocated_mcgs; + unsigned long *allocated_ahs; + unsigned long *allocated_srqs; + unsigned long *allocated_pris; + unsigned long *allocated_8k_idx; + + struct zxdh_sc_dev sc_dev; + struct pci_dev *pcidev; + struct zxdh_rdma_core_dev *zdev_info; + struct zxdh_hw hw; + + spinlock_t rsrc_lock; /* protects resource allocation arrays */ + spinlock_t qptable_lock; /* protects qp_table access */ + spinlock_t cqtable_lock; /* protects cq_table access */ + struct zxdh_qp **qp_table; + struct zxdh_cq **cq_table; + spinlock_t srqtable_lock; /* protects srq_table access */ + struct zxdh_srq **srq_table; + + struct msix_entry *msix_entries; + struct zxdh_gen_ops gen_ops; + struct zxdh_device *zdev; + u16 pcie_id; + u16 qp_index; + u32 srq_mem_size; + bool drv_np_cap; + bool net_irq_cap; + int common_func_num_max; + void *dh_dev; +}; + +struct zxdh_device { + struct ib_device ibdev; + struct zxdh_pci_f *rf; + struct net_device *netdev; + struct net_device *source_netdev; + struct zxdh_handler *hdl; + struct list_head ah_list; + struct mutex ah_list_lock; /* protects ah_list operations */ + u8 ibdev_status; + u8 roce_mode : 1; + enum init_completion_state init_state; + u32 netdev_speed; + struct zxdh_auxiliary_dev *zxdh_adev; +}; + +static inline struct zxdh_device *to_zdev(struct ib_device *ibdev) +{ + return container_of(ibdev, struct zxdh_device, ibdev); +} + +static inline int zxdh_alloc_rsrc(struct zxdh_pci_f *rf, + unsigned long *rsrc_array, u32 max_rsrc, + u32 *req_rsrc_num, u32 *next) +{ + unsigned long flags; + u32 rsrc_num; + + spin_lock_irqsave(&rf->rsrc_lock, flags); + rsrc_num =3D find_next_zero_bit(rsrc_array, max_rsrc, *next); + if (rsrc_num >=3D max_rsrc) { + rsrc_num =3D find_first_zero_bit(rsrc_array, max_rsrc); + if (rsrc_num >=3D max_rsrc) { + spin_unlock_irqrestore(&rf->rsrc_lock, flags); + pr_err("zrdma: resource [%d] allocation failed\n", + rsrc_num); + return -EOVERFLOW; + } + } + __set_bit(rsrc_num, rsrc_array); + *next =3D rsrc_num + 1; + if (*next =3D=3D max_rsrc) + *next =3D 0; + *req_rsrc_num =3D rsrc_num; + spin_unlock_irqrestore(&rf->rsrc_lock, flags); + + return 0; +} + +static inline void zxdh_free_rsrc(struct zxdh_pci_f *rf, + unsigned long *rsrc_array, u32 rsrc_num) +{ + unsigned long flags; + + spin_lock_irqsave(&rf->rsrc_lock, flags); + __clear_bit(rsrc_num, rsrc_array); + spin_unlock_irqrestore(&rf->rsrc_lock, flags); +} + +int zxdh_get_del_rdma_name(const char *sbdf, char *name); +int zxdh_ctrl_init_hw(struct zxdh_pci_f *rf); +int zxdh_ib_register_device(struct zxdh_device *zdev); +void zxdh_del_handler(struct zxdh_handler *hdl); + +#endif /* ZRDMA_MAIN_H */ --=20 2.27.0 From nobody Tue Sep 29 07:00:19 2026 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [160.30.148.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A69BB430CD0; Tue, 11 Aug 2026 09:43:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=160.30.148.35 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441386; cv=none; b=kIEJP5cDz/uXRKkLGI/K2klORR2JNVeiEW9/anpkHz/ck8pdiqVIolnry/c+DTwb3EG86Eb87RfggSmx55Pr5z2HPAyMcv5f/CU4ujtWkuLfgd++2xdZmh6xZrXeE/mAckdrX6LQtWT/BzC4wJ0lvkz43xEV/IBg9A29R/aD9DM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441386; c=relaxed/simple; bh=exbOpsXEOs28Aqyceof91u9IE45wAmkdZnqj40Npnik=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CZgZDDPZYFK+pIIQ4daus5zWFTcbplwMFgvW7BXupLm0lcPiywpuS0PLgW+PlJ6+TpVxeGgcI5RMjW8atbyu8dVW3kdhq3i/EKlT0aAZLOnlQB1WQHeXSVE2Bt8GCSUWSYbjqyFPAZzp4cCFaDru6qFI2n+wuBhje2J90CGL5FM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn; spf=pass smtp.mailfrom=zte.com.cn; arc=none smtp.client-ip=160.30.148.35 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zte.com.cn Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4hK69r17yHz8Xrrh; Tue, 11 Aug 2026 17:42:56 +0800 (CST) Received: from szxlzmapp02.zte.com.cn ([10.5.231.79]) by mse-fl2.zte.com.cn with SMTP id 67B9ggPD073903; Tue, 11 Aug 2026 17:42:42 +0800 (+08) (envelope-from zhang.yanze@zte.com.cn) Received: from localhost.localdomain (unknown [192.168.6.15]) by smtp (Zmail) with SMTP; Tue, 11 Aug 2026 17:42:44 +0800 X-Zmail-TransId: 3e816a7aee94013-44494 X-Zmail-LocalSMTP: 1 X-Zmail-RealSender: zhang.yanze@zte.com.cn From: Yanze Zhang To: jgg@ziepe.ca, leon@kernel.org, julianbraha@gmail.com, huangjunxian6@hisilicon.com, yanjun.zhu@linux.dev, lkp@intel.com Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, wei.quan@zte.com.cn, han.junyang@zte.com.cn, ran.ming@zte.com.cn, han.chengfei@zte.com.cn, zhang.yanze@zte.com.cn Subject: [PATCH rdma v5 4/7] RDMA/zrdma: Add hardware resource initialization and verbs skeleton Date: Tue, 11 Aug 2026 17:37:37 +0800 Message-ID: <20260811093740.2607476-5-zhang.yanze@zte.com.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> References: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ZMAIL-USEORIGINALEMLTOOUTBOUND: 1 Content-Transfer-Encoding: quoted-printable X-MAIL: mse-fl2.zte.com.cn 67B9ggPD073903 X-TLS: YES X-ENVELOPE-SENDER: zhang.yanze@zte.com.cn X-SOURCE-IP: 10.5.228.133 unknown Tue, 11 Aug 2026 17:42:56 +0800 X-CLEAN: YES X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 6A7AEEA0.001/4hK69r17yHz8Xrrh Content-Type: text/plain; charset="utf-8" Implement hardware resource initialization logic and provide a minimal but functional verbs layer skeleton. This includes basic device registration with an empty ops structure to satisfy the requirement that the driver must be functional and have a valid driver ID. Signed-off-by: Yanze Zhang --- drivers/infiniband/hw/zrdma/Makefile | 5 +- drivers/infiniband/hw/zrdma/zrdma_hw.c | 103 +++++++++++++++++ drivers/infiniband/hw/zrdma/zrdma_verbs.c | 41 +++++++ drivers/infiniband/hw/zrdma/zrdma_verbs.h | 128 ++++++++++++++++++++++ 4 files changed, 276 insertions(+), 1 deletion(-) create mode 100644 drivers/infiniband/hw/zrdma/zrdma_hw.c create mode 100644 drivers/infiniband/hw/zrdma/zrdma_verbs.c create mode 100644 drivers/infiniband/hw/zrdma/zrdma_verbs.h diff --git a/drivers/infiniband/hw/zrdma/Makefile b/drivers/infiniband/hw/z= rdma/Makefile index 2ddf50d1fd20..88be835f5c5c 100644 --- a/drivers/infiniband/hw/zrdma/Makefile +++ b/drivers/infiniband/hw/zrdma/Makefile @@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB =20 obj-$(CONFIG_INFINIBAND_ZRDMA) +=3D zrdma.o -zrdma-y :=3D zrdma_main.o +zrdma-y :=3D \ + zrdma_main.o \ + zrdma_hw.o \ + zrdma_verbs.o \ No newline at end of file diff --git a/drivers/infiniband/hw/zrdma/zrdma_hw.c b/drivers/infiniband/hw= /zrdma/zrdma_hw.c new file mode 100644 index 000000000000..c2432531f238 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_hw.c @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#include +#include "zrdma_hw.h" +#include "zrdma_ctrl.h" +#include "zrdma_main.h" + +static void zxdh_set_hw_rsrc(struct zxdh_pci_f *rf) +{ + rf->allocated_srqs =3D (void *)(rf->mem_rsrc); + rf->allocated_qps =3D &rf->allocated_srqs[BITS_TO_LONGS(rf->max_srq)]; + rf->allocated_cqs =3D &rf->allocated_qps[BITS_TO_LONGS(rf->max_qp)]; + rf->allocated_mrs =3D &rf->allocated_cqs[BITS_TO_LONGS(rf->max_cq)]; + rf->allocated_pds =3D &rf->allocated_mrs[BITS_TO_LONGS(rf->max_mr)]; + rf->allocated_ahs =3D &rf->allocated_pds[BITS_TO_LONGS(rf->max_pd)]; + rf->allocated_mcgs =3D &rf->allocated_ahs[BITS_TO_LONGS(rf->max_ah)]; + rf->allocated_pris =3D &rf->allocated_mcgs[BITS_TO_LONGS(rf->max_mcg)]; + rf->allocated_8k_idx =3D &rf->allocated_pris[BITS_TO_LONGS(rf->max_pri)]; + rf->qp_table =3D (struct zxdh_qp **) + (&rf->allocated_8k_idx[BITS_TO_LONGS(rf->max_8k_idx)]); + rf->cq_table =3D (struct zxdh_cq **)(&rf->qp_table[rf->max_qp]); + rf->srq_table =3D (struct zxdh_srq **)(&rf->cq_table[rf->max_cq]); + + spin_lock_init(&rf->rsrc_lock); + spin_lock_init(&rf->qptable_lock); + spin_lock_init(&rf->cqtable_lock); + spin_lock_init(&rf->srqtable_lock); +} + +static u32 zxdh_calc_mem_rsrc_size(struct zxdh_pci_f *rf) +{ + u32 rsrc_size; + + rsrc_size =3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_srq); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_qp); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_mr); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_cq); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_pd); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_ah); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_mcg); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_pri); + rsrc_size +=3D sizeof(unsigned long) * BITS_TO_LONGS(rf->max_8k_idx); + rsrc_size +=3D sizeof(struct zxdh_qp **) * rf->max_qp; + rsrc_size +=3D sizeof(struct zxdh_cq **) * rf->max_cq; + rsrc_size +=3D sizeof(struct zxdh_srq **) * rf->max_srq; + + return rsrc_size; +} + +static int zxdh_initialize_hw_rsrc(struct zxdh_pci_f *rf) +{ + u32 rsrc_size; + int ret; + + rf->max_cqe =3D ZXDH_MAX_CQE; + rf->max_qp =3D ZXDH_MAX_QP; + rf->max_mr =3D ZXDH_MAX_MR; + rf->max_cq =3D ZXDH_MAX_CQ; + rf->max_srq =3D ZXDH_MAX_SRQ; + rf->max_pd =3D ZXDH_MAX_PD; + rf->max_ah =3D ZXDH_MAX_AH; + rf->max_mcg =3D ZXDH_MAX_MCG; + rf->max_pri =3D ZXDH_MAX_USER_PRIORITY; + rf->max_8k_idx =3D ZXDH_MAX_8K_IDX; + + rf->qp_cnt_8k_idxs =3D vzalloc(rf->max_8k_idx * sizeof(u32)); + if (!rf->qp_cnt_8k_idxs) { + ret =3D -ENOMEM; + goto mem_8k_qp_cnt_vmalloc_fail; + } + + rsrc_size =3D zxdh_calc_mem_rsrc_size(rf); + rf->mem_rsrc =3D vzalloc(rsrc_size); + if (!rf->mem_rsrc) { + ret =3D -ENOMEM; + goto mem_rsrc_vmalloc_fail; + } + + zxdh_set_hw_rsrc(rf); + + set_bit(0, rf->allocated_mrs); + set_bit(1, rf->allocated_mrs); + set_bit(0, rf->allocated_pds); + set_bit(0, rf->allocated_qps); + set_bit(0, rf->allocated_ahs); + set_bit(0, rf->allocated_mcgs); + + return 0; + +mem_rsrc_vmalloc_fail: + vfree(rf->qp_cnt_8k_idxs); +mem_8k_qp_cnt_vmalloc_fail: + return ret; +} + +int zxdh_ctrl_init_hw(struct zxdh_pci_f *rf) +{ + return zxdh_initialize_hw_rsrc(rf); +} diff --git a/drivers/infiniband/hw/zrdma/zrdma_verbs.c b/drivers/infiniband= /hw/zrdma/zrdma_verbs.c new file mode 100644 index 000000000000..6db536c94e01 --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_verbs.c @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#include "zrdma_main.h" +#include "zrdma_verbs.h" +#include "zrdma_ctrl.h" + +static const struct ib_device_ops zxdh_dev_ops =3D { + .owner =3D THIS_MODULE, + .driver_id =3D RDMA_DRIVER_ZRDMA, + .uverbs_abi_ver =3D ZXDH_ABI_VER, +}; + +static int zxdh_init_rdma_device(struct zxdh_device *zdev) +{ + zdev->ibdev.phys_port_cnt =3D 1; + zdev->ibdev.num_comp_vectors =3D zdev->rf->ceqs_count; + zdev->ibdev.dev.parent =3D &zdev->rf->pcidev->dev; + + ib_set_device_ops(&zdev->ibdev, &zxdh_dev_ops); + + return 0; +} + +int zxdh_ib_register_device(struct zxdh_device *zdev) +{ + int ret; + + ret =3D zxdh_init_rdma_device(zdev); + if (ret) + return ret; + + ret =3D ib_register_device(&zdev->ibdev, "zrdma%d", zdev->rf->hw.device); + if (ret) + pr_err("zrdma: Register RDMA device fail\n"); + + return ret; +} diff --git a/drivers/infiniband/hw/zrdma/zrdma_verbs.h b/drivers/infiniband= /hw/zrdma/zrdma_verbs.h new file mode 100644 index 000000000000..7af678c2becb --- /dev/null +++ b/drivers/infiniband/hw/zrdma/zrdma_verbs.h @@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB + * + * ZTE DingHai Rdma driver + * Copyright (c) 2022-2026, ZTE Corporation + */ + +#ifndef ZRDMA_VERBS_H +#define ZRDMA_VERBS_H + +#include +#include + +#define CHIP_VERSION 1 +#define ZXDH_PAGE_OFFSET_NUM (PAGE_SIZE - 1) +#define MAX_HW_WQ_FRAGS 15 +#define MAX_HW_READ_SGES 15 +#define MAX_HW_INLINE 217 +#define MAX_HW_SRQ_WR 15 +#define MAX_HW_RQ_QUANTA 15 +#define MAX_HW_SRQ_QUANTA 15 +#define MAX_HW_WQ_QUANTA 15 +#define MAX_HW_SQ_CHUNK 0x8000 +#define MAX_HW_CQ_SIZE 0x200000 +#define MIN_HW_CQ_SIZE 4 + +#define ZXDH_MTU_HEADER_RSV 102 + +#define ZXDH_PKEY_TBL_SZ 1 +#define ZXDH_DEFAULT_PKEY 0xFFFF + +#define SQ_DB_BAR_OFF 0x4c000 +#define CQ_DB_BAR_OFF 0x43000 +#define SRQ_DB_MMAP_SIZE 0x10000 +#define SRQ_DB_BAR_OFF 0x62000 +#define SRQ_DB_OFFSET 0 + +#define MAX_MR_SIZE 0x200000000000 +#define MAX_QP 0x3fff +#define MAX_QP_WR 0x1000 +#define MAX_CQ 0x3fff +#define MAX_CQE 0x200000 +#define MAX_MR 0xfffe +#define MAX_MW 0xfffe +#define MAX_PD 0xfffff +#define MAX_SGE_RD 0xf +#define MAX_QP_RD_ATOM 0x10 +#define MAX_RES_RD_ATOM 0x3fff0 +#define MAX_QP_INIT_RD_ATOM 0x10 +#define MAX_SRQ 0x200 +#define MAX_SRQ_WR 0x7fff +#define MAX_SRQ_SGE 0xf +#define LOCAL_CA_ACK_DELAY 16 +#define HCA_CORE_CLOCK (1000 * 1000UL) +#define MAX_WQ_TYPE_RQ 0x3fff +#define MAX_AH 0x10000 +#define MAX_FAST_REG_PAGE_LIST_LEN 0x200 +#define MAX_CQ_MODERATION_COUNT 0xffff +#define MAX_CQ_MODERATION_PERIOD 0x1cc +#define TIMESTAMP_MASK 0x1ffff +#define MAX_MSG_SZ 0x100000 + +enum zxdh_mmap_flag { + ZXDH_MMAP_IO_NC, + ZXDH_MMAP_IO_WC, + ZXDH_MMAP_PFN, + ZXDH_MMAP_HMC, + ZXDH_MMAP_IO_NC_BY_SIZE, +}; + +struct zxdh_ucontext { + struct ib_ucontext ibucontext; + struct zxdh_device *zdev; + struct rdma_user_mmap_entry *sq_db_mmap_entry; + struct rdma_user_mmap_entry *cq_db_mmap_entry; + struct rdma_user_mmap_entry *srq_db_mmap_entry; + struct list_head cq_reg_mem_list; + spinlock_t cq_reg_mem_list_lock; /* protect CQ memory list */ + struct list_head qp_reg_mem_list; + spinlock_t qp_reg_mem_list_lock; /* protect QP memory list */ + struct list_head srq_reg_mem_list; + spinlock_t srq_reg_mem_list_lock; /* protect SRQ memory list */ + int abi_ver; +}; + +struct zxdh_pd { + struct ib_pd ibpd; + struct zxdh_sc_pd sc_pd; +}; + +struct zxdh_user_mmap_entry { + struct rdma_user_mmap_entry rdma_entry; + u64 bar_offset; + u8 mmap_flag; +}; + +static inline struct zxdh_pd *to_zpd(struct ib_pd *ibpd) +{ + return container_of(ibpd, struct zxdh_pd, ibpd); +} + +static inline struct zxdh_ucontext *to_ucontext(struct ib_ucontext *ibucon= text) +{ + return container_of(ibucontext, struct zxdh_ucontext, ibucontext); +} + +static inline struct zxdh_user_mmap_entry * +to_zxdh_mmap_entry(struct rdma_user_mmap_entry *rdma_entry) +{ + return container_of(rdma_entry, struct zxdh_user_mmap_entry, + rdma_entry); +} + +static inline enum ib_mtu zxdh_mtu_int_to_enum(int mtu) +{ + mtu =3D mtu - ZXDH_MTU_HEADER_RSV; + if (mtu >=3D 4096) + return IB_MTU_4096; + else if (mtu >=3D 2048) + return IB_MTU_2048; + else if (mtu >=3D 1024) + return IB_MTU_1024; + else if (mtu >=3D 512) + return IB_MTU_512; + else + return IB_MTU_256; +} + +#endif /* ZRDMA_VERBS_H */ --=20 2.27.0 From nobody Tue Sep 29 07:00:19 2026 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [160.30.148.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95C26430CC6; Tue, 11 Aug 2026 09:43:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=160.30.148.35 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441385; cv=none; b=RpBoLx2XzZO4NBx2xy37q19aNy2y17wvnCnABOqnp6TG62mrKxfh7PzjRP7NOWh2syDIJ/W02Dl8gaxatxQEXPUBIMSdTU0SWXJXDLDhAF+fgGN2VTwFJMsDbQ8NOZqrikKYP2Inietd8ciAjmw2xygelw4/VUYksBjw4l7M6rE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441385; c=relaxed/simple; bh=Zj8jkL4ub/Q9AuvAUCux4beVmXq4h282KwUMZVI3vQU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z5ONr35Jak+z4YlxPwXPcWBfPW+d27JVsQzUnDLgXsV5rBy+b05xYmCNkURW76ma343jI2O1WckKiNx56J2+d/kscFpLyC66z7jYCLvJPZz0/GPtWclc4s974m4ezs975CZj5fcFoRfxNCMe003w/72U64mUrL7WXsVr4zi2YxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn; spf=pass smtp.mailfrom=zte.com.cn; arc=none smtp.client-ip=160.30.148.35 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zte.com.cn Received: from mse-fl1.zte.com.cn (unknown [10.5.228.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4hK69r0j3Pz8Xrrd; Tue, 11 Aug 2026 17:42:56 +0800 (CST) Received: from szxl2zmapp05.zte.com.cn ([10.1.32.37]) by mse-fl1.zte.com.cn with SMTP id 67B9ggub089568; Tue, 11 Aug 2026 17:42:42 +0800 (+08) (envelope-from zhang.yanze@zte.com.cn) Received: from localhost.localdomain (unknown [192.168.6.15]) by smtp (Zmail) with SMTP; Tue, 11 Aug 2026 17:42:45 +0800 X-Zmail-TransId: 3e816a7aee94013-4449e X-Zmail-LocalSMTP: 1 X-Zmail-RealSender: zhang.yanze@zte.com.cn From: Yanze Zhang To: jgg@ziepe.ca, leon@kernel.org, julianbraha@gmail.com, huangjunxian6@hisilicon.com, yanjun.zhu@linux.dev, lkp@intel.com Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, wei.quan@zte.com.cn, han.junyang@zte.com.cn, ran.ming@zte.com.cn, han.chengfei@zte.com.cn, zhang.yanze@zte.com.cn Subject: [PATCH rdma v5 5/7] RDMA/zrdma: Add device probe and remove logic Date: Tue, 11 Aug 2026 17:37:38 +0800 Message-ID: <20260811093740.2607476-6-zhang.yanze@zte.com.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> References: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ZMAIL-USEORIGINALEMLTOOUTBOUND: 1 Content-Transfer-Encoding: quoted-printable X-MAIL: mse-fl1.zte.com.cn 67B9ggub089568 X-TLS: YES X-ENVELOPE-SENDER: zhang.yanze@zte.com.cn X-SOURCE-IP: 10.5.228.132 unknown Tue, 11 Aug 2026 17:42:56 +0800 X-CLEAN: YES X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 6A7AEEA0.000/4hK69r0j3Pz8Xrrd Content-Type: text/plain; charset="utf-8" Implement the auxiliary bus probe and remove functions, including device initialization, hardware resource setup, and IB device registration. This replaces the stub probe from the initial skeleton patch. Signed-off-by: Yanze Zhang --- drivers/infiniband/hw/zrdma/zrdma_main.c | 251 +++++++++++++++++++++-- 1 file changed, 237 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/hw/zrdma/zrdma_main.c b/drivers/infiniband/= hw/zrdma/zrdma_main.c index 7954d86573e2..c181736a9641 100644 --- a/drivers/infiniband/hw/zrdma/zrdma_main.c +++ b/drivers/infiniband/hw/zrdma/zrdma_main.c @@ -6,36 +6,246 @@ =20 #include #include +#include +#include +#include +#include =20 -#define ZXDH_PF_NAME "dinghai10e" -#define ZXDH_RDMA_DEV_NAME "rdma_aux" +#include "zrdma_main.h" =20 -MODULE_LICENSE("Dual BSD/GPL"); +MODULE_ALIAS("zrdma"); MODULE_AUTHOR("Yanze Zhang "); MODULE_DESCRIPTION("ZTE Ethernet Protocol Driver for RDMA"); +MODULE_LICENSE("Dual BSD/GPL"); =20 -static const struct auxiliary_device_id zxdh_auxiliary_id_table[] =3D { - { - .name =3D ZXDH_PF_NAME "." ZXDH_RDMA_DEV_NAME, - }, - {}, -}; +LIST_HEAD(zxdh_handlers); +DEFINE_SPINLOCK(zxdh_handler_lock); =20 -MODULE_DEVICE_TABLE(auxiliary, zxdh_auxiliary_id_table); +static struct mutex rdma_name_lock; +static struct list_head zxdh_rdma_list; + +static void zxdh_add_handler(struct zxdh_handler *hdl) +{ + unsigned long flags; + + spin_lock_irqsave(&zxdh_handler_lock, flags); + list_add(&hdl->list, &zxdh_handlers); + spin_unlock_irqrestore(&zxdh_handler_lock, flags); +} + +void zxdh_del_handler(struct zxdh_handler *hdl) +{ + unsigned long flags; + + spin_lock_irqsave(&zxdh_handler_lock, flags); + list_del(&hdl->list); + spin_unlock_irqrestore(&zxdh_handler_lock, flags); +} + +int zxdh_get_del_rdma_name(const char *sbdf, char *name) +{ + struct zxdh_rdma_node *pos, *tmp; + u8 len =3D 0; + + if (!sbdf || !name) + return -EINVAL; + + if (strlen(name) > IB_DEVICE_NAME_MAX) { + pr_err("zrdma: name len out of range\n"); + return -EINVAL; + } + + mutex_lock(&rdma_name_lock); + list_for_each_entry_safe(pos, tmp, &zxdh_rdma_list, list) { + len =3D strlen(pos->sbdf); + if (strlen(sbdf) =3D=3D len) { + if (strncasecmp(sbdf, pos->sbdf, len) =3D=3D 0) { + snprintf(name, IB_DEVICE_NAME_MAX, "%s", + pos->name); + list_del(&pos->list); + kfree(pos); + mutex_unlock(&rdma_name_lock); + return 0; + } + } + } + mutex_unlock(&rdma_name_lock); + return -ENOENT; +} + +static void zxdh_get_net_irq_cap(struct zxdh_pci_f *rf) +{ + u32 opcode =3D 0; + + if (rf->gen_ops.zxdh_common_func) { + opcode =3D rf->gen_ops.zxdh_common_func(NULL, NULL, + ZXDH_FUNC_NUM_REQUIRE); + if (opcode > ZXDH_FUNC_IRQ_FREE && opcode !=3D 0xFF) + rf->net_irq_cap =3D true; + } +} + +static void zxdh_get_common_func_num_max(struct zxdh_pci_f *rf) +{ + u32 num_max =3D 0; + + if (rf->gen_ops.zxdh_common_func) { + num_max =3D rf->gen_ops.zxdh_common_func(NULL, NULL, + ZXDH_FUNC_NUM_REQUIRE); + if (num_max !=3D 0xFF) { + rf->common_func_num_max =3D num_max; + return; + } + } + rf->common_func_num_max =3D -1; +} + +static void zxdh_fill_device_info(struct zxdh_device *zdev, + struct zxdh_rdma_core_dev *zdev_info) +{ + struct zxdh_pci_f *rf =3D zdev->rf; + + rf->ftype =3D ZXDH_FUNC_TYPE(zdev_info->vport_id); + rf->pf_id =3D ZXDH_PF_ID(zdev_info->vport_id); + rf->sc_dev.ep_id =3D ZXDH_EP_ID(zdev_info->vport_id); + rf->ep_id =3D rf->sc_dev.ep_id; + rf->sc_dev.driver_load =3D true; + + rf->zdev_info =3D zdev_info; + rf->pcidev =3D zdev_info->pdev; + + rf->msix_count =3D zdev_info->msix_count; + rf->msix_entries =3D zdev_info->msix_entries; + rf->sc_dev.max_ceqs =3D (rf->msix_count - 1); + rf->rdma_ver =3D ZXDH_GEN_2; + rf->rst_to =3D ZXDH_RST_TIMEOUT_HZ; + rf->qp_index =3D 0; + rf->gen_ops.zxdh_common_func =3D NULL; + rf->net_irq_cap =3D false; + rf->dh_dev =3D NULL; + rf->drv_np_cap =3D + (bool)FIELD_GET(ZXDH_RDMA_COMM_FUNC, zdev_info->ver.support); + if (rf->drv_np_cap =3D=3D ZXDH_RDMA_COMMON_FUNC_CAP) { + if (zdev_info->ops && zdev_info->ops->zxdh_common_func) + rf->gen_ops.zxdh_common_func =3D + zdev_info->ops->zxdh_common_func; + } + + zxdh_get_net_irq_cap(rf); + zxdh_get_common_func_num_max(rf); + + rf->zdev =3D zdev; + + INIT_LIST_HEAD(&zdev->ah_list); + mutex_init(&zdev->ah_list_lock); + zdev->init_state =3D INITIAL_STATE; + zdev->roce_mode =3D true; +} =20 static int zxdh_probe(struct auxiliary_device *aux_dev, const struct auxiliary_device_id *id) { - /* Placeholder: Real probe logic will be added in subsequent patches */ - dev_info(&aux_dev->dev, "ZTE DingHai RDMA device detected (stub)\n"); + struct zxdh_auxiliary_dev *zxdh_adev =3D + container_of(aux_dev, struct zxdh_auxiliary_dev, adev); + struct zxdh_handler *hdl; + struct zxdh_device *zdev; + struct zxdh_pci_f *rf; + struct net_device *netdev; + int err; + + if (!zxdh_adev->rdma_ops || !zxdh_adev->rdma_ops->get_rdma_netdev) { + pr_err("zrdma: rdma_ops or get_rdma_netdev is NULL\n"); + return -EINVAL; + } + + netdev =3D zxdh_adev->rdma_ops->get_rdma_netdev(zxdh_adev->parent); + if (!netdev) { + pr_err("zrdma: failed to get netdev\n"); + return -ENODEV; + } + + zdev =3D ib_alloc_device(zxdh_device, ibdev); + if (!zdev) + return -ENOMEM; + + zdev->zxdh_adev =3D zxdh_adev; + zdev->netdev =3D netdev; + + zdev->rf =3D kzalloc_obj(*zdev->rf); + if (!zdev->rf) { + ib_dealloc_device(&zdev->ibdev); + return -ENOMEM; + } + + zxdh_fill_device_info(zdev, zxdh_adev->zxdh_info); + + hdl =3D kzalloc_obj(*hdl); + if (!hdl) { + kfree(zdev->rf); + ib_dealloc_device(&zdev->ibdev); + return -ENOMEM; + } + + hdl->zdev =3D zdev; + zdev->hdl =3D hdl; + zdev->netdev_speed =3D SPEED_UNKNOWN; + + rf =3D zdev->rf; + err =3D zxdh_ctrl_init_hw(rf); + if (err) + goto err_ctrl_init; + + zxdh_add_handler(hdl); + + err =3D zxdh_ib_register_device(zdev); + if (err) + goto err_ib_reg; + + dev_set_drvdata(&aux_dev->dev, zdev); + return 0; + +err_ib_reg: + zxdh_del_handler(hdl); +err_ctrl_init: + kfree(zdev->rf); + kfree(hdl); + ib_dealloc_device(&zdev->ibdev); + + return err; } =20 static void zxdh_remove(struct auxiliary_device *aux_dev) { - dev_info(&aux_dev->dev, "ZTE DingHai RDMA device removed (stub)\n"); + struct zxdh_device *zdev =3D dev_get_drvdata(&aux_dev->dev); + + if (!zdev) { + dev_err(&aux_dev->dev, "%s: zdev is NULL\n", __func__); + return; + } + + ib_unregister_device(&zdev->ibdev); + zxdh_del_handler(zdev->hdl); + + kfree(zdev->rf); + kfree(zdev->hdl); + ib_dealloc_device(&zdev->ibdev); +} + +static void zxdh_shutdown(struct auxiliary_device *aux_dev) +{ + zxdh_remove(aux_dev); } =20 +static const struct auxiliary_device_id zxdh_auxiliary_id_table[] =3D { + { + .name =3D ZXDH_PF_NAME "." ZXDH_RDMA_DEV_NAME, + }, + {}, +}; + +MODULE_DEVICE_TABLE(auxiliary, zxdh_auxiliary_id_table); + static struct auxiliary_driver zxdh_auxiliary_drv =3D { .driver =3D { .name =3D ZXDH_RDMA_DEV_NAME, @@ -43,16 +253,29 @@ static struct auxiliary_driver zxdh_auxiliary_drv =3D { .id_table =3D zxdh_auxiliary_id_table, .probe =3D zxdh_probe, .remove =3D zxdh_remove, + .shutdown =3D zxdh_shutdown, }; =20 static int __init zrdma_module_init(void) { - return auxiliary_driver_register(&zxdh_auxiliary_drv); + int ret; + + INIT_LIST_HEAD(&zxdh_rdma_list); + mutex_init(&rdma_name_lock); + + ret =3D auxiliary_driver_register(&zxdh_auxiliary_drv); + if (ret) { + pr_err("zrdma: failed to register auxiliary driver\n"); + return ret; + } + + return 0; } =20 static void __exit zrdma_module_exit(void) { auxiliary_driver_unregister(&zxdh_auxiliary_drv); + mutex_destroy(&rdma_name_lock); } =20 module_init(zrdma_module_init); --=20 2.27.0 From nobody Tue Sep 29 07:00:19 2026 Received: from mxct.zte.com.cn (mxct.zte.com.cn [183.62.165.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 684A042B33E; Tue, 11 Aug 2026 09:42:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=183.62.165.209 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441381; cv=none; b=qygCaE2FCytMDjEAZsMmeAV1AxMYcNiHw1n80X+lfEf1MvN9XyWVPIUrx4ozmLJCyEe44MTUJo+WqhLwSU/l3D+QnI0thAUiu90huGuGi4nZEokeyEDhLFJ5HLyyzyLsPTiUPiTAA8vaci2ZbLWDn/eVNAKkZjxzL5R+iZULmPA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441381; c=relaxed/simple; bh=orndQq12vEmFMBEGGIFlyhuzxnppMtEakXAYWkLTD10=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=trUB2gL8gD4ulsl0pLD37ItFbwEUzR7OlumxL0qgOsGHwifMjMT3BSPQhIPuUWmLqXTz1v6H6Swq48CTADFYjn8B1ud4X2zvo5AzwqbZLN8kHg+t+H+QBQQ4JXLVNTmh0O9lq4fHBnZWsTQJGfzryC0VjPh/MOPSf2fUl9p4Zv4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn; spf=pass smtp.mailfrom=zte.com.cn; arc=none smtp.client-ip=183.62.165.209 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zte.com.cn Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxct.zte.com.cn (FangMail) with ESMTPS id 4hK69m6YNSz58fdM; Tue, 11 Aug 2026 17:42:52 +0800 (CST) Received: from szxlzmapp03.zte.com.cn ([10.5.231.207]) by mse-fl2.zte.com.cn with SMTP id 67B9gh1U073927; Tue, 11 Aug 2026 17:42:43 +0800 (+08) (envelope-from zhang.yanze@zte.com.cn) Received: from localhost.localdomain (unknown [192.168.6.15]) by smtp (Zmail) with SMTP; Tue, 11 Aug 2026 17:42:45 +0800 X-Zmail-TransId: 3e816a7aee95013-444ad X-Zmail-LocalSMTP: 1 X-Zmail-RealSender: zhang.yanze@zte.com.cn From: Yanze Zhang To: jgg@ziepe.ca, leon@kernel.org, julianbraha@gmail.com, huangjunxian6@hisilicon.com, yanjun.zhu@linux.dev, lkp@intel.com Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, wei.quan@zte.com.cn, han.junyang@zte.com.cn, ran.ming@zte.com.cn, han.chengfei@zte.com.cn, zhang.yanze@zte.com.cn Subject: [PATCH rdma v5 6/7] RDMA/zrdma: Implement context, PD, and mmap operations Date: Tue, 11 Aug 2026 17:37:39 +0800 Message-ID: <20260811093740.2607476-7-zhang.yanze@zte.com.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> References: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ZMAIL-USEORIGINALEMLTOOUTBOUND: 1 Content-Transfer-Encoding: quoted-printable X-MAIL: mse-fl2.zte.com.cn 67B9gh1U073927 X-TLS: YES X-ENVELOPE-SENDER: zhang.yanze@zte.com.cn X-SOURCE-IP: 10.5.228.133 unknown Tue, 11 Aug 2026 17:42:52 +0800 X-CLEAN: YES X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 6A7AEE9C.000/4hK69m6YNSz58fdM Content-Type: text/plain; charset="utf-8" Add implementation for allocation/deallocation of Protection Domains (PD), User Contexts, and memory mapping (mmap) for doorbells. This patch also includes the basic device initialization logic required for registration. Signed-off-by: Yanze Zhang --- drivers/infiniband/hw/zrdma/zrdma_verbs.c | 347 +++++++++++++++++++++- 1 file changed, 339 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/hw/zrdma/zrdma_verbs.c b/drivers/infiniband= /hw/zrdma/zrdma_verbs.c index 6db536c94e01..2194d34b6438 100644 --- a/drivers/infiniband/hw/zrdma/zrdma_verbs.c +++ b/drivers/infiniband/hw/zrdma/zrdma_verbs.c @@ -7,24 +7,353 @@ #include "zrdma_main.h" #include "zrdma_verbs.h" #include "zrdma_ctrl.h" +#include +#include +#include +#include =20 -static const struct ib_device_ops zxdh_dev_ops =3D { - .owner =3D THIS_MODULE, - .driver_id =3D RDMA_DRIVER_ZRDMA, - .uverbs_abi_ver =3D ZXDH_ABI_VER, -}; +static void zxdh_sc_pd_init(struct zxdh_sc_dev *dev, struct zxdh_sc_pd *pd, + u32 pd_id, int abi_ver) +{ + pd->pd_id =3D pd_id; + pd->abi_ver =3D abi_ver; + pd->dev =3D dev; +} + +static void zxdh_ib_dealloc_device(struct ib_device *ibdev) +{ + struct zxdh_device *zdev =3D to_zdev(ibdev); + + kfree(zdev->rf); + zdev->rf =3D NULL; +} + +static int zxdh_alloc_pd(struct ib_pd *pd, struct ib_udata *udata) +{ + struct zxdh_device *zdev =3D to_zdev(pd->device); + struct zxdh_sc_dev *dev =3D &zdev->rf->sc_dev; + struct zxdh_alloc_pd_resp uresp =3D {}; + struct zxdh_pd *zpd =3D to_zpd(pd); + struct zxdh_pci_f *rf =3D zdev->rf; + struct zxdh_sc_pd *sc_pd; + u32 pd_id; + int err; + + err =3D zxdh_alloc_rsrc(rf, rf->allocated_pds, rf->max_pd, &pd_id, + &rf->next_pd); + if (err) { + pr_warn("zrdma: zxdh_alloc_rsrc failed err=3D%d\n", err); + return err; + } + + sc_pd =3D &zpd->sc_pd; + if (udata) { + struct zxdh_ucontext *ucontext; + + ucontext =3D rdma_udata_to_drv_context(udata, + struct zxdh_ucontext, + ibucontext); + + zxdh_sc_pd_init(dev, sc_pd, pd_id, ucontext->abi_ver); + uresp.pd_id =3D pd_id; + if (ib_copy_to_udata(udata, &uresp, + min(sizeof(uresp), udata->outlen))) { + err =3D -EFAULT; + goto error; + } + } else { + zxdh_sc_pd_init(dev, sc_pd, pd_id, ZXDH_ABI_VER); + } + + return 0; + +error: + zxdh_free_rsrc(rf, rf->allocated_pds, pd_id); + + return err; +} + +static struct rdma_user_mmap_entry * +zxdh_user_mmap_entry_insert(struct zxdh_ucontext *ucontext, u64 bar_offset, + enum zxdh_mmap_flag mmap_flag, u64 *mmap_offset) +{ + struct zxdh_user_mmap_entry *entry =3D + kzalloc(sizeof(*entry), GFP_KERNEL); + int ret; + + if (!entry) + return NULL; + + entry->bar_offset =3D bar_offset; + entry->mmap_flag =3D mmap_flag; + + ret =3D rdma_user_mmap_entry_insert(&ucontext->ibucontext, + &entry->rdma_entry, PAGE_SIZE); + if (ret) { + kfree(entry); + return NULL; + } + *mmap_offset =3D rdma_user_mmap_get_offset(&entry->rdma_entry); + + return &entry->rdma_entry; +} + +static struct rdma_user_mmap_entry * +zxdh_mp_mmap_entry_insert(struct zxdh_ucontext *ucontext, u64 phy_addr, + size_t length, enum zxdh_mmap_flag mmap_flag, + u64 *mmap_offset) +{ + struct zxdh_user_mmap_entry *entry =3D + kzalloc(sizeof(*entry), GFP_KERNEL); + int ret; + + if (!entry) + return NULL; + + entry->bar_offset =3D phy_addr; + entry->mmap_flag =3D mmap_flag; + + ret =3D rdma_user_mmap_entry_insert(&ucontext->ibucontext, + &entry->rdma_entry, length); + if (ret) { + kfree(entry); + return NULL; + } + *mmap_offset =3D rdma_user_mmap_get_offset(&entry->rdma_entry); + + return &entry->rdma_entry; +} + +static int zxdh_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *= udata) +{ + struct zxdh_ucontext *ucontext =3D to_ucontext(uctx); + struct zxdh_alloc_ucontext_resp uresp =3D {}; + struct ib_device *ibdev =3D uctx->device; + struct zxdh_device *zdev =3D to_zdev(ibdev); + struct zxdh_alloc_ucontext_req req =3D {}; + + if (ib_copy_from_udata(&req, udata, min(sizeof(req), udata->inlen))) + return -EINVAL; + + if (req.userspace_ver !=3D ZXDH_CONTEXT_VER_V1 && + req.userspace_ver !=3D ZXDH_CONTEXT_VER_V2 && + req.userspace_ver !=3D ZXDH_CONTEXT_VER_V3) { + pr_err("zrdma: Invalid userspace_ver%d\n", req.userspace_ver); + return -EINVAL; + } + ucontext->zdev =3D zdev; + ucontext->abi_ver =3D req.userspace_ver; + + uresp.srq_db_bar_off =3D SRQ_DB_OFFSET; + + ucontext->sq_db_mmap_entry =3D + zxdh_user_mmap_entry_insert(ucontext, SQ_DB_BAR_OFF, + ZXDH_MMAP_IO_NC, + &uresp.sq_db_mmap_key); + if (!ucontext->sq_db_mmap_entry) + return -ENOMEM; + + ucontext->cq_db_mmap_entry =3D + zxdh_user_mmap_entry_insert(ucontext, CQ_DB_BAR_OFF, + ZXDH_MMAP_IO_NC, + &uresp.cq_db_mmap_key); + if (!ucontext->cq_db_mmap_entry) { + rdma_user_mmap_entry_remove(ucontext->sq_db_mmap_entry); + return -ENOMEM; + } + + if (ucontext->abi_ver =3D=3D ZXDH_CONTEXT_VER_V1 || + ucontext->abi_ver =3D=3D ZXDH_CONTEXT_VER_V2) { + ucontext->srq_db_mmap_entry =3D + zxdh_mp_mmap_entry_insert(ucontext, SRQ_DB_BAR_OFF, + PAGE_SIZE, + ZXDH_MMAP_IO_NC_BY_SIZE, + &uresp.srq_db_mmap_key); + } else { + ucontext->srq_db_mmap_entry =3D + zxdh_mp_mmap_entry_insert(ucontext, SRQ_DB_BAR_OFF, + SRQ_DB_MMAP_SIZE, + ZXDH_MMAP_IO_NC_BY_SIZE, + &uresp.srq_db_mmap_key); + } + if (!ucontext->srq_db_mmap_entry) { + rdma_user_mmap_entry_remove(ucontext->sq_db_mmap_entry); + rdma_user_mmap_entry_remove(ucontext->cq_db_mmap_entry); + pr_err("zrdma: srq_db_mmap_entry is NULL!\n"); + return -ENOMEM; + } + + uresp.srq_db_mmap_size =3D SRQ_DB_MMAP_SIZE; + uresp.kernel_ver =3D ZXDH_CONTEXT_VER_V3; + uresp.hw_rev =3D ZXDH_GEN_2; + uresp.chip_rev =3D CHIP_VERSION; + uresp.rdma_tool_flags =3D + ZXDH_QP_EXTEND_OP | ZXDH_CAPTURE | ZXDH_GET_HW_DATA | + ZXDH_GET_HW_OBJECT_DATA | ZXDH_CHECK_HW_HEALTH | + ZXDH_RDMA_TOOL_CFG_DEV_PARAM | ZXDH_RDMA_TOOL_READ_RAM | + ZXDH_RDMA_TOOL_DEVX_MODIFY_CQ | ZXDH_RDMA_SET_CREDIT_CAP | + ZXDH_RDMA_SWITCH | ZXDH_RDMA_TOOL_DEVX_EXT_MEM | + ZXDH_RDMA_TOOL_PRIV_EXT; + + uresp.feature_flags =3D ZXDH_FEATURE_RTS_AE | ZXDH_FEATURE_CQ_RESIZE | + ZXDH_FEATURE_64_BYTE_CQE; + uresp.max_hw_wq_frags =3D MAX_HW_WQ_FRAGS; + uresp.max_hw_read_sges =3D MAX_HW_READ_SGES; + uresp.max_hw_inline =3D MAX_HW_INLINE; + uresp.max_hw_srq_wr =3D MAX_HW_SRQ_WR; + uresp.max_hw_rq_quanta =3D MAX_HW_RQ_QUANTA; + uresp.max_hw_srq_quanta =3D MAX_HW_SRQ_QUANTA; + uresp.max_hw_wq_quanta =3D MAX_HW_WQ_QUANTA; + uresp.max_hw_sq_chunk =3D MAX_HW_SQ_CHUNK; + uresp.max_hw_cq_size =3D MAX_HW_CQ_SIZE; + uresp.min_hw_cq_size =3D MIN_HW_CQ_SIZE; + uresp.db_addr_type =3D ZXDH_DB_ADDR_BAR; + uresp.sq_db_bar_off =3D SQ_DB_BAR_OFF & ZXDH_PAGE_OFFSET_NUM; + uresp.cq_db_bar_off =3D CQ_DB_BAR_OFF & ZXDH_PAGE_OFFSET_NUM; + + if (ib_copy_to_udata(udata, &uresp, + min(sizeof(uresp), udata->outlen))) { + rdma_user_mmap_entry_remove(ucontext->sq_db_mmap_entry); + rdma_user_mmap_entry_remove(ucontext->cq_db_mmap_entry); + rdma_user_mmap_entry_remove(ucontext->srq_db_mmap_entry); + return -EFAULT; + } + + INIT_LIST_HEAD(&ucontext->cq_reg_mem_list); + spin_lock_init(&ucontext->cq_reg_mem_list_lock); + INIT_LIST_HEAD(&ucontext->qp_reg_mem_list); + spin_lock_init(&ucontext->qp_reg_mem_list_lock); + INIT_LIST_HEAD(&ucontext->srq_reg_mem_list); + spin_lock_init(&ucontext->srq_reg_mem_list_lock); + + return 0; +} + +static int zxdh_dealloc_pd(struct ib_pd *ibpd, struct ib_udata *udata) +{ + struct zxdh_pd *zpd =3D to_zpd(ibpd); + struct zxdh_device *zdev =3D to_zdev(ibpd->device); + + zxdh_free_rsrc(zdev->rf, zdev->rf->allocated_pds, zpd->sc_pd.pd_id); + return 0; +} + +static void zxdh_dealloc_ucontext(struct ib_ucontext *context) +{ + struct zxdh_ucontext *ucontext =3D to_ucontext(context); + + if (!ucontext) + return; + + if (ucontext->sq_db_mmap_entry) + rdma_user_mmap_entry_remove(ucontext->sq_db_mmap_entry); + if (ucontext->cq_db_mmap_entry) + rdma_user_mmap_entry_remove(ucontext->cq_db_mmap_entry); + if (ucontext->srq_db_mmap_entry) + rdma_user_mmap_entry_remove(ucontext->srq_db_mmap_entry); +} + +static void zxdh_mmap_free(struct rdma_user_mmap_entry *entry) +{ + struct zxdh_user_mmap_entry *zentry =3D to_zxdh_mmap_entry(entry); + + kfree(zentry); +} + +static int zxdh_mmap(struct ib_ucontext *context, struct vm_area_struct *v= ma) +{ + struct rdma_user_mmap_entry *rdma_entry; + struct zxdh_user_mmap_entry *entry; + struct zxdh_ucontext *ucontext; + u64 pfn; + int ret; + + ucontext =3D to_ucontext(context); + + rdma_entry =3D rdma_user_mmap_entry_get(&ucontext->ibucontext, vma); + if (!rdma_entry) { + pr_err("zrdma: pgoff[0x%lx] does not have valid entry\n", + vma->vm_pgoff); + return -EINVAL; + } + + entry =3D to_zxdh_mmap_entry(rdma_entry); + + pfn =3D (entry->bar_offset + + pci_resource_start(ucontext->zdev->rf->pcidev, 0)) >> + PAGE_SHIFT; + + switch (entry->mmap_flag) { + case ZXDH_MMAP_IO_NC: + ret =3D rdma_user_mmap_io(context, vma, pfn, PAGE_SIZE, + pgprot_noncached(vma->vm_page_prot), + rdma_entry); + break; + case ZXDH_MMAP_IO_NC_BY_SIZE: + ret =3D rdma_user_mmap_io(context, vma, pfn, + vma->vm_end - vma->vm_start, + pgprot_noncached(vma->vm_page_prot), + rdma_entry); + break; + case ZXDH_MMAP_IO_WC: + ret =3D rdma_user_mmap_io(context, vma, pfn, PAGE_SIZE, + pgprot_writecombine(vma->vm_page_prot), + rdma_entry); + break; + default: + pr_err("[zxdh_rdma] VERBS: unsupported mmap_flag[%d]\n", + entry->mmap_flag); + ret =3D -EINVAL; + } + + if (ret) + pr_err("[zxdh_rdma] VERBS: bar_offset [0x%llx] mmap_flag[%d] err[%d]\n", + entry->bar_offset, entry->mmap_flag, ret); + + rdma_user_mmap_entry_put(rdma_entry); + + return ret; +} + +static void zxdh_init_roce_device(struct zxdh_device *zdev) +{ + zdev->ibdev.node_type =3D RDMA_NODE_IB_CA; + addrconf_addr_eui48((u8 *)&zdev->ibdev.node_guid, + zdev->netdev->dev_addr); +} =20 static int zxdh_init_rdma_device(struct zxdh_device *zdev) { + struct pci_dev *pcidev =3D zdev->rf->pcidev; + + if (zdev->roce_mode) + zxdh_init_roce_device(zdev); + else + return -EPFNOSUPPORT; + zdev->ibdev.phys_port_cnt =3D 1; zdev->ibdev.num_comp_vectors =3D zdev->rf->ceqs_count; - zdev->ibdev.dev.parent =3D &zdev->rf->pcidev->dev; - - ib_set_device_ops(&zdev->ibdev, &zxdh_dev_ops); + zdev->ibdev.dev.parent =3D &pcidev->dev; =20 return 0; } =20 +static const struct ib_device_ops zxdh_dev_ops =3D { + .owner =3D THIS_MODULE, + .driver_id =3D RDMA_DRIVER_ZRDMA, + .uverbs_abi_ver =3D ZXDH_ABI_VER, + .alloc_pd =3D zxdh_alloc_pd, + .alloc_ucontext =3D zxdh_alloc_ucontext, + .dealloc_driver =3D zxdh_ib_dealloc_device, + .dealloc_pd =3D zxdh_dealloc_pd, + .dealloc_ucontext =3D zxdh_dealloc_ucontext, + .mmap =3D zxdh_mmap, + .mmap_free =3D zxdh_mmap_free, + + INIT_RDMA_OBJ_SIZE(ib_pd, zxdh_pd, ibpd), + INIT_RDMA_OBJ_SIZE(ib_ucontext, zxdh_ucontext, ibucontext), +}; + int zxdh_ib_register_device(struct zxdh_device *zdev) { int ret; @@ -33,6 +362,8 @@ int zxdh_ib_register_device(struct zxdh_device *zdev) if (ret) return ret; =20 + ib_set_device_ops(&zdev->ibdev, &zxdh_dev_ops); + ret =3D ib_register_device(&zdev->ibdev, "zrdma%d", zdev->rf->hw.device); if (ret) pr_err("zrdma: Register RDMA device fail\n"); --=20 2.27.0 From nobody Tue Sep 29 07:00:19 2026 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [160.30.148.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 761A542BEA0; Tue, 11 Aug 2026 09:43:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=160.30.148.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441384; cv=none; b=mDFObVU909kFkDwqB9pHaT1wG6kUQzA9v2xkY8w6cWnoK3Rh8jXGR+g68IfTW8T/KhU5nCpKcfRIGlUY4LQRvbWCjDZhB6gPuaqVeRc0iXjnMk2raGp9PQi68CfMyqqoCb6T779a/OMacratxcB/Lsg21wWTorIhLaeOsd6iXwA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786441384; c=relaxed/simple; bh=JCtLJJLGcnc2nyskBdNqhUkdgZRb0mxTV5Oylo08czc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CzjG4QnLgJCb3H1qGcD3BhRsBr4mmPZXWzcFAKopBjhq+p3Oib0V7+9UE/5vSUX+NdrngecwsOUE7fpOFdhmvWWbeRF/j+vCJQ8q5BJgeYwlG2gKGf9YsVVTlhOwr5Aj7Y3vl2YRFQgGbZjJlGjZCWyK4feLcsZBCDbLmCEebyE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn; spf=pass smtp.mailfrom=zte.com.cn; arc=none smtp.client-ip=160.30.148.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zte.com.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zte.com.cn Received: from mse-fl1.zte.com.cn (unknown [10.5.228.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4hK69v5XXjz4yjG7; Tue, 11 Aug 2026 17:42:59 +0800 (CST) Received: from szxlzmapp01.zte.com.cn ([10.5.231.85]) by mse-fl1.zte.com.cn with SMTP id 67B9ghmH089569; Tue, 11 Aug 2026 17:42:43 +0800 (+08) (envelope-from zhang.yanze@zte.com.cn) Received: from localhost.localdomain (unknown [192.168.6.15]) by smtp (Zmail) with SMTP; Tue, 11 Aug 2026 17:42:46 +0800 X-Zmail-TransId: 3e816a7aee95013-444b6 X-Zmail-LocalSMTP: 1 X-Zmail-RealSender: zhang.yanze@zte.com.cn From: Yanze Zhang To: jgg@ziepe.ca, leon@kernel.org, julianbraha@gmail.com, huangjunxian6@hisilicon.com, yanjun.zhu@linux.dev, lkp@intel.com Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, wei.quan@zte.com.cn, han.junyang@zte.com.cn, ran.ming@zte.com.cn, han.chengfei@zte.com.cn, zhang.yanze@zte.com.cn Subject: [PATCH rdma v5 7/7] RDMA/zrdma: Implement device and port query operations Date: Tue, 11 Aug 2026 17:37:40 +0800 Message-ID: <20260811093740.2607476-8-zhang.yanze@zte.com.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> References: <20260811093740.2607476-1-zhang.yanze@zte.com.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ZMAIL-USEORIGINALEMLTOOUTBOUND: 1 Content-Transfer-Encoding: quoted-printable X-MAIL: mse-fl1.zte.com.cn 67B9ghmH089569 X-TLS: YES X-ENVELOPE-SENDER: zhang.yanze@zte.com.cn X-SOURCE-IP: 10.5.228.132 unknown Tue, 11 Aug 2026 17:42:59 +0800 X-CLEAN: YES X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 6A7AEEA3.000/4hK69v5XXjz4yjG7 Content-Type: text/plain; charset="utf-8" Add implementation for querying device attributes, port attributes, GID, PKey, and link layer information. Also include firmware version string retrieval. Signed-off-by: Yanze Zhang --- drivers/infiniband/hw/zrdma/zrdma_verbs.c | 368 +++++++++++++++++++++- 1 file changed, 355 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/zrdma/zrdma_verbs.c b/drivers/infiniband= /hw/zrdma/zrdma_verbs.c index 2194d34b6438..de691662922f 100644 --- a/drivers/infiniband/hw/zrdma/zrdma_verbs.c +++ b/drivers/infiniband/hw/zrdma/zrdma_verbs.c @@ -315,29 +315,303 @@ static int zxdh_mmap(struct ib_ucontext *context, st= ruct vm_area_struct *vma) return ret; } =20 -static void zxdh_init_roce_device(struct zxdh_device *zdev) +static void extract_version(const char *input, char *output) { - zdev->ibdev.node_type =3D RDMA_NODE_IB_CA; - addrconf_addr_eui48((u8 *)&zdev->ibdev.node_guid, - zdev->netdev->dev_addr); + const char *last_dash_pos =3D strrchr(input, '-'); + + if (last_dash_pos) { + const char *v_pos =3D strstr(last_dash_pos, "V"); + + if (v_pos) + strscpy(output, v_pos + 1, 12); + else + output[0] =3D '\0'; + } else { + output[0] =3D '\0'; + } } =20 -static int zxdh_init_rdma_device(struct zxdh_device *zdev) +static int zxdh_query_device(struct ib_device *ibdev, + struct ib_device_attr *props, + struct ib_udata *udata) { + struct zxdh_device *zdev =3D to_zdev(ibdev); struct pci_dev *pcidev =3D zdev->rf->pcidev; + int major, sub_major, minor, sub_minor; + struct net_device *slave =3D NULL; + struct ethtool_drvinfo info; + char extracted_version[16]; + struct list_head *iter; + + memset(&info, 0, sizeof(info)); + if (zdev->netdev->priv_flags & IFF_BONDING) { + rcu_read_lock(); + netdev_for_each_lower_dev(zdev->netdev, slave, iter) { + slave->ethtool_ops->get_drvinfo(slave, &info); + break; + } + rcu_read_unlock(); + if (!slave) { + zdev->netdev->ethtool_ops->get_drvinfo(zdev->netdev, + &info); + } + } else { + zdev->netdev->ethtool_ops->get_drvinfo(zdev->netdev, &info); + } + extract_version(info.fw_version, extracted_version); + if (sscanf(extracted_version, "%d.%d.%d.%d", &major, &sub_major, &minor, + &sub_minor) !=3D 4) { + major =3D 0; + sub_major =3D 0; + minor =3D 0; + sub_minor =3D 0; + } =20 - if (zdev->roce_mode) - zxdh_init_roce_device(zdev); + if (udata->inlen || udata->outlen) + return -EINVAL; + + memset(props, 0, sizeof(*props)); + ether_addr_copy((u8 *)&props->sys_image_guid, zdev->netdev->dev_addr); + props->fw_ver =3D ((u64)major << 48 | (u64)sub_major << 32 | + (u64)minor << 16 | sub_minor); + props->device_cap_flags =3D + IB_DEVICE_MEM_WINDOW | IB_DEVICE_MEM_MGT_EXTENSIONS | + IB_DEVICE_BAD_QKEY_CNTR | IB_DEVICE_SYS_IMAGE_GUID | + IB_DEVICE_RC_RNR_NAK_GEN | IB_DEVICE_N_NOTIFY_CQ; + props->vendor_id =3D pcidev->vendor; + props->vendor_part_id =3D pcidev->device; + props->hw_ver =3D pcidev->revision; + props->page_size_cap =3D SZ_4K | SZ_2M | SZ_1G; + props->max_mr_size =3D MAX_MR_SIZE; + props->max_qp =3D MAX_QP; + props->max_qp_wr =3D MAX_QP_WR; + props->max_cq =3D MAX_CQ; + props->max_cqe =3D MAX_CQE; + props->max_mr =3D MAX_MR; + props->max_mw =3D MAX_MW; + props->max_pd =3D MAX_PD; + props->max_sge_rd =3D MAX_SGE_RD; + props->max_qp_rd_atom =3D MAX_QP_RD_ATOM; + props->max_res_rd_atom =3D MAX_RES_RD_ATOM; + props->max_qp_init_rd_atom =3D MAX_QP_INIT_RD_ATOM; + props->max_srq =3D MAX_SRQ; + props->max_srq_wr =3D MAX_SRQ_WR; + props->max_srq_sge =3D MAX_SRQ_SGE; + props->local_ca_ack_delay =3D LOCAL_CA_ACK_DELAY; + props->hca_core_clock =3D HCA_CORE_CLOCK; + props->max_wq_type_rq =3D MAX_WQ_TYPE_RQ; + if (rdma_protocol_roce(ibdev, 1)) { + props->max_pkeys =3D ZXDH_PKEY_TBL_SZ; + props->max_ah =3D MAX_AH; + props->max_mcast_grp =3D 0; + props->max_mcast_qp_attach =3D 0; + props->max_total_mcast_qp_attach =3D 0; + } + props->max_fast_reg_page_list_len =3D MAX_FAST_REG_PAGE_LIST_LEN; + props->cq_caps.max_cq_moderation_count =3D MAX_CQ_MODERATION_COUNT; + props->cq_caps.max_cq_moderation_period =3D MAX_CQ_MODERATION_PERIOD; + props->timestamp_mask =3D TIMESTAMP_MASK; + + return 0; +} + +static u32 zxdh_get_eth_netdev_speed(struct zxdh_device *zdev, u32 port_nu= m) +{ + struct ethtool_link_ksettings lksettings; + u32 netdev_speed =3D (u32)SPEED_UNKNOWN; + struct zxdh_pci_f *rf =3D zdev->rf; + int rc; + + if (rdma_port_get_link_layer(&zdev->ibdev, port_num) !=3D + IB_LINK_LAYER_ETHERNET) + return -EINVAL; + + if (zdev->netdev_speed !=3D (u32)SPEED_UNKNOWN) + return zdev->netdev_speed; + + if (rf->drv_np_cap =3D=3D ZXDH_RDMA_COMMON_FUNC_CAP && + rf->common_func_num_max > ZXDH_FUNC_NETDEV_SPEED_GET) { + rc =3D rf->gen_ops.zxdh_common_func(zdev->zxdh_adev->parent, + &netdev_speed, + ZXDH_FUNC_NETDEV_SPEED_GET); + + if (rc || netdev_speed =3D=3D (u32)SPEED_UNKNOWN) + netdev_speed =3D SPEED_1000; + else + zdev->netdev_speed =3D netdev_speed; + } else { + if (zdev->netdev_speed =3D=3D (u32)SPEED_UNKNOWN) { + rtnl_lock(); + rc =3D __ethtool_get_link_ksettings(zdev->netdev, + &lksettings); + rtnl_unlock(); + + if (!rc && + lksettings.base.speed !=3D (u32)SPEED_UNKNOWN) { + netdev_speed =3D lksettings.base.speed; + zdev->netdev_speed =3D netdev_speed; + } else { + netdev_speed =3D SPEED_1000; + } + } else { + netdev_speed =3D zdev->netdev_speed; + } + } + + return netdev_speed; +} + +static int zxdh_get_eth_speed(struct ib_device *dev, struct net_device *ne= tdev, + u32 port_num, u16 *speed, u8 *width) +{ + struct zxdh_device *zdev =3D to_zdev(dev); + u32 netdev_speed; + + if (zdev) + netdev_speed =3D zxdh_get_eth_netdev_speed(zdev, port_num); else - return -EPFNOSUPPORT; + netdev_speed =3D SPEED_1000; + + if (netdev_speed <=3D SPEED_1000) { + *width =3D IB_WIDTH_1X; + *speed =3D IB_SPEED_SDR; + } else if (netdev_speed <=3D SPEED_10000) { + *width =3D IB_WIDTH_1X; + *speed =3D IB_SPEED_FDR10; + } else if (netdev_speed <=3D SPEED_20000) { + *width =3D IB_WIDTH_4X; + *speed =3D IB_SPEED_DDR; + } else if (netdev_speed <=3D SPEED_25000) { + *width =3D IB_WIDTH_1X; + *speed =3D IB_SPEED_EDR; + } else if (netdev_speed <=3D SPEED_40000) { + *width =3D IB_WIDTH_4X; + *speed =3D IB_SPEED_FDR10; + } else if (netdev_speed <=3D SPEED_100000) { + *width =3D IB_WIDTH_4X; + *speed =3D IB_SPEED_EDR; + } else { + *width =3D IB_WIDTH_4X; + *speed =3D IB_SPEED_HDR; + } =20 - zdev->ibdev.phys_port_cnt =3D 1; - zdev->ibdev.num_comp_vectors =3D zdev->rf->ceqs_count; - zdev->ibdev.dev.parent =3D &pcidev->dev; + return 0; +} + +static int zxdh_query_port(struct ib_device *ibdev, u32 port, + struct ib_port_attr *props) +{ + struct zxdh_device *zdev =3D to_zdev(ibdev); + struct net_device *netdev =3D zdev->netdev; + + props->max_mtu =3D IB_MTU_4096; + props->active_mtu =3D zxdh_mtu_int_to_enum(netdev->mtu); + props->lid =3D 0; + props->lmc =3D 0; + props->sm_lid =3D 0; + props->sm_sl =3D 0; + if (netif_carrier_ok(netdev) && netif_running(netdev)) { + props->state =3D IB_PORT_ACTIVE; + props->phys_state =3D IB_PORT_PHYS_STATE_LINK_UP; + } else { + props->state =3D IB_PORT_DOWN; + props->phys_state =3D IB_PORT_PHYS_STATE_DISABLED; + } + zxdh_get_eth_speed(ibdev, netdev, port, &props->active_speed, + &props->active_width); + if (rdma_protocol_roce(ibdev, 1)) { + props->gid_tbl_len =3D 255; + props->ip_gids =3D true; + props->pkey_tbl_len =3D ZXDH_PKEY_TBL_SZ; + } else { + props->gid_tbl_len =3D 1; + } + props->port_cap_flags |=3D IB_PORT_CM_SUP; + props->max_msg_sz =3D MAX_MSG_SZ; + props->qkey_viol_cntr =3D 0; + + return 0; +} + +static enum rdma_link_layer zxdh_get_link_layer(struct ib_device *ibdev, + u32 port_num) +{ + return IB_LINK_LAYER_ETHERNET; +} + +static int zxdh_query_pkey(struct ib_device *ibdev, u32 port, u16 index, + u16 *pkey) +{ + if (index >=3D ZXDH_PKEY_TBL_SZ) + return -EINVAL; + + *pkey =3D ZXDH_DEFAULT_PKEY; + return 0; +} + +static int zxdh_query_gid_roce(struct ib_device *ibdev, u32 port, int inde= x, + union ib_gid *gid) +{ + int ret; + + ret =3D rdma_query_gid(ibdev, port, index, gid); + if (ret =3D=3D -EAGAIN) { + memcpy(gid, &zgid, sizeof(*gid)); + return 0; + } + + return ret; +} + +static int zxdh_roce_port_immutable(struct ib_device *ibdev, u32 port_num, + struct ib_port_immutable *immutable) +{ + struct ib_port_attr attr; + int err; + + immutable->core_cap_flags =3D RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP | + RDMA_CORE_CAP_PROT_ROCE; + err =3D ib_query_port(ibdev, port_num, &attr); + if (err) + return err; + + immutable->max_mad_size =3D IB_MGMT_MAD_SIZE; + immutable->pkey_tbl_len =3D attr.pkey_tbl_len; + immutable->gid_tbl_len =3D attr.gid_tbl_len; =20 return 0; } =20 +static void zxdh_get_dev_fw_str(struct ib_device *dev, char *str) +{ + struct zxdh_device *zdev =3D to_zdev(dev); + struct net_device *slave =3D NULL; + struct ethtool_drvinfo info; + char extracted_version[16]; + struct list_head *iter; + + memset(&info, 0, sizeof(info)); + + if (zdev->netdev->priv_flags & IFF_BONDING) { + rcu_read_lock(); + netdev_for_each_lower_dev(zdev->netdev, slave, iter) { + slave->ethtool_ops->get_drvinfo(slave, &info); + break; + } + rcu_read_unlock(); + if (!slave) { + zdev->netdev->ethtool_ops->get_drvinfo(zdev->netdev, + &info); + } + extract_version(info.fw_version, extracted_version); + snprintf(str, IB_FW_VERSION_NAME_MAX, "%s", extracted_version); + return; + } + zdev->netdev->ethtool_ops->get_drvinfo(zdev->netdev, &info); + extract_version(info.fw_version, extracted_version); + snprintf(str, IB_FW_VERSION_NAME_MAX, "%s", extracted_version); +} + static const struct ib_device_ops zxdh_dev_ops =3D { .owner =3D THIS_MODULE, .driver_id =3D RDMA_DRIVER_ZRDMA, @@ -347,6 +621,13 @@ static const struct ib_device_ops zxdh_dev_ops =3D { .dealloc_driver =3D zxdh_ib_dealloc_device, .dealloc_pd =3D zxdh_dealloc_pd, .dealloc_ucontext =3D zxdh_dealloc_ucontext, + .query_device =3D zxdh_query_device, + .query_port =3D zxdh_query_port, + .query_gid =3D zxdh_query_gid_roce, + .query_pkey =3D zxdh_query_pkey, + .get_link_layer =3D zxdh_get_link_layer, + .get_port_immutable =3D zxdh_roce_port_immutable, + .get_dev_fw_str =3D zxdh_get_dev_fw_str, .mmap =3D zxdh_mmap, .mmap_free =3D zxdh_mmap_free, =20 @@ -354,17 +635,78 @@ static const struct ib_device_ops zxdh_dev_ops =3D { INIT_RDMA_OBJ_SIZE(ib_ucontext, zxdh_ucontext, ibucontext), }; =20 +static void zxdh_init_roce_device(struct zxdh_device *zdev) +{ + zdev->ibdev.node_type =3D RDMA_NODE_IB_CA; + addrconf_addr_eui48((u8 *)&zdev->ibdev.node_guid, + zdev->netdev->dev_addr); +} + +static int zxdh_init_rdma_device(struct zxdh_device *zdev) +{ + struct pci_dev *pcidev =3D zdev->rf->pcidev; + + if (zdev->roce_mode) + zxdh_init_roce_device(zdev); + else + return -EPFNOSUPPORT; + + zdev->ibdev.phys_port_cnt =3D 1; + zdev->ibdev.num_comp_vectors =3D zdev->rf->ceqs_count; + zdev->ibdev.dev.parent =3D &pcidev->dev; + + ib_set_device_ops(&zdev->ibdev, &zxdh_dev_ops); + + return 0; +} + +static void zxdh_port_ibevent(struct zxdh_device *zdev) +{ + struct ib_event event; + + event.device =3D &zdev->ibdev; + event.element.port_num =3D 1; + event.event =3D zdev->ibdev_status ? IB_EVENT_PORT_ACTIVE : + IB_EVENT_PORT_ERR; + ib_dispatch_event(&event); +} + int zxdh_ib_register_device(struct zxdh_device *zdev) { + char name[IB_DEVICE_NAME_MAX] =3D {}; int ret; + int err; =20 ret =3D zxdh_init_rdma_device(zdev); if (ret) return ret; =20 - ib_set_device_ops(&zdev->ibdev, &zxdh_dev_ops); + ret =3D zxdh_get_del_rdma_name(pci_name(zdev->rf->pcidev), name); + if (ret) + snprintf(name, IB_DEVICE_NAME_MAX, "%s", "zrdma%d"); + + ret =3D ib_device_set_netdev(&zdev->ibdev, zdev->netdev, 1); + if (ret) + goto error; + + zdev->rf->hw.device =3D &zdev->rf->pcidev->dev; + dma_set_max_seg_size(zdev->rf->hw.device, SZ_1G); =20 - ret =3D ib_register_device(&zdev->ibdev, "zrdma%d", zdev->rf->hw.device); + ret =3D ib_register_device(&zdev->ibdev, name, zdev->rf->hw.device); + if (ret) { + snprintf(name, IB_DEVICE_NAME_MAX, "%s", "zrdma%d"); + err =3D ib_register_device(&zdev->ibdev, name, + zdev->rf->hw.device); + if (err) + goto error; + } + + zdev->ibdev_status =3D 1; + zxdh_port_ibevent(zdev); + + return 0; + +error: if (ret) pr_err("zrdma: Register RDMA device fail\n"); =20 --=20 2.27.0