From nobody Tue Nov 26 08:50:13 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D7A120CCC5; Fri, 18 Oct 2024 20:31:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283484; cv=none; b=JqqtTjzPjU641jJEH7W2Rxo8aFKcP+TTnP4W1oXM7d9onMMs2sZzxOyOeuOXdAirCkU6/D8lVOmDiWU65ZTKCHpTM9cXTGvjElVun+1zply7uWg56stqpuKGBmW4dX4+y2Yhi0+n+EWmXp3lQt3s2QwWTcK7dfRyo8JnjR/CYhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283484; c=relaxed/simple; bh=mdIVbytr6kIPewBcxaPOdfr9sudbKXGVgi+cW+YMGdA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=c3A5iumLQUAE4kLpfhJ9giOaLjoqVNrCIURdWepN4yMxq9M3WKD5tAKEbeaq56nQl3AidwYtrluKrGsVH9WJoX2ypf5TrZ17/XcIJGArL86bTRDz7z++5fMcyRvXwHWNyF4M/czHPvVWuF4e0VO0G9McZe9xzM/D+SYcvXwjXVE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=KMbPMQeI; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="KMbPMQeI" Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49I9R4H4008597; Fri, 18 Oct 2024 13:31:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=R xLZHCUdxSqURxF5tvaDu+NjdpoGZbV5gLB59XKQies=; b=KMbPMQeIOfBjfKYf5 uFVF6KydtgHlZDlfxO4FgnzcFNJFjNk8+NMJ1ton5c8FqdWgHkBo1br42PN4ad69 m7xBK0CD64JOB3dW1XzKcnzMjg3t4f+jQy2uu7kqyqpoBemQrWUyuxbi50f0/v3O PO2/bIPLjqrZVVP6ccziy6PeVuRfpY9O6hEaUNltVceJaLi46Hlj3jxFNnU+CqDi 9R8cuzF1GCpgmYpGDS/g0HM+7tNNOcvst0j6A3Dy9l4ktF9xJAfU0XtIzer5KEq1 SifHCbsXYSQnGscCLhENRFHEFUIbBoxNuLi0SI1Z0JRASs4UN8ooZrbKhAAvA8z0 Vid5w== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42bm18hjpc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 18 Oct 2024 13:31:10 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 18 Oct 2024 13:31:09 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 18 Oct 2024 13:31:09 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id 5BF5E3F704C; Fri, 18 Oct 2024 13:31:05 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH 1/6] octeontx2: Set appropriate PF, VF masks and shifts based on silicon Date: Sat, 19 Oct 2024 02:00:53 +0530 Message-ID: <20241018203058.3641959-2-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241018203058.3641959-1-saikrishnag@marvell.com> References: <20241018203058.3641959-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: SdNfn5iEoqBL1IR2AnWIH8lfNNcS4BWs X-Proofpoint-ORIG-GUID: SdNfn5iEoqBL1IR2AnWIH8lfNNcS4BWs X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 Content-Type: text/plain; charset="utf-8" From: Subbaraya Sundeep Number of RVU PFs on CN20K silicon have increased to 96 from maximum of 32 that were supported on earlier silicons. Every RVU PF and VF is identified by HW using a 16bit PF_FUNC value. Due to the change in Max number of PFs in CN20K, the bit encoding of this PF_FUNC has changed. This patch handles the change by exporting PF,VF masks and shifts present in mailbox module to all other modules. Also moved the NIX AF register offset macros to other files which will be posted in coming patches. Signed-off-by: Subbaraya Sundeep Signed-off-by: Sai Krishna --- .../net/ethernet/marvell/octeontx2/af/mbox.c | 15 ++++++++++ .../net/ethernet/marvell/octeontx2/af/mbox.h | 5 ++++ .../net/ethernet/marvell/octeontx2/af/rvu.c | 5 ---- .../net/ethernet/marvell/octeontx2/af/rvu.h | 14 +++++---- .../marvell/octeontx2/nic/otx2_common.h | 11 +------ .../ethernet/marvell/octeontx2/nic/otx2_reg.h | 30 ------------------- 6 files changed, 30 insertions(+), 50 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 1e5aa5397504..791c468a10c5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -13,6 +13,21 @@ #include "mbox.h" #include "rvu_trace.h" =20 +/* Default values of PF and VF bit encodings in PCIFUNC for + * CN9XXX and CN10K series silicons. + */ +u16 rvu_pcifunc_pf_shift =3D 10; +EXPORT_SYMBOL(rvu_pcifunc_pf_shift); + +u16 rvu_pcifunc_pf_mask =3D 0x3F; +EXPORT_SYMBOL(rvu_pcifunc_pf_mask); + +u16 rvu_pcifunc_func_shift; +EXPORT_SYMBOL(rvu_pcifunc_func_shift); + +u16 rvu_pcifunc_func_mask =3D 0x3FF; +EXPORT_SYMBOL(rvu_pcifunc_func_mask); + static const u16 msgs_offset =3D ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_A= LIGN); =20 void __otx2_mbox_reset(struct otx2_mbox *mbox, int devid) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.h index 6ea2f3071fe8..38a0badcdb68 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -50,6 +50,11 @@ #define MBOX_DIR_PFVF_UP 6 /* PF sends messages to VF */ #define MBOX_DIR_VFPF_UP 7 /* VF replies to PF */ =20 +extern u16 rvu_pcifunc_pf_shift; +extern u16 rvu_pcifunc_pf_mask; +extern u16 rvu_pcifunc_func_shift; +extern u16 rvu_pcifunc_func_mask; + struct otx2_mbox_dev { void *mbase; /* This dev's mbox region */ void *hwbase; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index 1a97fb9032fa..dcfc27a60b43 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -400,11 +400,6 @@ static void rvu_update_rsrc_map(struct rvu *rvu, struc= t rvu_pfvf *pfvf, rvu_write64(rvu, BLKADDR_RVUM, reg | (devnum << 16), num_lfs); } =20 -inline int rvu_get_pf(u16 pcifunc) -{ - return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; -} - void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf) { u64 cfg; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index 5016ba82e142..938a911cbf1c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -41,10 +41,10 @@ #define MAX_CPT_BLKS 2 =20 /* PF_FUNC */ -#define RVU_PFVF_PF_SHIFT 10 -#define RVU_PFVF_PF_MASK 0x3F -#define RVU_PFVF_FUNC_SHIFT 0 -#define RVU_PFVF_FUNC_MASK 0x3FF +#define RVU_PFVF_PF_SHIFT rvu_pcifunc_pf_shift +#define RVU_PFVF_PF_MASK rvu_pcifunc_pf_mask +#define RVU_PFVF_FUNC_SHIFT rvu_pcifunc_func_shift +#define RVU_PFVF_FUNC_MASK rvu_pcifunc_func_mask =20 #ifdef CONFIG_DEBUG_FS struct dump_ctx { @@ -819,7 +819,6 @@ int rvu_alloc_rsrc_contig(struct rsrc_bmap *rsrc, int n= rsrc); void rvu_free_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc, int start); bool rvu_rsrc_check_contig(struct rsrc_bmap *rsrc, int nrsrc); u16 rvu_get_rsrc_mapcount(struct rvu_pfvf *pfvf, int blkaddr); -int rvu_get_pf(u16 pcifunc); struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc); void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf); bool is_block_implemented(struct rvu_hwinfo *hw, int blkaddr); @@ -852,6 +851,11 @@ bool is_sdp_pfvf(u16 pcifunc); bool is_sdp_pf(u16 pcifunc); bool is_sdp_vf(struct rvu *rvu, u16 pcifunc); =20 +static inline int rvu_get_pf(u16 pcifunc) +{ + return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; +} + /* CGX APIs */ static inline bool is_pf_cgxmapped(struct rvu *rvu, u8 pf) { diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index f27a3456ae64..8e7ed3979f80 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -27,6 +27,7 @@ #include "otx2_reg.h" #include "otx2_txrx.h" #include "otx2_devlink.h" +#include #include #include "qos.h" =20 @@ -873,21 +874,11 @@ MBOX_UP_MCS_MESSAGES /* Time to wait before watchdog kicks off */ #define OTX2_TX_TIMEOUT (100 * HZ) =20 -#define RVU_PFVF_PF_SHIFT 10 -#define RVU_PFVF_PF_MASK 0x3F -#define RVU_PFVF_FUNC_SHIFT 0 -#define RVU_PFVF_FUNC_MASK 0x3FF - static inline bool is_otx2_vf(u16 pcifunc) { return !!(pcifunc & RVU_PFVF_FUNC_MASK); } =20 -static inline int rvu_get_pf(u16 pcifunc) -{ - return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; -} - static inline dma_addr_t otx2_dma_map_page(struct otx2_nic *pfvf, struct page *page, size_t offset, size_t size, diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h b/driver= s/net/ethernet/marvell/octeontx2/nic/otx2_reg.h index e3aee6e36215..858f084b9d47 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h @@ -138,36 +138,6 @@ #define NIX_LF_CINTX_ENA_W1S(a) (NIX_LFBASE | 0xD40 | (a) << 12) #define NIX_LF_CINTX_ENA_W1C(a) (NIX_LFBASE | 0xD50 | (a) << 12) =20 -/* NIX AF transmit scheduler registers */ -#define NIX_AF_SMQX_CFG(a) (0x700 | (u64)(a) << 16) -#define NIX_AF_TL4X_SDP_LINK_CFG(a) (0xB10 | (u64)(a) << 16) -#define NIX_AF_TL1X_SCHEDULE(a) (0xC00 | (u64)(a) << 16) -#define NIX_AF_TL1X_CIR(a) (0xC20 | (u64)(a) << 16) -#define NIX_AF_TL1X_TOPOLOGY(a) (0xC80 | (u64)(a) << 16) -#define NIX_AF_TL2X_PARENT(a) (0xE88 | (u64)(a) << 16) -#define NIX_AF_TL2X_SCHEDULE(a) (0xE00 | (u64)(a) << 16) -#define NIX_AF_TL2X_TOPOLOGY(a) (0xE80 | (u64)(a) << 16) -#define NIX_AF_TL2X_CIR(a) (0xE20 | (u64)(a) << 16) -#define NIX_AF_TL2X_PIR(a) (0xE30 | (u64)(a) << 16) -#define NIX_AF_TL3X_PARENT(a) (0x1088 | (u64)(a) << 16) -#define NIX_AF_TL3X_SCHEDULE(a) (0x1000 | (u64)(a) << 16) -#define NIX_AF_TL3X_SHAPE(a) (0x1010 | (u64)(a) << 16) -#define NIX_AF_TL3X_CIR(a) (0x1020 | (u64)(a) << 16) -#define NIX_AF_TL3X_PIR(a) (0x1030 | (u64)(a) << 16) -#define NIX_AF_TL3X_TOPOLOGY(a) (0x1080 | (u64)(a) << 16) -#define NIX_AF_TL4X_PARENT(a) (0x1288 | (u64)(a) << 16) -#define NIX_AF_TL4X_SCHEDULE(a) (0x1200 | (u64)(a) << 16) -#define NIX_AF_TL4X_SHAPE(a) (0x1210 | (u64)(a) << 16) -#define NIX_AF_TL4X_CIR(a) (0x1220 | (u64)(a) << 16) -#define NIX_AF_TL4X_PIR(a) (0x1230 | (u64)(a) << 16) -#define NIX_AF_TL4X_TOPOLOGY(a) (0x1280 | (u64)(a) << 16) -#define NIX_AF_MDQX_SCHEDULE(a) (0x1400 | (u64)(a) << 16) -#define NIX_AF_MDQX_SHAPE(a) (0x1410 | (u64)(a) << 16) -#define NIX_AF_MDQX_CIR(a) (0x1420 | (u64)(a) << 16) -#define NIX_AF_MDQX_PIR(a) (0x1430 | (u64)(a) << 16) -#define NIX_AF_MDQX_PARENT(a) (0x1480 | (u64)(a) << 16) -#define NIX_AF_TL3_TL2X_LINKX_CFG(a, b) (0x1700 | (u64)(a) << 16 | (b) << = 3) - /* LMT LF registers */ #define LMT_LFBASE BIT_ULL(RVU_FUNC_BLKADDR_SHIFT) #define LMT_LF_LMTLINEX(a) (LMT_LFBASE | 0x000 | (a) << 12) --=20 2.25.1 From nobody Tue Nov 26 08:50:13 2024 Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 329A5207A34; Fri, 18 Oct 2024 20:31:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.148.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283484; cv=none; b=Vu5bNdsD+I9TjKm1XgTCCQKXSWV6EzDLWkcbAtKJ6TsDy6bt7hPXIRcMKqgJ9GuULpJBfkSI38GJ+wNO1EOh5CMQItDMMqU53OTCae26fbrWSRTv0RNrbKSHOFsvC10k+qKi7a61EfFv/l3GQu3UeKVYtY092lH4g/0NbF8hYYw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283484; c=relaxed/simple; bh=GO6fxUH4cpR8B6o4P/Mrr3oLzpRR5KgppixuPQQYL+M=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WlROf88c83nA1vhphLXge3s77Snb/Oqo/3caLIE50zuh5VqC0lO3L/WacR9ozc1Qj7I/yd3UPwmlMY+D1xX6tx7OxQGCkdTStgn0mYpW4K0UgFccpZlgZY+fQ++ZuJgUEZGZGufmYweLe6EnVXJynVqJyV8cZ0lswtVwCpu+JBU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=gGAcBW3W; arc=none smtp.client-ip=67.231.148.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="gGAcBW3W" Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49IA8cAh032615; Fri, 18 Oct 2024 13:31:15 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=q eFlhB3Y+tGia3KluuOajnBOri30qUPwUEx0cz1Xt4c=; b=gGAcBW3WCbjPHxalW +joIBoh29Esy+Ln9hS6JSqugkClr1V3QdwD082Fi3xuI34gt9yfiVxukkCGwDjMi f9w8BZeo8RLQs5ZrTkgToGZ783h05eKX3TeV6HYalkweMQHvMfIhqG4QaHvU7tdG 4aSPaxbKcnlsd9puxLfe7GQuW0d7LCOjW4fs6DsOqHsOz+WVXJ8YgC30Gm20ZA2u hsRBlxVKbyRaXsvH/uDxwJKlp3QSs86Vsgjv4PdEP9vF++X7sgBgLq23xp6AZy1y Eib/sPwgc01eNochSafEooAZ3YkEoJS66Z7jQLFOwFH8txOIcNhcOHDuAyg1W0pH Ayvog== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 42bnnbhbp7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 18 Oct 2024 13:31:14 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 18 Oct 2024 13:31:13 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 18 Oct 2024 13:31:13 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id A409E3F7055; Fri, 18 Oct 2024 13:31:09 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH 2/6] octeontx2-af: CN20k basic mbox operations and structures Date: Sat, 19 Oct 2024 02:00:54 +0530 Message-ID: <20241018203058.3641959-3-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241018203058.3641959-1-saikrishnag@marvell.com> References: <20241018203058.3641959-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: a4slbULf2gORR76DL1h-DZCTx8Xv8r04 X-Proofpoint-ORIG-GUID: a4slbULf2gORR76DL1h-DZCTx8Xv8r04 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 Content-Type: text/plain; charset="utf-8" This patch adds basic mbox operation APIs and structures to add support for mbox module on CN20k silicon. There are few CSR offsets, interrupts changed between CN20k and prior Octeon series of devices. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/af/Makefile | 3 +- .../ethernet/marvell/octeontx2/af/cn20k/api.h | 22 +++++++ .../marvell/octeontx2/af/cn20k/mbox_init.c | 52 +++++++++++++++ .../ethernet/marvell/octeontx2/af/cn20k/reg.h | 27 ++++++++ .../net/ethernet/marvell/octeontx2/af/mbox.c | 3 + .../net/ethernet/marvell/octeontx2/af/mbox.h | 7 ++ .../net/ethernet/marvell/octeontx2/af/rvu.c | 65 +++++++++++++++---- .../net/ethernet/marvell/octeontx2/af/rvu.h | 22 +++++++ .../marvell/octeontx2/af/rvu_struct.h | 6 +- 9 files changed, 192 insertions(+), 15 deletions(-) create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_in= it.c create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h diff --git a/drivers/net/ethernet/marvell/octeontx2/af/Makefile b/drivers/n= et/ethernet/marvell/octeontx2/af/Makefile index 3cf4c8285c90..38d8599dc6eb 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/Makefile +++ b/drivers/net/ethernet/marvell/octeontx2/af/Makefile @@ -11,4 +11,5 @@ rvu_mbox-y :=3D mbox.o rvu_trace.o rvu_af-y :=3D cgx.o rvu.o rvu_cgx.o rvu_npa.o rvu_nix.o \ rvu_reg.o rvu_npc.o rvu_debugfs.o ptp.o rvu_npc_fs.o \ rvu_cpt.o rvu_devlink.o rpm.o rvu_cn10k.o rvu_switch.o \ - rvu_sdp.o rvu_npc_hash.o mcs.o mcs_rvu_if.o mcs_cnf10kb.o + rvu_sdp.o rvu_npc_hash.o mcs.o mcs_rvu_if.o mcs_cnf10kb.o \ + cn20k/mbox_init.o diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/api.h new file mode 100644 index 000000000000..b57bd38181aa --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef CN20K_API_H +#define CN20K_API_H + +#include "../rvu.h" + +struct ng_rvu { + struct mbox_ops *rvu_mbox_ops; + struct qmem *pf_mbox_addr; +}; + +/* Mbox related APIs */ +int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int num); +int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, + int num, int type, unsigned long *pf_bmap); +#endif /* CN20K_API_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c b/= drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c new file mode 100644 index 000000000000..0d7ad31e5dfb --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#include +#include + +#include "rvu_trace.h" +#include "mbox.h" +#include "reg.h" +#include "api.h" + +int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, + int num, int type, unsigned long *pf_bmap) +{ + int region; + u64 bar; + + for (region =3D 0; region < num; region++) { + if (!test_bit(region, pf_bmap)) + continue; + + bar =3D (u64)phys_to_virt((u64)rvu->ng_rvu->pf_mbox_addr->base); + bar +=3D region * MBOX_SIZE; + + mbox_addr[region] =3D (void *)bar; + + if (!mbox_addr[region]) + goto error; + } + return 0; + +error: + return -ENOMEM; +} + +int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int ndevs) +{ + int dev; + + if (!is_cn20k(rvu->pdev)) + return 0; + + for (dev =3D 0; dev < ndevs; dev++) + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); + + return 0; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/reg.h new file mode 100644 index 000000000000..58152a4024ec --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef RVU_MBOX_REG_H +#define RVU_MBOX_REG_H +#include "../rvu.h" +#include "../rvu_reg.h" + +/* RVUM block registers */ +#define RVU_PF_DISC (0x0) +#define RVU_PRIV_PFX_DISC(a) (0x8000208 | (a) << 16) +#define RVU_PRIV_HWVFX_DISC(a) (0xD000000 | (a) << 12) + +/* Mbox Registers */ +/* RVU AF BAR0 Mbox registers for AF =3D> PFx */ +#define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) +#define RVU_MBOX_AF_PFX_CFG(a) (0x6000 | (a) << 4) +#define RVU_AF_BAR2_SEL (0x9000000) +#define RVU_AF_BAR2_PFID (0x16400) +#define NIX_CINTX_INT_W1S(a) (0xd30 | (a) << 12) +#define NIX_QINTX_CNT(a) (0xc00 | (a) << 12) + +#endif /* RVU_MBOX_REG_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 791c468a10c5..1e3e72107a9d 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -10,8 +10,11 @@ #include =20 #include "rvu_reg.h" +#include "cn20k/reg.h" +#include "cn20k/api.h" #include "mbox.h" #include "rvu_trace.h" +#include "rvu.h" =20 /* Default values of PF and VF bit encodings in PCIFUNC for * CN9XXX and CN10K series silicons. diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.h index 38a0badcdb68..df64a18fe1d6 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -55,6 +55,11 @@ extern u16 rvu_pcifunc_pf_mask; extern u16 rvu_pcifunc_func_shift; extern u16 rvu_pcifunc_func_mask; =20 +enum { + TYPE_AFVF, + TYPE_AFPF, +}; + struct otx2_mbox_dev { void *mbase; /* This dev's mbox region */ void *hwbase; @@ -83,6 +88,8 @@ struct otx2_mbox { struct mbox_hdr { u64 msg_size; /* Total msgs size embedded */ u16 num_msgs; /* No of msgs embedded */ + u16 opt_msg; + u8 sig; }; =20 /* Header which precedes every msg and is also part of it */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index dcfc27a60b43..a5ebd7cd3a5c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -20,6 +20,8 @@ =20 #include "rvu_trace.h" #include "rvu_npc_hash.h" +#include "cn20k/reg.h" +#include "cn20k/api.h" =20 #define DRV_NAME "rvu_af" #define DRV_STRING "Marvell OcteonTX2 RVU Admin Function Driver" @@ -34,10 +36,8 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq= _info *mw, int type, int num, void (mbox_handler)(struct work_struct *), void (mbox_up_handler)(struct work_struct *)); -enum { - TYPE_AFVF, - TYPE_AFPF, -}; +static irqreturn_t rvu_mbox_pf_intr_handler(int irq, void *rvu_irq); +static irqreturn_t rvu_mbox_intr_handler(int irq, void *rvu_irq); =20 /* Supported devices */ static const struct pci_device_id rvu_id_table[] =3D { @@ -2212,6 +2212,22 @@ static void __rvu_mbox_handler(struct rvu_work *mwor= k, int type, bool poll) =20 offset =3D mbox->rx_start + ALIGN(sizeof(*req_hdr), MBOX_MSG_ALIGN); =20 + if (req_hdr->sig) { + req_hdr->opt_msg =3D mw->mbox_wrk[devid].num_msgs; + rvu_write64(rvu, BLKADDR_NIX0, RVU_AF_BAR2_SEL, + RVU_AF_BAR2_PFID); + if (type =3D=3D TYPE_AFPF) + rvu_write64(rvu, BLKADDR_NIX0, + AF_BAR2_ALIASX(0, NIX_CINTX_INT_W1S(devid)), + 0x1); + else + rvu_write64(rvu, BLKADDR_NIX0, + AF_BAR2_ALIASX(0, NIX_QINTX_CNT(devid)), + 0x1); + usleep_range(1000, 2000); + goto done; + } + for (id =3D 0; id < mw->mbox_wrk[devid].num_msgs; id++) { msg =3D mdev->mbase + offset; =20 @@ -2245,9 +2261,10 @@ static void __rvu_mbox_handler(struct rvu_work *mwor= k, int type, bool poll) err, otx2_mbox_id2name(msg->id), msg->id, devid); } +done: mw->mbox_wrk[devid].num_msgs =3D 0; =20 - if (poll) + if (!is_cn20k(mbox->pdev) && poll) otx2_mbox_wait_for_zero(mbox, devid); =20 /* Send mbox responses to VF/PF */ @@ -2360,6 +2377,10 @@ static int rvu_get_mbox_regions(struct rvu *rvu, voi= d **mbox_addr, int region; u64 bar4; =20 + if (is_cn20k(rvu->pdev)) + return cn20k_rvu_get_mbox_regions(rvu, mbox_addr, + num, type, pf_bmap); + /* For cn10k platform VF mailbox regions of a PF follows after the * PF <-> AF mailbox region. Whereas for Octeontx2 it is read from * RVU_PF_VF_BAR4_ADDR register. @@ -2413,12 +2434,17 @@ static int rvu_get_mbox_regions(struct rvu *rvu, vo= id **mbox_addr, return -ENOMEM; } =20 +static struct mbox_ops rvu_mbox_ops =3D { + .pf_intr_handler =3D rvu_mbox_pf_intr_handler, +}; + static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw, int type, int num, void (mbox_handler)(struct work_struct *), void (mbox_up_handler)(struct work_struct *)) { int err =3D -EINVAL, i, dir, dir_up; + struct ng_rvu *ng_rvu_mbox; void __iomem *reg_base; struct rvu_work *mwork; unsigned long *pf_bmap; @@ -2443,6 +2469,18 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbo= x_wq_info *mw, } } =20 + ng_rvu_mbox =3D kzalloc(sizeof(*ng_rvu_mbox), GFP_KERNEL); + if (!ng_rvu_mbox) { + err =3D -ENOMEM; + goto free_bitmap; + } + + rvu->ng_rvu =3D ng_rvu_mbox; + + rvu->ng_rvu->rvu_mbox_ops =3D &rvu_mbox_ops; + + cn20k_rvu_mbox_init(rvu, type, num); + mutex_init(&rvu->mbox_lock); =20 mbox_regions =3D kcalloc(num, sizeof(void *), GFP_KERNEL); @@ -2475,7 +2513,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox= _wq_info *mw, } =20 mw->mbox_wq =3D alloc_workqueue("%s", - WQ_UNBOUND | WQ_HIGHPRI | WQ_MEM_RECLAIM, + WQ_HIGHPRI | WQ_MEM_RECLAIM, num, name); if (!mw->mbox_wq) { err =3D -ENOMEM; @@ -2553,8 +2591,8 @@ static void rvu_mbox_destroy(struct mbox_wq_info *mw) otx2_mbox_destroy(&mw->mbox_up); } =20 -static void rvu_queue_work(struct mbox_wq_info *mw, int first, - int mdevs, u64 intr) +void rvu_queue_work(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr) { struct otx2_mbox_dev *mdev; struct otx2_mbox *mbox; @@ -2965,12 +3003,14 @@ static int rvu_register_interrupts(struct rvu *rvu) =20 /* Register mailbox interrupt handler */ sprintf(&rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], "RVUAF Mbox"); - ret =3D request_irq(pci_irq_vector(rvu->pdev, RVU_AF_INT_VEC_MBOX), - rvu_mbox_pf_intr_handler, 0, - &rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], rvu); + ret =3D request_irq(pci_irq_vector + (rvu->pdev, RVU_AF_INT_VEC_MBOX), + rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, + &rvu->irq_name[RVU_AF_INT_VEC_MBOX * + NAME_SIZE], rvu); if (ret) { dev_err(rvu->dev, - "RVUAF: IRQ registration failed for mbox irq\n"); + "RVUAF: IRQ registration failed for mbox\n"); goto fail; } =20 @@ -3478,6 +3518,7 @@ static void rvu_remove(struct pci_dev *pdev) pci_set_drvdata(pdev, NULL); =20 devm_kfree(&pdev->dev, rvu->hw); + kfree(rvu->ng_rvu); devm_kfree(&pdev->dev, rvu); } =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index 938a911cbf1c..9fd7aea8c481 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -444,6 +444,10 @@ struct mbox_wq_info { struct workqueue_struct *mbox_wq; }; =20 +struct mbox_ops { + irqreturn_t (*pf_intr_handler)(int irq, void *rvu_irq); +}; + struct channel_fwdata { struct sdp_node_info info; u8 valid; @@ -594,6 +598,7 @@ struct rvu { spinlock_t cpt_intr_lock; =20 struct mutex mbox_lock; /* Serialize mbox up and down msgs */ + struct ng_rvu *ng_rvu; }; =20 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64= val) @@ -875,11 +880,28 @@ static inline bool is_cgx_vf(struct rvu *rvu, u16 pci= func) is_pf_cgxmapped(rvu, rvu_get_pf(pcifunc))); } =20 +#define CN20K_CHIPID 0x20 + +/* + * Silicon check for CN20K family + */ +static inline bool is_cn20k(struct pci_dev *pdev) +{ + if ((pdev->subsystem_device & 0xFF) =3D=3D CN20K_CHIPID) + return true; + + return false; +} + #define M(_name, _id, fn_name, req, rsp) \ int rvu_mbox_handler_ ## fn_name(struct rvu *, struct req *, struct rsp *); MBOX_MESSAGES #undef M =20 +/* Mbox APIs */ +void rvu_queue_work(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr); + int rvu_cgx_init(struct rvu *rvu); int rvu_cgx_exit(struct rvu *rvu); void *rvu_cgx_pdata(u8 cgx_id, struct rvu *rvu); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_struct.h b/drive= rs/net/ethernet/marvell/octeontx2/af/rvu_struct.h index fc8da2090657..90cb063d00f0 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_struct.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_struct.h @@ -33,7 +33,8 @@ enum rvu_block_addr_e { BLKADDR_NDC_NIX1_RX =3D 0x10ULL, BLKADDR_NDC_NIX1_TX =3D 0x11ULL, BLKADDR_APR =3D 0x16ULL, - BLK_COUNT =3D 0x17ULL, + BLKADDR_MBOX =3D 0x1bULL, + BLK_COUNT =3D 0x1cULL, }; =20 /* RVU Block Type Enumeration */ @@ -49,7 +50,8 @@ enum rvu_block_type_e { BLKTYPE_TIM =3D 0x8, BLKTYPE_CPT =3D 0x9, BLKTYPE_NDC =3D 0xa, - BLKTYPE_MAX =3D 0xa, + BLKTYPE_MBOX =3D 0x13, + BLKTYPE_MAX =3D 0x13, }; =20 /* RVU Admin function Interrupt Vector Enumeration */ --=20 2.25.1 From nobody Tue Nov 26 08:50:13 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A57120CCFE; Fri, 18 Oct 2024 20:31:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283489; cv=none; b=MbLRkI94b7Bn9Uh0eFu1z5IO4pVLU37KF4oHC1dOxKZC0gyR1Tmtknbq2t3ME/XDFaqVMi10PygqKxGaYQjoT8ZqbiKx6b9x4FMx27nAFSi3UKITnQABoawk8s0uYit/LmHYMQLIG0ZihwHPDGI3qGnjhl0oHL0xcGvdgVkhLeY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283489; c=relaxed/simple; bh=wpym2DDLT7ySfySqf1iA9PzUfWQ/1OAxkVT+bod4IdM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QdilC3Kz/DKT6yBrS4gan07hNPYY50f8+PxnspQ7/k3HmgGpi8U3zNU0PLn93RUbZHd1fflPBo3Mbfb3zmt1uqW0Qmo9anttAwFPFVAfDbeHjvz1OxodxZWDX4iUFmXrFINu33kgHbJVKMaaNLP1lwf098j9W7GVkGgNekyWVMM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=ZcaYo3HX; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="ZcaYo3HX" Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49I9R4H5008597; Fri, 18 Oct 2024 13:31:19 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=F VNCjLYp7Mb5BxW79LsDSGZvXt28ETN+UtFqT3Lf2S0=; b=ZcaYo3HXZ21JVZLhJ 9LqCkYbhVs84NTFGkVkt1E2uZxjLcivzelAyy5zqko9234iLZMe7IIfcY43Rrm8C gkXf/UaUr2Mi79v9K4ZFAiCpRaZ1i399hrXqyKM1Nh5t8Qv8L3GUXq06ppoq5Y9r mxJPx9t2c4BPj56Yzrjs5Tgps/cCWAGfiIF7IrPRXex/LihJN4PZwIt1tO/DG37q Mv6GTPxpwBssgMEU+2gXzKWMjziLlmmgZ1ZNgkOKJ2VA3bYcewYQWc5t/oxQlmEv w2Hgrmy9EWNeYzO5BjG43gP6J5Bk8nyzX73EW4/zhsPlyTW/dNv3PY4VszuXMjbE D8U1w== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42bm18hjpf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 18 Oct 2024 13:31:18 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 18 Oct 2024 13:31:18 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 18 Oct 2024 13:31:18 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id F0B163F704C; Fri, 18 Oct 2024 13:31:13 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH 3/6] octeontx2-af: CN20k mbox to support AF REQ/ACK functionality Date: Sat, 19 Oct 2024 02:00:55 +0530 Message-ID: <20241018203058.3641959-4-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241018203058.3641959-1-saikrishnag@marvell.com> References: <20241018203058.3641959-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: d-q_-4HUEvrM8p2-xDRQHkVBFTp6zblt X-Proofpoint-ORIG-GUID: d-q_-4HUEvrM8p2-xDRQHkVBFTp6zblt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 Content-Type: text/plain; charset="utf-8" This implementation uses separate trigger interrupts for request, response MBOX messages against using trigger message data in CN10K. This patch adds support for basic mbox implementation for CN20K from AF side. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/af/cn20k/api.h | 8 + .../marvell/octeontx2/af/cn20k/mbox_init.c | 214 ++++++++++++++++++ .../ethernet/marvell/octeontx2/af/cn20k/reg.h | 17 ++ .../marvell/octeontx2/af/cn20k/struct.h | 25 ++ .../net/ethernet/marvell/octeontx2/af/mbox.c | 77 ++++++- .../net/ethernet/marvell/octeontx2/af/mbox.h | 1 + .../net/ethernet/marvell/octeontx2/af/rvu.c | 62 +++-- .../net/ethernet/marvell/octeontx2/af/rvu.h | 16 +- 8 files changed, 401 insertions(+), 19 deletions(-) create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/api.h index b57bd38181aa..9436a4a4d815 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h @@ -15,8 +15,16 @@ struct ng_rvu { struct qmem *pf_mbox_addr; }; =20 +struct rvu; + /* Mbox related APIs */ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int num); +int cn20k_register_afpf_mbox_intr(struct rvu *rvu); int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, int num, int type, unsigned long *pf_bmap); +void cn20k_rvu_enable_mbox_intr(struct rvu *rvu); +void cn20k_rvu_unregister_interrupts(struct rvu *rvu); +void cn20k_free_mbox_memory(struct rvu *rvu); +int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, + void *reg_base, int direction, int ndevs); #endif /* CN20K_API_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c b/= drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c index 0d7ad31e5dfb..e19de47da84a 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c @@ -13,6 +13,137 @@ #include "reg.h" #include "api.h" =20 +/* CN20K mbox PFx =3D> AF irq handler */ +static irqreturn_t cn20k_mbox_pf_common_intr_handler(int irq, void *rvu_ir= q) +{ + struct rvu_irq_data *rvu_irq_data =3D (struct rvu_irq_data *)rvu_irq; + struct rvu *rvu =3D rvu_irq_data->rvu; + u64 intr; + + /* Clear interrupts */ + intr =3D rvu_read64(rvu, BLKADDR_RVUM, rvu_irq_data->intr_status); + rvu_write64(rvu, BLKADDR_RVUM, rvu_irq_data->intr_status, intr); + + if (intr) + trace_otx2_msg_interrupt(rvu->pdev, "PF(s) to AF", intr); + + /* Sync with mbox memory region */ + rmb(); + + rvu_irq_data->rvu_queue_work_hdlr(&rvu->afpf_wq_info, + rvu_irq_data->start, + rvu_irq_data->mdevs, intr); + + return IRQ_HANDLED; +} + +void cn20k_rvu_enable_mbox_intr(struct rvu *rvu) +{ + struct rvu_hwinfo *hw =3D rvu->hw; + + /* Clear spurious irqs, if any */ + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF_INT(0), INTR_MASK(hw->total_pfs)); + + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF_INT(1), INTR_MASK(hw->total_pfs - 64)); + + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF1_INT(0), INTR_MASK(hw->total_pfs)); + + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF1_INT(1), INTR_MASK(hw->total_pfs - 64)); + + /* Enable mailbox interrupt for all PFs except PF0 i.e AF itself */ + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1S(0), + INTR_MASK(hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1S(1), + INTR_MASK(hw->total_pfs - 64)); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1S(0), + INTR_MASK(hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1S(1), + INTR_MASK(hw->total_pfs - 64)); +} + +void cn20k_rvu_unregister_interrupts(struct rvu *rvu) +{ + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1C(0), + INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1C(1), + INTR_MASK(rvu->hw->total_pfs - 64)); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1C(0), + INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1C(1), + INTR_MASK(rvu->hw->total_pfs - 64)); +} + +int cn20k_register_afpf_mbox_intr(struct rvu *rvu) +{ + struct rvu_irq_data *irq_data; + int intr_vec, ret, vec =3D 0; + + /* irq data for 4 PF intr vectors */ + irq_data =3D devm_kcalloc(rvu->dev, 4, + sizeof(struct rvu_irq_data), GFP_KERNEL); + if (!irq_data) + return -ENOMEM; + + for (intr_vec =3D RVU_AF_CN20K_INT_VEC_PFAF_MBOX0; intr_vec <=3D + RVU_AF_CN20K_INT_VEC_PFAF1_MBOX1; intr_vec++, + vec++) { + switch (intr_vec) { + case RVU_AF_CN20K_INT_VEC_PFAF_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF_INT(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_AF_CN20K_INT_VEC_PFAF_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF_INT(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + case RVU_AF_CN20K_INT_VEC_PFAF1_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF1_INT(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_AF_CN20K_INT_VEC_PFAF1_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF1_INT(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + } + irq_data[vec].rvu_queue_work_hdlr =3D rvu_queue_work; + irq_data[vec].vec_num =3D intr_vec; + irq_data[vec].rvu =3D rvu; + + /* Register mailbox interrupt handler */ + sprintf(&rvu->irq_name[intr_vec * NAME_SIZE], + "RVUAF PFAF%d Mbox%d", + vec / 2, vec % 2); + ret =3D request_irq(pci_irq_vector(rvu->pdev, intr_vec), + rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, + &rvu->irq_name[intr_vec * NAME_SIZE], + &irq_data[vec]); + if (ret) + return ret; + + rvu->irq_allocated[intr_vec] =3D true; + } + + return 0; +} + int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, int num, int type, unsigned long *pf_bmap) { @@ -37,6 +168,50 @@ int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **= mbox_addr, return -ENOMEM; } =20 +static struct mbox_ops cn20k_mbox_ops =3D { + .pf_intr_handler =3D cn20k_mbox_pf_common_intr_handler, +}; + +static int rvu_alloc_mbox_memory(struct rvu *rvu, int type, + int ndevs, int mbox_size) +{ + struct qmem *mbox_addr; + dma_addr_t iova; + int pf, err; + + /* Allocate contiguous memory for mailbox communication. + * eg: AF <=3D> PFx mbox memory + * This allocated memory is split into chunks of MBOX_SIZE + * and setup into each of the RVU PFs. In HW this memory will + * get aliased to an offset within BAR2 of those PFs. + * + * AF will access mbox memory using direct physical addresses + * and PFs will access the same shared memory from BAR2. + */ + + err =3D qmem_alloc(rvu->dev, &mbox_addr, ndevs, mbox_size); + if (err) { + dev_err(rvu->dev, "qmem alloc fail\n"); + return -ENOMEM; + } + + switch (type) { + case TYPE_AFPF: + rvu->ng_rvu->pf_mbox_addr =3D mbox_addr; + iova =3D (u64)mbox_addr->iova; + for (pf =3D 0; pf < ndevs; pf++) { + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFX_ADDR(pf), + (u64)iova); + iova +=3D mbox_size; + } + break; + default: + return 0; + } + + return 0; +} + int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int ndevs) { int dev; @@ -44,9 +219,48 @@ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int = ndevs) if (!is_cn20k(rvu->pdev)) return 0; =20 + rvu->ng_rvu->rvu_mbox_ops =3D &cn20k_mbox_ops; + for (dev =3D 0; dev < ndevs; dev++) rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); =20 + return rvu_alloc_mbox_memory(rvu, type, ndevs, MBOX_SIZE); +} + +void cn20k_free_mbox_memory(struct rvu *rvu) +{ + qmem_free(rvu->dev, rvu->ng_rvu->pf_mbox_addr); +} + +int rvu_alloc_cint_qint_mem(struct rvu *rvu, struct rvu_pfvf *pfvf, + int blkaddr, int nixlf) +{ + int qints, hwctx_size, err; + u64 cfg, ctx_cfg; + + ctx_cfg =3D rvu_read64(rvu, blkaddr, NIX_AF_CONST3); + /* Alloc memory for CQINT's HW contexts */ + cfg =3D rvu_read64(rvu, blkaddr, NIX_AF_CONST2); + qints =3D (cfg >> 24) & 0xFFF; + hwctx_size =3D 1UL << ((ctx_cfg >> 24) & 0xF); + err =3D qmem_alloc(rvu->dev, &pfvf->cq_ints_ctx, qints, hwctx_size); + if (err) + return -ENOMEM; + + rvu_write64(rvu, blkaddr, NIX_AF_LFX_CINTS_BASE(nixlf), + (u64)pfvf->cq_ints_ctx->iova); + + /* Alloc memory for QINT's HW contexts */ + cfg =3D rvu_read64(rvu, blkaddr, NIX_AF_CONST2); + qints =3D (cfg >> 12) & 0xFFF; + hwctx_size =3D 1UL << ((ctx_cfg >> 20) & 0xF); + err =3D qmem_alloc(rvu->dev, &pfvf->nix_qints_ctx, qints, hwctx_size); + if (err) + return -ENOMEM; + + rvu_write64(rvu, blkaddr, NIX_AF_LFX_QINTS_BASE(nixlf), + (u64)pfvf->nix_qints_ctx->iova); + return 0; } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/reg.h index 58152a4024ec..df2d52567da7 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h @@ -19,6 +19,23 @@ /* RVU AF BAR0 Mbox registers for AF =3D> PFx */ #define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) #define RVU_MBOX_AF_PFX_CFG(a) (0x6000 | (a) << 4) +#define RVU_MBOX_AF_AFPFX_TRIGX(a) (0x9000 | (a) << 3) +#define RVU_MBOX_AF_PFAF_INT(a) (0x2980 | (a) << 6) +#define RVU_MBOX_AF_PFAF_INT_W1S(a) (0x2988 | (a) << 6) +#define RVU_MBOX_AF_PFAF_INT_ENA_W1S(a) (0x2990 | (a) << 6) +#define RVU_MBOX_AF_PFAF_INT_ENA_W1C(a) (0x2998 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT(a) (0x29A0 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT_W1S(a) (0x29A8 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT_ENA_W1S(a) (0x29B0 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT_ENA_W1C(a) (0x29B8 | (a) << 6) + +/* RVU PF =3D> AF mbox registers */ +#define RVU_MBOX_PF_PFAF_TRIGX(a) (0xC00 | (a) << 3) +#define RVU_MBOX_PF_INT (0xC20) +#define RVU_MBOX_PF_INT_W1S (0xC28) +#define RVU_MBOX_PF_INT_ENA_W1S (0xC30) +#define RVU_MBOX_PF_INT_ENA_W1C (0xC38) + #define RVU_AF_BAR2_SEL (0x9000000) #define RVU_AF_BAR2_PFID (0x16400) #define NIX_CINTX_INT_W1S(a) (0xd30 | (a) << 12) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h b/dri= vers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h new file mode 100644 index 000000000000..fccad6e422e8 --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef STRUCT_H +#define STRUCT_H + +/* RVU Admin function Interrupt Vector Enumeration */ +enum rvu_af_cn20k_int_vec_e { + RVU_AF_CN20K_INT_VEC_POISON =3D 0x0, + RVU_AF_CN20K_INT_VEC_PFFLR0 =3D 0x1, + RVU_AF_CN20K_INT_VEC_PFFLR1 =3D 0x2, + RVU_AF_CN20K_INT_VEC_PFME0 =3D 0x3, + RVU_AF_CN20K_INT_VEC_PFME1 =3D 0x4, + RVU_AF_CN20K_INT_VEC_GEN =3D 0x5, + RVU_AF_CN20K_INT_VEC_PFAF_MBOX0 =3D 0x6, + RVU_AF_CN20K_INT_VEC_PFAF_MBOX1 =3D 0x7, + RVU_AF_CN20K_INT_VEC_PFAF1_MBOX0 =3D 0x8, + RVU_AF_CN20K_INT_VEC_PFAF1_MBOX1 =3D 0x9, + RVU_AF_CN20K_INT_VEC_CNT =3D 0xa, +}; +#endif diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 1e3e72107a9d..15aaa5e166e4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -46,8 +46,10 @@ void __otx2_mbox_reset(struct otx2_mbox *mbox, int devid) mdev->rsp_size =3D 0; tx_hdr->num_msgs =3D 0; tx_hdr->msg_size =3D 0; + tx_hdr->sig =3D 0; rx_hdr->num_msgs =3D 0; rx_hdr->msg_size =3D 0; + rx_hdr->sig =3D 0; } EXPORT_SYMBOL(__otx2_mbox_reset); =20 @@ -71,9 +73,78 @@ void otx2_mbox_destroy(struct otx2_mbox *mbox) } EXPORT_SYMBOL(otx2_mbox_destroy); =20 +int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, + void *reg_base, int direction, int ndevs) +{ + switch (direction) { + case MBOX_DIR_AFPF: + mbox->tx_start =3D MBOX_DOWN_TX_START; + mbox->rx_start =3D MBOX_DOWN_RX_START; + mbox->tx_size =3D MBOX_DOWN_TX_SIZE; + mbox->rx_size =3D MBOX_DOWN_RX_SIZE; + break; + case MBOX_DIR_PFAF: + mbox->tx_start =3D MBOX_DOWN_RX_START; + mbox->rx_start =3D MBOX_DOWN_TX_START; + mbox->tx_size =3D MBOX_DOWN_RX_SIZE; + mbox->rx_size =3D MBOX_DOWN_TX_SIZE; + break; + case MBOX_DIR_AFPF_UP: + mbox->tx_start =3D MBOX_UP_TX_START; + mbox->rx_start =3D MBOX_UP_RX_START; + mbox->tx_size =3D MBOX_UP_TX_SIZE; + mbox->rx_size =3D MBOX_UP_RX_SIZE; + break; + case MBOX_DIR_PFAF_UP: + mbox->tx_start =3D MBOX_UP_RX_START; + mbox->rx_start =3D MBOX_UP_TX_START; + mbox->tx_size =3D MBOX_UP_RX_SIZE; + mbox->rx_size =3D MBOX_UP_TX_SIZE; + break; + default: + return -ENODEV; + } + + switch (direction) { + case MBOX_DIR_AFPF: + mbox->trigger =3D RVU_MBOX_AF_AFPFX_TRIGX(1); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_AFPF_UP: + mbox->trigger =3D RVU_MBOX_AF_AFPFX_TRIGX(0); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_PFAF: + mbox->trigger =3D RVU_MBOX_PF_PFAF_TRIGX(0); + mbox->tr_shift =3D 0; + break; + case MBOX_DIR_PFAF_UP: + mbox->trigger =3D RVU_MBOX_PF_PFAF_TRIGX(1); + mbox->tr_shift =3D 0; + break; + default: + return -ENODEV; + } + mbox->reg_base =3D reg_base; + mbox->pdev =3D pdev; + + mbox->dev =3D kcalloc(ndevs, sizeof(struct otx2_mbox_dev), GFP_KERNEL); + if (!mbox->dev) { + otx2_mbox_destroy(mbox); + return -ENOMEM; + } + mbox->ndevs =3D ndevs; + + return 0; +} + static int otx2_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, void *reg_base, int direction, int ndevs) { + if (is_cn20k(pdev)) + return cn20k_mbox_setup(mbox, pdev, reg_base, + direction, ndevs); + switch (direction) { case MBOX_DIR_AFPF: case MBOX_DIR_PFVF: @@ -252,7 +323,10 @@ static void otx2_mbox_msg_send_data(struct otx2_mbox *= mbox, int devid, u64 data) =20 spin_lock(&mdev->mbox_lock); =20 - tx_hdr->msg_size =3D mdev->msg_size; + if (!tx_hdr->sig) { + tx_hdr->msg_size =3D mdev->msg_size; + tx_hdr->num_msgs =3D mdev->num_msgs; + } =20 /* Reset header for next messages */ mdev->msg_size =3D 0; @@ -266,7 +340,6 @@ static void otx2_mbox_msg_send_data(struct otx2_mbox *m= box, int devid, u64 data) * messages. So this should be written after writing all the messages * to the shared memory. */ - tx_hdr->num_msgs =3D mdev->num_msgs; rx_hdr->num_msgs =3D 0; =20 trace_otx2_msg_send(mbox->pdev, tx_hdr->num_msgs, tx_hdr->msg_size); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.h index df64a18fe1d6..86d07fc4a7ff 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -13,6 +13,7 @@ =20 #include "rvu_struct.h" #include "common.h" +#include "cn20k/struct.h" =20 #define MBOX_SIZE SZ_64K =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index a5ebd7cd3a5c..e49f9bc7ebda 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -755,6 +755,13 @@ static void rvu_free_hw_resources(struct rvu *rvu) =20 rvu_reset_msix(rvu); mutex_destroy(&rvu->rsrc_lock); + + /* Free the QINT/CINt memory */ + pfvf =3D &rvu->pf[RVU_AFPF]; + if (pfvf->nix_qints_ctx) + qmem_free(rvu->dev, pfvf->nix_qints_ctx); + if (pfvf->cq_ints_ctx) + qmem_free(rvu->dev, pfvf->cq_ints_ctx); } =20 static void rvu_setup_pfvf_macaddress(struct rvu *rvu) @@ -2683,6 +2690,11 @@ static void rvu_enable_mbox_intr(struct rvu *rvu) { struct rvu_hwinfo *hw =3D rvu->hw; =20 + if (is_cn20k(rvu->pdev)) { + cn20k_rvu_enable_mbox_intr(rvu); + return; + } + /* Clear spurious irqs, if any */ rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFAF_MBOX_INT, INTR_MASK(hw->total_pfs)); @@ -2936,9 +2948,12 @@ static void rvu_unregister_interrupts(struct rvu *rv= u) =20 rvu_cpt_unregister_interrupts(rvu); =20 - /* Disable the Mbox interrupt */ - rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFAF_MBOX_INT_ENA_W1C, - INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + if (!is_cn20k(rvu->pdev)) + /* Disable the Mbox interrupt */ + rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFAF_MBOX_INT_ENA_W1C, + INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + else + cn20k_rvu_unregister_interrupts(rvu); =20 /* Disable the PF FLR interrupt */ rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFFLR_INT_ENA_W1C, @@ -3001,20 +3016,30 @@ static int rvu_register_interrupts(struct rvu *rvu) return ret; } =20 - /* Register mailbox interrupt handler */ - sprintf(&rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], "RVUAF Mbox"); - ret =3D request_irq(pci_irq_vector - (rvu->pdev, RVU_AF_INT_VEC_MBOX), - rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, - &rvu->irq_name[RVU_AF_INT_VEC_MBOX * - NAME_SIZE], rvu); - if (ret) { - dev_err(rvu->dev, - "RVUAF: IRQ registration failed for mbox\n"); - goto fail; - } + if (!is_cn20k(rvu->pdev)) { + /* Register mailbox interrupt handler */ + sprintf(&rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], + "RVUAF Mbox"); + ret =3D request_irq(pci_irq_vector + (rvu->pdev, RVU_AF_INT_VEC_MBOX), + rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, + &rvu->irq_name[RVU_AF_INT_VEC_MBOX * + NAME_SIZE], rvu); + if (ret) { + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for mbox\n"); + goto fail; + } =20 - rvu->irq_allocated[RVU_AF_INT_VEC_MBOX] =3D true; + rvu->irq_allocated[RVU_AF_INT_VEC_MBOX] =3D true; + } else { + ret =3D cn20k_register_afpf_mbox_intr(rvu); + if (ret) { + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for mbox\n"); + goto fail; + } + } =20 /* Enable mailbox interrupts from all PFs */ rvu_enable_mbox_intr(rvu); @@ -3467,6 +3492,9 @@ static int rvu_probe(struct pci_dev *pdev, const stru= ct pci_device_id *id) ptp_start(rvu, rvu->fwdata->sclk, rvu->fwdata->ptp_ext_clk_rate, rvu->fwdata->ptp_ext_tstamp); =20 + /* Alloc CINT and QINT memory */ + rvu_alloc_cint_qint_mem(rvu, &rvu->pf[RVU_AFPF], BLKADDR_NIX0, + (rvu->hw->block[BLKADDR_NIX0].lf.max)); return 0; err_dl: rvu_unregister_dl(rvu); @@ -3518,6 +3546,8 @@ static void rvu_remove(struct pci_dev *pdev) pci_set_drvdata(pdev, NULL); =20 devm_kfree(&pdev->dev, rvu->hw); + if (is_cn20k(rvu->pdev)) + cn20k_free_mbox_memory(rvu); kfree(rvu->ng_rvu); devm_kfree(&pdev->dev, rvu); } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index 9fd7aea8c481..c3a6947e5e70 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -47,6 +47,9 @@ #define RVU_PFVF_FUNC_MASK rvu_pcifunc_func_mask =20 #ifdef CONFIG_DEBUG_FS + +#define RVU_AFPF 25 + struct dump_ctx { int lf; int id; @@ -444,6 +447,16 @@ struct mbox_wq_info { struct workqueue_struct *mbox_wq; }; =20 +struct rvu_irq_data { + u64 intr_status; + void (*rvu_queue_work_hdlr)(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr); + struct rvu *rvu; + int vec_num; + int start; + int mdevs; +}; + struct mbox_ops { irqreturn_t (*pf_intr_handler)(int irq, void *rvu_irq); }; @@ -956,7 +969,8 @@ int rvu_nix_mcast_get_mce_index(struct rvu *rvu, u16 pc= ifunc, int rvu_nix_mcast_update_mcam_entry(struct rvu *rvu, u16 pcifunc, u32 mcast_grp_idx, u16 mcam_index); void rvu_nix_flr_free_bpids(struct rvu *rvu, u16 pcifunc); - +int rvu_alloc_cint_qint_mem(struct rvu *rvu, struct rvu_pfvf *pfvf, + int blkaddr, int nixlf); /* NPC APIs */ void rvu_npc_freemem(struct rvu *rvu); int rvu_npc_get_pkind(struct rvu *rvu, u16 pf); --=20 2.25.1 From nobody Tue Nov 26 08:50:13 2024 Received: from mx0a-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3D1D20969C; Fri, 18 Oct 2024 20:31:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.148.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283494; cv=none; b=IRkFqgvMdTttmck4LdeaywVESTUCW9TTeSbcW/cCpdoX19mfSXxSmEcddUG1lekrHw9Wn2iDi+lPE+gwHnBggeMSVqKD86DVBA+TFZv0xr+lxB5s8B2tYkduwnz6KJ2Qwd4FebO4ZabZZYvdBVkWpyErMry+K2KajrsGOqhhtSE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283494; c=relaxed/simple; bh=aRrBf8Fg9CvwfIhPCy6HgJD/2AqEL65GLJvBeF86i1o=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IqNvnvQwgPow5ASSdu5xmP5K0jc3hIUb7fht39rYwzvQxbw01WIg7DaPLlCFFs4Gyej5Y4gKymwOz2pzlE5iZf4g/UQopWlkhYabjwucuQWrHYpqnfZ2XnOk3GDpSl9tOw37VWz30Ry8T3iMxFDQm1CevIYdwYGOjHz/fmG4OAc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=JMw9o6Lh; arc=none smtp.client-ip=67.231.148.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="JMw9o6Lh" Received: from pps.filterd (m0431384.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49IAPvws020149; Fri, 18 Oct 2024 13:31:24 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=C mTSg0D6zewqbLtzwCSzlfHbLOVfjAhhIOl4+22EdNA=; b=JMw9o6Lhpz74swfOV KCTLtnP1/wUG7ayMYJpkBj9q9+dUR1Cf6K6/eAptWKqNd4O6Jy2xZLOsTjaotCvW xn3mkGbFDULZ3U4JOhADxTlrAYt8ZRlGpyVfW3ORP3x0v1VB1YIpUqG2ZLMhf4An W0aXVLmbcgjQcA5vY7x8cwVXZB3HC3KFUPxVW+/HWa8j1Pw4AqxTI7x9EHa26IuU HMNQ8dBiK0p5hLvJexVZXn/aqudXzPyRnfivUi2/k4MoYFaFAz6oJ2p1HkQf30xL dgCtwtixDU+pUqmxm/tW3595H4sFspO9NJIXdEIpSc+k+vNIhjHS50WN+Wdr7W1L OABgA== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 42bnwt1as9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 18 Oct 2024 13:31:23 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 18 Oct 2024 13:31:22 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 18 Oct 2024 13:31:22 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id 8141F3F704C; Fri, 18 Oct 2024 13:31:18 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH 4/6] octeontx2-pf: CN20K mbox REQ/ACK implementation for NIC PF Date: Sat, 19 Oct 2024 02:00:56 +0530 Message-ID: <20241018203058.3641959-5-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241018203058.3641959-1-saikrishnag@marvell.com> References: <20241018203058.3641959-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: wB4kcxndR0tc-9Gz0f4lerYFNpC5bFyU X-Proofpoint-ORIG-GUID: wB4kcxndR0tc-9Gz0f4lerYFNpC5bFyU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 Content-Type: text/plain; charset="utf-8" This implementation uses separate trigger interrupts for request, response messages against using trigger message data in CN10K. This patch adds support for basic mbox implementation for CN20K from NIC PF side. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../marvell/octeontx2/af/cn20k/struct.h | 15 ++++ .../ethernet/marvell/octeontx2/nic/Makefile | 2 +- .../ethernet/marvell/octeontx2/nic/cn10k.c | 18 ++++- .../ethernet/marvell/octeontx2/nic/cn10k.h | 1 + .../ethernet/marvell/octeontx2/nic/cn20k.c | 65 +++++++++++++++ .../ethernet/marvell/octeontx2/nic/cn20k.h | 14 ++++ .../marvell/octeontx2/nic/otx2_common.h | 5 ++ .../ethernet/marvell/octeontx2/nic/otx2_pf.c | 80 +++++++++++++++---- .../ethernet/marvell/octeontx2/nic/otx2_reg.h | 4 + .../ethernet/marvell/octeontx2/nic/otx2_vf.c | 6 ++ 10 files changed, 188 insertions(+), 22 deletions(-) create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h b/dri= vers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h index fccad6e422e8..055ccc8c4689 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h @@ -8,6 +8,21 @@ #ifndef STRUCT_H #define STRUCT_H =20 +/** + * CN20k RVU PF MBOX Interrupt Vector Enumeration + * + * Vectors 0 - 3 are compatible with pre cn20k and hence + * existing macros are being reused. + */ +enum rvu_mbox_pf_int_vec_e { + RVU_MBOX_PF_INT_VEC_VFPF_MBOX0 =3D 0x4, + RVU_MBOX_PF_INT_VEC_VFPF_MBOX1 =3D 0x5, + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX0 =3D 0x6, + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1 =3D 0x7, + RVU_MBOX_PF_INT_VEC_AFPF_MBOX =3D 0x8, + RVU_MBOX_PF_INT_VEC_CNT =3D 0x9, +}; + /* RVU Admin function Interrupt Vector Enumeration */ enum rvu_af_cn20k_int_vec_e { RVU_AF_CN20K_INT_VEC_POISON =3D 0x0, diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/Makefile b/drivers/= net/ethernet/marvell/octeontx2/nic/Makefile index 64a97a0a10ed..1e2e838959f4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/Makefile +++ b/drivers/net/ethernet/marvell/octeontx2/nic/Makefile @@ -7,7 +7,7 @@ obj-$(CONFIG_OCTEONTX2_PF) +=3D rvu_nicpf.o otx2_ptp.o obj-$(CONFIG_OCTEONTX2_VF) +=3D rvu_nicvf.o otx2_ptp.o =20 rvu_nicpf-y :=3D otx2_pf.o otx2_common.o otx2_txrx.o otx2_ethtool.o \ - otx2_flows.o otx2_tc.o cn10k.o otx2_dmac_flt.o \ + otx2_flows.o otx2_tc.o cn10k.o cn20k.o otx2_dmac_flt.o \ otx2_devlink.o qos_sq.o qos.o rvu_nicvf-y :=3D otx2_vf.o =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn10k.c index c1c99d7054f8..52e0e036d00b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c @@ -14,6 +14,7 @@ static struct dev_hw_ops otx2_hw_ops =3D { .sqe_flush =3D otx2_sqe_flush, .aura_freeptr =3D otx2_aura_freeptr, .refill_pool_ptrs =3D otx2_refill_pool_ptrs, + .pfaf_mbox_intr_handler =3D otx2_pfaf_mbox_intr_handler, }; =20 static struct dev_hw_ops cn10k_hw_ops =3D { @@ -21,8 +22,20 @@ static struct dev_hw_ops cn10k_hw_ops =3D { .sqe_flush =3D cn10k_sqe_flush, .aura_freeptr =3D cn10k_aura_freeptr, .refill_pool_ptrs =3D cn10k_refill_pool_ptrs, + .pfaf_mbox_intr_handler =3D otx2_pfaf_mbox_intr_handler, }; =20 +void otx2_init_hw_ops(struct otx2_nic *pfvf) +{ + if (!test_bit(CN10K_LMTST, &pfvf->hw.cap_flag)) { + pfvf->hw_ops =3D &otx2_hw_ops; + return; + } + + pfvf->hw_ops =3D &cn10k_hw_ops; +} +EXPORT_SYMBOL(otx2_init_hw_ops); + int cn10k_lmtst_init(struct otx2_nic *pfvf) { =20 @@ -30,12 +43,9 @@ int cn10k_lmtst_init(struct otx2_nic *pfvf) struct otx2_lmt_info *lmt_info; int err, cpu; =20 - if (!test_bit(CN10K_LMTST, &pfvf->hw.cap_flag)) { - pfvf->hw_ops =3D &otx2_hw_ops; + if (!test_bit(CN10K_LMTST, &pfvf->hw.cap_flag)) return 0; - } =20 - pfvf->hw_ops =3D &cn10k_hw_ops; /* Total LMTLINES =3D num_online_cpus() * 32 (For Burst flush).*/ pfvf->tot_lmt_lines =3D (num_online_cpus() * LMT_BURST_SIZE); pfvf->hw.lmt_info =3D alloc_percpu(struct otx2_lmt_info); diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn10k.h index c1861f7de254..bb030816b523 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h @@ -39,4 +39,5 @@ int cn10k_alloc_leaf_profile(struct otx2_nic *pfvf, u16 *= leaf); int cn10k_set_ipolicer_rate(struct otx2_nic *pfvf, u16 profile, u32 burst, u64 rate, bool pps); int cn10k_free_leaf_profile(struct otx2_nic *pfvf, u16 leaf); +void otx2_init_hw_ops(struct otx2_nic *pfvf); #endif /* CN10K_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.c new file mode 100644 index 000000000000..44472a9f9a6e --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell RVU Ethernet driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#include "otx2_common.h" +#include "otx2_reg.h" +#include "otx2_struct.h" +#include "cn10k.h" + +static struct dev_hw_ops cn20k_hw_ops =3D { + .pfaf_mbox_intr_handler =3D cn20k_pfaf_mbox_intr_handler, +}; + +int cn20k_init(struct otx2_nic *pfvf) +{ + pfvf->hw_ops =3D &cn20k_hw_ops; + + return 0; +} +EXPORT_SYMBOL(cn20k_init); +/* CN20K mbox AF =3D> PFx irq handler */ +irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq) +{ + struct otx2_nic *pf =3D (struct otx2_nic *)pf_irq; + struct mbox *mw =3D &pf->mbox; + struct otx2_mbox_dev *mdev; + struct otx2_mbox *mbox; + struct mbox_hdr *hdr; + int pf_trig_val; + + pf_trig_val =3D otx2_read64(pf, RVU_PF_INT) & 0x3; + + /* Clear the IRQ */ + otx2_write64(pf, RVU_PF_INT, pf_trig_val); + + if (pf_trig_val & BIT_ULL(0)) { + mbox =3D &mw->mbox_up; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(pf->mbox_wq, &mw->mbox_up_wrk); + + trace_otx2_msg_interrupt(pf->pdev, "UP message from AF to PF", + BIT_ULL(0)); + } + + if (pf_trig_val & BIT_ULL(1)) { + mbox =3D &mw->mbox; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(pf->mbox_wq, &mw->mbox_wrk); + trace_otx2_msg_interrupt(pf->pdev, "DOWN reply from AF to PF", + BIT_ULL(1)); + } + + return IRQ_HANDLED; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.h new file mode 100644 index 000000000000..da6210042c62 --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Ethernet driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef CN20K_H +#define CN20K_H + +#include "otx2_common.h" + +int cn20k_init(struct otx2_nic *pfvf); +#endif /* CN20K_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 8e7ed3979f80..d0320192e7a5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -30,6 +30,7 @@ #include #include #include "qos.h" +#include "cn20k.h" =20 /* IPv4 flag more fragment bit */ #define IPV4_FLAG_MORE 0x20 @@ -53,6 +54,9 @@ #define NIX_PF_PFC_PRIO_MAX 8 #endif =20 +irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq); +irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq); + enum arua_mapped_qtypes { AURA_NIX_RQ, AURA_NIX_SQ, @@ -373,6 +377,7 @@ struct dev_hw_ops { int size, int qidx); int (*refill_pool_ptrs)(void *dev, struct otx2_cq_queue *cq); void (*aura_freeptr)(void *dev, int aura, u64 buf); + irqreturn_t (*pfaf_mbox_intr_handler)(int irq, void *pf_irq); }; =20 #define CN10K_MCS_SA_PER_SC 4 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 5492dea547a1..c0cf228ac714 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -958,7 +958,7 @@ static void otx2_pfaf_mbox_up_handler(struct work_struc= t *work) otx2_mbox_msg_send(mbox, 0); } =20 -static irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq) +irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq) { struct otx2_nic *pf =3D (struct otx2_nic *)pf_irq; struct mbox *mw =3D &pf->mbox; @@ -1010,10 +1010,18 @@ static irqreturn_t otx2_pfaf_mbox_intr_handler(int = irq, void *pf_irq) =20 static void otx2_disable_mbox_intr(struct otx2_nic *pf) { - int vector =3D pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX); + int vector; =20 /* Disable AF =3D> PF mailbox IRQ */ - otx2_write64(pf, RVU_PF_INT_ENA_W1C, BIT_ULL(0)); + if (!is_cn20k(pf->pdev)) { + vector =3D pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX); + otx2_write64(pf, RVU_PF_INT_ENA_W1C, BIT_ULL(0)); + } else { + vector =3D pci_irq_vector(pf->pdev, + RVU_MBOX_PF_INT_VEC_AFPF_MBOX); + otx2_write64(pf, RVU_PF_INT_ENA_W1C, + BIT_ULL(0) | BIT_ULL(1)); + } free_irq(vector, pf); } =20 @@ -1025,10 +1033,24 @@ static int otx2_register_mbox_intr(struct otx2_nic = *pf, bool probe_af) int err; =20 /* Register mailbox interrupt handler */ - irq_name =3D &hw->irq_name[RVU_PF_INT_VEC_AFPF_MBOX * NAME_SIZE]; - snprintf(irq_name, NAME_SIZE, "RVUPFAF Mbox"); - err =3D request_irq(pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX), - otx2_pfaf_mbox_intr_handler, 0, irq_name, pf); + if (!is_cn20k(pf->pdev)) { + irq_name =3D &hw->irq_name[RVU_PF_INT_VEC_AFPF_MBOX * NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d AFPF Mbox", + rvu_get_pf(pf->pcifunc)); + err =3D request_irq(pci_irq_vector + (pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX), + pf->hw_ops->pfaf_mbox_intr_handler, + 0, irq_name, pf); + } else { + irq_name =3D &hw->irq_name[RVU_MBOX_PF_INT_VEC_AFPF_MBOX * + NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d AFPF Mbox", + rvu_get_pf(pf->pcifunc)); + err =3D request_irq(pci_irq_vector + (pf->pdev, RVU_MBOX_PF_INT_VEC_AFPF_MBOX), + pf->hw_ops->pfaf_mbox_intr_handler, + 0, irq_name, pf); + } if (err) { dev_err(pf->dev, "RVUPF: IRQ registration failed for PFAF mbox irq\n"); @@ -1038,8 +1060,14 @@ static int otx2_register_mbox_intr(struct otx2_nic *= pf, bool probe_af) /* Enable mailbox interrupt for msgs coming from AF. * First clear to avoid spurious interrupts, if any. */ - otx2_write64(pf, RVU_PF_INT, BIT_ULL(0)); - otx2_write64(pf, RVU_PF_INT_ENA_W1S, BIT_ULL(0)); + if (!is_cn20k(pf->pdev)) { + otx2_write64(pf, RVU_PF_INT, BIT_ULL(0)); + otx2_write64(pf, RVU_PF_INT_ENA_W1S, BIT_ULL(0)); + } else { + otx2_write64(pf, RVU_PF_INT, BIT_ULL(0) | BIT_ULL(1)); + otx2_write64(pf, RVU_PF_INT_ENA_W1S, BIT_ULL(0) | + BIT_ULL(1)); + } =20 if (!probe_af) return 0; @@ -1070,7 +1098,7 @@ static void otx2_pfaf_mbox_destroy(struct otx2_nic *p= f) pf->mbox_wq =3D NULL; } =20 - if (mbox->mbox.hwbase) + if (mbox->mbox.hwbase && !is_cn20k(pf->pdev)) iounmap((void __iomem *)mbox->mbox.hwbase); =20 otx2_mbox_destroy(&mbox->mbox); @@ -1089,12 +1117,20 @@ static int otx2_pfaf_mbox_init(struct otx2_nic *pf) if (!pf->mbox_wq) return -ENOMEM; =20 - /* Mailbox is a reserved memory (in RAM) region shared between - * admin function (i.e AF) and this PF, shouldn't be mapped as - * device memory to allow unaligned accesses. + /* For CN20K, AF allocates mbox memory in DRAM and writes PF + * regions/offsets in RVU_MBOX_AF_PFX_ADDR, the RVU_PFX_FUNC_PFAF_MBOX + * gives the aliased address to access AF/PF mailbox regions. */ - hwbase =3D ioremap_wc(pci_resource_start(pf->pdev, PCI_MBOX_BAR_NUM), - MBOX_SIZE); + if (is_cn20k(pf->pdev)) + hwbase =3D pf->reg_base + RVU_PFX_FUNC_PFAF_MBOX + + ((u64)BLKADDR_MBOX << RVU_FUNC_BLKADDR_SHIFT); + else + /* Mailbox is a reserved memory (in RAM) region shared between + * admin function (i.e AF) and this PF, shouldn't be mapped as + * device memory to allow unaligned accesses. + */ + hwbase =3D ioremap_wc(pci_resource_start + (pf->pdev, PCI_MBOX_BAR_NUM), MBOX_SIZE); if (!hwbase) { dev_err(pf->dev, "Unable to map PFAF mailbox region\n"); err =3D -ENOMEM; @@ -2957,8 +2993,13 @@ static int otx2_probe(struct pci_dev *pdev, const st= ruct pci_device_id *id) if (err) goto err_free_netdev; =20 - err =3D pci_alloc_irq_vectors(hw->pdev, RVU_PF_INT_VEC_CNT, - RVU_PF_INT_VEC_CNT, PCI_IRQ_MSIX); + if (!is_cn20k(pf->pdev)) + err =3D pci_alloc_irq_vectors(hw->pdev, RVU_PF_INT_VEC_CNT, + RVU_PF_INT_VEC_CNT, PCI_IRQ_MSIX); + else + err =3D pci_alloc_irq_vectors(hw->pdev, RVU_MBOX_PF_INT_VEC_CNT, + RVU_MBOX_PF_INT_VEC_CNT, + PCI_IRQ_MSIX); if (err < 0) { dev_err(dev, "%s: Failed to alloc %d IRQ vectors\n", __func__, num_vec); @@ -2967,6 +3008,11 @@ static int otx2_probe(struct pci_dev *pdev, const st= ruct pci_device_id *id) =20 otx2_setup_dev_hw_settings(pf); =20 + if (is_cn20k(pf->pdev)) + cn20k_init(pf); + else + otx2_init_hw_ops(pf); + /* Init PF <=3D> AF mailbox stuff */ err =3D otx2_pfaf_mbox_init(pf); if (err) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h b/driver= s/net/ethernet/marvell/octeontx2/nic/otx2_reg.h index 858f084b9d47..901f8cf7f27a 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h @@ -58,6 +58,10 @@ #define RVU_VF_MSIX_PBAX(a) (0xF0000 | (a) << 3) #define RVU_VF_MBOX_REGION (0xC0000) =20 +/* CN20K RVU_MBOX_E: RVU PF/VF MBOX Address Range Enumeration */ +#define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) +#define RVU_PFX_FUNC_PFAF_MBOX (0x80000) + #define RVU_FUNC_BLKADDR_SHIFT 20 #define RVU_FUNC_BLKADDR_MASK 0x1FULL =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 99fcc5661674..76b97896ff1c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -616,6 +616,12 @@ static int otx2vf_probe(struct pci_dev *pdev, const st= ruct pci_device_id *id) } =20 otx2_setup_dev_hw_settings(vf); + + if (is_cn20k(vf->pdev)) + cn20k_init(vf); + else + otx2_init_hw_ops(vf); + /* Init VF <=3D> PF mailbox stuff */ err =3D otx2vf_vfaf_mbox_init(vf); if (err) --=20 2.25.1 From nobody Tue Nov 26 08:50:13 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60D0B212650; Fri, 18 Oct 2024 20:31:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283501; cv=none; b=dJ/LTTPTgJ6jvzXzwJvMoeP3caAr40VWyqBXp73Ae75svQPSDsvLXXq5FmWjpCTR+c+f+KiDccxw7vadsGZEMKjKB4FjNqA+LnfI1qI4QHLTaM8E6J2A1Smn2kcYL9Bp/Bum2X4ljhT+M9A5S/ZsTaKPQJKpCz/3qCUfNPW7G4s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283501; c=relaxed/simple; bh=X6gP0S/SB8k/BJPbtfYufJc5mf6O8ghVTetAiSgDcs0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=AYYhN2/HxoRdWlGjSoNMGui5Ym76vXM2TonTEPet6CLJ8I2DGkyaf1K3zxdNzRMXRF70W+PDLYGElgpteoxTKJpIUkviXRJYKPgd6cB6gxGmF25iuqiRoC4NtrPHm65w0fbOHsbtlQQgQSxMKfg3ARUVGg/2ztO5xJG1ApB1BOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=SBGg7ETk; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="SBGg7ETk" Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49I9R4H6008597; Fri, 18 Oct 2024 13:31:28 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=9 /dtVkMaV90OasPgCe/3SnGzimKOPmuUk5JaHmgf03k=; b=SBGg7ETkt39USSyOC +Hlj7g/U/WsCj1zEoJZyRNKwUMIBG2apsd9Hi5LFaSzjJM8MdnXr+pNLbVZN3EU9 PAw4+2FoiKv1gJJQIVzWkxYLxNNHDTigRYBjOBHH9MbG2KNpaooLkasiQix6kgn/ Ydt3UW4JriPhFDEjEd5IEo9K95ry15MA8csCEtoMfMw1vAVfrQKHGKgnBUgshX4x nRVaVnuwiLnyfD+HArLdXI3anFaKArD79xyCWTakLDgcxWGdbB3dzopXAAT1LqbZ fF92zVRs3l7xIl2X0k2ziCyg4X9czYLpkdCc4XSFBiwy97QrQVtrevst/NcAROZi 2neUw== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42bm18hjpn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 18 Oct 2024 13:31:28 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 18 Oct 2024 13:31:26 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 18 Oct 2024 13:31:26 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id CA1023F704C; Fri, 18 Oct 2024 13:31:22 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH 5/6] octeontx2-af: CN20K mbox implementation for AF's VF Date: Sat, 19 Oct 2024 02:00:57 +0530 Message-ID: <20241018203058.3641959-6-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241018203058.3641959-1-saikrishnag@marvell.com> References: <20241018203058.3641959-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: LjvCVMAwsBuYBbNU4vNFrvSkf1Dp_aTo X-Proofpoint-ORIG-GUID: LjvCVMAwsBuYBbNU4vNFrvSkf1Dp_aTo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 Content-Type: text/plain; charset="utf-8" This patch implements the CN20k MBOX communication between AF and AF's VFs. This implementation uses separate trigger interrupts for request, response messages against using trigger message data in CN10K. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/af/cn20k/api.h | 4 + .../marvell/octeontx2/af/cn20k/mbox_init.c | 163 +++++++++++++++++- .../ethernet/marvell/octeontx2/af/cn20k/reg.h | 37 ++++ .../net/ethernet/marvell/octeontx2/af/mbox.c | 28 +++ .../net/ethernet/marvell/octeontx2/af/rvu.c | 67 ++++--- .../net/ethernet/marvell/octeontx2/af/rvu.h | 3 + .../ethernet/marvell/octeontx2/nic/cn20k.c | 47 +++++ .../marvell/octeontx2/nic/otx2_common.h | 17 ++ .../ethernet/marvell/octeontx2/nic/otx2_pf.c | 6 +- .../ethernet/marvell/octeontx2/nic/otx2_reg.h | 15 ++ .../ethernet/marvell/octeontx2/nic/otx2_vf.c | 38 +++- 11 files changed, 388 insertions(+), 37 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/api.h index 9436a4a4d815..0617bf5a56e9 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h @@ -13,6 +13,7 @@ struct ng_rvu { struct mbox_ops *rvu_mbox_ops; struct qmem *pf_mbox_addr; + struct qmem *vf_mbox_addr; }; =20 struct rvu; @@ -20,6 +21,7 @@ struct rvu; /* Mbox related APIs */ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int num); int cn20k_register_afpf_mbox_intr(struct rvu *rvu); +int cn20k_register_afvf_mbox_intr(struct rvu *rvu, int pf_vec_start); int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, int num, int type, unsigned long *pf_bmap); void cn20k_rvu_enable_mbox_intr(struct rvu *rvu); @@ -27,4 +29,6 @@ void cn20k_rvu_unregister_interrupts(struct rvu *rvu); void cn20k_free_mbox_memory(struct rvu *rvu); int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, void *reg_base, int direction, int ndevs); +void cn20k_rvu_enable_afvf_intr(struct rvu *rvu, int vfs); +void cn20k_rvu_disable_afvf_intr(struct rvu *rvu, int vfs); #endif /* CN20K_API_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c b/= drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c index e19de47da84a..4aefec7f3eec 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c @@ -13,6 +13,91 @@ #include "reg.h" #include "api.h" =20 +static irqreturn_t cn20k_afvf_mbox_intr_handler(int irq, void *rvu_irq) +{ + struct rvu_irq_data *rvu_irq_data =3D (struct rvu_irq_data *)rvu_irq; + struct rvu *rvu =3D rvu_irq_data->rvu; + u64 intr; + + /* Sync with mbox memory region */ + rmb(); + + /* Clear interrupts */ + intr =3D rvupf_read64(rvu, rvu_irq_data->intr_status); + rvupf_write64(rvu, rvu_irq_data->intr_status, intr); + + if (intr) + trace_otx2_msg_interrupt(rvu->pdev, "VF(s) to AF", intr); + + rvu_irq_data->afvf_queue_work_hdlr(&rvu->afvf_wq_info, rvu_irq_data->star= t, + rvu_irq_data->mdevs, intr); + + return IRQ_HANDLED; +} + +int cn20k_register_afvf_mbox_intr(struct rvu *rvu, int pf_vec_start) +{ + struct rvu_irq_data *irq_data; + int intr_vec, offset, vec =3D 0; + int err; + + /* irq data for 4 VFPF intr vectors */ + irq_data =3D devm_kcalloc(rvu->dev, 4, + sizeof(struct rvu_irq_data), GFP_KERNEL); + if (!irq_data) + return -ENOMEM; + + for (intr_vec =3D RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; intr_vec <=3D + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1; + intr_vec++, vec++) { + switch (intr_vec) { + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF1_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1: + irq_data[vec].intr_status =3D RVU_MBOX_PF_VFPF1_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 64; + break; + } + irq_data[vec].afvf_queue_work_hdlr =3D + rvu_queue_work; + offset =3D pf_vec_start + intr_vec; + irq_data[vec].vec_num =3D offset; + irq_data[vec].rvu =3D rvu; + + sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAF VFAF%d Mbox%d", + vec / 2, vec % 2); + err =3D request_irq(pci_irq_vector(rvu->pdev, offset), + rvu->ng_rvu->rvu_mbox_ops->afvf_intr_handler, 0, + &rvu->irq_name[offset * NAME_SIZE], + &irq_data[vec]); + if (err) { + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for AFVF mbox irq\n"); + return err; + } + rvu->irq_allocated[offset] =3D true; + } + + return 0; +} + /* CN20K mbox PFx =3D> AF irq handler */ static irqreturn_t cn20k_mbox_pf_common_intr_handler(int irq, void *rvu_ir= q) { @@ -150,6 +235,21 @@ int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void *= *mbox_addr, int region; u64 bar; =20 + if (type =3D=3D TYPE_AFVF) { + for (region =3D 0; region < num; region++) { + if (!test_bit(region, pf_bmap)) + continue; + + bar =3D (u64)phys_to_virt((u64)rvu->ng_rvu->vf_mbox_addr->base); + bar +=3D region * MBOX_SIZE; + mbox_addr[region] =3D (void *)bar; + + if (!mbox_addr[region]) + goto error; + } + return 0; + } + for (region =3D 0; region < num; region++) { if (!test_bit(region, pf_bmap)) continue; @@ -170,6 +270,7 @@ int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **= mbox_addr, =20 static struct mbox_ops cn20k_mbox_ops =3D { .pf_intr_handler =3D cn20k_mbox_pf_common_intr_handler, + .afvf_intr_handler =3D cn20k_afvf_mbox_intr_handler, }; =20 static int rvu_alloc_mbox_memory(struct rvu *rvu, int type, @@ -187,6 +288,9 @@ static int rvu_alloc_mbox_memory(struct rvu *rvu, int t= ype, * * AF will access mbox memory using direct physical addresses * and PFs will access the same shared memory from BAR2. + * + * PF <=3D> VF mbox memory also works in the same fashion. + * AFPF, PFVF requires IOVA to be used to maintain the mailbox msgs */ =20 err =3D qmem_alloc(rvu->dev, &mbox_addr, ndevs, mbox_size); @@ -205,6 +309,10 @@ static int rvu_alloc_mbox_memory(struct rvu *rvu, int = type, iova +=3D mbox_size; } break; + case TYPE_AFVF: + rvu->ng_rvu->vf_mbox_addr =3D mbox_addr; + rvupf_write64(rvu, RVU_PF_VF_MBOX_ADDR, (u64)mbox_addr->iova); + break; default: return 0; } @@ -221,9 +329,13 @@ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int= ndevs) =20 rvu->ng_rvu->rvu_mbox_ops =3D &cn20k_mbox_ops; =20 - for (dev =3D 0; dev < ndevs; dev++) - rvu_write64(rvu, BLKADDR_RVUM, - RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); + if (type =3D=3D TYPE_AFVF) { + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_PF_VF_CFG, ilog2(MBOX_SIZE)); + } else { + for (dev =3D 0; dev < ndevs; dev++) + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); + } =20 return rvu_alloc_mbox_memory(rvu, type, ndevs, MBOX_SIZE); } @@ -231,6 +343,51 @@ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int= ndevs) void cn20k_free_mbox_memory(struct rvu *rvu) { qmem_free(rvu->dev, rvu->ng_rvu->pf_mbox_addr); + qmem_free(rvu->dev, rvu->ng_rvu->vf_mbox_addr); +} + +void cn20k_rvu_disable_afvf_intr(struct rvu *rvu, int vfs) +{ + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1CX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1CX(0), INTR_MASK(vfs)); + + if (vfs <=3D 64) + return; + + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)= ); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); +} + +void cn20k_rvu_enable_afvf_intr(struct rvu *rvu, int vfs) +{ + /* Clear any pending interrupts and enable AF VF interrupts for + * the first 64 VFs. + */ + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INTX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INTX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(0), INTR_MASK(vfs)); + + /* FLR */ + rvupf_write64(rvu, RVU_PF_VFFLR_INTX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1SX(0), INTR_MASK(vfs)); + + /* Same for remaining VFs, if any. */ + if (vfs <=3D 64) + return; + + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INTX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INTX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)= ); + + rvupf_write64(rvu, RVU_PF_VFFLR_INTX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)); } =20 int rvu_alloc_cint_qint_mem(struct rvu *rvu, struct rvu_pfvf *pfvf, diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/reg.h index df2d52567da7..affb39803120 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h @@ -41,4 +41,41 @@ #define NIX_CINTX_INT_W1S(a) (0xd30 | (a) << 12) #define NIX_QINTX_CNT(a) (0xc00 | (a) << 12) =20 +#define RVU_MBOX_AF_VFAF_INT(a) (0x3000 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_W1S(a) (0x3008 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_ENA_W1S(a) (0x3010 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_ENA_W1C(a) (0x3018 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_ENA_W1C(a) (0x3018 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_INT(a) (0x3020 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_INT_W1S(a) (0x3028 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_IN_ENA_W1S(a) (0x3030 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_IN_ENA_W1C(a) (0x3038 | (a) << 6) + +#define RVU_MBOX_AF_AFVFX_TRIG(a, b) (0x10000 | (a) << 4 | (b) << 3) +#define RVU_MBOX_AF_VFX_ADDR(a) (0x20000 | (a) << 4) +#define RVU_MBOX_AF_VFX_CFG(a) (0x28000 | (a) << 4) + +#define RVU_MBOX_PF_VFX_PFVF_TRIGX(a) (0x2000 | (a) << 3) + +#define RVU_MBOX_PF_VFPF_INTX(a) (0x1000 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_W1SX(a) (0x1020 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1SX(a) (0x1040 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1CX(a) (0x1060 | (a) << 3) + +#define RVU_MBOX_PF_VFPF1_INTX(a) (0x1080 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_W1SX(a) (0x10a0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(a) (0x10c0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(a) (0x10e0 | (a) << 3) + +#define RVU_MBOX_PF_VF_ADDR (0xC40) +#define RVU_MBOX_PF_LMTLINE_ADDR (0xC48) +#define RVU_MBOX_PF_VF_CFG (0xC60) + +#define RVU_MBOX_VF_VFPF_TRIGX(a) (0x3000 | (a) << 3) +#define RVU_MBOX_VF_INT (0x20) +#define RVU_MBOX_VF_INT_W1S (0x28) +#define RVU_MBOX_VF_INT_ENA_W1S (0x30) +#define RVU_MBOX_VF_INT_ENA_W1C (0x38) + +#define RVU_MBOX_VF_VFAF_TRIGX(a) (0x2000 | (a) << 3) #endif /* RVU_MBOX_REG_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 15aaa5e166e4..18b40a9a9784 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -76,26 +76,38 @@ EXPORT_SYMBOL(otx2_mbox_destroy); int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, void *reg_base, int direction, int ndevs) { + /* For CN20K, PF and VF bit encodings in PCIFUNC are changed. + * Hence set the PF and VF masks here. + */ + rvu_pcifunc_pf_shift =3D 9; + rvu_pcifunc_pf_mask =3D 0x7F; + rvu_pcifunc_func_shift =3D 0; + rvu_pcifunc_func_mask =3D 0x1FF; + switch (direction) { case MBOX_DIR_AFPF: + case MBOX_DIR_PFVF: mbox->tx_start =3D MBOX_DOWN_TX_START; mbox->rx_start =3D MBOX_DOWN_RX_START; mbox->tx_size =3D MBOX_DOWN_TX_SIZE; mbox->rx_size =3D MBOX_DOWN_RX_SIZE; break; case MBOX_DIR_PFAF: + case MBOX_DIR_VFPF: mbox->tx_start =3D MBOX_DOWN_RX_START; mbox->rx_start =3D MBOX_DOWN_TX_START; mbox->tx_size =3D MBOX_DOWN_RX_SIZE; mbox->rx_size =3D MBOX_DOWN_TX_SIZE; break; case MBOX_DIR_AFPF_UP: + case MBOX_DIR_PFVF_UP: mbox->tx_start =3D MBOX_UP_TX_START; mbox->rx_start =3D MBOX_UP_RX_START; mbox->tx_size =3D MBOX_UP_TX_SIZE; mbox->rx_size =3D MBOX_UP_RX_SIZE; break; case MBOX_DIR_PFAF_UP: + case MBOX_DIR_VFPF_UP: mbox->tx_start =3D MBOX_UP_RX_START; mbox->rx_start =3D MBOX_UP_TX_START; mbox->tx_size =3D MBOX_UP_RX_SIZE; @@ -122,6 +134,22 @@ int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pc= i_dev *pdev, mbox->trigger =3D RVU_MBOX_PF_PFAF_TRIGX(1); mbox->tr_shift =3D 0; break; + case MBOX_DIR_PFVF: + mbox->trigger =3D RVU_MBOX_PF_VFX_PFVF_TRIGX(1); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_PFVF_UP: + mbox->trigger =3D RVU_MBOX_PF_VFX_PFVF_TRIGX(0); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_VFPF: + mbox->trigger =3D RVU_MBOX_VF_VFPF_TRIGX(0); + mbox->tr_shift =3D 0; + break; + case MBOX_DIR_VFPF_UP: + mbox->trigger =3D RVU_MBOX_VF_VFPF_TRIGX(1); + mbox->tr_shift =3D 0; + break; default: return -ENODEV; } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index e49f9bc7ebda..1ddd12e49cac 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -2443,6 +2443,7 @@ static int rvu_get_mbox_regions(struct rvu *rvu, void= **mbox_addr, =20 static struct mbox_ops rvu_mbox_ops =3D { .pf_intr_handler =3D rvu_mbox_pf_intr_handler, + .afvf_intr_handler =3D rvu_mbox_intr_handler, }; =20 static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw, @@ -2986,6 +2987,10 @@ static int rvu_afvf_msix_vectors_num_ok(struct rvu *= rvu) * VF interrupts can be handled. Offset equal to zero means * that PF vectors are not configured and overlapping AF vectors. */ + if (is_cn20k(rvu->pdev)) + return (pfvf->msix.max >=3D RVU_AF_CN20K_INT_VEC_CNT + + RVU_MBOX_PF_INT_VEC_CNT) && offset; + return (pfvf->msix.max >=3D RVU_AF_INT_VEC_CNT + RVU_PF_INT_VEC_CNT) && offset; } @@ -3094,34 +3099,40 @@ static int rvu_register_interrupts(struct rvu *rvu) /* Get PF MSIX vectors offset. */ pf_vec_start =3D rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_INT_CFG(0)) & 0x3ff; + if (!is_cn20k(rvu->pdev)) { + /* Register MBOX0 interrupt. */ + offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX0; + sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox0"); + ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), + rvu->ng_rvu->rvu_mbox_ops->afvf_intr_handler, 0, + &rvu->irq_name[offset * NAME_SIZE], + rvu); + if (ret) + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for Mbox0\n"); =20 - /* Register MBOX0 interrupt. */ - offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX0; - sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox0"); - ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), - rvu_mbox_intr_handler, 0, - &rvu->irq_name[offset * NAME_SIZE], - rvu); - if (ret) - dev_err(rvu->dev, - "RVUAF: IRQ registration failed for Mbox0\n"); - - rvu->irq_allocated[offset] =3D true; + rvu->irq_allocated[offset] =3D true; =20 - /* Register MBOX1 interrupt. MBOX1 IRQ number follows MBOX0 so - * simply increment current offset by 1. - */ - offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX1; - sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox1"); - ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), - rvu_mbox_intr_handler, 0, - &rvu->irq_name[offset * NAME_SIZE], - rvu); - if (ret) - dev_err(rvu->dev, - "RVUAF: IRQ registration failed for Mbox1\n"); + /* Register MBOX1 interrupt. MBOX1 IRQ number follows MBOX0 so + * simply increment current offset by 1. + */ + offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX1; + sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox1"); + ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), + rvu->ng_rvu->rvu_mbox_ops->afvf_intr_handler, 0, + &rvu->irq_name[offset * NAME_SIZE], + rvu); + if (ret) + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for Mbox1\n"); =20 - rvu->irq_allocated[offset] =3D true; + rvu->irq_allocated[offset] =3D true; + } else { + ret =3D cn20k_register_afvf_mbox_intr(rvu, pf_vec_start); + if (ret) + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for Mbox\n"); + } =20 /* Register FLR interrupt handler for AF's VFs */ offset =3D pf_vec_start + RVU_PF_INT_VEC_VFFLR0; @@ -3232,6 +3243,9 @@ static void rvu_disable_afvf_intr(struct rvu *rvu) { int vfs =3D rvu->vfs; =20 + if (is_cn20k(rvu->pdev)) + return cn20k_rvu_disable_afvf_intr(rvu, vfs); + rvupf_write64(rvu, RVU_PF_VFPF_MBOX_INT_ENA_W1CX(0), INTR_MASK(vfs)); rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1CX(0), INTR_MASK(vfs)); rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1CX(0), INTR_MASK(vfs)); @@ -3248,6 +3262,9 @@ static void rvu_enable_afvf_intr(struct rvu *rvu) { int vfs =3D rvu->vfs; =20 + if (is_cn20k(rvu->pdev)) + return cn20k_rvu_enable_afvf_intr(rvu, vfs); + /* Clear any pending interrupts and enable AF VF interrupts for * the first 64 VFs. */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index c3a6947e5e70..b22425b36675 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -451,6 +451,8 @@ struct rvu_irq_data { u64 intr_status; void (*rvu_queue_work_hdlr)(struct mbox_wq_info *mw, int first, int mdevs, u64 intr); + void (*afvf_queue_work_hdlr)(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr); struct rvu *rvu; int vec_num; int start; @@ -459,6 +461,7 @@ struct rvu_irq_data { =20 struct mbox_ops { irqreturn_t (*pf_intr_handler)(int irq, void *rvu_irq); + irqreturn_t (*afvf_intr_handler)(int irq, void *rvu_irq); }; =20 struct channel_fwdata { diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.c index 44472a9f9a6e..219a0b1d0c9b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c @@ -12,6 +12,7 @@ =20 static struct dev_hw_ops cn20k_hw_ops =3D { .pfaf_mbox_intr_handler =3D cn20k_pfaf_mbox_intr_handler, + .vfaf_mbox_intr_handler =3D cn20k_vfaf_mbox_intr_handler, }; =20 int cn20k_init(struct otx2_nic *pfvf) @@ -63,3 +64,49 @@ irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *= pf_irq) =20 return IRQ_HANDLED; } + +irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, void *vf_irq) +{ + struct otx2_nic *vf =3D (struct otx2_nic *)vf_irq; + struct otx2_mbox_dev *mdev; + struct otx2_mbox *mbox; + struct mbox_hdr *hdr; + int vf_trig_val; + + vf_trig_val =3D otx2_read64(vf, RVU_VF_INT) & 0x3; + /* Clear the IRQ */ + otx2_write64(vf, RVU_VF_INT, vf_trig_val); + + /* Read latest mbox data */ + smp_rmb(); + + if (vf_trig_val & BIT_ULL(1)) { + /* Check for PF =3D> VF response messages */ + mbox =3D &vf->mbox.mbox; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(vf->mbox_wq, &vf->mbox.mbox_wrk); + + trace_otx2_msg_interrupt(mbox->pdev, "DOWN reply from PF0 to VF", + BIT_ULL(1)); + } + + if (vf_trig_val & BIT_ULL(0)) { + /* Check for PF =3D> VF notification messages */ + mbox =3D &vf->mbox.mbox_up; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(vf->mbox_wq, &vf->mbox.mbox_up_wrk); + + trace_otx2_msg_interrupt(mbox->pdev, "UP message from PF0 to VF", + BIT_ULL(0)); + } + + return IRQ_HANDLED; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index d0320192e7a5..edebdd9ce1e1 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -56,6 +56,8 @@ =20 irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq); +irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, void *vf_irq); +irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq); =20 enum arua_mapped_qtypes { AURA_NIX_RQ, @@ -258,6 +260,7 @@ struct otx2_hw { u16 nix_msixoff; /* Offset of NIX vectors */ char *irq_name; cpumask_var_t *affinity_mask; + struct pf_irq_data *pfvf_irq_devid[4]; =20 /* Stats */ struct otx2_dev_stats dev_stats; @@ -378,6 +381,7 @@ struct dev_hw_ops { int (*refill_pool_ptrs)(void *dev, struct otx2_cq_queue *cq); void (*aura_freeptr)(void *dev, int aura, u64 buf); irqreturn_t (*pfaf_mbox_intr_handler)(int irq, void *pf_irq); + irqreturn_t (*vfaf_mbox_intr_handler)(int irq, void *pf_irq); }; =20 #define CN10K_MCS_SA_PER_SC 4 @@ -445,6 +449,16 @@ struct cn10k_mcs_cfg { struct list_head rxsc_list; }; =20 +struct pf_irq_data { + u64 intr_status; + void (*pf_queue_work_hdlr)(struct mbox *mb, struct workqueue_struct *mw, + int first, int mdevs, u64 intr); + struct otx2_nic *pf; + int vec_num; + int start; + int mdevs; +}; + struct otx2_nic { void __iomem *reg_base; struct net_device *netdev; @@ -485,6 +499,7 @@ struct otx2_nic { struct mbox *mbox_pfvf; struct workqueue_struct *mbox_wq; struct workqueue_struct *mbox_pfvf_wq; + struct qmem *pfvf_mbox_addr; =20 u8 total_vfs; u16 pcifunc; /* RVU PF_FUNC */ @@ -1123,4 +1138,6 @@ u16 otx2_select_queue(struct net_device *netdev, stru= ct sk_buff *skb, int otx2_get_txq_by_classid(struct otx2_nic *pfvf, u16 classid); void otx2_qos_config_txschq(struct otx2_nic *pfvf); void otx2_clean_qos_queues(struct otx2_nic *pfvf); +void otx2_queue_vf_work(struct mbox *mw, struct workqueue_struct *mbox_wq, + int first, int mdevs, u64 intr); #endif /* OTX2_COMMON_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_pf.c index c0cf228ac714..148a5c91af55 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -292,8 +292,8 @@ static int otx2_pf_flr_init(struct otx2_nic *pf, int nu= m_vfs) return 0; } =20 -static void otx2_queue_vf_work(struct mbox *mw, struct workqueue_struct *m= box_wq, - int first, int mdevs, u64 intr) +void otx2_queue_vf_work(struct mbox *mw, struct workqueue_struct *mbox_wq, + int first, int mdevs, u64 intr) { struct otx2_mbox_dev *mdev; struct otx2_mbox *mbox; @@ -536,7 +536,7 @@ static void otx2_pfvf_mbox_up_handler(struct work_struc= t *work) } } =20 -static irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq) +irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq) { struct otx2_nic *pf =3D (struct otx2_nic *)(pf_irq); int vfs =3D pf->total_vfs; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h b/driver= s/net/ethernet/marvell/octeontx2/nic/otx2_reg.h index 901f8cf7f27a..1cd576fd09c5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h @@ -44,6 +44,17 @@ #define RVU_PF_VF_MBOX_ADDR (0xC40) #define RVU_PF_LMTLINE_ADDR (0xC48) =20 +#define RVU_MBOX_PF_VFX_PFVF_TRIGX(a) (0x2000 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INTX(a) (0x1000 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_W1SX(a) (0x1020 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1SX(a) (0x1040 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1CX(a) (0x1060 | (a) << 3) + +#define RVU_MBOX_PF_VFPF1_INTX(a) (0x1080 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_W1SX(a) (0x10a0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(a) (0x10c0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(a) (0x10e0 | (a) << 3) + /* RVU VF registers */ #define RVU_VF_VFPF_MBOX0 (0x00000) #define RVU_VF_VFPF_MBOX1 (0x00008) @@ -61,6 +72,7 @@ /* CN20K RVU_MBOX_E: RVU PF/VF MBOX Address Range Enumeration */ #define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) #define RVU_PFX_FUNC_PFAF_MBOX (0x80000) +#define RVU_PFX_FUNCX_VFAF_MBOX (0x40000) =20 #define RVU_FUNC_BLKADDR_SHIFT 20 #define RVU_FUNC_BLKADDR_MASK 0x1FULL @@ -147,4 +159,7 @@ #define LMT_LF_LMTLINEX(a) (LMT_LFBASE | 0x000 | (a) << 12) #define LMT_LF_LMTCANCEL (LMT_LFBASE | 0x400) =20 +/* CN20K registers */ +#define RVU_PF_DISC (0x0) + #endif /* OTX2_REG_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 76b97896ff1c..95bdc65a989c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -238,6 +238,10 @@ static void otx2vf_disable_mbox_intr(struct otx2_nic *= vf) =20 /* Disable VF =3D> PF mailbox IRQ */ otx2_write64(vf, RVU_VF_INT_ENA_W1C, BIT_ULL(0)); + + if (is_cn20k(vf->pdev)) + otx2_write64(vf, RVU_VF_INT_ENA_W1C, BIT_ULL(0) | BIT_ULL(1)); + free_irq(vector, vf); } =20 @@ -250,9 +254,18 @@ static int otx2vf_register_mbox_intr(struct otx2_nic *= vf, bool probe_pf) =20 /* Register mailbox interrupt handler */ irq_name =3D &hw->irq_name[RVU_VF_INT_VEC_MBOX * NAME_SIZE]; - snprintf(irq_name, NAME_SIZE, "RVUVFAF Mbox"); - err =3D request_irq(pci_irq_vector(vf->pdev, RVU_VF_INT_VEC_MBOX), - otx2vf_vfaf_mbox_intr_handler, 0, irq_name, vf); + snprintf(irq_name, NAME_SIZE, "RVUVF%d AFVF Mbox", ((vf->pcifunc & + RVU_PFVF_FUNC_MASK) - 1)); + + if (!is_cn20k(vf->pdev)) { + err =3D request_irq(pci_irq_vector(vf->pdev, RVU_VF_INT_VEC_MBOX), + otx2vf_vfaf_mbox_intr_handler, 0, irq_name, vf); + } else { + err =3D request_irq(pci_irq_vector(vf->pdev, RVU_VF_INT_VEC_MBOX), + vf->hw_ops->vfaf_mbox_intr_handler, 0, irq_name, + vf); + } + if (err) { dev_err(vf->dev, "RVUPF: IRQ registration failed for VFAF mbox irq\n"); @@ -262,8 +275,15 @@ static int otx2vf_register_mbox_intr(struct otx2_nic *= vf, bool probe_pf) /* Enable mailbox interrupt for msgs coming from PF. * First clear to avoid spurious interrupts, if any. */ - otx2_write64(vf, RVU_VF_INT, BIT_ULL(0)); - otx2_write64(vf, RVU_VF_INT_ENA_W1S, BIT_ULL(0)); + if (!is_cn20k(vf->pdev)) { + otx2_write64(vf, RVU_VF_INT, BIT_ULL(0)); + otx2_write64(vf, RVU_VF_INT_ENA_W1S, BIT_ULL(0)); + } else { + otx2_write64(vf, RVU_VF_INT, BIT_ULL(0) | BIT_ULL(1) | + BIT_ULL(2) | BIT_ULL(3)); + otx2_write64(vf, RVU_VF_INT_ENA_W1S, BIT_ULL(0) | + BIT_ULL(1) | BIT_ULL(2) | BIT_ULL(3)); + } =20 if (!probe_pf) return 0; @@ -313,7 +333,13 @@ static int otx2vf_vfaf_mbox_init(struct otx2_nic *vf) if (!vf->mbox_wq) return -ENOMEM; =20 - if (test_bit(CN10K_MBOX, &vf->hw.cap_flag)) { + /* For cn20k platform, VF mailbox region is in dram aliased from AF + * VF MBOX ADDR, MBOX is a separate RVU block. + */ + if (is_cn20k(vf->pdev)) { + hwbase =3D vf->reg_base + RVU_VF_MBOX_REGION + ((u64)BLKADDR_MBOX << + RVU_FUNC_BLKADDR_SHIFT); + } else if (test_bit(CN10K_MBOX, &vf->hw.cap_flag)) { /* For cn10k platform, VF mailbox region is in its BAR2 * register space */ --=20 2.25.1 From nobody Tue Nov 26 08:50:13 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9088A212D1A; Fri, 18 Oct 2024 20:31:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283502; cv=none; b=npA5Hy/KJBSqBvs2u1VLBk3FFqfhuyGlmgJfbG/jWOtWsU+ENn8hB7l+/vTM30tCikaGiLz8kbSbfPbBdQNEuagFMX6hqJhr6IzIU4DOzAgN8It17IZ1VuGiY6gG8xu7kpFIlQytM229KsXUc2h8YJPY0oLiVPzRqVAjYBaj3h0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729283502; c=relaxed/simple; bh=Fj/8NKTV5uK5CJ66XD0//mKsX90Okyu8DByGYjiDSeA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=npkELlfQ+xV4y2pWNpQWOM5NOiVFuB9SSxsZux1mUn+ezFbacAbhaag6crPn3+s2Ufh6zaDkZx45WTv2dFNE5jic2OMKnL37xInnIdF19aCzZAiq2GmSG0tIvrW+rN9GnXcV0iE1vpe9H9STIFOXb5l6O1u9ulY+CCGUgolwBf4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=ByTccqxF; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="ByTccqxF" Received: from pps.filterd (m0431383.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49IEPYPH001361; Fri, 18 Oct 2024 13:31:32 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=m aWJua9+JGzP7BnlMk+LeZpIIL5TRzHIue5sz8vH9Q4=; b=ByTccqxF1mzqhN4mn sm9mzihO3PdSh1h7lUmKLirSG1gXYO8w20bnxTsvQkej1rpPLlfIrY09DJsSR7WP 1jWdoM46W9/QfAmgljP/n4u+d+L/YeJzAxqm4lMk+0DuRgnRQj8Nsc0QRhxgyGDN WXlxGagrXb0ltM8WMxcznkcxrevghPmynIthpLUHJ18TUwJ9La55RFUdss8osMhx TLyWjK8oBzoceZZexVpZV31eivIOt7E+dCPwm1ELH0y1nF5M4uMIEd+WwT/WeROx +IxzP9thImHhFqKMU1i2wytWPRl4b5/TM/+l4vlmcaYKCDPYysVwycXZS5OKr9cE CA5qg== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42bse6ru97-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 18 Oct 2024 13:31:32 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 18 Oct 2024 13:31:30 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 18 Oct 2024 13:31:30 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id 1E8593F704C; Fri, 18 Oct 2024 13:31:26 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH 6/6] octeontx2-pf: CN20K mbox implementation between PF-VF Date: Sat, 19 Oct 2024 02:00:58 +0530 Message-ID: <20241018203058.3641959-7-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241018203058.3641959-1-saikrishnag@marvell.com> References: <20241018203058.3641959-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-ORIG-GUID: Z-MdKzW6TrQSXnP-PYgimHTu7-OtU5mK X-Proofpoint-GUID: Z-MdKzW6TrQSXnP-PYgimHTu7-OtU5mK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 Content-Type: text/plain; charset="utf-8" This patch implements the CN20k MBOX communication between PF and it's VFs. CN20K silicon got extra interrupt of MBOX response for trigger interrupt. Also few of the CSR offsets got changed in CN20K against prior series of silicons. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/nic/cn20k.c | 142 ++++++++++++++++++ .../ethernet/marvell/octeontx2/nic/cn20k.h | 3 + .../marvell/octeontx2/nic/otx2_common.h | 2 + .../ethernet/marvell/octeontx2/nic/otx2_pf.c | 59 ++++++-- 4 files changed, 193 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.c index 219a0b1d0c9b..8a130a2771b1 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c @@ -13,6 +13,7 @@ static struct dev_hw_ops cn20k_hw_ops =3D { .pfaf_mbox_intr_handler =3D cn20k_pfaf_mbox_intr_handler, .vfaf_mbox_intr_handler =3D cn20k_vfaf_mbox_intr_handler, + .pfvf_mbox_intr_handler =3D cn20k_pfvf_mbox_intr_handler, }; =20 int cn20k_init(struct otx2_nic *pfvf) @@ -110,3 +111,144 @@ irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, voi= d *vf_irq) =20 return IRQ_HANDLED; } + +void cn20k_enable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs) +{ + /* Clear PF <=3D> VF mailbox IRQ */ + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(1), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(1), ~0ull); + + /* Enable PF <=3D> VF mailbox IRQ */ + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(0), INTR_MASK(numvfs)); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(0), INTR_MASK(numvfs)); + if (numvfs > 64) { + numvfs -=3D 64; + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(1), + INTR_MASK(numvfs)); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(1), + INTR_MASK(numvfs)); + } +} + +void cn20k_disable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs) +{ + int vector, intr_vec, vec =3D 0; + + /* Disable PF <=3D> VF mailbox IRQ */ + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(1), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(1), ~0ull); + + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(0), ~0ull); + + if (numvfs > 64) { + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(1), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(1), ~0ull); + } + + for (intr_vec =3D RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; intr_vec <=3D + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1; intr_vec++, vec++) { + vector =3D pci_irq_vector(pf->pdev, intr_vec); + free_irq(vector, pf->hw.pfvf_irq_devid[vec]); + } +} + +irqreturn_t cn20k_pfvf_mbox_intr_handler(int irq, void *pf_irq) +{ + struct pf_irq_data *irq_data =3D (struct pf_irq_data *)(pf_irq); + struct otx2_nic *pf =3D irq_data->pf; + struct mbox *mbox; + u64 intr; + + /* Sync with mbox memory region */ + rmb(); + + /* Clear interrupts */ + intr =3D otx2_read64(pf, irq_data->intr_status); + otx2_write64(pf, irq_data->intr_status, intr); + mbox =3D pf->mbox_pfvf; + + if (intr) + trace_otx2_msg_interrupt(pf->pdev, "VF(s) to PF", intr); + + irq_data->pf_queue_work_hdlr(mbox, pf->mbox_pfvf_wq, irq_data->start, + irq_data->mdevs, intr); + + return IRQ_HANDLED; +} + +int cn20k_register_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs) +{ + struct otx2_hw *hw =3D &pf->hw; + struct pf_irq_data *irq_data; + int intr_vec, ret, vec =3D 0; + char *irq_name; + + /* irq data for 4 PF intr vectors */ + irq_data =3D devm_kcalloc(pf->dev, 4, + sizeof(struct pf_irq_data), GFP_KERNEL); + if (!irq_data) + return -ENOMEM; + + for (intr_vec =3D RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; intr_vec <=3D + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1; intr_vec++, vec++) { + switch (intr_vec) { + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF1_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF1_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + } + irq_data[vec].pf_queue_work_hdlr =3D otx2_queue_vf_work; + irq_data[vec].vec_num =3D intr_vec; + irq_data[vec].pf =3D pf; + + /* Register mailbox interrupt handler */ + irq_name =3D &hw->irq_name[intr_vec * NAME_SIZE]; + if (pf->pcifunc) + snprintf(irq_name, NAME_SIZE, + "RVUPF%d_VF%d Mbox%d", rvu_get_pf(pf->pcifunc), + vec / 2, vec % 2); + else + snprintf(irq_name, NAME_SIZE, "RVUPF_VF%d Mbox%d", + vec / 2, vec % 2); + + hw->pfvf_irq_devid[vec] =3D &irq_data[vec]; + ret =3D request_irq(pci_irq_vector(pf->pdev, intr_vec), + pf->hw_ops->pfvf_mbox_intr_handler, 0, + irq_name, + &irq_data[vec]); + if (ret) { + dev_err(pf->dev, + "RVUPF: IRQ registration failed for PFVF mbox0 irq\n"); + return ret; + } + } + + cn20k_enable_pfvf_mbox_intr(pf, numvfs); + + return 0; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.h index da6210042c62..763acaa31650 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h @@ -11,4 +11,7 @@ #include "otx2_common.h" =20 int cn20k_init(struct otx2_nic *pfvf); +int cn20k_register_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs); +void cn20k_disable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs); +void cn20k_enable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs); #endif /* CN20K_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index edebdd9ce1e1..e56df7aa1d57 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -57,6 +57,7 @@ irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, void *vf_irq); +irqreturn_t cn20k_pfvf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq); =20 enum arua_mapped_qtypes { @@ -382,6 +383,7 @@ struct dev_hw_ops { void (*aura_freeptr)(void *dev, int aura, u64 buf); irqreturn_t (*pfaf_mbox_intr_handler)(int irq, void *pf_irq); irqreturn_t (*vfaf_mbox_intr_handler)(int irq, void *pf_irq); + irqreturn_t (*pfvf_mbox_intr_handler)(int irq, void *pf_irq); }; =20 #define CN10K_MCS_SA_PER_SC 4 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 148a5c91af55..cd7d6472f5e1 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -565,6 +565,23 @@ irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void = *pf_irq) return IRQ_HANDLED; } =20 +static void *cn20k_pfvf_mbox_alloc(struct otx2_nic *pf, int numvfs) +{ + struct qmem *mbox_addr; + int err; + + err =3D qmem_alloc(&pf->pdev->dev, &mbox_addr, numvfs, MBOX_SIZE); + if (err) { + dev_err(pf->dev, "qmem alloc fail\n"); + return ERR_PTR(-ENOMEM); + } + + otx2_write64(pf, RVU_PF_VF_MBOX_ADDR, (u64)mbox_addr->iova); + pf->pfvf_mbox_addr =3D mbox_addr; + + return mbox_addr->base; +} + static int otx2_pfvf_mbox_init(struct otx2_nic *pf, int numvfs) { void __iomem *hwbase; @@ -586,20 +603,28 @@ static int otx2_pfvf_mbox_init(struct otx2_nic *pf, i= nt numvfs) if (!pf->mbox_pfvf_wq) return -ENOMEM; =20 - /* On CN10K platform, PF <-> VF mailbox region follows after - * PF <-> AF mailbox region. + /* For CN20K, PF allocates mbox memory in DRAM and writes PF/VF + * regions/offsets in RVU_PF_VF_MBOX_ADDR, the RVU_PFX_FUNC_PFAF_MBOX + * gives the aliased address to access PF/VF mailbox regions. */ - if (test_bit(CN10K_MBOX, &pf->hw.cap_flag)) - base =3D pci_resource_start(pf->pdev, PCI_MBOX_BAR_NUM) + - MBOX_SIZE; - else - base =3D readq((void __iomem *)((u64)pf->reg_base + + if (is_cn20k(pf->pdev)) { + hwbase =3D cn20k_pfvf_mbox_alloc(pf, numvfs); + } else { + /* On CN10K platform, PF <-> VF mailbox region follows after + * PF <-> AF mailbox region. + */ + if (test_bit(CN10K_MBOX, &pf->hw.cap_flag)) + base =3D pci_resource_start(pf->pdev, PCI_MBOX_BAR_NUM) + + MBOX_SIZE; + else + base =3D readq((void __iomem *)((u64)pf->reg_base + RVU_PF_VF_BAR4_ADDR)); =20 - hwbase =3D ioremap_wc(base, MBOX_SIZE * pf->total_vfs); - if (!hwbase) { - err =3D -ENOMEM; - goto free_wq; + hwbase =3D ioremap_wc(base, MBOX_SIZE * pf->total_vfs); + if (!hwbase) { + err =3D -ENOMEM; + goto free_wq; + } } =20 mbox =3D &pf->mbox_pfvf[0]; @@ -623,7 +648,7 @@ static int otx2_pfvf_mbox_init(struct otx2_nic *pf, int= numvfs) return 0; =20 free_iomem: - if (hwbase) + if (hwbase && !(is_cn20k(pf->pdev))) iounmap(hwbase); free_wq: destroy_workqueue(pf->mbox_pfvf_wq); @@ -642,8 +667,10 @@ static void otx2_pfvf_mbox_destroy(struct otx2_nic *pf) pf->mbox_pfvf_wq =3D NULL; } =20 - if (mbox->mbox.hwbase) + if (mbox->mbox.hwbase && !is_cn20k(pf->pdev)) iounmap(mbox->mbox.hwbase); + else + qmem_free(&pf->pdev->dev, pf->pfvf_mbox_addr); =20 otx2_mbox_destroy(&mbox->mbox); } @@ -667,6 +694,9 @@ static void otx2_disable_pfvf_mbox_intr(struct otx2_nic= *pf, int numvfs) { int vector; =20 + if (is_cn20k(pf->pdev)) + return cn20k_disable_pfvf_mbox_intr(pf, numvfs); + /* Disable PF <=3D> VF mailbox IRQ */ otx2_write64(pf, RVU_PF_VFPF_MBOX_INT_ENA_W1CX(0), ~0ull); otx2_write64(pf, RVU_PF_VFPF_MBOX_INT_ENA_W1CX(1), ~0ull); @@ -688,6 +718,9 @@ static int otx2_register_pfvf_mbox_intr(struct otx2_nic= *pf, int numvfs) char *irq_name; int err; =20 + if (is_cn20k(pf->pdev)) + return cn20k_register_pfvf_mbox_intr(pf, numvfs); + /* Register MBOX0 interrupt handler */ irq_name =3D &hw->irq_name[RVU_PF_INT_VEC_VFPF_MBOX0 * NAME_SIZE]; if (pf->pcifunc) --=20 2.25.1