From nobody Tue Nov 26 01:47:17 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2D081C9DC5; Tue, 22 Oct 2024 18:54:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623284; cv=none; b=RlpbHHMSHDCMjqHdwJ93oOJxtu5ykeX/Gz3PzpvNMetO4lyun/pKkbTTCSK2bB+tAdRHwUnMJtUH8xdVq0jqkQYuoVe3mlL7eImwluZyzMfSLnmVO7ZF7Skj5Lzogy2Klj9EPUu/MDTpFghBX7lCK0MG4yIVso2VgCPSfiWwgxI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623284; c=relaxed/simple; bh=mdIVbytr6kIPewBcxaPOdfr9sudbKXGVgi+cW+YMGdA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UVfVOk0e7im1BWAWXavGhfxLO13K9WFV4fG8LEdArd7I2WShVQhBAKEhf8XgdT0pBIHlgH+NsngeAPuQTXTrNoR8LkciNri6UahN9I/BV7iSuT7peztTMcsJItemi/ITvCcutytDHlj4iNZF6+nKuDeWCC+aKi0nnvibe/y7Uy0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=WyPlLI/K; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="WyPlLI/K" Received: from pps.filterd (m0431383.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49MBsmAr017699; Tue, 22 Oct 2024 11:54:24 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=R xLZHCUdxSqURxF5tvaDu+NjdpoGZbV5gLB59XKQies=; b=WyPlLI/KRufQB2rVe 6Q3weiMzDEJrABqICFSxxAkbK6jqi2xVHn/uqOx7Y+dA88mxy777OrtrcQr8rhG0 Xs59bAPWuj8JUQt4wed/zj/r4u8bkmXNmDGoGcBlopW9NAI3c48uVAC2r7/2HASP CZw0zvKapmQYmdXOKH+lEVoHlGXsZCw2A6MGQeZDWLdqbe0RScAZoVeJdGeAUgvd JNdWrcRBXsAz/uyav2h2aboHByv4vMVhSz6+iYZ7vDNA9hvFckOdOVQx8+4Ie8o3 oG8ZNyuTaKDuFcfRufD8iiN4mpCH+sFoHY7vevge96P1LjY0w8hQkIBL5kG2MC2Y DmwFg== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42eb66s55r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Oct 2024 11:54:23 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 22 Oct 2024 11:54:23 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 22 Oct 2024 11:54:22 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id ACE2F3F7067; Tue, 22 Oct 2024 11:54:18 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH v2 1/6] octeontx2: Set appropriate PF, VF masks and shifts based on silicon Date: Wed, 23 Oct 2024 00:24:05 +0530 Message-ID: <20241022185410.4036100-2-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241022185410.4036100-1-saikrishnag@marvell.com> References: <20241022185410.4036100-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: AzGs-qMNkwA-qnDKwWRPmrcsJ_ot6kbr X-Proofpoint-ORIG-GUID: AzGs-qMNkwA-qnDKwWRPmrcsJ_ot6kbr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 Content-Type: text/plain; charset="utf-8" From: Subbaraya Sundeep Number of RVU PFs on CN20K silicon have increased to 96 from maximum of 32 that were supported on earlier silicons. Every RVU PF and VF is identified by HW using a 16bit PF_FUNC value. Due to the change in Max number of PFs in CN20K, the bit encoding of this PF_FUNC has changed. This patch handles the change by exporting PF,VF masks and shifts present in mailbox module to all other modules. Also moved the NIX AF register offset macros to other files which will be posted in coming patches. Signed-off-by: Subbaraya Sundeep Signed-off-by: Sai Krishna --- .../net/ethernet/marvell/octeontx2/af/mbox.c | 15 ++++++++++ .../net/ethernet/marvell/octeontx2/af/mbox.h | 5 ++++ .../net/ethernet/marvell/octeontx2/af/rvu.c | 5 ---- .../net/ethernet/marvell/octeontx2/af/rvu.h | 14 +++++---- .../marvell/octeontx2/nic/otx2_common.h | 11 +------ .../ethernet/marvell/octeontx2/nic/otx2_reg.h | 30 ------------------- 6 files changed, 30 insertions(+), 50 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 1e5aa5397504..791c468a10c5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -13,6 +13,21 @@ #include "mbox.h" #include "rvu_trace.h" =20 +/* Default values of PF and VF bit encodings in PCIFUNC for + * CN9XXX and CN10K series silicons. + */ +u16 rvu_pcifunc_pf_shift =3D 10; +EXPORT_SYMBOL(rvu_pcifunc_pf_shift); + +u16 rvu_pcifunc_pf_mask =3D 0x3F; +EXPORT_SYMBOL(rvu_pcifunc_pf_mask); + +u16 rvu_pcifunc_func_shift; +EXPORT_SYMBOL(rvu_pcifunc_func_shift); + +u16 rvu_pcifunc_func_mask =3D 0x3FF; +EXPORT_SYMBOL(rvu_pcifunc_func_mask); + static const u16 msgs_offset =3D ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_A= LIGN); =20 void __otx2_mbox_reset(struct otx2_mbox *mbox, int devid) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.h index 6ea2f3071fe8..38a0badcdb68 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -50,6 +50,11 @@ #define MBOX_DIR_PFVF_UP 6 /* PF sends messages to VF */ #define MBOX_DIR_VFPF_UP 7 /* VF replies to PF */ =20 +extern u16 rvu_pcifunc_pf_shift; +extern u16 rvu_pcifunc_pf_mask; +extern u16 rvu_pcifunc_func_shift; +extern u16 rvu_pcifunc_func_mask; + struct otx2_mbox_dev { void *mbase; /* This dev's mbox region */ void *hwbase; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index 1a97fb9032fa..dcfc27a60b43 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -400,11 +400,6 @@ static void rvu_update_rsrc_map(struct rvu *rvu, struc= t rvu_pfvf *pfvf, rvu_write64(rvu, BLKADDR_RVUM, reg | (devnum << 16), num_lfs); } =20 -inline int rvu_get_pf(u16 pcifunc) -{ - return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; -} - void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf) { u64 cfg; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index 5016ba82e142..938a911cbf1c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -41,10 +41,10 @@ #define MAX_CPT_BLKS 2 =20 /* PF_FUNC */ -#define RVU_PFVF_PF_SHIFT 10 -#define RVU_PFVF_PF_MASK 0x3F -#define RVU_PFVF_FUNC_SHIFT 0 -#define RVU_PFVF_FUNC_MASK 0x3FF +#define RVU_PFVF_PF_SHIFT rvu_pcifunc_pf_shift +#define RVU_PFVF_PF_MASK rvu_pcifunc_pf_mask +#define RVU_PFVF_FUNC_SHIFT rvu_pcifunc_func_shift +#define RVU_PFVF_FUNC_MASK rvu_pcifunc_func_mask =20 #ifdef CONFIG_DEBUG_FS struct dump_ctx { @@ -819,7 +819,6 @@ int rvu_alloc_rsrc_contig(struct rsrc_bmap *rsrc, int n= rsrc); void rvu_free_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc, int start); bool rvu_rsrc_check_contig(struct rsrc_bmap *rsrc, int nrsrc); u16 rvu_get_rsrc_mapcount(struct rvu_pfvf *pfvf, int blkaddr); -int rvu_get_pf(u16 pcifunc); struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc); void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf); bool is_block_implemented(struct rvu_hwinfo *hw, int blkaddr); @@ -852,6 +851,11 @@ bool is_sdp_pfvf(u16 pcifunc); bool is_sdp_pf(u16 pcifunc); bool is_sdp_vf(struct rvu *rvu, u16 pcifunc); =20 +static inline int rvu_get_pf(u16 pcifunc) +{ + return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; +} + /* CGX APIs */ static inline bool is_pf_cgxmapped(struct rvu *rvu, u8 pf) { diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index f27a3456ae64..8e7ed3979f80 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -27,6 +27,7 @@ #include "otx2_reg.h" #include "otx2_txrx.h" #include "otx2_devlink.h" +#include #include #include "qos.h" =20 @@ -873,21 +874,11 @@ MBOX_UP_MCS_MESSAGES /* Time to wait before watchdog kicks off */ #define OTX2_TX_TIMEOUT (100 * HZ) =20 -#define RVU_PFVF_PF_SHIFT 10 -#define RVU_PFVF_PF_MASK 0x3F -#define RVU_PFVF_FUNC_SHIFT 0 -#define RVU_PFVF_FUNC_MASK 0x3FF - static inline bool is_otx2_vf(u16 pcifunc) { return !!(pcifunc & RVU_PFVF_FUNC_MASK); } =20 -static inline int rvu_get_pf(u16 pcifunc) -{ - return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; -} - static inline dma_addr_t otx2_dma_map_page(struct otx2_nic *pfvf, struct page *page, size_t offset, size_t size, diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h b/driver= s/net/ethernet/marvell/octeontx2/nic/otx2_reg.h index e3aee6e36215..858f084b9d47 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h @@ -138,36 +138,6 @@ #define NIX_LF_CINTX_ENA_W1S(a) (NIX_LFBASE | 0xD40 | (a) << 12) #define NIX_LF_CINTX_ENA_W1C(a) (NIX_LFBASE | 0xD50 | (a) << 12) =20 -/* NIX AF transmit scheduler registers */ -#define NIX_AF_SMQX_CFG(a) (0x700 | (u64)(a) << 16) -#define NIX_AF_TL4X_SDP_LINK_CFG(a) (0xB10 | (u64)(a) << 16) -#define NIX_AF_TL1X_SCHEDULE(a) (0xC00 | (u64)(a) << 16) -#define NIX_AF_TL1X_CIR(a) (0xC20 | (u64)(a) << 16) -#define NIX_AF_TL1X_TOPOLOGY(a) (0xC80 | (u64)(a) << 16) -#define NIX_AF_TL2X_PARENT(a) (0xE88 | (u64)(a) << 16) -#define NIX_AF_TL2X_SCHEDULE(a) (0xE00 | (u64)(a) << 16) -#define NIX_AF_TL2X_TOPOLOGY(a) (0xE80 | (u64)(a) << 16) -#define NIX_AF_TL2X_CIR(a) (0xE20 | (u64)(a) << 16) -#define NIX_AF_TL2X_PIR(a) (0xE30 | (u64)(a) << 16) -#define NIX_AF_TL3X_PARENT(a) (0x1088 | (u64)(a) << 16) -#define NIX_AF_TL3X_SCHEDULE(a) (0x1000 | (u64)(a) << 16) -#define NIX_AF_TL3X_SHAPE(a) (0x1010 | (u64)(a) << 16) -#define NIX_AF_TL3X_CIR(a) (0x1020 | (u64)(a) << 16) -#define NIX_AF_TL3X_PIR(a) (0x1030 | (u64)(a) << 16) -#define NIX_AF_TL3X_TOPOLOGY(a) (0x1080 | (u64)(a) << 16) -#define NIX_AF_TL4X_PARENT(a) (0x1288 | (u64)(a) << 16) -#define NIX_AF_TL4X_SCHEDULE(a) (0x1200 | (u64)(a) << 16) -#define NIX_AF_TL4X_SHAPE(a) (0x1210 | (u64)(a) << 16) -#define NIX_AF_TL4X_CIR(a) (0x1220 | (u64)(a) << 16) -#define NIX_AF_TL4X_PIR(a) (0x1230 | (u64)(a) << 16) -#define NIX_AF_TL4X_TOPOLOGY(a) (0x1280 | (u64)(a) << 16) -#define NIX_AF_MDQX_SCHEDULE(a) (0x1400 | (u64)(a) << 16) -#define NIX_AF_MDQX_SHAPE(a) (0x1410 | (u64)(a) << 16) -#define NIX_AF_MDQX_CIR(a) (0x1420 | (u64)(a) << 16) -#define NIX_AF_MDQX_PIR(a) (0x1430 | (u64)(a) << 16) -#define NIX_AF_MDQX_PARENT(a) (0x1480 | (u64)(a) << 16) -#define NIX_AF_TL3_TL2X_LINKX_CFG(a, b) (0x1700 | (u64)(a) << 16 | (b) << = 3) - /* LMT LF registers */ #define LMT_LFBASE BIT_ULL(RVU_FUNC_BLKADDR_SHIFT) #define LMT_LF_LMTLINEX(a) (LMT_LFBASE | 0x000 | (a) << 12) --=20 2.25.1 From nobody Tue Nov 26 01:47:17 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA2A71CB32F; Tue, 22 Oct 2024 18:54:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623286; cv=none; b=urAuR+YFN81qCnG2lwZqQEmruDsxuVkb4YHs66Ldr9nyNiRZXfDJ9ozHmDTbH5Nlm9DliGnLg69iayVJ6iq1yJBvaDuXKxu9q6qfFf+VW5u7slTYbCYLnHRr9ihcz8ZjYwuPyWaHMfQysRQ+pmNJosJRtYlqM21SBE0acnr+/y4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623286; c=relaxed/simple; bh=5T6sZvzxJ/fgemmr7Iz/hGivgpGvGPvDTfIrQI+dmLc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BXV/+AauYxjFFR0NzR4nN28Gh/3wcOUB82gUpM17h4F+naj1NKl8rPkLZFW2DKhp2esyw7w9EtXOwtPb47V9ZCicRFh9N8vg3XuSkeeSgxh7ICcx9gsXANEkNYSTKQhLnx1DkhzYeANtwFTO3B/IzAORp+wfSoDEdOBY6s+z3JE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=DQ4WdIXH; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="DQ4WdIXH" Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49M97f0n014323; Tue, 22 Oct 2024 11:54:29 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=5 umdT488ztOfPEYhsP/2p8atlUt0eOXMtvTyu5LnypM=; b=DQ4WdIXH5sU2TQrRM 3TYqLmy82c6txLAOPR4cho1hLk4bXOxrndeNihEPKBDZQ7vdXrYoY/omdOhAlKPL xD0zPAfM25oeAcO2qbZXR39Jk7YjgojfYzLh7YHSC+/+NM1TDwWEbJz/IdZ2sVIq gj6/DjAo9SVkQtiXpfStyJK6uE7rDWzSe3oEHlN5sg/cc/e/x5s1/EwmsvbEGMbu eQKpFi83UCrTAcXiF1uw+5tFj1qwlFjJvUHfUwgakurYyoHPFmZZ+at3QmTHJGAI 3c8z5M2V/ZmcHAnXZg2KlQsJMugiFd2kMn22VtJCfVYfEJ1BIk4fh4Y36pjIysEL SNDig== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42e5291x5m-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Oct 2024 11:54:28 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 22 Oct 2024 11:54:27 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 22 Oct 2024 11:54:27 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id 4178A3F7067; Tue, 22 Oct 2024 11:54:23 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH v2 2/6] octeontx2-af: CN20k basic mbox operations and structures Date: Wed, 23 Oct 2024 00:24:06 +0530 Message-ID: <20241022185410.4036100-3-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241022185410.4036100-1-saikrishnag@marvell.com> References: <20241022185410.4036100-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-ORIG-GUID: ifWuidbkPg8OLaDasLwaD1Yvo3kHkJVv X-Proofpoint-GUID: ifWuidbkPg8OLaDasLwaD1Yvo3kHkJVv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 Content-Type: text/plain; charset="utf-8" This patch adds basic mbox operation APIs and structures to add support for mbox module on CN20k silicon. There are few CSR offsets, interrupts changed between CN20k and prior Octeon series of devices. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/af/Makefile | 3 +- .../ethernet/marvell/octeontx2/af/cn20k/api.h | 22 +++++++ .../marvell/octeontx2/af/cn20k/mbox_init.c | 49 ++++++++++++++ .../ethernet/marvell/octeontx2/af/cn20k/reg.h | 27 ++++++++ .../net/ethernet/marvell/octeontx2/af/mbox.c | 3 + .../net/ethernet/marvell/octeontx2/af/mbox.h | 7 ++ .../net/ethernet/marvell/octeontx2/af/rvu.c | 65 +++++++++++++++---- .../net/ethernet/marvell/octeontx2/af/rvu.h | 19 ++++++ .../marvell/octeontx2/af/rvu_struct.h | 6 +- 9 files changed, 186 insertions(+), 15 deletions(-) create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_in= it.c create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h diff --git a/drivers/net/ethernet/marvell/octeontx2/af/Makefile b/drivers/n= et/ethernet/marvell/octeontx2/af/Makefile index 3cf4c8285c90..38d8599dc6eb 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/Makefile +++ b/drivers/net/ethernet/marvell/octeontx2/af/Makefile @@ -11,4 +11,5 @@ rvu_mbox-y :=3D mbox.o rvu_trace.o rvu_af-y :=3D cgx.o rvu.o rvu_cgx.o rvu_npa.o rvu_nix.o \ rvu_reg.o rvu_npc.o rvu_debugfs.o ptp.o rvu_npc_fs.o \ rvu_cpt.o rvu_devlink.o rpm.o rvu_cn10k.o rvu_switch.o \ - rvu_sdp.o rvu_npc_hash.o mcs.o mcs_rvu_if.o mcs_cnf10kb.o + rvu_sdp.o rvu_npc_hash.o mcs.o mcs_rvu_if.o mcs_cnf10kb.o \ + cn20k/mbox_init.o diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/api.h new file mode 100644 index 000000000000..b57bd38181aa --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef CN20K_API_H +#define CN20K_API_H + +#include "../rvu.h" + +struct ng_rvu { + struct mbox_ops *rvu_mbox_ops; + struct qmem *pf_mbox_addr; +}; + +/* Mbox related APIs */ +int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int num); +int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, + int num, int type, unsigned long *pf_bmap); +#endif /* CN20K_API_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c b/= drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c new file mode 100644 index 000000000000..0e128013a03f --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#include +#include + +#include "rvu_trace.h" +#include "mbox.h" +#include "reg.h" +#include "api.h" + +int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, + int num, int type, unsigned long *pf_bmap) +{ + int region; + u64 bar; + + for (region =3D 0; region < num; region++) { + if (!test_bit(region, pf_bmap)) + continue; + + bar =3D (u64)phys_to_virt((u64)rvu->ng_rvu->pf_mbox_addr->base); + bar +=3D region * MBOX_SIZE; + + mbox_addr[region] =3D (void *)bar; + + if (!mbox_addr[region]) + return -ENOMEM; + } + return 0; +} + +int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int ndevs) +{ + int dev; + + if (!is_cn20k(rvu->pdev)) + return 0; + + for (dev =3D 0; dev < ndevs; dev++) + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); + + return 0; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/reg.h new file mode 100644 index 000000000000..58152a4024ec --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef RVU_MBOX_REG_H +#define RVU_MBOX_REG_H +#include "../rvu.h" +#include "../rvu_reg.h" + +/* RVUM block registers */ +#define RVU_PF_DISC (0x0) +#define RVU_PRIV_PFX_DISC(a) (0x8000208 | (a) << 16) +#define RVU_PRIV_HWVFX_DISC(a) (0xD000000 | (a) << 12) + +/* Mbox Registers */ +/* RVU AF BAR0 Mbox registers for AF =3D> PFx */ +#define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) +#define RVU_MBOX_AF_PFX_CFG(a) (0x6000 | (a) << 4) +#define RVU_AF_BAR2_SEL (0x9000000) +#define RVU_AF_BAR2_PFID (0x16400) +#define NIX_CINTX_INT_W1S(a) (0xd30 | (a) << 12) +#define NIX_QINTX_CNT(a) (0xc00 | (a) << 12) + +#endif /* RVU_MBOX_REG_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 791c468a10c5..1e3e72107a9d 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -10,8 +10,11 @@ #include =20 #include "rvu_reg.h" +#include "cn20k/reg.h" +#include "cn20k/api.h" #include "mbox.h" #include "rvu_trace.h" +#include "rvu.h" =20 /* Default values of PF and VF bit encodings in PCIFUNC for * CN9XXX and CN10K series silicons. diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.h index 38a0badcdb68..df64a18fe1d6 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -55,6 +55,11 @@ extern u16 rvu_pcifunc_pf_mask; extern u16 rvu_pcifunc_func_shift; extern u16 rvu_pcifunc_func_mask; =20 +enum { + TYPE_AFVF, + TYPE_AFPF, +}; + struct otx2_mbox_dev { void *mbase; /* This dev's mbox region */ void *hwbase; @@ -83,6 +88,8 @@ struct otx2_mbox { struct mbox_hdr { u64 msg_size; /* Total msgs size embedded */ u16 num_msgs; /* No of msgs embedded */ + u16 opt_msg; + u8 sig; }; =20 /* Header which precedes every msg and is also part of it */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index dcfc27a60b43..a5ebd7cd3a5c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -20,6 +20,8 @@ =20 #include "rvu_trace.h" #include "rvu_npc_hash.h" +#include "cn20k/reg.h" +#include "cn20k/api.h" =20 #define DRV_NAME "rvu_af" #define DRV_STRING "Marvell OcteonTX2 RVU Admin Function Driver" @@ -34,10 +36,8 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq= _info *mw, int type, int num, void (mbox_handler)(struct work_struct *), void (mbox_up_handler)(struct work_struct *)); -enum { - TYPE_AFVF, - TYPE_AFPF, -}; +static irqreturn_t rvu_mbox_pf_intr_handler(int irq, void *rvu_irq); +static irqreturn_t rvu_mbox_intr_handler(int irq, void *rvu_irq); =20 /* Supported devices */ static const struct pci_device_id rvu_id_table[] =3D { @@ -2212,6 +2212,22 @@ static void __rvu_mbox_handler(struct rvu_work *mwor= k, int type, bool poll) =20 offset =3D mbox->rx_start + ALIGN(sizeof(*req_hdr), MBOX_MSG_ALIGN); =20 + if (req_hdr->sig) { + req_hdr->opt_msg =3D mw->mbox_wrk[devid].num_msgs; + rvu_write64(rvu, BLKADDR_NIX0, RVU_AF_BAR2_SEL, + RVU_AF_BAR2_PFID); + if (type =3D=3D TYPE_AFPF) + rvu_write64(rvu, BLKADDR_NIX0, + AF_BAR2_ALIASX(0, NIX_CINTX_INT_W1S(devid)), + 0x1); + else + rvu_write64(rvu, BLKADDR_NIX0, + AF_BAR2_ALIASX(0, NIX_QINTX_CNT(devid)), + 0x1); + usleep_range(1000, 2000); + goto done; + } + for (id =3D 0; id < mw->mbox_wrk[devid].num_msgs; id++) { msg =3D mdev->mbase + offset; =20 @@ -2245,9 +2261,10 @@ static void __rvu_mbox_handler(struct rvu_work *mwor= k, int type, bool poll) err, otx2_mbox_id2name(msg->id), msg->id, devid); } +done: mw->mbox_wrk[devid].num_msgs =3D 0; =20 - if (poll) + if (!is_cn20k(mbox->pdev) && poll) otx2_mbox_wait_for_zero(mbox, devid); =20 /* Send mbox responses to VF/PF */ @@ -2360,6 +2377,10 @@ static int rvu_get_mbox_regions(struct rvu *rvu, voi= d **mbox_addr, int region; u64 bar4; =20 + if (is_cn20k(rvu->pdev)) + return cn20k_rvu_get_mbox_regions(rvu, mbox_addr, + num, type, pf_bmap); + /* For cn10k platform VF mailbox regions of a PF follows after the * PF <-> AF mailbox region. Whereas for Octeontx2 it is read from * RVU_PF_VF_BAR4_ADDR register. @@ -2413,12 +2434,17 @@ static int rvu_get_mbox_regions(struct rvu *rvu, vo= id **mbox_addr, return -ENOMEM; } =20 +static struct mbox_ops rvu_mbox_ops =3D { + .pf_intr_handler =3D rvu_mbox_pf_intr_handler, +}; + static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw, int type, int num, void (mbox_handler)(struct work_struct *), void (mbox_up_handler)(struct work_struct *)) { int err =3D -EINVAL, i, dir, dir_up; + struct ng_rvu *ng_rvu_mbox; void __iomem *reg_base; struct rvu_work *mwork; unsigned long *pf_bmap; @@ -2443,6 +2469,18 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbo= x_wq_info *mw, } } =20 + ng_rvu_mbox =3D kzalloc(sizeof(*ng_rvu_mbox), GFP_KERNEL); + if (!ng_rvu_mbox) { + err =3D -ENOMEM; + goto free_bitmap; + } + + rvu->ng_rvu =3D ng_rvu_mbox; + + rvu->ng_rvu->rvu_mbox_ops =3D &rvu_mbox_ops; + + cn20k_rvu_mbox_init(rvu, type, num); + mutex_init(&rvu->mbox_lock); =20 mbox_regions =3D kcalloc(num, sizeof(void *), GFP_KERNEL); @@ -2475,7 +2513,7 @@ static int rvu_mbox_init(struct rvu *rvu, struct mbox= _wq_info *mw, } =20 mw->mbox_wq =3D alloc_workqueue("%s", - WQ_UNBOUND | WQ_HIGHPRI | WQ_MEM_RECLAIM, + WQ_HIGHPRI | WQ_MEM_RECLAIM, num, name); if (!mw->mbox_wq) { err =3D -ENOMEM; @@ -2553,8 +2591,8 @@ static void rvu_mbox_destroy(struct mbox_wq_info *mw) otx2_mbox_destroy(&mw->mbox_up); } =20 -static void rvu_queue_work(struct mbox_wq_info *mw, int first, - int mdevs, u64 intr) +void rvu_queue_work(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr) { struct otx2_mbox_dev *mdev; struct otx2_mbox *mbox; @@ -2965,12 +3003,14 @@ static int rvu_register_interrupts(struct rvu *rvu) =20 /* Register mailbox interrupt handler */ sprintf(&rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], "RVUAF Mbox"); - ret =3D request_irq(pci_irq_vector(rvu->pdev, RVU_AF_INT_VEC_MBOX), - rvu_mbox_pf_intr_handler, 0, - &rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], rvu); + ret =3D request_irq(pci_irq_vector + (rvu->pdev, RVU_AF_INT_VEC_MBOX), + rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, + &rvu->irq_name[RVU_AF_INT_VEC_MBOX * + NAME_SIZE], rvu); if (ret) { dev_err(rvu->dev, - "RVUAF: IRQ registration failed for mbox irq\n"); + "RVUAF: IRQ registration failed for mbox\n"); goto fail; } =20 @@ -3478,6 +3518,7 @@ static void rvu_remove(struct pci_dev *pdev) pci_set_drvdata(pdev, NULL); =20 devm_kfree(&pdev->dev, rvu->hw); + kfree(rvu->ng_rvu); devm_kfree(&pdev->dev, rvu); } =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index 938a911cbf1c..2533e294149c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -444,6 +444,10 @@ struct mbox_wq_info { struct workqueue_struct *mbox_wq; }; =20 +struct mbox_ops { + irqreturn_t (*pf_intr_handler)(int irq, void *rvu_irq); +}; + struct channel_fwdata { struct sdp_node_info info; u8 valid; @@ -594,6 +598,7 @@ struct rvu { spinlock_t cpt_intr_lock; =20 struct mutex mbox_lock; /* Serialize mbox up and down msgs */ + struct ng_rvu *ng_rvu; }; =20 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64= val) @@ -875,11 +880,25 @@ static inline bool is_cgx_vf(struct rvu *rvu, u16 pci= func) is_pf_cgxmapped(rvu, rvu_get_pf(pcifunc))); } =20 +#define CN20K_CHIPID 0x20 + +/* + * Silicon check for CN20K family + */ +static inline bool is_cn20k(struct pci_dev *pdev) +{ + return (pdev->subsystem_device & 0xFF) =3D=3D CN20K_CHIPID; +} + #define M(_name, _id, fn_name, req, rsp) \ int rvu_mbox_handler_ ## fn_name(struct rvu *, struct req *, struct rsp *); MBOX_MESSAGES #undef M =20 +/* Mbox APIs */ +void rvu_queue_work(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr); + int rvu_cgx_init(struct rvu *rvu); int rvu_cgx_exit(struct rvu *rvu); void *rvu_cgx_pdata(u8 cgx_id, struct rvu *rvu); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_struct.h b/drive= rs/net/ethernet/marvell/octeontx2/af/rvu_struct.h index fc8da2090657..90cb063d00f0 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_struct.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_struct.h @@ -33,7 +33,8 @@ enum rvu_block_addr_e { BLKADDR_NDC_NIX1_RX =3D 0x10ULL, BLKADDR_NDC_NIX1_TX =3D 0x11ULL, BLKADDR_APR =3D 0x16ULL, - BLK_COUNT =3D 0x17ULL, + BLKADDR_MBOX =3D 0x1bULL, + BLK_COUNT =3D 0x1cULL, }; =20 /* RVU Block Type Enumeration */ @@ -49,7 +50,8 @@ enum rvu_block_type_e { BLKTYPE_TIM =3D 0x8, BLKTYPE_CPT =3D 0x9, BLKTYPE_NDC =3D 0xa, - BLKTYPE_MAX =3D 0xa, + BLKTYPE_MBOX =3D 0x13, + BLKTYPE_MAX =3D 0x13, }; =20 /* RVU Admin function Interrupt Vector Enumeration */ --=20 2.25.1 From nobody Tue Nov 26 01:47:17 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA2FC1CB331; Tue, 22 Oct 2024 18:54:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623287; cv=none; b=t7BLzbyJbTMzy8PCaHH0yPuf69J4/BUJx3AnPbdoXB+lOKz/Dy9+ISwvSB8vh3j1nEgUHkRwKooqeZ8GbEvSeUFnV9tKLHrhoHl+Oy5VL9JtrXdw3aoa3Y5TUma5pD0UL84viQb+uyhFDHim5f+w30TbsJTUf97m8Z12MwIF49k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623287; c=relaxed/simple; bh=jX4eV1zneefwcsqdD7kZf5lZO1Qjd8m1jmB965WHsds=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CjZnNbGP4hrD0nOUllNCM42auo9BQxIAio451+jb8IUx9AUq4I4aNXfnisX0j5HpBGVJVLT3aQZAiTwxiRXLrEYrscXLACvjExP5sy+CCaMBX1C9NBt5kw46wSRNbfSUyM+tmFBXMUKRg1ZDv02iNFBu8GTP5iZyXRYrV5PMzR8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=VuoOkkjB; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="VuoOkkjB" Received: from pps.filterd (m0431383.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49MBRe1g009419; Tue, 22 Oct 2024 11:54:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=+ uvIE03sgzUUswRmDIFO7B6QzXGDHYVXOj1A0V07u5I=; b=VuoOkkjBjPsW+J9tl +OY44UIlJoW+wwQfv2mIEmDbuGB3KxE35me0dE7xw1yIbW23UrGjpWf+jIUNWG+w 5yCEUcHlAgoTSwnqrD7mdPvM7zRPx2jrnNyauaeKotiUvdoii548cc1icCwON3// PkZEFIHjkluwwqh9O7wUUEYJs4sPhDgr4cIC97IPBY9QAm9MSJ9yqL3fUF8rVHeW pll9T1txn01HaviXjBGZR1btEMt7/t3CL9D1VMscEUco2vH0C//qaGyJMjnTYdTy QIsmZV1wLwY4ieXf839Z6RvSE6XGdHTZA4Oh7E/s5gmZ8OkxvG0QC/hdJCFISbD6 WzISg== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42eb66s55w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Oct 2024 11:54:33 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 22 Oct 2024 11:54:32 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 22 Oct 2024 11:54:32 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id C8A9F3F7067; Tue, 22 Oct 2024 11:54:27 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH v2 3/6] octeontx2-af: CN20k mbox to support AF REQ/ACK functionality Date: Wed, 23 Oct 2024 00:24:07 +0530 Message-ID: <20241022185410.4036100-4-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241022185410.4036100-1-saikrishnag@marvell.com> References: <20241022185410.4036100-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: T7Fcy12YwiJLDYAGX0AfahSUHijtiKWM X-Proofpoint-ORIG-GUID: T7Fcy12YwiJLDYAGX0AfahSUHijtiKWM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 Content-Type: text/plain; charset="utf-8" This implementation uses separate trigger interrupts for request, response MBOX messages against using trigger message data in CN10K. This patch adds support for basic mbox implementation for CN20K from AF side. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/af/cn20k/api.h | 8 + .../marvell/octeontx2/af/cn20k/mbox_init.c | 212 ++++++++++++++++++ .../ethernet/marvell/octeontx2/af/cn20k/reg.h | 17 ++ .../marvell/octeontx2/af/cn20k/struct.h | 25 +++ .../net/ethernet/marvell/octeontx2/af/mbox.c | 83 ++++++- .../net/ethernet/marvell/octeontx2/af/mbox.h | 1 + .../net/ethernet/marvell/octeontx2/af/rvu.c | 60 +++-- .../net/ethernet/marvell/octeontx2/af/rvu.h | 16 +- 8 files changed, 403 insertions(+), 19 deletions(-) create mode 100644 drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/api.h index b57bd38181aa..9436a4a4d815 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h @@ -15,8 +15,16 @@ struct ng_rvu { struct qmem *pf_mbox_addr; }; =20 +struct rvu; + /* Mbox related APIs */ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int num); +int cn20k_register_afpf_mbox_intr(struct rvu *rvu); int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, int num, int type, unsigned long *pf_bmap); +void cn20k_rvu_enable_mbox_intr(struct rvu *rvu); +void cn20k_rvu_unregister_interrupts(struct rvu *rvu); +void cn20k_free_mbox_memory(struct rvu *rvu); +int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, + void *reg_base, int direction, int ndevs); #endif /* CN20K_API_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c b/= drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c index 0e128013a03f..0c1ea6923043 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c @@ -13,6 +13,137 @@ #include "reg.h" #include "api.h" =20 +/* CN20K mbox PFx =3D> AF irq handler */ +static irqreturn_t cn20k_mbox_pf_common_intr_handler(int irq, void *rvu_ir= q) +{ + struct rvu_irq_data *rvu_irq_data =3D (struct rvu_irq_data *)rvu_irq; + struct rvu *rvu =3D rvu_irq_data->rvu; + u64 intr; + + /* Clear interrupts */ + intr =3D rvu_read64(rvu, BLKADDR_RVUM, rvu_irq_data->intr_status); + rvu_write64(rvu, BLKADDR_RVUM, rvu_irq_data->intr_status, intr); + + if (intr) + trace_otx2_msg_interrupt(rvu->pdev, "PF(s) to AF", intr); + + /* Sync with mbox memory region */ + rmb(); + + rvu_irq_data->rvu_queue_work_hdlr(&rvu->afpf_wq_info, + rvu_irq_data->start, + rvu_irq_data->mdevs, intr); + + return IRQ_HANDLED; +} + +void cn20k_rvu_enable_mbox_intr(struct rvu *rvu) +{ + struct rvu_hwinfo *hw =3D rvu->hw; + + /* Clear spurious irqs, if any */ + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF_INT(0), INTR_MASK(hw->total_pfs)); + + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF_INT(1), INTR_MASK(hw->total_pfs - 64)); + + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF1_INT(0), INTR_MASK(hw->total_pfs)); + + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFAF1_INT(1), INTR_MASK(hw->total_pfs - 64)); + + /* Enable mailbox interrupt for all PFs except PF0 i.e AF itself */ + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1S(0), + INTR_MASK(hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1S(1), + INTR_MASK(hw->total_pfs - 64)); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1S(0), + INTR_MASK(hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1S(1), + INTR_MASK(hw->total_pfs - 64)); +} + +void cn20k_rvu_unregister_interrupts(struct rvu *rvu) +{ + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1C(0), + INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF_INT_ENA_W1C(1), + INTR_MASK(rvu->hw->total_pfs - 64)); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1C(0), + INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFAF1_INT_ENA_W1C(1), + INTR_MASK(rvu->hw->total_pfs - 64)); +} + +int cn20k_register_afpf_mbox_intr(struct rvu *rvu) +{ + struct rvu_irq_data *irq_data; + int intr_vec, ret, vec =3D 0; + + /* irq data for 4 PF intr vectors */ + irq_data =3D devm_kcalloc(rvu->dev, 4, + sizeof(struct rvu_irq_data), GFP_KERNEL); + if (!irq_data) + return -ENOMEM; + + for (intr_vec =3D RVU_AF_CN20K_INT_VEC_PFAF_MBOX0; intr_vec <=3D + RVU_AF_CN20K_INT_VEC_PFAF1_MBOX1; intr_vec++, + vec++) { + switch (intr_vec) { + case RVU_AF_CN20K_INT_VEC_PFAF_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF_INT(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_AF_CN20K_INT_VEC_PFAF_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF_INT(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + case RVU_AF_CN20K_INT_VEC_PFAF1_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF1_INT(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_AF_CN20K_INT_VEC_PFAF1_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_AF_PFAF1_INT(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + } + irq_data[vec].rvu_queue_work_hdlr =3D rvu_queue_work; + irq_data[vec].vec_num =3D intr_vec; + irq_data[vec].rvu =3D rvu; + + /* Register mailbox interrupt handler */ + sprintf(&rvu->irq_name[intr_vec * NAME_SIZE], + "RVUAF PFAF%d Mbox%d", + vec / 2, vec % 2); + ret =3D request_irq(pci_irq_vector(rvu->pdev, intr_vec), + rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, + &rvu->irq_name[intr_vec * NAME_SIZE], + &irq_data[vec]); + if (ret) + return ret; + + rvu->irq_allocated[intr_vec] =3D true; + } + + return 0; +} + int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, int num, int type, unsigned long *pf_bmap) { @@ -34,6 +165,48 @@ int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **= mbox_addr, return 0; } =20 +static struct mbox_ops cn20k_mbox_ops =3D { + .pf_intr_handler =3D cn20k_mbox_pf_common_intr_handler, +}; + +static int rvu_alloc_mbox_memory(struct rvu *rvu, int type, + int ndevs, int mbox_size) +{ + struct qmem *mbox_addr; + dma_addr_t iova; + int pf, err; + + /* Allocate contiguous memory for mailbox communication. + * eg: AF <=3D> PFx mbox memory + * This allocated memory is split into chunks of MBOX_SIZE + * and setup into each of the RVU PFs. In HW this memory will + * get aliased to an offset within BAR2 of those PFs. + * + * AF will access mbox memory using direct physical addresses + * and PFs will access the same shared memory from BAR2. + */ + + err =3D qmem_alloc(rvu->dev, &mbox_addr, ndevs, mbox_size); + if (err) + return -ENOMEM; + + switch (type) { + case TYPE_AFPF: + rvu->ng_rvu->pf_mbox_addr =3D mbox_addr; + iova =3D (u64)mbox_addr->iova; + for (pf =3D 0; pf < ndevs; pf++) { + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFX_ADDR(pf), + (u64)iova); + iova +=3D mbox_size; + } + break; + default: + return 0; + } + + return 0; +} + int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int ndevs) { int dev; @@ -41,9 +214,48 @@ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int = ndevs) if (!is_cn20k(rvu->pdev)) return 0; =20 + rvu->ng_rvu->rvu_mbox_ops =3D &cn20k_mbox_ops; + for (dev =3D 0; dev < ndevs; dev++) rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); =20 + return rvu_alloc_mbox_memory(rvu, type, ndevs, MBOX_SIZE); +} + +void cn20k_free_mbox_memory(struct rvu *rvu) +{ + qmem_free(rvu->dev, rvu->ng_rvu->pf_mbox_addr); +} + +int rvu_alloc_cint_qint_mem(struct rvu *rvu, struct rvu_pfvf *pfvf, + int blkaddr, int nixlf) +{ + int qints, hwctx_size, err; + u64 cfg, ctx_cfg; + + ctx_cfg =3D rvu_read64(rvu, blkaddr, NIX_AF_CONST3); + /* Alloc memory for CQINT's HW contexts */ + cfg =3D rvu_read64(rvu, blkaddr, NIX_AF_CONST2); + qints =3D (cfg >> 24) & 0xFFF; + hwctx_size =3D 1UL << ((ctx_cfg >> 24) & 0xF); + err =3D qmem_alloc(rvu->dev, &pfvf->cq_ints_ctx, qints, hwctx_size); + if (err) + return -ENOMEM; + + rvu_write64(rvu, blkaddr, NIX_AF_LFX_CINTS_BASE(nixlf), + (u64)pfvf->cq_ints_ctx->iova); + + /* Alloc memory for QINT's HW contexts */ + cfg =3D rvu_read64(rvu, blkaddr, NIX_AF_CONST2); + qints =3D (cfg >> 12) & 0xFFF; + hwctx_size =3D 1UL << ((ctx_cfg >> 20) & 0xF); + err =3D qmem_alloc(rvu->dev, &pfvf->nix_qints_ctx, qints, hwctx_size); + if (err) + return -ENOMEM; + + rvu_write64(rvu, blkaddr, NIX_AF_LFX_QINTS_BASE(nixlf), + (u64)pfvf->nix_qints_ctx->iova); + return 0; } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/reg.h index 58152a4024ec..df2d52567da7 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h @@ -19,6 +19,23 @@ /* RVU AF BAR0 Mbox registers for AF =3D> PFx */ #define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) #define RVU_MBOX_AF_PFX_CFG(a) (0x6000 | (a) << 4) +#define RVU_MBOX_AF_AFPFX_TRIGX(a) (0x9000 | (a) << 3) +#define RVU_MBOX_AF_PFAF_INT(a) (0x2980 | (a) << 6) +#define RVU_MBOX_AF_PFAF_INT_W1S(a) (0x2988 | (a) << 6) +#define RVU_MBOX_AF_PFAF_INT_ENA_W1S(a) (0x2990 | (a) << 6) +#define RVU_MBOX_AF_PFAF_INT_ENA_W1C(a) (0x2998 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT(a) (0x29A0 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT_W1S(a) (0x29A8 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT_ENA_W1S(a) (0x29B0 | (a) << 6) +#define RVU_MBOX_AF_PFAF1_INT_ENA_W1C(a) (0x29B8 | (a) << 6) + +/* RVU PF =3D> AF mbox registers */ +#define RVU_MBOX_PF_PFAF_TRIGX(a) (0xC00 | (a) << 3) +#define RVU_MBOX_PF_INT (0xC20) +#define RVU_MBOX_PF_INT_W1S (0xC28) +#define RVU_MBOX_PF_INT_ENA_W1S (0xC30) +#define RVU_MBOX_PF_INT_ENA_W1C (0xC38) + #define RVU_AF_BAR2_SEL (0x9000000) #define RVU_AF_BAR2_PFID (0x16400) #define NIX_CINTX_INT_W1S(a) (0xd30 | (a) << 12) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h b/dri= vers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h new file mode 100644 index 000000000000..fccad6e422e8 --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Admin Function driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef STRUCT_H +#define STRUCT_H + +/* RVU Admin function Interrupt Vector Enumeration */ +enum rvu_af_cn20k_int_vec_e { + RVU_AF_CN20K_INT_VEC_POISON =3D 0x0, + RVU_AF_CN20K_INT_VEC_PFFLR0 =3D 0x1, + RVU_AF_CN20K_INT_VEC_PFFLR1 =3D 0x2, + RVU_AF_CN20K_INT_VEC_PFME0 =3D 0x3, + RVU_AF_CN20K_INT_VEC_PFME1 =3D 0x4, + RVU_AF_CN20K_INT_VEC_GEN =3D 0x5, + RVU_AF_CN20K_INT_VEC_PFAF_MBOX0 =3D 0x6, + RVU_AF_CN20K_INT_VEC_PFAF_MBOX1 =3D 0x7, + RVU_AF_CN20K_INT_VEC_PFAF1_MBOX0 =3D 0x8, + RVU_AF_CN20K_INT_VEC_PFAF1_MBOX1 =3D 0x9, + RVU_AF_CN20K_INT_VEC_CNT =3D 0xa, +}; +#endif diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 1e3e72107a9d..4ff3aa58d3d4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -46,8 +46,10 @@ void __otx2_mbox_reset(struct otx2_mbox *mbox, int devid) mdev->rsp_size =3D 0; tx_hdr->num_msgs =3D 0; tx_hdr->msg_size =3D 0; + tx_hdr->sig =3D 0; rx_hdr->num_msgs =3D 0; rx_hdr->msg_size =3D 0; + rx_hdr->sig =3D 0; } EXPORT_SYMBOL(__otx2_mbox_reset); =20 @@ -71,9 +73,78 @@ void otx2_mbox_destroy(struct otx2_mbox *mbox) } EXPORT_SYMBOL(otx2_mbox_destroy); =20 +int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, + void *reg_base, int direction, int ndevs) +{ + switch (direction) { + case MBOX_DIR_AFPF: + mbox->tx_start =3D MBOX_DOWN_TX_START; + mbox->rx_start =3D MBOX_DOWN_RX_START; + mbox->tx_size =3D MBOX_DOWN_TX_SIZE; + mbox->rx_size =3D MBOX_DOWN_RX_SIZE; + break; + case MBOX_DIR_PFAF: + mbox->tx_start =3D MBOX_DOWN_RX_START; + mbox->rx_start =3D MBOX_DOWN_TX_START; + mbox->tx_size =3D MBOX_DOWN_RX_SIZE; + mbox->rx_size =3D MBOX_DOWN_TX_SIZE; + break; + case MBOX_DIR_AFPF_UP: + mbox->tx_start =3D MBOX_UP_TX_START; + mbox->rx_start =3D MBOX_UP_RX_START; + mbox->tx_size =3D MBOX_UP_TX_SIZE; + mbox->rx_size =3D MBOX_UP_RX_SIZE; + break; + case MBOX_DIR_PFAF_UP: + mbox->tx_start =3D MBOX_UP_RX_START; + mbox->rx_start =3D MBOX_UP_TX_START; + mbox->tx_size =3D MBOX_UP_RX_SIZE; + mbox->rx_size =3D MBOX_UP_TX_SIZE; + break; + default: + return -ENODEV; + } + + switch (direction) { + case MBOX_DIR_AFPF: + mbox->trigger =3D RVU_MBOX_AF_AFPFX_TRIGX(1); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_AFPF_UP: + mbox->trigger =3D RVU_MBOX_AF_AFPFX_TRIGX(0); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_PFAF: + mbox->trigger =3D RVU_MBOX_PF_PFAF_TRIGX(0); + mbox->tr_shift =3D 0; + break; + case MBOX_DIR_PFAF_UP: + mbox->trigger =3D RVU_MBOX_PF_PFAF_TRIGX(1); + mbox->tr_shift =3D 0; + break; + default: + return -ENODEV; + } + mbox->reg_base =3D reg_base; + mbox->pdev =3D pdev; + + mbox->dev =3D kcalloc(ndevs, sizeof(struct otx2_mbox_dev), GFP_KERNEL); + if (!mbox->dev) { + otx2_mbox_destroy(mbox); + return -ENOMEM; + } + mbox->ndevs =3D ndevs; + + return 0; +} + static int otx2_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, void *reg_base, int direction, int ndevs) { + if (is_cn20k(pdev)) + return cn20k_mbox_setup(mbox, pdev, reg_base, + direction, ndevs); + switch (direction) { case MBOX_DIR_AFPF: case MBOX_DIR_PFVF: @@ -252,7 +323,10 @@ static void otx2_mbox_msg_send_data(struct otx2_mbox *= mbox, int devid, u64 data) =20 spin_lock(&mdev->mbox_lock); =20 - tx_hdr->msg_size =3D mdev->msg_size; + if (!tx_hdr->sig) { + tx_hdr->msg_size =3D mdev->msg_size; + tx_hdr->num_msgs =3D mdev->num_msgs; + } =20 /* Reset header for next messages */ mdev->msg_size =3D 0; @@ -266,7 +340,6 @@ static void otx2_mbox_msg_send_data(struct otx2_mbox *m= box, int devid, u64 data) * messages. So this should be written after writing all the messages * to the shared memory. */ - tx_hdr->num_msgs =3D mdev->num_msgs; rx_hdr->num_msgs =3D 0; =20 trace_otx2_msg_send(mbox->pdev, tx_hdr->num_msgs, tx_hdr->msg_size); @@ -324,6 +397,7 @@ struct mbox_msghdr *otx2_mbox_alloc_msg_rsp(struct otx2= _mbox *mbox, int devid, { struct otx2_mbox_dev *mdev =3D &mbox->dev[devid]; struct mbox_msghdr *msghdr =3D NULL; + struct mbox_hdr *mboxhdr =3D NULL; =20 spin_lock(&mdev->mbox_lock); size =3D ALIGN(size, MBOX_MSG_ALIGN); @@ -347,6 +421,11 @@ struct mbox_msghdr *otx2_mbox_alloc_msg_rsp(struct otx= 2_mbox *mbox, int devid, mdev->msg_size +=3D size; mdev->rsp_size +=3D size_rsp; msghdr->next_msgoff =3D mdev->msg_size + msgs_offset; + + mboxhdr =3D mdev->mbase + mbox->tx_start; + /* Clear the msg header region */ + memset(mboxhdr, 0, msgs_offset); + exit: spin_unlock(&mdev->mbox_lock); =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.h index df64a18fe1d6..86d07fc4a7ff 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -13,6 +13,7 @@ =20 #include "rvu_struct.h" #include "common.h" +#include "cn20k/struct.h" =20 #define MBOX_SIZE SZ_64K =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index a5ebd7cd3a5c..85d2e42b115e 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -755,6 +755,11 @@ static void rvu_free_hw_resources(struct rvu *rvu) =20 rvu_reset_msix(rvu); mutex_destroy(&rvu->rsrc_lock); + + /* Free the QINT/CINt memory */ + pfvf =3D &rvu->pf[RVU_AFPF]; + qmem_free(rvu->dev, pfvf->nix_qints_ctx); + qmem_free(rvu->dev, pfvf->cq_ints_ctx); } =20 static void rvu_setup_pfvf_macaddress(struct rvu *rvu) @@ -2683,6 +2688,11 @@ static void rvu_enable_mbox_intr(struct rvu *rvu) { struct rvu_hwinfo *hw =3D rvu->hw; =20 + if (is_cn20k(rvu->pdev)) { + cn20k_rvu_enable_mbox_intr(rvu); + return; + } + /* Clear spurious irqs, if any */ rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFAF_MBOX_INT, INTR_MASK(hw->total_pfs)); @@ -2936,9 +2946,12 @@ static void rvu_unregister_interrupts(struct rvu *rv= u) =20 rvu_cpt_unregister_interrupts(rvu); =20 - /* Disable the Mbox interrupt */ - rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFAF_MBOX_INT_ENA_W1C, - INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + if (!is_cn20k(rvu->pdev)) + /* Disable the Mbox interrupt */ + rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFAF_MBOX_INT_ENA_W1C, + INTR_MASK(rvu->hw->total_pfs) & ~1ULL); + else + cn20k_rvu_unregister_interrupts(rvu); =20 /* Disable the PF FLR interrupt */ rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_PFFLR_INT_ENA_W1C, @@ -3001,20 +3014,30 @@ static int rvu_register_interrupts(struct rvu *rvu) return ret; } =20 - /* Register mailbox interrupt handler */ - sprintf(&rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], "RVUAF Mbox"); - ret =3D request_irq(pci_irq_vector - (rvu->pdev, RVU_AF_INT_VEC_MBOX), - rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, - &rvu->irq_name[RVU_AF_INT_VEC_MBOX * - NAME_SIZE], rvu); - if (ret) { - dev_err(rvu->dev, - "RVUAF: IRQ registration failed for mbox\n"); - goto fail; - } + if (!is_cn20k(rvu->pdev)) { + /* Register mailbox interrupt handler */ + sprintf(&rvu->irq_name[RVU_AF_INT_VEC_MBOX * NAME_SIZE], + "RVUAF Mbox"); + ret =3D request_irq(pci_irq_vector + (rvu->pdev, RVU_AF_INT_VEC_MBOX), + rvu->ng_rvu->rvu_mbox_ops->pf_intr_handler, 0, + &rvu->irq_name[RVU_AF_INT_VEC_MBOX * + NAME_SIZE], rvu); + if (ret) { + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for mbox\n"); + goto fail; + } =20 - rvu->irq_allocated[RVU_AF_INT_VEC_MBOX] =3D true; + rvu->irq_allocated[RVU_AF_INT_VEC_MBOX] =3D true; + } else { + ret =3D cn20k_register_afpf_mbox_intr(rvu); + if (ret) { + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for mbox\n"); + goto fail; + } + } =20 /* Enable mailbox interrupts from all PFs */ rvu_enable_mbox_intr(rvu); @@ -3467,6 +3490,9 @@ static int rvu_probe(struct pci_dev *pdev, const stru= ct pci_device_id *id) ptp_start(rvu, rvu->fwdata->sclk, rvu->fwdata->ptp_ext_clk_rate, rvu->fwdata->ptp_ext_tstamp); =20 + /* Alloc CINT and QINT memory */ + rvu_alloc_cint_qint_mem(rvu, &rvu->pf[RVU_AFPF], BLKADDR_NIX0, + (rvu->hw->block[BLKADDR_NIX0].lf.max)); return 0; err_dl: rvu_unregister_dl(rvu); @@ -3518,6 +3544,8 @@ static void rvu_remove(struct pci_dev *pdev) pci_set_drvdata(pdev, NULL); =20 devm_kfree(&pdev->dev, rvu->hw); + if (is_cn20k(rvu->pdev)) + cn20k_free_mbox_memory(rvu); kfree(rvu->ng_rvu); devm_kfree(&pdev->dev, rvu); } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index 2533e294149c..6407d102e1ef 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -47,6 +47,9 @@ #define RVU_PFVF_FUNC_MASK rvu_pcifunc_func_mask =20 #ifdef CONFIG_DEBUG_FS + +#define RVU_AFPF 25 + struct dump_ctx { int lf; int id; @@ -444,6 +447,16 @@ struct mbox_wq_info { struct workqueue_struct *mbox_wq; }; =20 +struct rvu_irq_data { + u64 intr_status; + void (*rvu_queue_work_hdlr)(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr); + struct rvu *rvu; + int vec_num; + int start; + int mdevs; +}; + struct mbox_ops { irqreturn_t (*pf_intr_handler)(int irq, void *rvu_irq); }; @@ -953,7 +966,8 @@ int rvu_nix_mcast_get_mce_index(struct rvu *rvu, u16 pc= ifunc, int rvu_nix_mcast_update_mcam_entry(struct rvu *rvu, u16 pcifunc, u32 mcast_grp_idx, u16 mcam_index); void rvu_nix_flr_free_bpids(struct rvu *rvu, u16 pcifunc); - +int rvu_alloc_cint_qint_mem(struct rvu *rvu, struct rvu_pfvf *pfvf, + int blkaddr, int nixlf); /* NPC APIs */ void rvu_npc_freemem(struct rvu *rvu); int rvu_npc_get_pkind(struct rvu *rvu, u16 pf); --=20 2.25.1 From nobody Tue Nov 26 01:47:17 2024 Received: from mx0a-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7957B1CCEE7; Tue, 22 Oct 2024 18:54:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.148.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623299; cv=none; b=AN+/vmb021WUsE9nceH8kt2KOk0iMo1ewSUX28dDSXdSkcgg9FhxkSatYLTyJ/ZWxa08BpZp3Pm/geUgJV1Fm7Ka58E8/qWpg5Nw2MUUXO2FgZAfBXM0ZdeTYhIVI/ouLKOFoPBoTaAAfq8f532gR4XriKHPQVZtJf9cCIxpvaE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623299; c=relaxed/simple; bh=zNLroXTzLmwqg7o964g/sMkIHb+2ljb8xphnCIvp7V8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BNA8uxDWcMFMvZxdm/BAbQHptUD7Q2aSWDMwYwUOHEM1JdKJ9tkIUAIDEzj5m/mXgHcMuFfwCg6HQEK8zEm/iIGHgMeU3KCz3WxQn3VkgL0Z3meBPjw2iAhZQS8newMar3EPDWRB4i11LGlSIDfmlym9jagrmHebWWIAPEVW6Kw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=VzCHx6ei; arc=none smtp.client-ip=67.231.148.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="VzCHx6ei" Received: from pps.filterd (m0431384.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49MFURZO004379; Tue, 22 Oct 2024 11:54:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=f dIXt75EUbxNybjVNvpTDEE0S6W14znguVh7d6/zb5E=; b=VzCHx6eiVkg3VFvlZ vB1vVEHLLhvx7ZavUjgamKlEaEghQAK9NWFN4OS2bw7HGbsukwrn29236+UCL72J dQkkD3WW4jkb4SCoY7T6euQ7REgWQZNLfWJiICNAbrNt45V7vKO8yCVRvGtPrmLe rZ+6hQEYk2bKfiH4GtFoLDdjl5l9/oCFGGh+ym/lGSRctJAl2/YjR5nfwWLk+hRm CSdKAyK6E8oZHLhv3asYVSpJygKkLb6YAtL43UVbQqMCr6RFeu2N0RIZ70nE1Bam 9zdvK7oBDo40a7mfBKoq3YHy1TCU7HWZI+rtxicktO2W26ILxoQsltMQFN2eXTu+ qHvpA== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 42eera0hxp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Oct 2024 11:54:37 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 22 Oct 2024 11:54:36 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 22 Oct 2024 11:54:36 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id 988433F7067; Tue, 22 Oct 2024 11:54:32 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH v2 4/6] octeontx2-pf: CN20K mbox REQ/ACK implementation for NIC PF Date: Wed, 23 Oct 2024 00:24:08 +0530 Message-ID: <20241022185410.4036100-5-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241022185410.4036100-1-saikrishnag@marvell.com> References: <20241022185410.4036100-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-ORIG-GUID: n7SFiwDXPYDzdzEX1nCpLy7lBEprw48- X-Proofpoint-GUID: n7SFiwDXPYDzdzEX1nCpLy7lBEprw48- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 Content-Type: text/plain; charset="utf-8" This implementation uses separate trigger interrupts for request, response messages against using trigger message data in CN10K. This patch adds support for basic mbox implementation for CN20K from NIC PF side. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../marvell/octeontx2/af/cn20k/struct.h | 15 ++++ .../ethernet/marvell/octeontx2/nic/Makefile | 2 +- .../ethernet/marvell/octeontx2/nic/cn10k.c | 18 ++++- .../ethernet/marvell/octeontx2/nic/cn10k.h | 1 + .../ethernet/marvell/octeontx2/nic/cn20k.c | 63 +++++++++++++++ .../ethernet/marvell/octeontx2/nic/cn20k.h | 14 ++++ .../marvell/octeontx2/nic/otx2_common.h | 5 ++ .../ethernet/marvell/octeontx2/nic/otx2_pf.c | 80 +++++++++++++++---- .../ethernet/marvell/octeontx2/nic/otx2_reg.h | 4 + .../ethernet/marvell/octeontx2/nic/otx2_vf.c | 6 ++ 10 files changed, 186 insertions(+), 22 deletions(-) create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h b/dri= vers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h index fccad6e422e8..055ccc8c4689 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/struct.h @@ -8,6 +8,21 @@ #ifndef STRUCT_H #define STRUCT_H =20 +/** + * CN20k RVU PF MBOX Interrupt Vector Enumeration + * + * Vectors 0 - 3 are compatible with pre cn20k and hence + * existing macros are being reused. + */ +enum rvu_mbox_pf_int_vec_e { + RVU_MBOX_PF_INT_VEC_VFPF_MBOX0 =3D 0x4, + RVU_MBOX_PF_INT_VEC_VFPF_MBOX1 =3D 0x5, + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX0 =3D 0x6, + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1 =3D 0x7, + RVU_MBOX_PF_INT_VEC_AFPF_MBOX =3D 0x8, + RVU_MBOX_PF_INT_VEC_CNT =3D 0x9, +}; + /* RVU Admin function Interrupt Vector Enumeration */ enum rvu_af_cn20k_int_vec_e { RVU_AF_CN20K_INT_VEC_POISON =3D 0x0, diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/Makefile b/drivers/= net/ethernet/marvell/octeontx2/nic/Makefile index 64a97a0a10ed..1e2e838959f4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/Makefile +++ b/drivers/net/ethernet/marvell/octeontx2/nic/Makefile @@ -7,7 +7,7 @@ obj-$(CONFIG_OCTEONTX2_PF) +=3D rvu_nicpf.o otx2_ptp.o obj-$(CONFIG_OCTEONTX2_VF) +=3D rvu_nicvf.o otx2_ptp.o =20 rvu_nicpf-y :=3D otx2_pf.o otx2_common.o otx2_txrx.o otx2_ethtool.o \ - otx2_flows.o otx2_tc.o cn10k.o otx2_dmac_flt.o \ + otx2_flows.o otx2_tc.o cn10k.o cn20k.o otx2_dmac_flt.o \ otx2_devlink.o qos_sq.o qos.o rvu_nicvf-y :=3D otx2_vf.o =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn10k.c index c1c99d7054f8..52e0e036d00b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c @@ -14,6 +14,7 @@ static struct dev_hw_ops otx2_hw_ops =3D { .sqe_flush =3D otx2_sqe_flush, .aura_freeptr =3D otx2_aura_freeptr, .refill_pool_ptrs =3D otx2_refill_pool_ptrs, + .pfaf_mbox_intr_handler =3D otx2_pfaf_mbox_intr_handler, }; =20 static struct dev_hw_ops cn10k_hw_ops =3D { @@ -21,8 +22,20 @@ static struct dev_hw_ops cn10k_hw_ops =3D { .sqe_flush =3D cn10k_sqe_flush, .aura_freeptr =3D cn10k_aura_freeptr, .refill_pool_ptrs =3D cn10k_refill_pool_ptrs, + .pfaf_mbox_intr_handler =3D otx2_pfaf_mbox_intr_handler, }; =20 +void otx2_init_hw_ops(struct otx2_nic *pfvf) +{ + if (!test_bit(CN10K_LMTST, &pfvf->hw.cap_flag)) { + pfvf->hw_ops =3D &otx2_hw_ops; + return; + } + + pfvf->hw_ops =3D &cn10k_hw_ops; +} +EXPORT_SYMBOL(otx2_init_hw_ops); + int cn10k_lmtst_init(struct otx2_nic *pfvf) { =20 @@ -30,12 +43,9 @@ int cn10k_lmtst_init(struct otx2_nic *pfvf) struct otx2_lmt_info *lmt_info; int err, cpu; =20 - if (!test_bit(CN10K_LMTST, &pfvf->hw.cap_flag)) { - pfvf->hw_ops =3D &otx2_hw_ops; + if (!test_bit(CN10K_LMTST, &pfvf->hw.cap_flag)) return 0; - } =20 - pfvf->hw_ops =3D &cn10k_hw_ops; /* Total LMTLINES =3D num_online_cpus() * 32 (For Burst flush).*/ pfvf->tot_lmt_lines =3D (num_online_cpus() * LMT_BURST_SIZE); pfvf->hw.lmt_info =3D alloc_percpu(struct otx2_lmt_info); diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn10k.h index c1861f7de254..bb030816b523 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.h @@ -39,4 +39,5 @@ int cn10k_alloc_leaf_profile(struct otx2_nic *pfvf, u16 *= leaf); int cn10k_set_ipolicer_rate(struct otx2_nic *pfvf, u16 profile, u32 burst, u64 rate, bool pps); int cn10k_free_leaf_profile(struct otx2_nic *pfvf, u16 leaf); +void otx2_init_hw_ops(struct otx2_nic *pfvf); #endif /* CN10K_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.c new file mode 100644 index 000000000000..5d2cfcd67e91 --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Marvell RVU Ethernet driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#include "otx2_common.h" +#include "otx2_reg.h" +#include "otx2_struct.h" +#include "cn10k.h" + +static struct dev_hw_ops cn20k_hw_ops =3D { + .pfaf_mbox_intr_handler =3D cn20k_pfaf_mbox_intr_handler, +}; + +void cn20k_init(struct otx2_nic *pfvf) +{ + pfvf->hw_ops =3D &cn20k_hw_ops; +} +EXPORT_SYMBOL(cn20k_init); +/* CN20K mbox AF =3D> PFx irq handler */ +irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq) +{ + struct otx2_nic *pf =3D (struct otx2_nic *)pf_irq; + struct mbox *mw =3D &pf->mbox; + struct otx2_mbox_dev *mdev; + struct otx2_mbox *mbox; + struct mbox_hdr *hdr; + int pf_trig_val; + + pf_trig_val =3D otx2_read64(pf, RVU_PF_INT) & 0x3; + + /* Clear the IRQ */ + otx2_write64(pf, RVU_PF_INT, pf_trig_val); + + if (pf_trig_val & BIT_ULL(0)) { + mbox =3D &mw->mbox_up; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(pf->mbox_wq, &mw->mbox_up_wrk); + + trace_otx2_msg_interrupt(pf->pdev, "UP message from AF to PF", + BIT_ULL(0)); + } + + if (pf_trig_val & BIT_ULL(1)) { + mbox =3D &mw->mbox; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(pf->mbox_wq, &mw->mbox_wrk); + trace_otx2_msg_interrupt(pf->pdev, "DOWN reply from AF to PF", + BIT_ULL(1)); + } + + return IRQ_HANDLED; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.h new file mode 100644 index 000000000000..712bb2b5e2ae --- /dev/null +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Marvell RVU Ethernet driver + * + * Copyright (C) 2024 Marvell. + * + */ + +#ifndef CN20K_H +#define CN20K_H + +#include "otx2_common.h" + +void cn20k_init(struct otx2_nic *pfvf); +#endif /* CN20K_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 8e7ed3979f80..d0320192e7a5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -30,6 +30,7 @@ #include #include #include "qos.h" +#include "cn20k.h" =20 /* IPv4 flag more fragment bit */ #define IPV4_FLAG_MORE 0x20 @@ -53,6 +54,9 @@ #define NIX_PF_PFC_PRIO_MAX 8 #endif =20 +irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq); +irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq); + enum arua_mapped_qtypes { AURA_NIX_RQ, AURA_NIX_SQ, @@ -373,6 +377,7 @@ struct dev_hw_ops { int size, int qidx); int (*refill_pool_ptrs)(void *dev, struct otx2_cq_queue *cq); void (*aura_freeptr)(void *dev, int aura, u64 buf); + irqreturn_t (*pfaf_mbox_intr_handler)(int irq, void *pf_irq); }; =20 #define CN10K_MCS_SA_PER_SC 4 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 5492dea547a1..c0cf228ac714 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -958,7 +958,7 @@ static void otx2_pfaf_mbox_up_handler(struct work_struc= t *work) otx2_mbox_msg_send(mbox, 0); } =20 -static irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq) +irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq) { struct otx2_nic *pf =3D (struct otx2_nic *)pf_irq; struct mbox *mw =3D &pf->mbox; @@ -1010,10 +1010,18 @@ static irqreturn_t otx2_pfaf_mbox_intr_handler(int = irq, void *pf_irq) =20 static void otx2_disable_mbox_intr(struct otx2_nic *pf) { - int vector =3D pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX); + int vector; =20 /* Disable AF =3D> PF mailbox IRQ */ - otx2_write64(pf, RVU_PF_INT_ENA_W1C, BIT_ULL(0)); + if (!is_cn20k(pf->pdev)) { + vector =3D pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX); + otx2_write64(pf, RVU_PF_INT_ENA_W1C, BIT_ULL(0)); + } else { + vector =3D pci_irq_vector(pf->pdev, + RVU_MBOX_PF_INT_VEC_AFPF_MBOX); + otx2_write64(pf, RVU_PF_INT_ENA_W1C, + BIT_ULL(0) | BIT_ULL(1)); + } free_irq(vector, pf); } =20 @@ -1025,10 +1033,24 @@ static int otx2_register_mbox_intr(struct otx2_nic = *pf, bool probe_af) int err; =20 /* Register mailbox interrupt handler */ - irq_name =3D &hw->irq_name[RVU_PF_INT_VEC_AFPF_MBOX * NAME_SIZE]; - snprintf(irq_name, NAME_SIZE, "RVUPFAF Mbox"); - err =3D request_irq(pci_irq_vector(pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX), - otx2_pfaf_mbox_intr_handler, 0, irq_name, pf); + if (!is_cn20k(pf->pdev)) { + irq_name =3D &hw->irq_name[RVU_PF_INT_VEC_AFPF_MBOX * NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d AFPF Mbox", + rvu_get_pf(pf->pcifunc)); + err =3D request_irq(pci_irq_vector + (pf->pdev, RVU_PF_INT_VEC_AFPF_MBOX), + pf->hw_ops->pfaf_mbox_intr_handler, + 0, irq_name, pf); + } else { + irq_name =3D &hw->irq_name[RVU_MBOX_PF_INT_VEC_AFPF_MBOX * + NAME_SIZE]; + snprintf(irq_name, NAME_SIZE, "RVUPF%d AFPF Mbox", + rvu_get_pf(pf->pcifunc)); + err =3D request_irq(pci_irq_vector + (pf->pdev, RVU_MBOX_PF_INT_VEC_AFPF_MBOX), + pf->hw_ops->pfaf_mbox_intr_handler, + 0, irq_name, pf); + } if (err) { dev_err(pf->dev, "RVUPF: IRQ registration failed for PFAF mbox irq\n"); @@ -1038,8 +1060,14 @@ static int otx2_register_mbox_intr(struct otx2_nic *= pf, bool probe_af) /* Enable mailbox interrupt for msgs coming from AF. * First clear to avoid spurious interrupts, if any. */ - otx2_write64(pf, RVU_PF_INT, BIT_ULL(0)); - otx2_write64(pf, RVU_PF_INT_ENA_W1S, BIT_ULL(0)); + if (!is_cn20k(pf->pdev)) { + otx2_write64(pf, RVU_PF_INT, BIT_ULL(0)); + otx2_write64(pf, RVU_PF_INT_ENA_W1S, BIT_ULL(0)); + } else { + otx2_write64(pf, RVU_PF_INT, BIT_ULL(0) | BIT_ULL(1)); + otx2_write64(pf, RVU_PF_INT_ENA_W1S, BIT_ULL(0) | + BIT_ULL(1)); + } =20 if (!probe_af) return 0; @@ -1070,7 +1098,7 @@ static void otx2_pfaf_mbox_destroy(struct otx2_nic *p= f) pf->mbox_wq =3D NULL; } =20 - if (mbox->mbox.hwbase) + if (mbox->mbox.hwbase && !is_cn20k(pf->pdev)) iounmap((void __iomem *)mbox->mbox.hwbase); =20 otx2_mbox_destroy(&mbox->mbox); @@ -1089,12 +1117,20 @@ static int otx2_pfaf_mbox_init(struct otx2_nic *pf) if (!pf->mbox_wq) return -ENOMEM; =20 - /* Mailbox is a reserved memory (in RAM) region shared between - * admin function (i.e AF) and this PF, shouldn't be mapped as - * device memory to allow unaligned accesses. + /* For CN20K, AF allocates mbox memory in DRAM and writes PF + * regions/offsets in RVU_MBOX_AF_PFX_ADDR, the RVU_PFX_FUNC_PFAF_MBOX + * gives the aliased address to access AF/PF mailbox regions. */ - hwbase =3D ioremap_wc(pci_resource_start(pf->pdev, PCI_MBOX_BAR_NUM), - MBOX_SIZE); + if (is_cn20k(pf->pdev)) + hwbase =3D pf->reg_base + RVU_PFX_FUNC_PFAF_MBOX + + ((u64)BLKADDR_MBOX << RVU_FUNC_BLKADDR_SHIFT); + else + /* Mailbox is a reserved memory (in RAM) region shared between + * admin function (i.e AF) and this PF, shouldn't be mapped as + * device memory to allow unaligned accesses. + */ + hwbase =3D ioremap_wc(pci_resource_start + (pf->pdev, PCI_MBOX_BAR_NUM), MBOX_SIZE); if (!hwbase) { dev_err(pf->dev, "Unable to map PFAF mailbox region\n"); err =3D -ENOMEM; @@ -2957,8 +2993,13 @@ static int otx2_probe(struct pci_dev *pdev, const st= ruct pci_device_id *id) if (err) goto err_free_netdev; =20 - err =3D pci_alloc_irq_vectors(hw->pdev, RVU_PF_INT_VEC_CNT, - RVU_PF_INT_VEC_CNT, PCI_IRQ_MSIX); + if (!is_cn20k(pf->pdev)) + err =3D pci_alloc_irq_vectors(hw->pdev, RVU_PF_INT_VEC_CNT, + RVU_PF_INT_VEC_CNT, PCI_IRQ_MSIX); + else + err =3D pci_alloc_irq_vectors(hw->pdev, RVU_MBOX_PF_INT_VEC_CNT, + RVU_MBOX_PF_INT_VEC_CNT, + PCI_IRQ_MSIX); if (err < 0) { dev_err(dev, "%s: Failed to alloc %d IRQ vectors\n", __func__, num_vec); @@ -2967,6 +3008,11 @@ static int otx2_probe(struct pci_dev *pdev, const st= ruct pci_device_id *id) =20 otx2_setup_dev_hw_settings(pf); =20 + if (is_cn20k(pf->pdev)) + cn20k_init(pf); + else + otx2_init_hw_ops(pf); + /* Init PF <=3D> AF mailbox stuff */ err =3D otx2_pfaf_mbox_init(pf); if (err) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h b/driver= s/net/ethernet/marvell/octeontx2/nic/otx2_reg.h index 858f084b9d47..901f8cf7f27a 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h @@ -58,6 +58,10 @@ #define RVU_VF_MSIX_PBAX(a) (0xF0000 | (a) << 3) #define RVU_VF_MBOX_REGION (0xC0000) =20 +/* CN20K RVU_MBOX_E: RVU PF/VF MBOX Address Range Enumeration */ +#define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) +#define RVU_PFX_FUNC_PFAF_MBOX (0x80000) + #define RVU_FUNC_BLKADDR_SHIFT 20 #define RVU_FUNC_BLKADDR_MASK 0x1FULL =20 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 99fcc5661674..76b97896ff1c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -616,6 +616,12 @@ static int otx2vf_probe(struct pci_dev *pdev, const st= ruct pci_device_id *id) } =20 otx2_setup_dev_hw_settings(vf); + + if (is_cn20k(vf->pdev)) + cn20k_init(vf); + else + otx2_init_hw_ops(vf); + /* Init VF <=3D> PF mailbox stuff */ err =3D otx2vf_vfaf_mbox_init(vf); if (err) --=20 2.25.1 From nobody Tue Nov 26 01:47:17 2024 Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED3121CC178; Tue, 22 Oct 2024 18:54:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.156.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623295; cv=none; b=uoDCn3P91S318VK7/wk1GX2cepF4J+9Q0fzPDjpjoL3VdCL3y0ndq/xTD+Zifk8vb4M8FTDiT/PiuYcTOQFQ8P7BWcS33q6ur9JmX9cMfyZWkyhAp9btfwj75Z2SJyRPKWi+qtMuv2bJH8uMJ5l+uC8FesolFzJQW2Mws8BwoKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623295; c=relaxed/simple; bh=XqlsNP6gWKCaTmGjw8oQyB2v5zm5qetbAo1YNd7w+vY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=SZVFlrWsT3S+T3axp3RP4srK7Z6w2aR49dBoGmWFoWfQ31M4qCV5sNfh3a9NGEFPdv50D7KFZ6bsRLiAgRaeqZwM47/h5ffdaTYdzQvAdy/ocEyQp4F4ssqIGix/3vny64kMyf3qu4Yfr1ipGbJZmHPgJOi3isC1xe7rgmOZoHo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=WLBQADzV; arc=none smtp.client-ip=67.231.156.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="WLBQADzV" Received: from pps.filterd (m0431383.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49MBRki5009482; Tue, 22 Oct 2024 11:54:43 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=x vmIIaIsLZ+rrTuhcakL7OkkOgpo/jXVd3LqjbeTEKc=; b=WLBQADzVwUhvXN0CY Rcb2YIJx5IpwbRYxIB4oCQAwIzh02HXBbratf79bLMj3FMbIzaLiN1iYbpetT+xB 8/dxo+4AW8tfWNCi1MPDwFFkoMSJ/NOQxW5xfsijhtdEA6kV9QEGrqNShZ1lYP24 wwD88822Ee+82N3xkPa9kHr52zugUxmCp8QRpuytO9weZIJMRNFRAfvOlgVHzV+2 hQtTU6AaGiHQYyvSC4ZQRI59ipMPukKO8+e9+dgx3EhJWpw3Rm9/ONbEN4b8CaWG Py/5uHBXo9v3RhYGfM1B+PFjz8U10IkRcLyT6qBQVqgyZhDh3N+HUdf3/AgIrMXx YjXBg== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42eb66s57k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Oct 2024 11:54:42 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 22 Oct 2024 11:54:41 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 22 Oct 2024 11:54:41 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id 2B6943F7067; Tue, 22 Oct 2024 11:54:36 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH v2 5/6] octeontx2-af: CN20K mbox implementation for AF's VF Date: Wed, 23 Oct 2024 00:24:09 +0530 Message-ID: <20241022185410.4036100-6-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241022185410.4036100-1-saikrishnag@marvell.com> References: <20241022185410.4036100-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: k3L5KBU47SL9a6WlLrnrkHK3spOxpfwE X-Proofpoint-ORIG-GUID: k3L5KBU47SL9a6WlLrnrkHK3spOxpfwE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 Content-Type: text/plain; charset="utf-8" This patch implements the CN20k MBOX communication between AF and AF's VFs. This implementation uses separate trigger interrupts for request, response messages against using trigger message data in CN10K. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/af/cn20k/api.h | 4 + .../marvell/octeontx2/af/cn20k/mbox_init.c | 163 +++++++++++++++++- .../ethernet/marvell/octeontx2/af/cn20k/reg.h | 37 ++++ .../net/ethernet/marvell/octeontx2/af/mbox.c | 28 +++ .../net/ethernet/marvell/octeontx2/af/rvu.c | 67 ++++--- .../net/ethernet/marvell/octeontx2/af/rvu.h | 3 + .../ethernet/marvell/octeontx2/nic/cn20k.c | 47 +++++ .../marvell/octeontx2/nic/otx2_common.h | 17 ++ .../ethernet/marvell/octeontx2/nic/otx2_pf.c | 6 +- .../ethernet/marvell/octeontx2/nic/otx2_reg.h | 15 ++ .../ethernet/marvell/octeontx2/nic/otx2_vf.c | 38 +++- 11 files changed, 388 insertions(+), 37 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/api.h index 9436a4a4d815..0617bf5a56e9 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/api.h @@ -13,6 +13,7 @@ struct ng_rvu { struct mbox_ops *rvu_mbox_ops; struct qmem *pf_mbox_addr; + struct qmem *vf_mbox_addr; }; =20 struct rvu; @@ -20,6 +21,7 @@ struct rvu; /* Mbox related APIs */ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int num); int cn20k_register_afpf_mbox_intr(struct rvu *rvu); +int cn20k_register_afvf_mbox_intr(struct rvu *rvu, int pf_vec_start); int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **mbox_addr, int num, int type, unsigned long *pf_bmap); void cn20k_rvu_enable_mbox_intr(struct rvu *rvu); @@ -27,4 +29,6 @@ void cn20k_rvu_unregister_interrupts(struct rvu *rvu); void cn20k_free_mbox_memory(struct rvu *rvu); int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, void *reg_base, int direction, int ndevs); +void cn20k_rvu_enable_afvf_intr(struct rvu *rvu, int vfs); +void cn20k_rvu_disable_afvf_intr(struct rvu *rvu, int vfs); #endif /* CN20K_API_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c b/= drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c index 0c1ea6923043..b4eb8f2d3c94 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/mbox_init.c @@ -13,6 +13,91 @@ #include "reg.h" #include "api.h" =20 +static irqreturn_t cn20k_afvf_mbox_intr_handler(int irq, void *rvu_irq) +{ + struct rvu_irq_data *rvu_irq_data =3D (struct rvu_irq_data *)rvu_irq; + struct rvu *rvu =3D rvu_irq_data->rvu; + u64 intr; + + /* Sync with mbox memory region */ + rmb(); + + /* Clear interrupts */ + intr =3D rvupf_read64(rvu, rvu_irq_data->intr_status); + rvupf_write64(rvu, rvu_irq_data->intr_status, intr); + + if (intr) + trace_otx2_msg_interrupt(rvu->pdev, "VF(s) to AF", intr); + + rvu_irq_data->afvf_queue_work_hdlr(&rvu->afvf_wq_info, rvu_irq_data->star= t, + rvu_irq_data->mdevs, intr); + + return IRQ_HANDLED; +} + +int cn20k_register_afvf_mbox_intr(struct rvu *rvu, int pf_vec_start) +{ + struct rvu_irq_data *irq_data; + int intr_vec, offset, vec =3D 0; + int err; + + /* irq data for 4 VFPF intr vectors */ + irq_data =3D devm_kcalloc(rvu->dev, 4, + sizeof(struct rvu_irq_data), GFP_KERNEL); + if (!irq_data) + return -ENOMEM; + + for (intr_vec =3D RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; intr_vec <=3D + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1; + intr_vec++, vec++) { + switch (intr_vec) { + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF1_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1: + irq_data[vec].intr_status =3D RVU_MBOX_PF_VFPF1_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 64; + break; + } + irq_data[vec].afvf_queue_work_hdlr =3D + rvu_queue_work; + offset =3D pf_vec_start + intr_vec; + irq_data[vec].vec_num =3D offset; + irq_data[vec].rvu =3D rvu; + + sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAF VFAF%d Mbox%d", + vec / 2, vec % 2); + err =3D request_irq(pci_irq_vector(rvu->pdev, offset), + rvu->ng_rvu->rvu_mbox_ops->afvf_intr_handler, 0, + &rvu->irq_name[offset * NAME_SIZE], + &irq_data[vec]); + if (err) { + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for AFVF mbox irq\n"); + return err; + } + rvu->irq_allocated[offset] =3D true; + } + + return 0; +} + /* CN20K mbox PFx =3D> AF irq handler */ static irqreturn_t cn20k_mbox_pf_common_intr_handler(int irq, void *rvu_ir= q) { @@ -150,6 +235,21 @@ int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void *= *mbox_addr, int region; u64 bar; =20 + if (type =3D=3D TYPE_AFVF) { + for (region =3D 0; region < num; region++) { + if (!test_bit(region, pf_bmap)) + continue; + + bar =3D (u64)phys_to_virt((u64)rvu->ng_rvu->vf_mbox_addr->base); + bar +=3D region * MBOX_SIZE; + mbox_addr[region] =3D (void *)bar; + + if (!mbox_addr[region]) + return -ENOMEM; + } + return 0; + } + for (region =3D 0; region < num; region++) { if (!test_bit(region, pf_bmap)) continue; @@ -167,6 +267,7 @@ int cn20k_rvu_get_mbox_regions(struct rvu *rvu, void **= mbox_addr, =20 static struct mbox_ops cn20k_mbox_ops =3D { .pf_intr_handler =3D cn20k_mbox_pf_common_intr_handler, + .afvf_intr_handler =3D cn20k_afvf_mbox_intr_handler, }; =20 static int rvu_alloc_mbox_memory(struct rvu *rvu, int type, @@ -184,6 +285,9 @@ static int rvu_alloc_mbox_memory(struct rvu *rvu, int t= ype, * * AF will access mbox memory using direct physical addresses * and PFs will access the same shared memory from BAR2. + * + * PF <=3D> VF mbox memory also works in the same fashion. + * AFPF, PFVF requires IOVA to be used to maintain the mailbox msgs */ =20 err =3D qmem_alloc(rvu->dev, &mbox_addr, ndevs, mbox_size); @@ -200,6 +304,10 @@ static int rvu_alloc_mbox_memory(struct rvu *rvu, int = type, iova +=3D mbox_size; } break; + case TYPE_AFVF: + rvu->ng_rvu->vf_mbox_addr =3D mbox_addr; + rvupf_write64(rvu, RVU_PF_VF_MBOX_ADDR, (u64)mbox_addr->iova); + break; default: return 0; } @@ -216,9 +324,13 @@ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int= ndevs) =20 rvu->ng_rvu->rvu_mbox_ops =3D &cn20k_mbox_ops; =20 - for (dev =3D 0; dev < ndevs; dev++) - rvu_write64(rvu, BLKADDR_RVUM, - RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); + if (type =3D=3D TYPE_AFVF) { + rvu_write64(rvu, BLKADDR_RVUM, RVU_MBOX_PF_VF_CFG, ilog2(MBOX_SIZE)); + } else { + for (dev =3D 0; dev < ndevs; dev++) + rvu_write64(rvu, BLKADDR_RVUM, + RVU_MBOX_AF_PFX_CFG(dev), ilog2(MBOX_SIZE)); + } =20 return rvu_alloc_mbox_memory(rvu, type, ndevs, MBOX_SIZE); } @@ -226,6 +338,51 @@ int cn20k_rvu_mbox_init(struct rvu *rvu, int type, int= ndevs) void cn20k_free_mbox_memory(struct rvu *rvu) { qmem_free(rvu->dev, rvu->ng_rvu->pf_mbox_addr); + qmem_free(rvu->dev, rvu->ng_rvu->vf_mbox_addr); +} + +void cn20k_rvu_disable_afvf_intr(struct rvu *rvu, int vfs) +{ + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1CX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1CX(0), INTR_MASK(vfs)); + + if (vfs <=3D 64) + return; + + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)= ); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1CX(1), INTR_MASK(vfs - 64)); +} + +void cn20k_rvu_enable_afvf_intr(struct rvu *rvu, int vfs) +{ + /* Clear any pending interrupts and enable AF VF interrupts for + * the first 64 VFs. + */ + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INTX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INTX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(0), INTR_MASK(vfs)); + + /* FLR */ + rvupf_write64(rvu, RVU_PF_VFFLR_INTX(0), INTR_MASK(vfs)); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1SX(0), INTR_MASK(vfs)); + + /* Same for remaining VFs, if any. */ + if (vfs <=3D 64) + return; + + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INTX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INTX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)= ); + + rvupf_write64(rvu, RVU_PF_VFFLR_INTX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)); + rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1SX(1), INTR_MASK(vfs - 64)); } =20 int rvu_alloc_cint_qint_mem(struct rvu *rvu, struct rvu_pfvf *pfvf, diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h b/driver= s/net/ethernet/marvell/octeontx2/af/cn20k/reg.h index df2d52567da7..affb39803120 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/cn20k/reg.h @@ -41,4 +41,41 @@ #define NIX_CINTX_INT_W1S(a) (0xd30 | (a) << 12) #define NIX_QINTX_CNT(a) (0xc00 | (a) << 12) =20 +#define RVU_MBOX_AF_VFAF_INT(a) (0x3000 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_W1S(a) (0x3008 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_ENA_W1S(a) (0x3010 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_ENA_W1C(a) (0x3018 | (a) << 6) +#define RVU_MBOX_AF_VFAF_INT_ENA_W1C(a) (0x3018 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_INT(a) (0x3020 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_INT_W1S(a) (0x3028 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_IN_ENA_W1S(a) (0x3030 | (a) << 6) +#define RVU_MBOX_AF_VFAF1_IN_ENA_W1C(a) (0x3038 | (a) << 6) + +#define RVU_MBOX_AF_AFVFX_TRIG(a, b) (0x10000 | (a) << 4 | (b) << 3) +#define RVU_MBOX_AF_VFX_ADDR(a) (0x20000 | (a) << 4) +#define RVU_MBOX_AF_VFX_CFG(a) (0x28000 | (a) << 4) + +#define RVU_MBOX_PF_VFX_PFVF_TRIGX(a) (0x2000 | (a) << 3) + +#define RVU_MBOX_PF_VFPF_INTX(a) (0x1000 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_W1SX(a) (0x1020 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1SX(a) (0x1040 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1CX(a) (0x1060 | (a) << 3) + +#define RVU_MBOX_PF_VFPF1_INTX(a) (0x1080 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_W1SX(a) (0x10a0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(a) (0x10c0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(a) (0x10e0 | (a) << 3) + +#define RVU_MBOX_PF_VF_ADDR (0xC40) +#define RVU_MBOX_PF_LMTLINE_ADDR (0xC48) +#define RVU_MBOX_PF_VF_CFG (0xC60) + +#define RVU_MBOX_VF_VFPF_TRIGX(a) (0x3000 | (a) << 3) +#define RVU_MBOX_VF_INT (0x20) +#define RVU_MBOX_VF_INT_W1S (0x28) +#define RVU_MBOX_VF_INT_ENA_W1S (0x30) +#define RVU_MBOX_VF_INT_ENA_W1C (0x38) + +#define RVU_MBOX_VF_VFAF_TRIGX(a) (0x2000 | (a) << 3) #endif /* RVU_MBOX_REG_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c b/drivers/net= /ethernet/marvell/octeontx2/af/mbox.c index 4ff3aa58d3d4..e142f8d99d77 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.c @@ -76,26 +76,38 @@ EXPORT_SYMBOL(otx2_mbox_destroy); int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pci_dev *pdev, void *reg_base, int direction, int ndevs) { + /* For CN20K, PF and VF bit encodings in PCIFUNC are changed. + * Hence set the PF and VF masks here. + */ + rvu_pcifunc_pf_shift =3D 9; + rvu_pcifunc_pf_mask =3D 0x7F; + rvu_pcifunc_func_shift =3D 0; + rvu_pcifunc_func_mask =3D 0x1FF; + switch (direction) { case MBOX_DIR_AFPF: + case MBOX_DIR_PFVF: mbox->tx_start =3D MBOX_DOWN_TX_START; mbox->rx_start =3D MBOX_DOWN_RX_START; mbox->tx_size =3D MBOX_DOWN_TX_SIZE; mbox->rx_size =3D MBOX_DOWN_RX_SIZE; break; case MBOX_DIR_PFAF: + case MBOX_DIR_VFPF: mbox->tx_start =3D MBOX_DOWN_RX_START; mbox->rx_start =3D MBOX_DOWN_TX_START; mbox->tx_size =3D MBOX_DOWN_RX_SIZE; mbox->rx_size =3D MBOX_DOWN_TX_SIZE; break; case MBOX_DIR_AFPF_UP: + case MBOX_DIR_PFVF_UP: mbox->tx_start =3D MBOX_UP_TX_START; mbox->rx_start =3D MBOX_UP_RX_START; mbox->tx_size =3D MBOX_UP_TX_SIZE; mbox->rx_size =3D MBOX_UP_RX_SIZE; break; case MBOX_DIR_PFAF_UP: + case MBOX_DIR_VFPF_UP: mbox->tx_start =3D MBOX_UP_RX_START; mbox->rx_start =3D MBOX_UP_TX_START; mbox->tx_size =3D MBOX_UP_RX_SIZE; @@ -122,6 +134,22 @@ int cn20k_mbox_setup(struct otx2_mbox *mbox, struct pc= i_dev *pdev, mbox->trigger =3D RVU_MBOX_PF_PFAF_TRIGX(1); mbox->tr_shift =3D 0; break; + case MBOX_DIR_PFVF: + mbox->trigger =3D RVU_MBOX_PF_VFX_PFVF_TRIGX(1); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_PFVF_UP: + mbox->trigger =3D RVU_MBOX_PF_VFX_PFVF_TRIGX(0); + mbox->tr_shift =3D 4; + break; + case MBOX_DIR_VFPF: + mbox->trigger =3D RVU_MBOX_VF_VFPF_TRIGX(0); + mbox->tr_shift =3D 0; + break; + case MBOX_DIR_VFPF_UP: + mbox->trigger =3D RVU_MBOX_VF_VFPF_TRIGX(1); + mbox->tr_shift =3D 0; + break; default: return -ENODEV; } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.c index 85d2e42b115e..f8dc2ace4387 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -2441,6 +2441,7 @@ static int rvu_get_mbox_regions(struct rvu *rvu, void= **mbox_addr, =20 static struct mbox_ops rvu_mbox_ops =3D { .pf_intr_handler =3D rvu_mbox_pf_intr_handler, + .afvf_intr_handler =3D rvu_mbox_intr_handler, }; =20 static int rvu_mbox_init(struct rvu *rvu, struct mbox_wq_info *mw, @@ -2984,6 +2985,10 @@ static int rvu_afvf_msix_vectors_num_ok(struct rvu *= rvu) * VF interrupts can be handled. Offset equal to zero means * that PF vectors are not configured and overlapping AF vectors. */ + if (is_cn20k(rvu->pdev)) + return (pfvf->msix.max >=3D RVU_AF_CN20K_INT_VEC_CNT + + RVU_MBOX_PF_INT_VEC_CNT) && offset; + return (pfvf->msix.max >=3D RVU_AF_INT_VEC_CNT + RVU_PF_INT_VEC_CNT) && offset; } @@ -3092,34 +3097,40 @@ static int rvu_register_interrupts(struct rvu *rvu) /* Get PF MSIX vectors offset. */ pf_vec_start =3D rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_INT_CFG(0)) & 0x3ff; + if (!is_cn20k(rvu->pdev)) { + /* Register MBOX0 interrupt. */ + offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX0; + sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox0"); + ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), + rvu->ng_rvu->rvu_mbox_ops->afvf_intr_handler, 0, + &rvu->irq_name[offset * NAME_SIZE], + rvu); + if (ret) + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for Mbox0\n"); =20 - /* Register MBOX0 interrupt. */ - offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX0; - sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox0"); - ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), - rvu_mbox_intr_handler, 0, - &rvu->irq_name[offset * NAME_SIZE], - rvu); - if (ret) - dev_err(rvu->dev, - "RVUAF: IRQ registration failed for Mbox0\n"); - - rvu->irq_allocated[offset] =3D true; + rvu->irq_allocated[offset] =3D true; =20 - /* Register MBOX1 interrupt. MBOX1 IRQ number follows MBOX0 so - * simply increment current offset by 1. - */ - offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX1; - sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox1"); - ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), - rvu_mbox_intr_handler, 0, - &rvu->irq_name[offset * NAME_SIZE], - rvu); - if (ret) - dev_err(rvu->dev, - "RVUAF: IRQ registration failed for Mbox1\n"); + /* Register MBOX1 interrupt. MBOX1 IRQ number follows MBOX0 so + * simply increment current offset by 1. + */ + offset =3D pf_vec_start + RVU_PF_INT_VEC_VFPF_MBOX1; + sprintf(&rvu->irq_name[offset * NAME_SIZE], "RVUAFVF Mbox1"); + ret =3D request_irq(pci_irq_vector(rvu->pdev, offset), + rvu->ng_rvu->rvu_mbox_ops->afvf_intr_handler, 0, + &rvu->irq_name[offset * NAME_SIZE], + rvu); + if (ret) + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for Mbox1\n"); =20 - rvu->irq_allocated[offset] =3D true; + rvu->irq_allocated[offset] =3D true; + } else { + ret =3D cn20k_register_afvf_mbox_intr(rvu, pf_vec_start); + if (ret) + dev_err(rvu->dev, + "RVUAF: IRQ registration failed for Mbox\n"); + } =20 /* Register FLR interrupt handler for AF's VFs */ offset =3D pf_vec_start + RVU_PF_INT_VEC_VFFLR0; @@ -3230,6 +3241,9 @@ static void rvu_disable_afvf_intr(struct rvu *rvu) { int vfs =3D rvu->vfs; =20 + if (is_cn20k(rvu->pdev)) + return cn20k_rvu_disable_afvf_intr(rvu, vfs); + rvupf_write64(rvu, RVU_PF_VFPF_MBOX_INT_ENA_W1CX(0), INTR_MASK(vfs)); rvupf_write64(rvu, RVU_PF_VFFLR_INT_ENA_W1CX(0), INTR_MASK(vfs)); rvupf_write64(rvu, RVU_PF_VFME_INT_ENA_W1CX(0), INTR_MASK(vfs)); @@ -3246,6 +3260,9 @@ static void rvu_enable_afvf_intr(struct rvu *rvu) { int vfs =3D rvu->vfs; =20 + if (is_cn20k(rvu->pdev)) + return cn20k_rvu_enable_afvf_intr(rvu, vfs); + /* Clear any pending interrupts and enable AF VF interrupts for * the first 64 VFs. */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/= ethernet/marvell/octeontx2/af/rvu.h index 6407d102e1ef..ebf191c70804 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -451,6 +451,8 @@ struct rvu_irq_data { u64 intr_status; void (*rvu_queue_work_hdlr)(struct mbox_wq_info *mw, int first, int mdevs, u64 intr); + void (*afvf_queue_work_hdlr)(struct mbox_wq_info *mw, int first, + int mdevs, u64 intr); struct rvu *rvu; int vec_num; int start; @@ -459,6 +461,7 @@ struct rvu_irq_data { =20 struct mbox_ops { irqreturn_t (*pf_intr_handler)(int irq, void *rvu_irq); + irqreturn_t (*afvf_intr_handler)(int irq, void *rvu_irq); }; =20 struct channel_fwdata { diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.c index 5d2cfcd67e91..84ecbfb92449 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c @@ -12,6 +12,7 @@ =20 static struct dev_hw_ops cn20k_hw_ops =3D { .pfaf_mbox_intr_handler =3D cn20k_pfaf_mbox_intr_handler, + .vfaf_mbox_intr_handler =3D cn20k_vfaf_mbox_intr_handler, }; =20 void cn20k_init(struct otx2_nic *pfvf) @@ -61,3 +62,49 @@ irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *= pf_irq) =20 return IRQ_HANDLED; } + +irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, void *vf_irq) +{ + struct otx2_nic *vf =3D (struct otx2_nic *)vf_irq; + struct otx2_mbox_dev *mdev; + struct otx2_mbox *mbox; + struct mbox_hdr *hdr; + int vf_trig_val; + + vf_trig_val =3D otx2_read64(vf, RVU_VF_INT) & 0x3; + /* Clear the IRQ */ + otx2_write64(vf, RVU_VF_INT, vf_trig_val); + + /* Read latest mbox data */ + smp_rmb(); + + if (vf_trig_val & BIT_ULL(1)) { + /* Check for PF =3D> VF response messages */ + mbox =3D &vf->mbox.mbox; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(vf->mbox_wq, &vf->mbox.mbox_wrk); + + trace_otx2_msg_interrupt(mbox->pdev, "DOWN reply from PF0 to VF", + BIT_ULL(1)); + } + + if (vf_trig_val & BIT_ULL(0)) { + /* Check for PF =3D> VF notification messages */ + mbox =3D &vf->mbox.mbox_up; + mdev =3D &mbox->dev[0]; + otx2_sync_mbox_bbuf(mbox, 0); + + hdr =3D (struct mbox_hdr *)(mdev->mbase + mbox->rx_start); + if (hdr->num_msgs) + queue_work(vf->mbox_wq, &vf->mbox.mbox_up_wrk); + + trace_otx2_msg_interrupt(mbox->pdev, "UP message from PF0 to VF", + BIT_ULL(0)); + } + + return IRQ_HANDLED; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index d0320192e7a5..edebdd9ce1e1 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -56,6 +56,8 @@ =20 irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq); +irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, void *vf_irq); +irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq); =20 enum arua_mapped_qtypes { AURA_NIX_RQ, @@ -258,6 +260,7 @@ struct otx2_hw { u16 nix_msixoff; /* Offset of NIX vectors */ char *irq_name; cpumask_var_t *affinity_mask; + struct pf_irq_data *pfvf_irq_devid[4]; =20 /* Stats */ struct otx2_dev_stats dev_stats; @@ -378,6 +381,7 @@ struct dev_hw_ops { int (*refill_pool_ptrs)(void *dev, struct otx2_cq_queue *cq); void (*aura_freeptr)(void *dev, int aura, u64 buf); irqreturn_t (*pfaf_mbox_intr_handler)(int irq, void *pf_irq); + irqreturn_t (*vfaf_mbox_intr_handler)(int irq, void *pf_irq); }; =20 #define CN10K_MCS_SA_PER_SC 4 @@ -445,6 +449,16 @@ struct cn10k_mcs_cfg { struct list_head rxsc_list; }; =20 +struct pf_irq_data { + u64 intr_status; + void (*pf_queue_work_hdlr)(struct mbox *mb, struct workqueue_struct *mw, + int first, int mdevs, u64 intr); + struct otx2_nic *pf; + int vec_num; + int start; + int mdevs; +}; + struct otx2_nic { void __iomem *reg_base; struct net_device *netdev; @@ -485,6 +499,7 @@ struct otx2_nic { struct mbox *mbox_pfvf; struct workqueue_struct *mbox_wq; struct workqueue_struct *mbox_pfvf_wq; + struct qmem *pfvf_mbox_addr; =20 u8 total_vfs; u16 pcifunc; /* RVU PF_FUNC */ @@ -1123,4 +1138,6 @@ u16 otx2_select_queue(struct net_device *netdev, stru= ct sk_buff *skb, int otx2_get_txq_by_classid(struct otx2_nic *pfvf, u16 classid); void otx2_qos_config_txschq(struct otx2_nic *pfvf); void otx2_clean_qos_queues(struct otx2_nic *pfvf); +void otx2_queue_vf_work(struct mbox *mw, struct workqueue_struct *mbox_wq, + int first, int mdevs, u64 intr); #endif /* OTX2_COMMON_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_pf.c index c0cf228ac714..148a5c91af55 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -292,8 +292,8 @@ static int otx2_pf_flr_init(struct otx2_nic *pf, int nu= m_vfs) return 0; } =20 -static void otx2_queue_vf_work(struct mbox *mw, struct workqueue_struct *m= box_wq, - int first, int mdevs, u64 intr) +void otx2_queue_vf_work(struct mbox *mw, struct workqueue_struct *mbox_wq, + int first, int mdevs, u64 intr) { struct otx2_mbox_dev *mdev; struct otx2_mbox *mbox; @@ -536,7 +536,7 @@ static void otx2_pfvf_mbox_up_handler(struct work_struc= t *work) } } =20 -static irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq) +irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq) { struct otx2_nic *pf =3D (struct otx2_nic *)(pf_irq); int vfs =3D pf->total_vfs; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h b/driver= s/net/ethernet/marvell/octeontx2/nic/otx2_reg.h index 901f8cf7f27a..1cd576fd09c5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h @@ -44,6 +44,17 @@ #define RVU_PF_VF_MBOX_ADDR (0xC40) #define RVU_PF_LMTLINE_ADDR (0xC48) =20 +#define RVU_MBOX_PF_VFX_PFVF_TRIGX(a) (0x2000 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INTX(a) (0x1000 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_W1SX(a) (0x1020 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1SX(a) (0x1040 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1CX(a) (0x1060 | (a) << 3) + +#define RVU_MBOX_PF_VFPF1_INTX(a) (0x1080 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_W1SX(a) (0x10a0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(a) (0x10c0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(a) (0x10e0 | (a) << 3) + /* RVU VF registers */ #define RVU_VF_VFPF_MBOX0 (0x00000) #define RVU_VF_VFPF_MBOX1 (0x00008) @@ -61,6 +72,7 @@ /* CN20K RVU_MBOX_E: RVU PF/VF MBOX Address Range Enumeration */ #define RVU_MBOX_AF_PFX_ADDR(a) (0x5000 | (a) << 4) #define RVU_PFX_FUNC_PFAF_MBOX (0x80000) +#define RVU_PFX_FUNCX_VFAF_MBOX (0x40000) =20 #define RVU_FUNC_BLKADDR_SHIFT 20 #define RVU_FUNC_BLKADDR_MASK 0x1FULL @@ -147,4 +159,7 @@ #define LMT_LF_LMTLINEX(a) (LMT_LFBASE | 0x000 | (a) << 12) #define LMT_LF_LMTCANCEL (LMT_LFBASE | 0x400) =20 +/* CN20K registers */ +#define RVU_PF_DISC (0x0) + #endif /* OTX2_REG_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 76b97896ff1c..95bdc65a989c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -238,6 +238,10 @@ static void otx2vf_disable_mbox_intr(struct otx2_nic *= vf) =20 /* Disable VF =3D> PF mailbox IRQ */ otx2_write64(vf, RVU_VF_INT_ENA_W1C, BIT_ULL(0)); + + if (is_cn20k(vf->pdev)) + otx2_write64(vf, RVU_VF_INT_ENA_W1C, BIT_ULL(0) | BIT_ULL(1)); + free_irq(vector, vf); } =20 @@ -250,9 +254,18 @@ static int otx2vf_register_mbox_intr(struct otx2_nic *= vf, bool probe_pf) =20 /* Register mailbox interrupt handler */ irq_name =3D &hw->irq_name[RVU_VF_INT_VEC_MBOX * NAME_SIZE]; - snprintf(irq_name, NAME_SIZE, "RVUVFAF Mbox"); - err =3D request_irq(pci_irq_vector(vf->pdev, RVU_VF_INT_VEC_MBOX), - otx2vf_vfaf_mbox_intr_handler, 0, irq_name, vf); + snprintf(irq_name, NAME_SIZE, "RVUVF%d AFVF Mbox", ((vf->pcifunc & + RVU_PFVF_FUNC_MASK) - 1)); + + if (!is_cn20k(vf->pdev)) { + err =3D request_irq(pci_irq_vector(vf->pdev, RVU_VF_INT_VEC_MBOX), + otx2vf_vfaf_mbox_intr_handler, 0, irq_name, vf); + } else { + err =3D request_irq(pci_irq_vector(vf->pdev, RVU_VF_INT_VEC_MBOX), + vf->hw_ops->vfaf_mbox_intr_handler, 0, irq_name, + vf); + } + if (err) { dev_err(vf->dev, "RVUPF: IRQ registration failed for VFAF mbox irq\n"); @@ -262,8 +275,15 @@ static int otx2vf_register_mbox_intr(struct otx2_nic *= vf, bool probe_pf) /* Enable mailbox interrupt for msgs coming from PF. * First clear to avoid spurious interrupts, if any. */ - otx2_write64(vf, RVU_VF_INT, BIT_ULL(0)); - otx2_write64(vf, RVU_VF_INT_ENA_W1S, BIT_ULL(0)); + if (!is_cn20k(vf->pdev)) { + otx2_write64(vf, RVU_VF_INT, BIT_ULL(0)); + otx2_write64(vf, RVU_VF_INT_ENA_W1S, BIT_ULL(0)); + } else { + otx2_write64(vf, RVU_VF_INT, BIT_ULL(0) | BIT_ULL(1) | + BIT_ULL(2) | BIT_ULL(3)); + otx2_write64(vf, RVU_VF_INT_ENA_W1S, BIT_ULL(0) | + BIT_ULL(1) | BIT_ULL(2) | BIT_ULL(3)); + } =20 if (!probe_pf) return 0; @@ -313,7 +333,13 @@ static int otx2vf_vfaf_mbox_init(struct otx2_nic *vf) if (!vf->mbox_wq) return -ENOMEM; =20 - if (test_bit(CN10K_MBOX, &vf->hw.cap_flag)) { + /* For cn20k platform, VF mailbox region is in dram aliased from AF + * VF MBOX ADDR, MBOX is a separate RVU block. + */ + if (is_cn20k(vf->pdev)) { + hwbase =3D vf->reg_base + RVU_VF_MBOX_REGION + ((u64)BLKADDR_MBOX << + RVU_FUNC_BLKADDR_SHIFT); + } else if (test_bit(CN10K_MBOX, &vf->hw.cap_flag)) { /* For cn10k platform, VF mailbox region is in its BAR2 * register space */ --=20 2.25.1 From nobody Tue Nov 26 01:47:17 2024 Received: from mx0a-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48E651CDA25; Tue, 22 Oct 2024 18:55:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.148.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623302; cv=none; b=Re2e1i5wu15JppGGKhiaKurz/kf6OYw2f+sftP/zu5671TwpF4sF9yZ35Ezjp9Iaea4pdM04Reh2QGzpvAqPg70HKzctsACeBKG/XeyuHHWzwddjlKMB8dSikQM71W+RpEHPZS6w81m6Ixe9g6wF/cCoYDHcdHlopaA2rkkN2to= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729623302; c=relaxed/simple; bh=Sso6AS+yelqSl4LmNgrcBfSnOdzbWeLCP+A4wEx8STw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=G2zS6TAIYhvIeYPwVBqMxMtaeeRWzDYgWx+gSYTFon7QP7gNJH2kSmh8iYqKBaZ+/97mXEDlV23tIhYFDy/FsxCUeDniB3mPSD0k+9AoCPG6rqcpg69o8RV9rsLrtXGulfE9s96gK18Pv1E/g8utkRVeaCtBrl0MKHx84WxRtz8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=KKOy7yDC; arc=none smtp.client-ip=67.231.148.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="KKOy7yDC" Received: from pps.filterd (m0431384.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49MFU98t004127; Tue, 22 Oct 2024 11:54:47 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=m zdAkdplWcXxSRYswNZAXcgRm/cicuqcQR0EDOVYyYg=; b=KKOy7yDCekQaRO+Li Vp+TgOYfyS7iPPtGYlX7Z6JsN9ZDZyFg2LEMh6s7Zo3MkmplnYrEedEsKSRPzBOU 0t7LRMKcyMMlgwd4EXFhZW8iD+512Su+NCAnUeP3MQGiK7ychr2fAHtPjjRrusub xTnRyA5SXKVDb1dlYZCqWppix1kDECE0Atiqi5I8WM7OZol3qwV0uGevdLLiu47W aMUoDix00i/9+wB9rdJbVUitYVfcio2IFswWQ8I4GXhDjQ6IP56Q+Ar330R69ohu 1sIQyuKOnHmce9mFo1CJZMdy+OkcYqUkc+xJAt8ErGXStf0LtfjXHFvSQmhQx9Ge /3fvA== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 42eera0hy1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 22 Oct 2024 11:54:46 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 22 Oct 2024 11:54:45 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 22 Oct 2024 11:54:45 -0700 Received: from hyd1425.marvell.com (unknown [10.29.37.152]) by maili.marvell.com (Postfix) with ESMTP id B1EB83F7067; Tue, 22 Oct 2024 11:54:41 -0700 (PDT) From: Sai Krishna To: , , , , , , , , , , , , CC: Sai Krishna Subject: [net-next PATCH v2 6/6] octeontx2-pf: CN20K mbox implementation between PF-VF Date: Wed, 23 Oct 2024 00:24:10 +0530 Message-ID: <20241022185410.4036100-7-saikrishnag@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241022185410.4036100-1-saikrishnag@marvell.com> References: <20241022185410.4036100-1-saikrishnag@marvell.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-ORIG-GUID: NdQ_hft3-LWBlS1_D7BubePoMjwNwcVV X-Proofpoint-GUID: NdQ_hft3-LWBlS1_D7BubePoMjwNwcVV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 Content-Type: text/plain; charset="utf-8" This patch implements the CN20k MBOX communication between PF and it's VFs. CN20K silicon got extra interrupt of MBOX response for trigger interrupt. Also few of the CSR offsets got changed in CN20K against prior series of silicons. Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Sai Krishna --- .../ethernet/marvell/octeontx2/nic/cn20k.c | 142 ++++++++++++++++++ .../ethernet/marvell/octeontx2/nic/cn20k.h | 3 + .../marvell/octeontx2/nic/otx2_common.h | 2 + .../ethernet/marvell/octeontx2/nic/otx2_pf.c | 59 ++++++-- 4 files changed, 193 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.c index 84ecbfb92449..6e47698ec5c7 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.c @@ -13,6 +13,7 @@ static struct dev_hw_ops cn20k_hw_ops =3D { .pfaf_mbox_intr_handler =3D cn20k_pfaf_mbox_intr_handler, .vfaf_mbox_intr_handler =3D cn20k_vfaf_mbox_intr_handler, + .pfvf_mbox_intr_handler =3D cn20k_pfvf_mbox_intr_handler, }; =20 void cn20k_init(struct otx2_nic *pfvf) @@ -108,3 +109,144 @@ irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, voi= d *vf_irq) =20 return IRQ_HANDLED; } + +void cn20k_enable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs) +{ + /* Clear PF <=3D> VF mailbox IRQ */ + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(1), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(1), ~0ull); + + /* Enable PF <=3D> VF mailbox IRQ */ + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(0), INTR_MASK(numvfs)); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(0), INTR_MASK(numvfs)); + if (numvfs > 64) { + numvfs -=3D 64; + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1SX(1), + INTR_MASK(numvfs)); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(1), + INTR_MASK(numvfs)); + } +} + +void cn20k_disable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs) +{ + int vector, intr_vec, vec =3D 0; + + /* Disable PF <=3D> VF mailbox IRQ */ + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF_INT_ENA_W1CX(1), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(1), ~0ull); + + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(0), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(0), ~0ull); + + if (numvfs > 64) { + otx2_write64(pf, RVU_MBOX_PF_VFPF_INTX(1), ~0ull); + otx2_write64(pf, RVU_MBOX_PF_VFPF1_INTX(1), ~0ull); + } + + for (intr_vec =3D RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; intr_vec <=3D + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1; intr_vec++, vec++) { + vector =3D pci_irq_vector(pf->pdev, intr_vec); + free_irq(vector, pf->hw.pfvf_irq_devid[vec]); + } +} + +irqreturn_t cn20k_pfvf_mbox_intr_handler(int irq, void *pf_irq) +{ + struct pf_irq_data *irq_data =3D (struct pf_irq_data *)(pf_irq); + struct otx2_nic *pf =3D irq_data->pf; + struct mbox *mbox; + u64 intr; + + /* Sync with mbox memory region */ + rmb(); + + /* Clear interrupts */ + intr =3D otx2_read64(pf, irq_data->intr_status); + otx2_write64(pf, irq_data->intr_status, intr); + mbox =3D pf->mbox_pfvf; + + if (intr) + trace_otx2_msg_interrupt(pf->pdev, "VF(s) to PF", intr); + + irq_data->pf_queue_work_hdlr(mbox, pf->mbox_pfvf_wq, irq_data->start, + irq_data->mdevs, intr); + + return IRQ_HANDLED; +} + +int cn20k_register_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs) +{ + struct otx2_hw *hw =3D &pf->hw; + struct pf_irq_data *irq_data; + int intr_vec, ret, vec =3D 0; + char *irq_name; + + /* irq data for 4 PF intr vectors */ + irq_data =3D devm_kcalloc(pf->dev, 4, + sizeof(struct pf_irq_data), GFP_KERNEL); + if (!irq_data) + return -ENOMEM; + + for (intr_vec =3D RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; intr_vec <=3D + RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1; intr_vec++, vec++) { + switch (intr_vec) { + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX0: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF1_INTX(0); + irq_data[vec].start =3D 0; + irq_data[vec].mdevs =3D 64; + break; + case RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1: + irq_data[vec].intr_status =3D + RVU_MBOX_PF_VFPF1_INTX(1); + irq_data[vec].start =3D 64; + irq_data[vec].mdevs =3D 96; + break; + } + irq_data[vec].pf_queue_work_hdlr =3D otx2_queue_vf_work; + irq_data[vec].vec_num =3D intr_vec; + irq_data[vec].pf =3D pf; + + /* Register mailbox interrupt handler */ + irq_name =3D &hw->irq_name[intr_vec * NAME_SIZE]; + if (pf->pcifunc) + snprintf(irq_name, NAME_SIZE, + "RVUPF%d_VF%d Mbox%d", rvu_get_pf(pf->pcifunc), + vec / 2, vec % 2); + else + snprintf(irq_name, NAME_SIZE, "RVUPF_VF%d Mbox%d", + vec / 2, vec % 2); + + hw->pfvf_irq_devid[vec] =3D &irq_data[vec]; + ret =3D request_irq(pci_irq_vector(pf->pdev, intr_vec), + pf->hw_ops->pfvf_mbox_intr_handler, 0, + irq_name, + &irq_data[vec]); + if (ret) { + dev_err(pf->dev, + "RVUPF: IRQ registration failed for PFVF mbox0 irq\n"); + return ret; + } + } + + cn20k_enable_pfvf_mbox_intr(pf, numvfs); + + return 0; +} diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h b/drivers/n= et/ethernet/marvell/octeontx2/nic/cn20k.h index 712bb2b5e2ae..832adaf8c57f 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn20k.h @@ -11,4 +11,7 @@ #include "otx2_common.h" =20 void cn20k_init(struct otx2_nic *pfvf); +int cn20k_register_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs); +void cn20k_disable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs); +void cn20k_enable_pfvf_mbox_intr(struct otx2_nic *pf, int numvfs); #endif /* CN20K_H */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/dri= vers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index edebdd9ce1e1..e56df7aa1d57 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -57,6 +57,7 @@ irqreturn_t otx2_pfaf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t cn20k_pfaf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t cn20k_vfaf_mbox_intr_handler(int irq, void *vf_irq); +irqreturn_t cn20k_pfvf_mbox_intr_handler(int irq, void *pf_irq); irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void *pf_irq); =20 enum arua_mapped_qtypes { @@ -382,6 +383,7 @@ struct dev_hw_ops { void (*aura_freeptr)(void *dev, int aura, u64 buf); irqreturn_t (*pfaf_mbox_intr_handler)(int irq, void *pf_irq); irqreturn_t (*vfaf_mbox_intr_handler)(int irq, void *pf_irq); + irqreturn_t (*pfvf_mbox_intr_handler)(int irq, void *pf_irq); }; =20 #define CN10K_MCS_SA_PER_SC 4 diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers= /net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 148a5c91af55..1a7920327fd5 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -565,6 +565,23 @@ irqreturn_t otx2_pfvf_mbox_intr_handler(int irq, void = *pf_irq) return IRQ_HANDLED; } =20 +static void *cn20k_pfvf_mbox_alloc(struct otx2_nic *pf, int numvfs) +{ + struct qmem *mbox_addr; + int err; + + err =3D qmem_alloc(&pf->pdev->dev, &mbox_addr, numvfs, MBOX_SIZE); + if (err) { + dev_err(pf->dev, "qmem alloc fail\n"); + return ERR_PTR(-ENOMEM); + } + + otx2_write64(pf, RVU_PF_VF_MBOX_ADDR, (u64)mbox_addr->iova); + pf->pfvf_mbox_addr =3D mbox_addr; + + return mbox_addr->base; +} + static int otx2_pfvf_mbox_init(struct otx2_nic *pf, int numvfs) { void __iomem *hwbase; @@ -586,20 +603,28 @@ static int otx2_pfvf_mbox_init(struct otx2_nic *pf, i= nt numvfs) if (!pf->mbox_pfvf_wq) return -ENOMEM; =20 - /* On CN10K platform, PF <-> VF mailbox region follows after - * PF <-> AF mailbox region. + /* For CN20K, PF allocates mbox memory in DRAM and writes PF/VF + * regions/offsets in RVU_PF_VF_MBOX_ADDR, the RVU_PFX_FUNC_PFAF_MBOX + * gives the aliased address to access PF/VF mailbox regions. */ - if (test_bit(CN10K_MBOX, &pf->hw.cap_flag)) - base =3D pci_resource_start(pf->pdev, PCI_MBOX_BAR_NUM) + - MBOX_SIZE; - else - base =3D readq((void __iomem *)((u64)pf->reg_base + + if (is_cn20k(pf->pdev)) { + hwbase =3D (void __iomem *)cn20k_pfvf_mbox_alloc(pf, numvfs); + } else { + /* On CN10K platform, PF <-> VF mailbox region follows after + * PF <-> AF mailbox region. + */ + if (test_bit(CN10K_MBOX, &pf->hw.cap_flag)) + base =3D pci_resource_start(pf->pdev, PCI_MBOX_BAR_NUM) + + MBOX_SIZE; + else + base =3D readq((void __iomem *)((u64)pf->reg_base + RVU_PF_VF_BAR4_ADDR)); =20 - hwbase =3D ioremap_wc(base, MBOX_SIZE * pf->total_vfs); - if (!hwbase) { - err =3D -ENOMEM; - goto free_wq; + hwbase =3D ioremap_wc(base, MBOX_SIZE * pf->total_vfs); + if (!hwbase) { + err =3D -ENOMEM; + goto free_wq; + } } =20 mbox =3D &pf->mbox_pfvf[0]; @@ -623,7 +648,7 @@ static int otx2_pfvf_mbox_init(struct otx2_nic *pf, int= numvfs) return 0; =20 free_iomem: - if (hwbase) + if (hwbase && !(is_cn20k(pf->pdev))) iounmap(hwbase); free_wq: destroy_workqueue(pf->mbox_pfvf_wq); @@ -642,8 +667,10 @@ static void otx2_pfvf_mbox_destroy(struct otx2_nic *pf) pf->mbox_pfvf_wq =3D NULL; } =20 - if (mbox->mbox.hwbase) + if (mbox->mbox.hwbase && !is_cn20k(pf->pdev)) iounmap(mbox->mbox.hwbase); + else + qmem_free(&pf->pdev->dev, pf->pfvf_mbox_addr); =20 otx2_mbox_destroy(&mbox->mbox); } @@ -667,6 +694,9 @@ static void otx2_disable_pfvf_mbox_intr(struct otx2_nic= *pf, int numvfs) { int vector; =20 + if (is_cn20k(pf->pdev)) + return cn20k_disable_pfvf_mbox_intr(pf, numvfs); + /* Disable PF <=3D> VF mailbox IRQ */ otx2_write64(pf, RVU_PF_VFPF_MBOX_INT_ENA_W1CX(0), ~0ull); otx2_write64(pf, RVU_PF_VFPF_MBOX_INT_ENA_W1CX(1), ~0ull); @@ -688,6 +718,9 @@ static int otx2_register_pfvf_mbox_intr(struct otx2_nic= *pf, int numvfs) char *irq_name; int err; =20 + if (is_cn20k(pf->pdev)) + return cn20k_register_pfvf_mbox_intr(pf, numvfs); + /* Register MBOX0 interrupt handler */ irq_name =3D &hw->irq_name[RVU_PF_INT_VEC_VFPF_MBOX0 * NAME_SIZE]; if (pf->pcifunc) --=20 2.25.1