From nobody Mon Jun 29 07:49:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 964ACC433EF for ; Mon, 14 Feb 2022 11:46:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352366AbiBNLjt (ORCPT ); Mon, 14 Feb 2022 06:39:49 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:43238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351967AbiBNLj3 (ORCPT ); Mon, 14 Feb 2022 06:39:29 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FFEEBF56; Mon, 14 Feb 2022 03:29:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644838177; x=1676374177; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=U2uuZbQHEdb5rmolpAMFUIlVMwjCd15kzHvEeAFLHvA=; b=G4KYOy9NrM95Ee3RpNcvvPPFrqDL/aNl1rpKP32CXjW/O1KqxYU/h1AT E3vwJdwS+wm1WWrpD7BzHw4uHC5Qx7AVf0Nfb85yLO7fuDtPBu0IADBkv YdWiDr8DnfYK3M/8dyzigF1bsOg4DPBwUq8rHv4YVov35SkAPGMgYY3By 4mvveyjSq6S9QZxd0PNmxmBrQEnPL2HqroZigw9tg8/i+ZLznBfRY2Jfm 23EN3qFgky8ETsWA9oClnXIbc99CW1NkZoHu/YvByxorg4dHD1D/YWtij MKDTh2rScgnIFsq4lIvkCBXv6Lh2COrJqo9MMfOKQGUMwOLuWu8KQnn6X g==; X-IronPort-AV: E=McAfee;i="6200,9189,10257"; a="250276727" X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="250276727" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Feb 2022 03:29:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="587166967" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by fmsmga008.fm.intel.com with ESMTP; 14 Feb 2022 03:29:35 -0800 From: Tianfei zhang To: hao.wu@intel.com, trix@redhat.com, mdf@kernel.org, yilun.xu@intel.com, linux-fpga@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: corbet@lwn.net, Tianfei Zhang Subject: [PATCH v1 1/7] Documentation: fpga: dfl: add description of IOFS Date: Mon, 14 Feb 2022 06:26:13 -0500 Message-Id: <20220214112619.219761-2-tianfei.zhang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220214112619.219761-1-tianfei.zhang@intel.com> References: <20220214112619.219761-1-tianfei.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianfei Zhang This patch adds description about IOFS support for DFL. Signed-off-by: Tianfei Zhang --- Documentation/fpga/dfl.rst | 99 +++++++++++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 2 deletions(-) diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst index ef9eec71f6f3..6f9eae1c1697 100644 --- a/Documentation/fpga/dfl.rst +++ b/Documentation/fpga/dfl.rst @@ -58,7 +58,10 @@ interface to FPGA, e.g. the FPGA Management Engine (FME)= and Port (more descriptions on FME and Port in later sections). =20 Accelerated Function Unit (AFU) represents an FPGA programmable region and -always connects to a FIU (e.g. a Port) as its child as illustrated above. +always connects to a FIU (e.g. a Port) as its child as illustrated above, = but +on IOFS design, it introducing Port Gasket which contains AFUs. For DFL pe= rspective, +the Next_AFU pointer on FIU feature header can point to NULL so the AFU is= not +connects to a FIU(more descriptions on IOFS in later section). =20 Private Features represent sub features of the FIU and AFU. They could be various function blocks with different IDs, but all private features which @@ -134,6 +137,9 @@ reconfigurable region containing an AFU. It controls th= e communication from SW to the accelerator and exposes features such as reset and debug. Each FPGA device may have more than one port, but always one AFU per port. =20 +On IOFS, it introducing a new hardware unit, Port Gasket, which contains a= ll +the PR specific modules and regions (more descriptions on IOFS in later se= ction). + =20 AFU =3D=3D=3D @@ -143,6 +149,9 @@ used for accelerator-specific control registers. User-space applications can acquire exclusive access to an AFU attached to= a port by using open() on the port device node and release it using close(). =20 +On IOFS, the AFU is embedded in a Port Gasket. The AFU resource can expose= via +VFs with SRIOV support (more descriptions on IOFS in later section). + The following functions are exposed through ioctls: =20 - Get driver API version (DFL_FPGA_GET_API_VERSION) @@ -284,7 +293,8 @@ FME is always accessed through the physical function (P= F). =20 Ports (and related AFUs) are accessed via PF by default, but could be expo= sed through virtual function (VF) devices via PCIe SRIOV. Each VF only contains -1 Port and 1 AFU for isolation. Users could assign individual VFs (acceler= ators) +1 Port (On IOFS design, the VF is designs without Port) and 1 AFU for isol= ation. +Users could assign individual VFs (accelerators) created via PCIe SRIOV interface, to virtual machines. =20 The driver organization in virtualization case is illustrated below: @@ -389,6 +399,91 @@ The device nodes used for ioctl() or mmap() can be ref= erenced through:: /sys/class/fpga_region///dev /sys/class/fpga_region///dev =20 +Intel Open FPGA stack +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Intel Open FPGA stack aka IOFS, Intel's version of a common core set of +RTL to allow customers to easily interface to logic and IP on the FPGA. +IOFS leverage the DFL for the implementation of the FPGA RTL design. + +IOFS designs allow for the arrangement of software interfaces across multi= ple +PCIe endpoints. Some of these interfaces may be PFs defined in the static = region +that connect to interfaces in an IP that is loaded via Partial Reconfigura= tion (PR). +And some of these interfaces may be VFs defined in the PR region that can = be +reconfigured by the end-user. Furthermore, these PFs/VFs may also be arran= ged +using a DFL such that features may be discovered and accessed in user space +(with the aid of a generic kernel driver like vfio-pci). The diagram below= depicts +an example design with two PFs and two VFs. In this example, PF1 implement= s its +MMIO space such that it is compatible with the VirtIO framework. The other= functions, +VF0 and VF1, leverage VFIO to export the MMIO space to an application or a= hypervisor. + + +-----------------+ +--------------+ +-------------+ +------------+ + | FPGA Managerment| | VirtIO | | User App | | Virtual | + | App | | App | | | | Machine | + +--------+--------+ +------+-------+ +------+------+ +-----+------+ + | | | | + | | | | + +--------+--------+ +------+-------+ +------+------+ | + | DFL Driver | |VirtIO driver | | VFIO | | + +--------+--------+ +------+-------+ +------+------+ | + | | | | + | | | | + +--------+--------+ +------+-------+ +------+------+ +----+------+ + | PF0 | | PF1 | | PF0_VF0 | | PF0_VF1 | + +-----------------+ +--------------+ +-------------+ +-----------+ + +On IOFS, it introducing some enhancements compared with original DFL desig= n. +1. It introducing Port Gasket in PF0 which is responsible for FPGA managem= ent, +like FME and Port management. The Port Gasket contains all the PR specific= modules +and logic, e.g., PR slot reset/freeze control, user clock, remote STP etc. +Architecturally, a Port Gasket can have multiple PR slots where user workl= oad can +be programmed into. +2. To expend the scalable of FPGA, it can support multiple FPs in static r= egion +which contain some static functions like VirtIO, diagnostic test, and acce= ss over +VFIO or assigned to VMs easily. Those PFs will not have a Port Unit which = without +PR region (AFU) connected to those PFs, and the end-user cannot partial re= configurate +those PFs. +3. In our previous DFL design, it can only create one VF based in an AFU. = To raise +the efficiency usage of AFU, it can create more than one VFs in an AFU via= PCIe +SRIOV, so those VFs share the PR region and resource. + +There is one reference architecture design for IOFS as illustrated below: + + +----------------------+ + | PF/VF mux/demux | + +--+--+-----+------+-+-+ + | | | | | + +------------------------+ | | | | + PF0 | +---------+ +-+ | | + +---+---+ | +---+----+ | | + | DFH | | | DFH | | | + +-------+ +-----+----+ +--------+ | | + | FME | | VirtIO | | Test | | | + +-------+ +----------+ +--------+ | | + | Port | PF1 PF2 | | + +---+---+ | | + | +----------+ | + | | ++ + | | | + | | PF0_VF0 | PF0_VF1 + | +-----------------+-----------+------------+ + | | +-----+-----------+--------+ | + | | | | | | | + | | +------+ | +--+ -+ +--+---+ | | + | | | CSR | | | DFH | | DFH | | | + +-----------+ +------+ | +-----+ +------+ | | + | | | DEV | | DEV | | | + | | +-----+ +------+ | | + | | PR Slot | | + | +--------------------------+ | + | Port Gasket | + +------------------------------------------+ + +Here are the major changes about DFL structures on IOFS implementation des= ign: +1. The Port Gasket connects to FIU Port in DFL, but the Next_AFU pointer in +FIU feature header can point to NULL so that it is no AFU connects to a FIU +Port. +2. The VF which include in PR region can start with AFU feature header wit= hout +a FIU Port feature header. =20 Performance Counters =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.17.1 From nobody Mon Jun 29 07:49:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6240BC433EF for ; Mon, 14 Feb 2022 11:39:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352562AbiBNLjz (ORCPT ); Mon, 14 Feb 2022 06:39:55 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:41562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1351978AbiBNLja (ORCPT ); Mon, 14 Feb 2022 06:39:30 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECA6FBF54; Mon, 14 Feb 2022 03:29:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644838179; x=1676374179; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=W/BuSL2ztgck7pI4bKqfzYgV6SD1PByKo0yDS6YFAvM=; b=kWaBzsfZDnPZiamSMsDiPc9pO/zLu6tflW3pHZQ4PzSkri7mPNKb8KpV 2KioEe51A5gf5NnS649M8VbeO4+Ve9m70D6VsL186J3UiYofRZlVja8XM BcBE1thBiGoQ01NHGEKg0iYZxelmWNrRY+dcLgghTBP9BlbFxNeqdW+6X +5SjUthcdzWdfqxx/4MJ7OSW0Pi/iMi9SQvnVHbQkAIWE/11Ma9X2ZjXk l7AALrT+ba6zHFQ0dVg3HnB21WpCnxU/Hx1ndVM8FvwQzJMQgD3ek8tn6 t0csY+7MBSmszDCfI3Vi3MDseaFGeUqj9aWz1aOZimvcx1l2vKU5rPyTA Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10257"; a="250276732" X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="250276732" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Feb 2022 03:29:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="587166995" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by fmsmga008.fm.intel.com with ESMTP; 14 Feb 2022 03:29:37 -0800 From: Tianfei zhang To: hao.wu@intel.com, trix@redhat.com, mdf@kernel.org, yilun.xu@intel.com, linux-fpga@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: corbet@lwn.net, Tianfei Zhang Subject: [PATCH v1 2/7] fpga: dfl: check feature type before parse irq info Date: Mon, 14 Feb 2022 06:26:14 -0500 Message-Id: <20220214112619.219761-3-tianfei.zhang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220214112619.219761-1-tianfei.zhang@intel.com> References: <20220214112619.219761-1-tianfei.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Tianfei Zhang The feature ID of "Port User Interrupt" and the "PMCI Subsystem" are identical, 0x12, but one is for FME, other is for Port. It should check the feature type While parsing the irq info in parse_feature_irqs(). Signed-off-by: Tianfei Zhang --- drivers/fpga/dfl.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index 599bb21d86af..26f8cf890700 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -940,9 +940,14 @@ static int parse_feature_irqs(struct build_feature_dev= s_info *binfo, { void __iomem *base =3D binfo->ioaddr + ofst; unsigned int i, ibase, inr =3D 0; + enum dfl_id_type type; int virq; u64 v; =20 + type =3D feature_dev_id_type(binfo->feature_dev); + if (type >=3D DFL_ID_MAX) + return -EINVAL; + /* * Ideally DFL framework should only read info from DFL header, but * current version DFL only provides mmio resources information for @@ -959,16 +964,22 @@ static int parse_feature_irqs(struct build_feature_de= vs_info *binfo, */ switch (fid) { case PORT_FEATURE_ID_UINT: + if (type !=3D PORT_ID) + break; v =3D readq(base + PORT_UINT_CAP); ibase =3D FIELD_GET(PORT_UINT_CAP_FST_VECT, v); inr =3D FIELD_GET(PORT_UINT_CAP_INT_NUM, v); break; case PORT_FEATURE_ID_ERROR: + if (type !=3D PORT_ID) + break; v =3D readq(base + PORT_ERROR_CAP); ibase =3D FIELD_GET(PORT_ERROR_CAP_INT_VECT, v); inr =3D FIELD_GET(PORT_ERROR_CAP_SUPP_INT, v); break; case FME_FEATURE_ID_GLOBAL_ERR: + if (type !=3D FME_ID) + break; v =3D readq(base + FME_ERROR_CAP); ibase =3D FIELD_GET(FME_ERROR_CAP_INT_VECT, v); inr =3D FIELD_GET(FME_ERROR_CAP_SUPP_INT, v); --=20 2.17.1 From nobody Mon Jun 29 07:49:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA666C433F5 for ; Mon, 14 Feb 2022 11:39:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352472AbiBNLjv (ORCPT ); Mon, 14 Feb 2022 06:39:51 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:42720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352013AbiBNLja (ORCPT ); Mon, 14 Feb 2022 06:39:30 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79E3CDE8B; Mon, 14 Feb 2022 03:29:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644838182; x=1676374182; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L8LQgDjx0EOOe2M9p7V6FYObzOK0Cj9rYbobse8T9IQ=; b=hwIot3nRZlzQeibazRS60y4VaAjyAAxVFr6W6tB2unGRT68e1Gsxwn3h gSvb5kk2nM96FDGq4m17wKW2fz4uJWTtglYy9emUeGx6oA1m4pr4jeGD7 NdKVSmN/DQR9LVj4QCVsRKDjam5A7PqX2lpRL+oQ5txOAwqe1ozH3rOiF cEXFl9SAHIVKFAzJML2n4lfObyuw370OZD8VUy6jdrgEOsxSjpXt7xwYJ YffOIn0ATsEfBcya35jUGfVq0l7LUtPQ+m4d5yGBBAtstPnTsxL53SOUT KS0EN1TPzKy6YjWNsOZAaDjnDXPtzgQYcZdYbeqCmR3PwLyOdGa17FR1z g==; X-IronPort-AV: E=McAfee;i="6200,9189,10257"; a="250276735" X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="250276735" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Feb 2022 03:29:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="587167012" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by fmsmga008.fm.intel.com with ESMTP; 14 Feb 2022 03:29:39 -0800 From: Tianfei zhang To: hao.wu@intel.com, trix@redhat.com, mdf@kernel.org, yilun.xu@intel.com, linux-fpga@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: corbet@lwn.net, Matthew Gerlach , Tianfei Zhang Subject: [PATCH v1 3/7] fpga: dfl: Allow for ports with no local bar space. Date: Mon, 14 Feb 2022 06:26:15 -0500 Message-Id: <20220214112619.219761-4-tianfei.zhang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220214112619.219761-1-tianfei.zhang@intel.com> References: <20220214112619.219761-1-tianfei.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Matthew Gerlach From a fpga partial reconfiguration standpoint, a port may not be connected any local BAR space. The port could be connected to a different PCIe Physical Function (PF) or Virtual Function (VF), in which case another driver instance would manage the endpoint. Signed-off-by: Matthew Gerlach Signed-off-by: Tianfei Zhang --- drivers/fpga/dfl-pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c index 4d68719e608f..8abd9b408403 100644 --- a/drivers/fpga/dfl-pci.c +++ b/drivers/fpga/dfl-pci.c @@ -243,6 +243,7 @@ static int find_dfls_by_default(struct pci_dev *pcidev, v =3D readq(base + FME_HDR_CAP); port_num =3D FIELD_GET(FME_CAP_NUM_PORTS, v); =20 + dev_info(&pcidev->dev, "port_num =3D %d\n", port_num); WARN_ON(port_num > MAX_DFL_FPGA_PORT_NUM); =20 for (i =3D 0; i < port_num; i++) { @@ -258,6 +259,13 @@ static int find_dfls_by_default(struct pci_dev *pcidev, */ bar =3D FIELD_GET(FME_PORT_OFST_BAR_ID, v); offset =3D FIELD_GET(FME_PORT_OFST_DFH_OFST, v); + if (bar >=3D PCI_STD_NUM_BARS) { + dev_info(&pcidev->dev, "skipping port without local BAR space %d\n", + bar); + continue; + } else { + dev_info(&pcidev->dev, "BAR %d offset %u\n", bar, offset); + } start =3D pci_resource_start(pcidev, bar) + offset; len =3D pci_resource_len(pcidev, bar) - offset; =20 --=20 2.17.1 From nobody Mon Jun 29 07:49:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9F48C433EF for ; Mon, 14 Feb 2022 11:43:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352575AbiBNLkB (ORCPT ); Mon, 14 Feb 2022 06:40:01 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:43628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352021AbiBNLja (ORCPT ); Mon, 14 Feb 2022 06:39:30 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4D07DF40; Mon, 14 Feb 2022 03:29:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644838185; x=1676374185; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LyVZUfq/4Kt0u89YufhzKsBwro574htmaUqau0RQcl0=; b=bV7YjWRDXHWJeQFEoqNSyv7beWasxmTvNQB33sA6mqmBi/z0raVD8Tck X+wxmHFswIP8z4CE2M8b3pGxh8xncZ3VD5IUvqsDNKhbHcAHv7Oesi9IX W+q/TVfTehdEYK1zNrtXMCMhb2UL9UZg141z5AQThVSXdZvBo32EWwNqa Shvl5CWJeE25o+LNPR1M84/qiKoVXOc9zq4OeRoYQFAQKsKFt/8jzyFgX E/+w4vXb0iFR3s6Ol3QWntomASwZnZACMQasFI2ZldQG7OqQWFcd9545H dLFHfxs9jijYX3Ciopc6wPivu3q30f7MJLWb7a/qUa5Fiuc6bsuvn9Uoh w==; X-IronPort-AV: E=McAfee;i="6200,9189,10257"; a="250276737" X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="250276737" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Feb 2022 03:29:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="587167021" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by fmsmga008.fm.intel.com with ESMTP; 14 Feb 2022 03:29:42 -0800 From: Tianfei zhang To: hao.wu@intel.com, trix@redhat.com, mdf@kernel.org, yilun.xu@intel.com, linux-fpga@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: corbet@lwn.net, Matthew Gerlach , Tianfei Zhang Subject: [PATCH v1 4/7] fpga: dfl: fix VF creation when ports have no local BAR space Date: Mon, 14 Feb 2022 06:26:16 -0500 Message-Id: <20220214112619.219761-5-tianfei.zhang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220214112619.219761-1-tianfei.zhang@intel.com> References: <20220214112619.219761-1-tianfei.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Matthew Gerlach When a port is not connected to the same PCIe endpoint as the FME, the port does not need to be released before being virtualized. Fix VF creation code to handle this new use case. Signed-off-by: Matthew Gerlach Signed-off-by: Tianfei Zhang --- drivers/fpga/dfl.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index 26f8cf890700..cfc539a656f0 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -1705,15 +1705,22 @@ EXPORT_SYMBOL_GPL(dfl_fpga_cdev_config_ports_pf); int dfl_fpga_cdev_config_ports_vf(struct dfl_fpga_cdev *cdev, int num_vfs) { struct dfl_feature_platform_data *pdata; - int ret =3D 0; + int ret =3D 0, port_count =3D 0; =20 mutex_lock(&cdev->lock); + + list_for_each_entry(pdata, &cdev->port_dev_list, node) { + if (pdata->dev) + continue; + port_count++; + } + /* * can't turn multiple ports into 1 VF device, only 1 port for 1 VF * device, so if released port number doesn't match VF device number, * then reject the request with -EINVAL error code. */ - if (cdev->released_port_num !=3D num_vfs) { + if (port_count && cdev->released_port_num !=3D num_vfs) { ret =3D -EINVAL; goto done; } --=20 2.17.1 From nobody Mon Jun 29 07:49:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A952C433F5 for ; Mon, 14 Feb 2022 11:40:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352708AbiBNLkM (ORCPT ); Mon, 14 Feb 2022 06:40:12 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:42234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352036AbiBNLja (ORCPT ); Mon, 14 Feb 2022 06:39:30 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E112E0B2; Mon, 14 Feb 2022 03:29:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644838187; x=1676374187; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Jy4XCIjAms8bGp66mP+cp45glqNgaC+j1VDAlljF7SE=; b=nFnNAXzGUvE1MBoDbq7nO7Qgf35ALCx+ygRNdHuohWD3xhxmV4IFz+g/ Pg7RI9QRZwQTRl/zx5OctrqO6nU91HvW/KaJDH2DQNeX3ASJ0iNDPVehb xAu1QM8oXw5Dz/1c5QkfGREAaDZ/9m5SgNji4jIzYlJ2SzZ45rEO66YNC Bu5kXictDl6u/TxtUs8KRio7G/BMbQJ0wVdusRt3F7fjLX8JlwcqpqjO5 fpkstiEzK/igFDN3QAI3SNYINgM3EHIVhzvQ6Flk+9gyxUD1Bz1xVJs87 NWxbfpWIirpE98xpR0ZgmUOEcClUWjKZR3heZI4EKNXeKBRt9csj8FlCe A==; X-IronPort-AV: E=McAfee;i="6200,9189,10257"; a="250276742" X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="250276742" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Feb 2022 03:29:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="587167031" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by fmsmga008.fm.intel.com with ESMTP; 14 Feb 2022 03:29:45 -0800 From: Tianfei zhang To: hao.wu@intel.com, trix@redhat.com, mdf@kernel.org, yilun.xu@intel.com, linux-fpga@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: corbet@lwn.net, Matthew Gerlach , Tianfei Zhang Subject: [PATCH v1 5/7] drivers: fpga: dfl: handle empty port list Date: Mon, 14 Feb 2022 06:26:17 -0500 Message-Id: <20220214112619.219761-6-tianfei.zhang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220214112619.219761-1-tianfei.zhang@intel.com> References: <20220214112619.219761-1-tianfei.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Matthew Gerlach Not all FPGA designs managed by the DFL driver have a port. In these cases, don't write the Port Access Control register when enabling SRIOV. Signed-off-by: Matthew Gerlach Signed-off-by: Tianfei Zhang --- drivers/fpga/dfl.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index cfc539a656f0..a5263ac258c5 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -1708,6 +1708,8 @@ int dfl_fpga_cdev_config_ports_vf(struct dfl_fpga_cde= v *cdev, int num_vfs) int ret =3D 0, port_count =3D 0; =20 mutex_lock(&cdev->lock); + if (list_empty(&cdev->port_dev_list)) + goto done; =20 list_for_each_entry(pdata, &cdev->port_dev_list, node) { if (pdata->dev) --=20 2.17.1 From nobody Mon Jun 29 07:49:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 702E1C433F5 for ; Mon, 14 Feb 2022 11:39:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352590AbiBNLkD (ORCPT ); Mon, 14 Feb 2022 06:40:03 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:43236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352040AbiBNLja (ORCPT ); Mon, 14 Feb 2022 06:39:30 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 272F5E0DA; Mon, 14 Feb 2022 03:29:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644838190; x=1676374190; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GVA2HyRU3/ajvARmfnCSRplK7C2pWCzhOUpUp/it5+c=; b=HR02gycDgQcunl+0FFC76j5vrzy03mXTJXPpz6qslCf/Y2pt2LE76ivj ks3szI31XFfAaxyKDDygmGxusOkkqLUji0MqS9fpSX+3QAszDjaTZmOSW 6R0WteyQWByHtgXK5iUf7+GVo9XYHU6PqnHwOPh1BUpCmUBKFBCkSatop bPfX/ivjVkTKwxOW/Ccaj0aCgPCSlsvMqG9C2tbCb2kflaBXLq6stCfuW IEdEBcjf6HAVNfeUjNE60kbFEKOQB41n6BLP2j3lCsssGZwpqGOLmcZEC lZ5HMP5s/oeFuwFufoCLLq/b+iES0R4KrILE97v3LK1TSX2Ap3IkRqLe5 w==; X-IronPort-AV: E=McAfee;i="6200,9189,10257"; a="250276748" X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="250276748" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Feb 2022 03:29:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="587167044" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by fmsmga008.fm.intel.com with ESMTP; 14 Feb 2022 03:29:47 -0800 From: Tianfei zhang To: hao.wu@intel.com, trix@redhat.com, mdf@kernel.org, yilun.xu@intel.com, linux-fpga@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: corbet@lwn.net, Matthew Gerlach , Tianfei Zhang Subject: [PATCH v1 6/7] fpga: dfl: Handle dfl's starting with AFU Date: Mon, 14 Feb 2022 06:26:18 -0500 Message-Id: <20220214112619.219761-7-tianfei.zhang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220214112619.219761-1-tianfei.zhang@intel.com> References: <20220214112619.219761-1-tianfei.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Matthew Gerlach Allow for a Device Feature List (DFL) to start with a Device Feature Header (DFH) of type Accelerator Function Unit (AFU) by doing nothing. This allows for PCIe VFs to be created. Signed-off-by: Matthew Gerlach Signed-off-by: Tianfei Zhang --- drivers/fpga/dfl-pci.c | 7 ++++++- drivers/fpga/dfl.c | 23 ++++++++++++++--------- 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c index 8abd9b408403..83b604d6dbe6 100644 --- a/drivers/fpga/dfl-pci.c +++ b/drivers/fpga/dfl-pci.c @@ -277,7 +277,12 @@ static int find_dfls_by_default(struct pci_dev *pcidev, =20 dfl_fpga_enum_info_add_dfl(info, start, len); } else { - ret =3D -ENODEV; + v =3D readq(base + DFH); + if (FIELD_GET(DFH_TYPE, v) !=3D DFH_TYPE_AFU) { + dev_info(&pcidev->dev, "Unknown feature type 0x%llx id 0x%llx\n", + FIELD_GET(DFH_TYPE, v), FIELD_GET(DFH_ID, v)); + ret =3D -ENODEV; + } } =20 /* release I/O mappings for next step enumeration */ diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c index a5263ac258c5..25bd24a4cca0 100644 --- a/drivers/fpga/dfl.c +++ b/drivers/fpga/dfl.c @@ -900,9 +900,11 @@ static void build_info_free(struct build_feature_devs_= info *binfo) dfl_id_free(feature_dev_id_type(binfo->feature_dev), binfo->feature_dev->id); =20 - list_for_each_entry_safe(finfo, p, &binfo->sub_features, node) { - list_del(&finfo->node); - kfree(finfo); + if (!list_empty(&binfo->sub_features)) { + list_for_each_entry_safe(finfo, p, &binfo->sub_features, node) { + list_del(&finfo->node); + kfree(finfo); + } } } =20 @@ -1437,6 +1439,7 @@ dfl_fpga_feature_devs_enumerate(struct dfl_fpga_enum_= info *info) =20 binfo->dev =3D info->dev; binfo->cdev =3D cdev; + INIT_LIST_HEAD(&binfo->sub_features); =20 binfo->nr_irqs =3D info->nr_irqs; if (info->nr_irqs) @@ -1446,12 +1449,14 @@ dfl_fpga_feature_devs_enumerate(struct dfl_fpga_enu= m_info *info) * start enumeration for all feature devices based on Device Feature * Lists. */ - list_for_each_entry(dfl, &info->dfls, node) { - ret =3D parse_feature_list(binfo, dfl->start, dfl->len); - if (ret) { - remove_feature_devs(cdev); - build_info_free(binfo); - goto unregister_region_exit; + if (!list_empty(&info->dfls)) { + list_for_each_entry(dfl, &info->dfls, node) { + ret =3D parse_feature_list(binfo, dfl->start, dfl->len); + if (ret) { + remove_feature_devs(cdev); + build_info_free(binfo); + goto unregister_region_exit; + } } } =20 --=20 2.17.1 From nobody Mon Jun 29 07:49:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CF23C433EF for ; Mon, 14 Feb 2022 11:40:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352653AbiBNLkH (ORCPT ); Mon, 14 Feb 2022 06:40:07 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:40848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352042AbiBNLja (ORCPT ); Mon, 14 Feb 2022 06:39:30 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EFF0E0DE; Mon, 14 Feb 2022 03:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644838192; x=1676374192; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=STRRUYI+lvRTyBLH3L2y01xRRbP5DjQI/AMuUvI0bDw=; b=eDZQsVWozUdsj8XzvuCDCd3XqrPARMAD49GAwTnKOo/mcDNNbbxY6Azv 33XXoIiSuJDBDgbpaEbnIVwtcQhVYnqYCwYcm0NcPtEB/gyap22iqOMtS 1g4KVl6URPskv/xYvdZ5zaHRUbAM8xHZ2+xAewd3PRaQ8/6i1MM4WDE2M /grU2jwdHRmjfiymKnC4JlWlYga/H2ASYqWoJPXUH8vat/g9i8HZewjkF q9Uab2YrMsYMYBd7oZRQjH4wRgA677G15+7oUeEbyq8jPKoAM15hG+pg0 AuL7ETVdp5d8j9Xrj8mXtRhbKDRxvm7BAXATU6NfTMIxKo2yRLITNpjvP Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10257"; a="250276749" X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="250276749" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Feb 2022 03:29:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,367,1635231600"; d="scan'208";a="587167074" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by fmsmga008.fm.intel.com with ESMTP; 14 Feb 2022 03:29:50 -0800 From: Tianfei zhang To: hao.wu@intel.com, trix@redhat.com, mdf@kernel.org, yilun.xu@intel.com, linux-fpga@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Cc: corbet@lwn.net, Matthew Gerlach , Tianfei Zhang Subject: [PATCH v1 7/7] fpga: dfl: pci: Add generic OFS PCI PID Date: Mon, 14 Feb 2022 06:26:19 -0500 Message-Id: <20220214112619.219761-8-tianfei.zhang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220214112619.219761-1-tianfei.zhang@intel.com> References: <20220214112619.219761-1-tianfei.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Matthew Gerlach Add the PCI product id for an Open FPGA Stack PCI card. Signed-off-by: Matthew Gerlach Signed-off-by: Tianfei Zhang --- drivers/fpga/dfl-pci.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c index 83b604d6dbe6..cb2fbf3eb918 100644 --- a/drivers/fpga/dfl-pci.c +++ b/drivers/fpga/dfl-pci.c @@ -76,12 +76,14 @@ static void cci_pci_free_irq(struct pci_dev *pcidev) #define PCIE_DEVICE_ID_INTEL_PAC_D5005 0x0B2B #define PCIE_DEVICE_ID_SILICOM_PAC_N5010 0x1000 #define PCIE_DEVICE_ID_SILICOM_PAC_N5011 0x1001 +#define PCIE_DEVICE_ID_INTEL_OFS 0xbcce =20 /* VF Device */ #define PCIE_DEVICE_ID_VF_INT_5_X 0xBCBF #define PCIE_DEVICE_ID_VF_INT_6_X 0xBCC1 #define PCIE_DEVICE_ID_VF_DSC_1_X 0x09C5 #define PCIE_DEVICE_ID_INTEL_PAC_D5005_VF 0x0B2C +#define PCIE_DEVICE_ID_INTEL_OFS_VF 0xbccf =20 static struct pci_device_id cci_pcie_id_tbl[] =3D { {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_PF_INT_5_X),}, @@ -95,6 +97,8 @@ static struct pci_device_id cci_pcie_id_tbl[] =3D { {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_INTEL_PAC_D5005_VF),}, {PCI_DEVICE(PCI_VENDOR_ID_SILICOM_DENMARK, PCIE_DEVICE_ID_SILICOM_PAC_N50= 10),}, {PCI_DEVICE(PCI_VENDOR_ID_SILICOM_DENMARK, PCIE_DEVICE_ID_SILICOM_PAC_N50= 11),}, + {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_INTEL_OFS),}, + {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_INTEL_OFS_VF),}, {0,} }; MODULE_DEVICE_TABLE(pci, cci_pcie_id_tbl); --=20 2.17.1