From nobody Sat May 18 22:14:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1554981049; cv=none; d=zoho.com; s=zohoarc; b=Iz0mcxq/l4Ylx+05mtaRoi+Y+Yvcmzz7FRuwyTEy73esBWo/syNYMxTN0AvLJAvGWAjc+9FWKEvwNjfc5focMN6AEpkkAEHe/IoF2iGCJlmxL15N4evec3Ysd47WW/oR35tUKcErPpYh4fVUPQX8ZDsIxJnjHe9g+l87FD3Ti1s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1554981049; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=1lQQU8onCKOeE0LqztxuYSPLnayuzTvoGHT3m+GuAYM=; b=gftRcOQ/BKAue9p7BuXA7A5iFH1QnauGHa3lU6KRaStkvU1JnB9xA6IzFewitbTHG0H3EsEhjvOdsHW4eQQtex0G5wnYYUDMu0qXVVenzU0eVhQIdjx2fJt0Vd99w4h0m07qqWUE209T5RHNPoYyK2P9NIZu/Xxyq+WdhxnlP+k= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155498104919813.436611216613073; Thu, 11 Apr 2019 04:10:49 -0700 (PDT) Received: from localhost ([127.0.0.1]:46584 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEXbP-0000hk-6F for importer@patchew.org; Thu, 11 Apr 2019 07:10:43 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38214) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEXTT-0003F6-OT for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEXTP-0000tB-89 for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:30 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:58966) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEXTN-0007Hb-Jj for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:27 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x3BArwCC090728; Thu, 11 Apr 2019 11:02:08 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 2rpmrqg6s8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Apr 2019 11:02:07 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x3BB1k5i060041; Thu, 11 Apr 2019 11:02:07 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3020.oracle.com with ESMTP id 2rpytcq8tp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Apr 2019 11:02:06 +0000 Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x3BB25iP014715; Thu, 11 Apr 2019 11:02:05 GMT Received: from host4.lan (/77.138.183.59) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 11 Apr 2019 04:02:04 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=1lQQU8onCKOeE0LqztxuYSPLnayuzTvoGHT3m+GuAYM=; b=AQ1IwIwnVKXev37V2LLMBBSyRqh+i9auAZi0vm3Gl/3DfeD5jQgH/zvLponypZkrdEJm JwRxf3byDJbvJALK4cJjSBXyfr5vc+/kljOyGSCiZqAAa+S9obL9EFr7nJfhav7mSvQk ho+YuYxqBcIr7cUbNdD1bJcnaiR7eaN5jwFEK3P34sUIzq2B4nH0ejiP6EgqKSOjMmhA XencES8PwIjzrpBFeC/LX4yW9vuxlJT9Dupq7xKYNHPWUImguYuinG2QRW3R5ELO8IQC 0S4Rzpsw78pJSyoVcZ5Rc0ABC5uPYycrjdiAd18i7ar17fwEeRFoW8dYN+k3Wp9r4Qxc 4g== From: Yuval Shaia To: virtualization@lists.linux-foundation.org, qemu-devel@nongnu.org, mst@redhat.com, cohuck@redhat.com, marcel.apfelbaum@gmail.com, linux-rdma@vger.kernel.org, jgg@mellanox.com Date: Thu, 11 Apr 2019 14:01:55 +0300 Message-Id: <20190411110157.14252-2-yuval.shaia@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190411110157.14252-1-yuval.shaia@oracle.com> References: <20190411110157.14252-1-yuval.shaia@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9223 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904110077 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9223 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904110077 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 156.151.31.85 Subject: [Qemu-devel] [RFC 1/3] virtio-net: Move some virtio-net-pci decl to include/hw/virtio X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yuval Shaia Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Signed-off-by: Yuval Shaia --- hw/virtio/virtio-net-pci.c | 18 ++------------- include/hw/virtio/virtio-net-pci.h | 35 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 16 deletions(-) create mode 100644 include/hw/virtio/virtio-net-pci.h diff --git a/hw/virtio/virtio-net-pci.c b/hw/virtio/virtio-net-pci.c index db07ab9e21..63617d5550 100644 --- a/hw/virtio/virtio-net-pci.c +++ b/hw/virtio/virtio-net-pci.c @@ -17,24 +17,10 @@ =20 #include "qemu/osdep.h" =20 -#include "hw/virtio/virtio-net.h" +#include "hw/virtio/virtio-net-pci.h" #include "virtio-pci.h" #include "qapi/error.h" =20 -typedef struct VirtIONetPCI VirtIONetPCI; - -/* - * virtio-net-pci: This extends VirtioPCIProxy. - */ -#define TYPE_VIRTIO_NET_PCI "virtio-net-pci-base" -#define VIRTIO_NET_PCI(obj) \ - OBJECT_CHECK(VirtIONetPCI, (obj), TYPE_VIRTIO_NET_PCI) - -struct VirtIONetPCI { - VirtIOPCIProxy parent_obj; - VirtIONet vdev; -}; - static Property virtio_net_properties[] =3D { DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags, VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true), @@ -82,7 +68,7 @@ static void virtio_net_pci_instance_init(Object *obj) =20 static const VirtioPCIDeviceTypeInfo virtio_net_pci_info =3D { .base_name =3D TYPE_VIRTIO_NET_PCI, - .generic_name =3D "virtio-net-pci", + .generic_name =3D TYPE_VIRTIO_NET_PCI_GENERIC, .transitional_name =3D "virtio-net-pci-transitional", .non_transitional_name =3D "virtio-net-pci-non-transitional", .instance_size =3D sizeof(VirtIONetPCI), diff --git a/include/hw/virtio/virtio-net-pci.h b/include/hw/virtio/virtio-= net-pci.h new file mode 100644 index 0000000000..f14e6ed992 --- /dev/null +++ b/include/hw/virtio/virtio-net-pci.h @@ -0,0 +1,35 @@ +/* + * PCI Virtio Network Device + * + * Copyright IBM, Corp. 2007 + * + * Authors: + * Anthony Liguori + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +#ifndef QEMU_VIRTIO_NET_PCI_H +#define QEMU_VIRTIO_NET_PCI_H + +#include "hw/virtio/virtio-net.h" +#include "virtio-pci.h" + +typedef struct VirtIONetPCI VirtIONetPCI; + +/* + * virtio-net-pci: This extends VirtioPCIProxy. + */ +#define TYPE_VIRTIO_NET_PCI_GENERIC "virtio-net-pci" +#define TYPE_VIRTIO_NET_PCI "virtio-net-pci-base" +#define VIRTIO_NET_PCI(obj) \ + OBJECT_CHECK(VirtIONetPCI, (obj), TYPE_VIRTIO_NET_PCI) + +struct VirtIONetPCI { + VirtIOPCIProxy parent_obj; + VirtIONet vdev; +}; + +#endif --=20 2.20.1 From nobody Sat May 18 22:14:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1554980680; cv=none; d=zoho.com; s=zohoarc; b=Wd5c/A2iZMEL2B8tkbww70tJcVfCq3Lc7Kg07eeYSJ0Qaw8OaW/+9R1EmERyDztQZ4PhWxhfeIjSa+rF0A72kfBSX98+EYc2YDcjwtI1UiLQmcdXIHamzxWSMsV7aLWwuGqjHXxd0GCy59SsCoJtodSW2R1ZheVfBE3q6ov2Ogo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1554980680; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=S2u7RqPhmQ3j/mb66ctCfnUp9jyf44k+/Hb1+bpBH3E=; b=Aycz+fMSIKVLLiarHWomp1mjJeLrj2iI0aepwHUBQgB+eH2fiDBgXMh4JWlHiyUgLjtmhX1mGSNMLvdTTph04ZXRoGsxgefH5ClQ8c1SZXJ9XhLdBcp+RrIU6+zrHddomeVhZY+/AMzGFHmdtRE2HhT7dHwdoMSitKdxkilfbQM= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1554980680723958.8952614308567; Thu, 11 Apr 2019 04:04:40 -0700 (PDT) Received: from localhost ([127.0.0.1]:46453 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEXVS-0004DF-8u for importer@patchew.org; Thu, 11 Apr 2019 07:04:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38191) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEXTR-0003Cj-Vj for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEXTL-0000Km-H0 for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:27 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:42600) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEXTL-0007id-5A for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:23 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x3BAri94084216; Thu, 11 Apr 2019 11:02:10 GMT Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 2rpkht8b0s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Apr 2019 11:02:09 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x3BB28wA001689; Thu, 11 Apr 2019 11:02:08 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 2rpkekcngc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Apr 2019 11:02:08 +0000 Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x3BB27QK014732; Thu, 11 Apr 2019 11:02:07 GMT Received: from host4.lan (/77.138.183.59) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 11 Apr 2019 04:02:07 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=S2u7RqPhmQ3j/mb66ctCfnUp9jyf44k+/Hb1+bpBH3E=; b=O4yvzB9dzjPiofgy9U5ZNMMXpdsDJ964Zx/p/cYY8EjF+p5O2yTkjyqbhB711jrz+3th hW16hDLQZEYsPT8FLOkXZ4wTMhJIJqFK0a5g4yHp49JWUGCCBmHPqdPXCBz+WzVnoGCv f4mcD3Btt25qtrbHRL3vUCUUX41GaapEkI3zM2pV5ZkuLw15+5Lnufdc2JmsGwh1BxnI mb+ysq54VfxLmJaoXe/S0s5uFLsjbrg2ha7PIsG8G2WrbEgXDXEYZr7U2bL97NTdEHRL Wp15hnwaG8KN5ys32lsQw/kOj4Zv95Hun5YAiocCZwUcVWQUU6la+lG946ftqNtJ2AVi 2w== From: Yuval Shaia To: virtualization@lists.linux-foundation.org, qemu-devel@nongnu.org, mst@redhat.com, cohuck@redhat.com, marcel.apfelbaum@gmail.com, linux-rdma@vger.kernel.org, jgg@mellanox.com Date: Thu, 11 Apr 2019 14:01:56 +0300 Message-Id: <20190411110157.14252-3-yuval.shaia@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190411110157.14252-1-yuval.shaia@oracle.com> References: <20190411110157.14252-1-yuval.shaia@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9223 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904110077 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9223 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904110077 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 156.151.31.86 Subject: [Qemu-devel] [RFC 2/3] hw/virtio-rdma: VirtIO rdma device X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yuval Shaia Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Signed-off-by: Yuval Shaia --- hw/Kconfig | 1 + hw/rdma/Kconfig | 4 + hw/rdma/Makefile.objs | 2 + hw/rdma/virtio/virtio-rdma-ib.c | 287 ++++++++++++++++++++ hw/rdma/virtio/virtio-rdma-ib.h | 93 +++++++ hw/rdma/virtio/virtio-rdma-main.c | 185 +++++++++++++ hw/virtio/Makefile.objs | 1 + hw/virtio/virtio-rdma-pci.c | 108 ++++++++ include/hw/pci/pci.h | 1 + include/hw/virtio/virtio-rdma.h | 44 +++ include/standard-headers/linux/virtio_ids.h | 1 + 11 files changed, 727 insertions(+) create mode 100644 hw/rdma/Kconfig create mode 100644 hw/rdma/virtio/virtio-rdma-ib.c create mode 100644 hw/rdma/virtio/virtio-rdma-ib.h create mode 100644 hw/rdma/virtio/virtio-rdma-main.c create mode 100644 hw/virtio/virtio-rdma-pci.c create mode 100644 include/hw/virtio/virtio-rdma.h diff --git a/hw/Kconfig b/hw/Kconfig index d5ecd02070..88b9f15007 100644 --- a/hw/Kconfig +++ b/hw/Kconfig @@ -26,6 +26,7 @@ source pci-bridge/Kconfig source pci-host/Kconfig source pcmcia/Kconfig source pci/Kconfig +source rdma/Kconfig source scsi/Kconfig source sd/Kconfig source smbios/Kconfig diff --git a/hw/rdma/Kconfig b/hw/rdma/Kconfig new file mode 100644 index 0000000000..b10bd7182b --- /dev/null +++ b/hw/rdma/Kconfig @@ -0,0 +1,4 @@ +config VIRTIO_RDMA + bool + default y + depends on VIRTIO diff --git a/hw/rdma/Makefile.objs b/hw/rdma/Makefile.objs index c354e60e5b..ed640882be 100644 --- a/hw/rdma/Makefile.objs +++ b/hw/rdma/Makefile.objs @@ -3,3 +3,5 @@ obj-$(CONFIG_PCI) +=3D rdma_utils.o rdma_backend.o rdma_rm.= o rdma.o obj-$(CONFIG_PCI) +=3D vmw/pvrdma_dev_ring.o vmw/pvrdma_cmd.o \ vmw/pvrdma_qp_ops.o vmw/pvrdma_main.o endif +obj-$(CONFIG_VIRTIO_RDMA) +=3D virtio/virtio-rdma-main.o \ + virtio/virtio-rdma-ib.o diff --git a/hw/rdma/virtio/virtio-rdma-ib.c b/hw/rdma/virtio/virtio-rdma-i= b.c new file mode 100644 index 0000000000..2590a831a2 --- /dev/null +++ b/hw/rdma/virtio/virtio-rdma-ib.c @@ -0,0 +1,287 @@ +/* + * Virtio RDMA Device - IB verbs + * + * Copyright (C) 2019 Oracle + * + * Authors: + * Yuval Shaia + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +#include + +#include "qemu/osdep.h" + +#include "virtio-rdma-ib.h" +#include "../rdma_utils.h" +#include "../rdma_rm.h" +#include "../rdma_backend.h" + +int virtio_rdma_query_device(VirtIORdma *rdev, struct iovec *in, + struct iovec *out) +{ + struct ibv_device_attr attr =3D {}; + int offs; + size_t s; + + addrconf_addr_eui48((unsigned char *)&attr.sys_image_guid, + (const char *)&rdev->netdev->mac); + + attr.max_mr_size =3D 4096; + attr.page_size_cap =3D 4096; + attr.vendor_id =3D 1; + attr.vendor_part_id =3D 1; + attr.hw_ver =3D VIRTIO_RDMA_HW_VER; + attr.max_qp =3D 1024; + attr.max_qp_wr =3D 1024; + attr.device_cap_flags =3D 0; + attr.max_sge =3D 64; + attr.max_sge_rd =3D 64; + attr.max_cq =3D 1024; + attr.max_cqe =3D 64; + attr.max_mr =3D 1024; + attr.max_pd =3D 1024; + attr.max_qp_rd_atom =3D 0; + attr.max_ee_rd_atom =3D 0; + attr.max_res_rd_atom =3D 0; + attr.max_qp_init_rd_atom =3D 0; + attr.max_ee_init_rd_atom =3D 0; + attr.atomic_cap =3D IBV_ATOMIC_NONE; + attr.max_ee =3D 0; + attr.max_rdd =3D 0; + attr.max_mw =3D 0; + attr.max_raw_ipv6_qp =3D 0; + attr.max_raw_ethy_qp =3D 0; + attr.max_mcast_grp =3D 0; + attr.max_mcast_qp_attach =3D 0; + attr.max_total_mcast_qp_attach =3D 0; + attr.max_ah =3D 1024; + attr.max_fmr =3D 0; + attr.max_map_per_fmr =3D 0; + attr.max_srq =3D 0; + attr.max_srq_wr =3D 0; + attr.max_srq_sge =3D 0; + attr.max_pkeys =3D 1; + attr.local_ca_ack_delay =3D 0; + attr.phys_port_cnt =3D VIRTIO_RDMA_PORT_CNT; + + offs =3D offsetof(struct ibv_device_attr, sys_image_guid); + s =3D iov_from_buf(out, 1, 0, (void *)&attr + offs, sizeof(attr) - off= s); + + return s =3D=3D sizeof(attr) - offs ? VIRTIO_RDMA_CTRL_OK : + VIRTIO_RDMA_CTRL_ERR; +} + +int virtio_rdma_query_port(VirtIORdma *rdev, struct iovec *in, + struct iovec *out) +{ + struct ibv_port_attr attr =3D {}; + struct cmd_query_port cmd =3D {}; + int offs; + size_t s; + + s =3D iov_to_buf(in, 1, 0, &cmd, sizeof(cmd)); + if (s !=3D sizeof(cmd)) { + return VIRTIO_RDMA_CTRL_ERR; + } + + if (cmd.port !=3D 1) { + return VIRTIO_RDMA_CTRL_ERR; + } + + attr.state =3D IBV_PORT_ACTIVE; + attr.max_mtu =3D attr.active_mtu =3D IBV_MTU_1024; + attr.gid_tbl_len =3D 256; + attr.port_cap_flags =3D 0; + attr.max_msg_sz =3D 1024; + attr.bad_pkey_cntr =3D 0; + attr.qkey_viol_cntr =3D 0; + attr.pkey_tbl_len =3D 1; + attr.lid =3D 0; + attr.sm_lid =3D 0; + attr.lmc =3D 0; + attr.max_vl_num =3D 1; + attr.sm_sl =3D 0; + attr.subnet_timeout =3D 0; + attr.init_type_reply =3D 0; + attr.active_width =3D 0; + attr.active_speed =3D 0; + attr.phys_state =3D 0; + + offs =3D offsetof(struct ibv_port_attr, state); + s =3D iov_from_buf(out, 1, 0, (void *)&attr + offs, sizeof(attr) - off= s); + + return s =3D=3D sizeof(attr) - offs ? VIRTIO_RDMA_CTRL_OK : + VIRTIO_RDMA_CTRL_ERR; +} + +int virtio_rdma_create_cq(VirtIORdma *rdev, struct iovec *in, + struct iovec *out) +{ + struct cmd_create_cq cmd =3D {}; + struct rsp_create_cq rsp =3D {}; + size_t s; + int rc; + + s =3D iov_to_buf(in, 1, 0, &cmd, sizeof(cmd)); + if (s !=3D sizeof(cmd)) { + return VIRTIO_RDMA_CTRL_ERR; + } + + /* TODO: Define MAX_CQE */ +#define MAX_CQE 1024 + /* TODO: Check MAX_CQ */ + if (cmd.cqe > MAX_CQE) { + return VIRTIO_RDMA_CTRL_ERR; + } + + printf("%s: %d\n", __func__, cmd.cqe); + + /* TODO: Create VirtQ */ + + rc =3D rdma_rm_alloc_cq(rdev->rdma_dev_res, rdev->backend_dev, cmd.cqe, + &rsp.cqn, NULL); + if (rc) { + /* TODO: Destroy VirtQ */ + return VIRTIO_RDMA_CTRL_ERR; + } + + printf("%s: %d\n", __func__, rsp.cqn); + + s =3D iov_from_buf(out, 1, 0, &rsp, sizeof(rsp)); + + return s =3D=3D sizeof(rsp) ? VIRTIO_RDMA_CTRL_OK : + VIRTIO_RDMA_CTRL_ERR; +} + +int virtio_rdma_destroy_cq(VirtIORdma *rdev, struct iovec *in, + struct iovec *out) +{ + struct cmd_destroy_cq cmd =3D {}; + size_t s; + + s =3D iov_to_buf(in, 1, 0, &cmd, sizeof(cmd)); + if (s !=3D sizeof(cmd)) { + return VIRTIO_RDMA_CTRL_ERR; + } + + printf("%s: %d\n", __func__, cmd.cqn); + + /* TODO: Destroy VirtQ */ + + rdma_rm_dealloc_cq(rdev->rdma_dev_res, cmd.cqn); + + return VIRTIO_RDMA_CTRL_OK; +} + +int virtio_rdma_create_pd(VirtIORdma *rdev, struct iovec *in, + struct iovec *out) +{ + struct rsp_create_pd rsp =3D {}; + size_t s; + int rc; + + /* TODO: Check MAX_PD */ + + /* TODO: ctx */ + rc =3D rdma_rm_alloc_pd(rdev->rdma_dev_res, rdev->backend_dev, &rsp.pd= n, + 0); + if (rc) + return VIRTIO_RDMA_CTRL_ERR; + + printf("%s: %d\n", __func__, rsp.pdn); + + s =3D iov_from_buf(out, 1, 0, &rsp, sizeof(rsp)); + + return s =3D=3D sizeof(rsp) ? VIRTIO_RDMA_CTRL_OK : + VIRTIO_RDMA_CTRL_ERR; +} + +int virtio_rdma_destroy_pd(VirtIORdma *rdev, struct iovec *in, + struct iovec *out) +{ + struct cmd_destroy_pd cmd =3D {}; + size_t s; + + s =3D iov_to_buf(in, 1, 0, &cmd, sizeof(cmd)); + if (s !=3D sizeof(cmd)) { + return VIRTIO_RDMA_CTRL_ERR; + } + + printf("%s: %d\n", __func__, cmd.pdn); + + rdma_rm_dealloc_cq(rdev->rdma_dev_res, cmd.pdn); + + return VIRTIO_RDMA_CTRL_OK; +} + +int virtio_rdma_get_dma_mr(VirtIORdma *rdev, struct iovec *in, + struct iovec *out) +{ + struct cmd_get_dma_mr cmd =3D {}; + struct rsp_get_dma_mr rsp =3D {}; + size_t s; + + s =3D iov_to_buf(in, 1, 0, &cmd, sizeof(cmd)); + if (s !=3D sizeof(cmd)) { + return VIRTIO_RDMA_CTRL_ERR; + } + + /* TODO: Call rdma_rm_alloc_mr */ + rsp.mrn =3D 0x10; + rsp.lkey =3D 0x11; + rsp.rkey =3D 0x12; + printf("%s: 0x%x\n", __func__, rsp.mrn); + + s =3D iov_from_buf(out, 1, 0, &rsp, sizeof(rsp)); + + return s =3D=3D sizeof(rsp) ? VIRTIO_RDMA_CTRL_OK : + VIRTIO_RDMA_CTRL_ERR; +} + +static void virtio_rdma_init_dev_caps(VirtIORdma *rdev) +{ + rdev->dev_attr.max_qp_wr =3D 1024; +} + +int virtio_rdma_init_ib(VirtIORdma *rdev) +{ + int rc; + + virtio_rdma_init_dev_caps(rdev); + + rdev->rdma_dev_res =3D g_malloc0(sizeof(RdmaDeviceResources)); + rdev->backend_dev =3D g_malloc0(sizeof(RdmaBackendDev)); + + rc =3D rdma_backend_init(rdev->backend_dev, NULL, rdev->rdma_dev_res, + rdev->backend_device_name, + rdev->backend_port_num, &rdev->dev_attr, + &rdev->mad_chr); + if (rc) { + rdma_error_report("Fail to initialize backend device"); + return rc; + } + + rc =3D rdma_rm_init(rdev->rdma_dev_res, &rdev->dev_attr); + if (rc) { + rdma_error_report("Fail to initialize resource manager"); + return rc; + } + + /* rdma_backend_start(rdev->backend_dev); */ + + return 0; +} + +void virtio_rdma_fini_ib(VirtIORdma *rdev) +{ + /* rdma_backend_stop(rdev->backend_dev); */ + rdma_rm_fini(rdev->rdma_dev_res, rdev->backend_dev, + rdev->backend_eth_device_name); + rdma_backend_fini(rdev->backend_dev); + g_free(rdev->rdma_dev_res); + g_free(rdev->backend_dev); +} diff --git a/hw/rdma/virtio/virtio-rdma-ib.h b/hw/rdma/virtio/virtio-rdma-i= b.h new file mode 100644 index 0000000000..c4bdc063ac --- /dev/null +++ b/hw/rdma/virtio/virtio-rdma-ib.h @@ -0,0 +1,93 @@ +/* + * Virtio RDMA Device - IB verbs + * + * Copyright (C) 2019 Oracle + * + * Authors: + * Yuval Shaia + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu/iov.h" +#include "hw/virtio/virtio-rdma.h" + +/* TODO: Move to uapi header file */ +#define VIRTIO_RDMA_CTRL_OK 0 +#define VIRTIO_RDMA_CTRL_ERR 1 + +enum { + VIRTIO_CMD_QUERY_DEVICE =3D 10, + VIRTIO_CMD_QUERY_PORT, + VIRTIO_CMD_CREATE_CQ, + VIRTIO_CMD_DESTROY_CQ, + VIRTIO_CMD_CREATE_PD, + VIRTIO_CMD_DESTROY_PD, + VIRTIO_CMD_GET_DMA_MR, +}; + +struct control_buf { + uint8_t cmd; + uint8_t status; +}; + +struct cmd_query_port { + uint8_t port; +}; + +struct cmd_create_cq { + uint32_t cqe; +}; + +struct rsp_create_cq { + uint32_t cqn; +}; + +struct cmd_destroy_cq { + uint32_t cqn; +}; + +struct rsp_create_pd { + uint32_t pdn; +}; + +struct cmd_destroy_pd { + uint32_t pdn; +}; + +struct cmd_get_dma_mr { + uint32_t pdn; + uint32_t access_flags; +}; + +struct rsp_get_dma_mr { + uint32_t mrn; + uint32_t lkey; + uint32_t rkey; +}; + +/* TODO: Move to uapi header file */ + +#define VIRTIO_RDMA_PORT_CNT 1 +#define VIRTIO_RDMA_HW_VER 1 + +int virtio_rdma_init_ib(VirtIORdma *rdev); +void virtio_rdma_fini_ib(VirtIORdma *rdev); + +int virtio_rdma_query_device(VirtIORdma *rdev, struct iovec *in, + struct iovec *out); +int virtio_rdma_query_port(VirtIORdma *rdev, struct iovec *in, + struct iovec *out); +int virtio_rdma_create_cq(VirtIORdma *rdev, struct iovec *in, + struct iovec *out); +int virtio_rdma_destroy_cq(VirtIORdma *rdev, struct iovec *in, + struct iovec *out); +int virtio_rdma_create_pd(VirtIORdma *rdev, struct iovec *in, + struct iovec *out); +int virtio_rdma_destroy_pd(VirtIORdma *rdev, struct iovec *in, + struct iovec *out); +int virtio_rdma_get_dma_mr(VirtIORdma *rdev, struct iovec *in, + struct iovec *out); diff --git a/hw/rdma/virtio/virtio-rdma-main.c b/hw/rdma/virtio/virtio-rdma= -main.c new file mode 100644 index 0000000000..54f75b14c0 --- /dev/null +++ b/hw/rdma/virtio/virtio-rdma-main.c @@ -0,0 +1,185 @@ +/* + * Virtio RDMA Device + * + * Copyright (C) 2019 Oracle + * + * Authors: + * Yuval Shaia + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +#include + +#include "qemu/osdep.h" +#include "hw/virtio/virtio.h" +#include "qemu/error-report.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/virtio/virtio-rdma.h" +#include "include/standard-headers/linux/virtio_ids.h" + +#include "virtio-rdma-ib.h" +#include "../rdma_rm_defs.h" +#include "../rdma_utils.h" + +static void virtio_rdma_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq) +{ + VirtIORdma *r =3D VIRTIO_RDMA(vdev); + struct control_buf cb; + VirtQueueElement *e; + size_t s; + + virtio_queue_set_notification(vq, 0); + + for (;;) { + e =3D virtqueue_pop(vq, sizeof(VirtQueueElement)); + if (!e) { + break; + } + + if (iov_size(e->in_sg, e->in_num) < sizeof(cb.status) || + iov_size(e->out_sg, e->out_num) < sizeof(cb.cmd)) { + virtio_error(vdev, "Got invalid message size"); + virtqueue_detach_element(vq, e, 0); + g_free(e); + break; + } + + s =3D iov_to_buf(&e->out_sg[0], 1, 0, &cb.cmd, sizeof(cb.cmd)); + if (s !=3D sizeof(cb.cmd)) { + cb.status =3D VIRTIO_RDMA_CTRL_ERR; + } else { + printf("cmd=3D%d\n", cb.cmd); + switch (cb.cmd) { + case VIRTIO_CMD_QUERY_DEVICE: + cb.status =3D virtio_rdma_query_device(r, &e->out_sg[1], + &e->in_sg[0]); + break; + case VIRTIO_CMD_QUERY_PORT: + cb.status =3D virtio_rdma_query_port(r, &e->out_sg[1], + &e->in_sg[0]); + break; + case VIRTIO_CMD_CREATE_CQ: + cb.status =3D virtio_rdma_create_cq(r, &e->out_sg[1], + &e->in_sg[0]); + break; + case VIRTIO_CMD_DESTROY_CQ: + cb.status =3D virtio_rdma_destroy_cq(r, &e->out_sg[1], + &e->in_sg[0]); + break; + case VIRTIO_CMD_CREATE_PD: + cb.status =3D virtio_rdma_create_pd(r, &e->out_sg[1], + &e->in_sg[0]); + break; + case VIRTIO_CMD_DESTROY_PD: + cb.status =3D virtio_rdma_destroy_pd(r, &e->out_sg[1], + &e->in_sg[0]); + break; + case VIRTIO_CMD_GET_DMA_MR: + cb.status =3D virtio_rdma_get_dma_mr(r, &e->out_sg[1], + &e->in_sg[0]); + break; + default: + cb.status =3D VIRTIO_RDMA_CTRL_ERR; + } + } + printf("status=3D%d\n", cb.status); + s =3D iov_from_buf(&e->in_sg[1], 1, 0, &cb.status, sizeof(cb.statu= s)); + assert(s =3D=3D sizeof(cb.status)); + + virtqueue_push(vq, e, sizeof(cb.status)); + virtio_notify(vdev, vq); + } + + virtio_queue_set_notification(vq, 1); +} + +static void virtio_rdma_device_realize(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev =3D VIRTIO_DEVICE(dev); + VirtIORdma *r =3D VIRTIO_RDMA(dev); + int rc; + + rc =3D virtio_rdma_init_ib(r); + if (rc) { + rdma_error_report("Fail to initialize IB layer"); + return; + } + + virtio_init(vdev, "virtio-rdma", VIRTIO_ID_RDMA, 1024); + + r->ctrl_vq =3D virtio_add_queue(vdev, 64, virtio_rdma_handle_ctrl); +} + +static void virtio_rdma_device_unrealize(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev =3D VIRTIO_DEVICE(dev); + VirtIORdma *r =3D VIRTIO_RDMA(dev); + + virtio_del_queue(vdev, 0); + + virtio_cleanup(vdev); + + virtio_rdma_fini_ib(r); +} + +static uint64_t virtio_rdma_get_features(VirtIODevice *vdev, uint64_t feat= ures, + Error **errp) +{ + /* virtio_add_feature(&features, VIRTIO_NET_F_MAC); */ + + vdev->backend_features =3D features; + + return features; +} + + +static Property virtio_rdma_dev_properties[] =3D { + DEFINE_PROP_STRING("netdev", VirtIORdma, backend_eth_device_name), + DEFINE_PROP_STRING("ibdev",VirtIORdma, backend_device_name), + DEFINE_PROP_UINT8("ibport", VirtIORdma, backend_port_num, 1), + DEFINE_PROP_UINT64("dev-caps-max-mr-size", VirtIORdma, dev_attr.max_mr= _size, + MAX_MR_SIZE), + DEFINE_PROP_INT32("dev-caps-max-qp", VirtIORdma, dev_attr.max_qp, MAX_= QP), + DEFINE_PROP_INT32("dev-caps-max-cq", VirtIORdma, dev_attr.max_cq, MAX_= CQ), + DEFINE_PROP_INT32("dev-caps-max-mr", VirtIORdma, dev_attr.max_mr, MAX_= MR), + DEFINE_PROP_INT32("dev-caps-max-pd", VirtIORdma, dev_attr.max_pd, MAX_= PD), + DEFINE_PROP_INT32("dev-caps-qp-rd-atom", VirtIORdma, + dev_attr.max_qp_rd_atom, MAX_QP_RD_ATOM), + DEFINE_PROP_INT32("dev-caps-max-qp-init-rd-atom", VirtIORdma, + dev_attr.max_qp_init_rd_atom, MAX_QP_INIT_RD_ATOM), + DEFINE_PROP_INT32("dev-caps-max-ah", VirtIORdma, dev_attr.max_ah, MAX_= AH), + DEFINE_PROP_CHR("mad-chardev", VirtIORdma, mad_chr), + DEFINE_PROP_END_OF_LIST(), +}; + +static void virtio_rdma_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + VirtioDeviceClass *vdc =3D VIRTIO_DEVICE_CLASS(klass); + + set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); + vdc->realize =3D virtio_rdma_device_realize; + vdc->unrealize =3D virtio_rdma_device_unrealize; + vdc->get_features =3D virtio_rdma_get_features; + + dc->desc =3D "Virtio RDMA Device"; + dc->props =3D virtio_rdma_dev_properties; + set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); +} + +static const TypeInfo virtio_rdma_info =3D { + .name =3D TYPE_VIRTIO_RDMA, + .parent =3D TYPE_VIRTIO_DEVICE, + .instance_size =3D sizeof(VirtIORdma), + .class_init =3D virtio_rdma_class_init, +}; + +static void virtio_register_types(void) +{ + type_register_static(&virtio_rdma_info); +} + +type_init(virtio_register_types) diff --git a/hw/virtio/Makefile.objs b/hw/virtio/Makefile.objs index f2ab667a21..fd701feb9f 100644 --- a/hw/virtio/Makefile.objs +++ b/hw/virtio/Makefile.objs @@ -27,6 +27,7 @@ obj-$(CONFIG_VIRTIO_9P) +=3D virtio-9p-pci.o obj-$(CONFIG_VIRTIO_SCSI) +=3D virtio-scsi-pci.o obj-$(CONFIG_VIRTIO_BLK) +=3D virtio-blk-pci.o obj-$(CONFIG_VIRTIO_NET) +=3D virtio-net-pci.o +obj-$(CONFIG_VIRTIO_RDMA) +=3D virtio-rdma-pci.o obj-$(CONFIG_VIRTIO_SERIAL) +=3D virtio-serial-pci.o endif else diff --git a/hw/virtio/virtio-rdma-pci.c b/hw/virtio/virtio-rdma-pci.c new file mode 100644 index 0000000000..36efce285b --- /dev/null +++ b/hw/virtio/virtio-rdma-pci.c @@ -0,0 +1,108 @@ +/* + * Virtio rdma PCI Bindings + * + * Copyright (C) 2019 Oracle + * + * Authors: + * Yuval Shaia + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" + +#include "hw/virtio/virtio-net-pci.h" +#include "hw/virtio/virtio-rdma.h" +#include "virtio-pci.h" +#include "qapi/error.h" + +typedef struct VirtIORdmaPCI VirtIORdmaPCI; + +/* + * virtio-rdma-pci: This extends VirtioPCIProxy. + */ +#define TYPE_VIRTIO_RDMA_PCI "virtio-rdma-pci-base" +#define VIRTIO_RDMA_PCI(obj) \ + OBJECT_CHECK(VirtIORdmaPCI, (obj), TYPE_VIRTIO_RDMA_PCI) + +struct VirtIORdmaPCI { + VirtIOPCIProxy parent_obj; + VirtIORdma vdev; +}; + +static Property virtio_rdma_properties[] =3D { + DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags, + VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true), + DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, 3), + DEFINE_PROP_END_OF_LIST(), +}; + +static void virtio_rdma_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) +{ + VirtIORdmaPCI *dev =3D VIRTIO_RDMA_PCI(vpci_dev); + DeviceState *vdev =3D DEVICE(&dev->vdev); + VirtIONetPCI *vnet_pci; + PCIDevice *func0; + + qdev_set_parent_bus(vdev, BUS(&vpci_dev->bus)); + object_property_set_bool(OBJECT(vdev), true, "realized", errp); + + func0 =3D pci_get_function_0(&vpci_dev->pci_dev); + /* Break if not virtio device in slot 0 */ + if (strcmp(object_get_typename(OBJECT(func0)), + TYPE_VIRTIO_NET_PCI_GENERIC)) { + error_setg(errp, "Device on %x.0 is type %s but must be %s", + PCI_SLOT(vpci_dev->pci_dev.devfn), + object_get_typename(OBJECT(func0)), + TYPE_VIRTIO_NET_PCI_GENERIC); + return; + } + vnet_pci =3D VIRTIO_NET_PCI(func0); + dev->vdev.netdev =3D &vnet_pci->vdev; +} + +static void virtio_rdma_pci_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + PCIDeviceClass *k =3D PCI_DEVICE_CLASS(klass); + VirtioPCIClass *vpciklass =3D VIRTIO_PCI_CLASS(klass); + + k->vendor_id =3D PCI_VENDOR_ID_REDHAT_QUMRANET; + k->device_id =3D PCI_DEVICE_ID_VIRTIO_RDMA; + k->revision =3D VIRTIO_PCI_ABI_VERSION; + k->class_id =3D PCI_CLASS_NETWORK_OTHER; + set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); + dc->props =3D virtio_rdma_properties; + vpciklass->realize =3D virtio_rdma_pci_realize; +} + +static void virtio_rdma_pci_instance_init(Object *obj) +{ + VirtIORdmaPCI *dev =3D VIRTIO_RDMA_PCI(obj); + + virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev), + TYPE_VIRTIO_RDMA); + /* + object_property_add_alias(obj, "bootindex", OBJECT(&dev->vdev), + "bootindex", &error_abort); + */ +} + +static const VirtioPCIDeviceTypeInfo virtio_rdma_pci_info =3D { + .base_name =3D TYPE_VIRTIO_RDMA_PCI, + .generic_name =3D "virtio-rdma-pci", + .transitional_name =3D "virtio-rdma-pci-transitional", + .non_transitional_name =3D "virtio-rdma-pci-non-transitional", + .instance_size =3D sizeof(VirtIORdmaPCI), + .instance_init =3D virtio_rdma_pci_instance_init, + .class_init =3D virtio_rdma_pci_class_init, +}; + +static void virtio_rdma_pci_register(void) +{ + virtio_pci_types_register(&virtio_rdma_pci_info); +} + +type_init(virtio_rdma_pci_register) diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index d87f5f93e9..c2d34c382f 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -85,6 +85,7 @@ extern bool pci_available; #define PCI_DEVICE_ID_VIRTIO_RNG 0x1005 #define PCI_DEVICE_ID_VIRTIO_9P 0x1009 #define PCI_DEVICE_ID_VIRTIO_VSOCK 0x1012 +#define PCI_DEVICE_ID_VIRTIO_RDMA 0x1013 =20 #define PCI_VENDOR_ID_REDHAT 0x1b36 #define PCI_DEVICE_ID_REDHAT_BRIDGE 0x0001 diff --git a/include/hw/virtio/virtio-rdma.h b/include/hw/virtio/virtio-rdm= a.h new file mode 100644 index 0000000000..3c7534cd8a --- /dev/null +++ b/include/hw/virtio/virtio-rdma.h @@ -0,0 +1,44 @@ +/* + * Virtio RDMA Device + * + * Copyright (C) 2019 Oracle + * + * Authors: + * Yuval Shaia + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +#ifndef QEMU_VIRTIO_RDMA_H +#define QEMU_VIRTIO_RDMA_H + +#include + +#include "chardev/char-fe.h" +#include "hw/virtio/virtio.h" +#include "hw/virtio/virtio-net.h" + +#define TYPE_VIRTIO_RDMA "virtio-rdma-device" +#define VIRTIO_RDMA(obj) \ + OBJECT_CHECK(VirtIORdma, (obj), TYPE_VIRTIO_RDMA) + +typedef struct RdmaBackendDev RdmaBackendDev; +typedef struct RdmaDeviceResources RdmaDeviceResources; +struct ibv_device_attr; + +typedef struct VirtIORdma { + VirtIODevice parent_obj; + VirtQueue *ctrl_vq; + VirtIONet *netdev; + RdmaBackendDev *backend_dev; + RdmaDeviceResources *rdma_dev_res; + CharBackend mad_chr; + char *backend_eth_device_name; + char *backend_device_name; + uint8_t backend_port_num; + struct ibv_device_attr dev_attr; +} VirtIORdma; + +#endif diff --git a/include/standard-headers/linux/virtio_ids.h b/include/standard= -headers/linux/virtio_ids.h index 6d5c3b2d4f..bd2c699450 100644 --- a/include/standard-headers/linux/virtio_ids.h +++ b/include/standard-headers/linux/virtio_ids.h @@ -43,5 +43,6 @@ #define VIRTIO_ID_INPUT 18 /* virtio input */ #define VIRTIO_ID_VSOCK 19 /* virtio vsock transport */ #define VIRTIO_ID_CRYPTO 20 /* virtio crypto */ +#define VIRTIO_ID_RDMA 26 /* virtio crypto */ =20 #endif /* _LINUX_VIRTIO_IDS_H */ --=20 2.20.1 From nobody Sat May 18 22:14:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=oracle.com ARC-Seal: i=1; a=rsa-sha256; t=1554981186; cv=none; d=zoho.com; s=zohoarc; b=QS0yHgAd6/6SfjvzZxQNK896TLTQVMwAirhLESBsOh/eXzUWVmjixqqjOqKYq/G1jgL36XNNMyzMWbHflZPYBWA++tVXwdA/4s8dFPb1otuk/7JXj+LIABlNh4sGJwK6SUd0rhMhPHwFcv5EQIPoVG2+1KZqrJi010jL/f7zp+4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1554981186; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Eg/wEs0utPORKGoF88LpTrGrUTtR/V5GQyoGvYxNZGA=; b=Pp3o4L1l3lZIOYYg9PzS7WukRm7hnksADblxnv11Tfso2mqej0yYEkbhXvCdirrtGKzk2yzWajuoQjtfbGXtZu8xzjdoY3+Ap8MuNw0qlrgmUX7Fmu20OdTQeTRsQc7tkyYZXvnqtRewWsMAwBoh8r7vkfbuNArzL3GbLi8VzFk= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155498118686929.120496971706075; Thu, 11 Apr 2019 04:13:06 -0700 (PDT) Received: from localhost ([127.0.0.1]:46601 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEXdc-0002B2-Lk for importer@patchew.org; Thu, 11 Apr 2019 07:13:00 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38256) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEXTZ-0003Il-PR for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEXTP-0000uL-Bw for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:33 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:42616) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEXTO-00082s-Gt for qemu-devel@nongnu.org; Thu, 11 Apr 2019 07:02:27 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x3BAriEe084211; Thu, 11 Apr 2019 11:02:12 GMT Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 2rpkht8b11-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Apr 2019 11:02:11 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x3BB1gYA059789; Thu, 11 Apr 2019 11:02:11 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 2rpytcq8vd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Apr 2019 11:02:10 +0000 Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x3BB2APN005791; Thu, 11 Apr 2019 11:02:10 GMT Received: from host4.lan (/77.138.183.59) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 11 Apr 2019 04:02:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=Eg/wEs0utPORKGoF88LpTrGrUTtR/V5GQyoGvYxNZGA=; b=cZoFiRgVtmZtq9zfV4T+IcL7EpOd8B9QAoVDHN5asWOD6JrL84tuty8RsB/WOKwZKDwB ZIoR34bts2eSsmaEnhT8oGBL2Avu5EkYiB+VYx4APRGgvrwpoROkNtIIJxDVlRF4gJX2 OMlNH9L8UD/MHWkg2IlC8vod25bPwczX3q6lPAXTfpMLOYlHvZyAGyoHrYiHf0+XwSmR fu4qtD/9yxcN+lRVbBY+wpWbo+5efOBU51Ekz6FlnxD6oRc+86L6s57QiDeVDoh9ljbu 0BeHJ+Cj4ybQ5xJku18X4ua64kyqn+4tz2y/M+FfPJKKjtrAeYuygdbOLnlkx3j8QOid Qw== From: Yuval Shaia To: virtualization@lists.linux-foundation.org, qemu-devel@nongnu.org, mst@redhat.com, cohuck@redhat.com, marcel.apfelbaum@gmail.com, linux-rdma@vger.kernel.org, jgg@mellanox.com Date: Thu, 11 Apr 2019 14:01:57 +0300 Message-Id: <20190411110157.14252-4-yuval.shaia@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190411110157.14252-1-yuval.shaia@oracle.com> References: <20190411110157.14252-1-yuval.shaia@oracle.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9223 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904110077 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9223 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904110077 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 156.151.31.86 Subject: [Qemu-devel] [RFC 3/3] RDMA/virtio-rdma: VirtIO rdma driver X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yuval Shaia Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Signed-off-by: Yuval Shaia --- drivers/infiniband/Kconfig | 1 + drivers/infiniband/hw/Makefile | 1 + drivers/infiniband/hw/virtio/Kconfig | 6 + drivers/infiniband/hw/virtio/Makefile | 4 + drivers/infiniband/hw/virtio/virtio_rdma.h | 40 + .../infiniband/hw/virtio/virtio_rdma_device.c | 59 ++ .../infiniband/hw/virtio/virtio_rdma_device.h | 32 + drivers/infiniband/hw/virtio/virtio_rdma_ib.c | 711 ++++++++++++++++++ drivers/infiniband/hw/virtio/virtio_rdma_ib.h | 48 ++ .../infiniband/hw/virtio/virtio_rdma_main.c | 149 ++++ .../infiniband/hw/virtio/virtio_rdma_netdev.c | 44 ++ .../infiniband/hw/virtio/virtio_rdma_netdev.h | 33 + include/uapi/linux/virtio_ids.h | 1 + 13 files changed, 1129 insertions(+) create mode 100644 drivers/infiniband/hw/virtio/Kconfig create mode 100644 drivers/infiniband/hw/virtio/Makefile create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma.h create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma_device.c create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma_device.h create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma_ib.c create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma_ib.h create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma_main.c create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma_netdev.c create mode 100644 drivers/infiniband/hw/virtio/virtio_rdma_netdev.h diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig index a1fb840de45d..218a47d4cecf 100644 --- a/drivers/infiniband/Kconfig +++ b/drivers/infiniband/Kconfig @@ -107,6 +107,7 @@ source "drivers/infiniband/hw/hfi1/Kconfig" source "drivers/infiniband/hw/qedr/Kconfig" source "drivers/infiniband/sw/rdmavt/Kconfig" source "drivers/infiniband/sw/rxe/Kconfig" +source "drivers/infiniband/hw/virtio/Kconfig" endif =20 source "drivers/infiniband/ulp/ipoib/Kconfig" diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile index e4f31c1be8f7..10ffb2c421e4 100644 --- a/drivers/infiniband/hw/Makefile +++ b/drivers/infiniband/hw/Makefile @@ -14,3 +14,4 @@ obj-$(CONFIG_INFINIBAND_HFI1) +=3D hfi1/ obj-$(CONFIG_INFINIBAND_HNS) +=3D hns/ obj-$(CONFIG_INFINIBAND_QEDR) +=3D qedr/ obj-$(CONFIG_INFINIBAND_BNXT_RE) +=3D bnxt_re/ +obj-$(CONFIG_INFINIBAND_VIRTIO_RDMA) +=3D virtio/ diff --git a/drivers/infiniband/hw/virtio/Kconfig b/drivers/infiniband/hw/v= irtio/Kconfig new file mode 100644 index 000000000000..92e41691cf5d --- /dev/null +++ b/drivers/infiniband/hw/virtio/Kconfig @@ -0,0 +1,6 @@ +config INFINIBAND_VIRTIO_RDMA + tristate "VirtIO Paravirtualized RDMA Driver" + depends on NETDEVICES && ETHERNET && PCI && INET + ---help--- + This driver provides low-level support for VirtIO Paravirtual + RDMA adapter. diff --git a/drivers/infiniband/hw/virtio/Makefile b/drivers/infiniband/hw/= virtio/Makefile new file mode 100644 index 000000000000..fb637e467167 --- /dev/null +++ b/drivers/infiniband/hw/virtio/Makefile @@ -0,0 +1,4 @@ +obj-$(CONFIG_INFINIBAND_VIRTIO_RDMA) +=3D virtio_rdma.o + +virtio_rdma-y :=3D virtio_rdma_main.o virtio_rdma_device.o virtio_rdma_ib.= o \ + virtio_rdma_netdev.o diff --git a/drivers/infiniband/hw/virtio/virtio_rdma.h b/drivers/infiniban= d/hw/virtio/virtio_rdma.h new file mode 100644 index 000000000000..7896a2dfb812 --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma.h @@ -0,0 +1,40 @@ +/* + * Virtio RDMA device: Driver main data types + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#ifndef __VIRTIO_RDMA__ +#define __VIRTIO_RDMA__ + +#include +#include + +struct virtio_rdma_info { + struct ib_device ib_dev; + struct virtio_device *vdev; + struct virtqueue *ctrl_vq; + wait_queue_head_t acked; /* arm on send to host, release on recv */ + struct net_device *netdev; +}; + +static inline struct virtio_rdma_info *to_vdev(struct ib_device *ibdev) +{ + return container_of(ibdev, struct virtio_rdma_info, ib_dev); +} + +#endif diff --git a/drivers/infiniband/hw/virtio/virtio_rdma_device.c b/drivers/in= finiband/hw/virtio/virtio_rdma_device.c new file mode 100644 index 000000000000..ae41e530644f --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma_device.c @@ -0,0 +1,59 @@ +/* + * Virtio RDMA device: Device related functions and data + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#include + +#include "virtio_rdma.h" + +static void rdma_ctrl_ack(struct virtqueue *vq) +{ + struct virtio_rdma_info *dev =3D vq->vdev->priv; + + wake_up(&dev->acked); + + printk("%s\n", __func__); +} + +int init_device(struct virtio_rdma_info *dev) +{ +#define TMP_MAX_VQ 1 + int rc; + struct virtqueue *vqs[TMP_MAX_VQ]; + vq_callback_t *cbs[TMP_MAX_VQ]; + const char *names[TMP_MAX_VQ]; + + names[0] =3D "ctrl"; + cbs[0] =3D rdma_ctrl_ack; + cbs[0] =3D NULL; + + rc =3D virtio_find_vqs(dev->vdev, TMP_MAX_VQ, vqs, cbs, names, NULL); + if (rc) + return rc; + + dev->ctrl_vq =3D vqs[0]; + + return 0; +} + +void fini_device(struct virtio_rdma_info *dev) +{ + dev->vdev->config->reset(dev->vdev); + dev->vdev->config->del_vqs(dev->vdev); +} diff --git a/drivers/infiniband/hw/virtio/virtio_rdma_device.h b/drivers/in= finiband/hw/virtio/virtio_rdma_device.h new file mode 100644 index 000000000000..d9b1240daf92 --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma_device.h @@ -0,0 +1,32 @@ +/* + * Virtio RDMA device: Device related functions and data + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#ifndef __VIRTIO_RDMA_DEVICE__ +#define __VIRTIO_RDMA_DEVICE__ + +#define VIRTIO_RDMA_BOARD_ID 1 +#define VIRTIO_RDMA_HW_NAME "virtio-rdma" +#define VIRTIO_RDMA_HW_REV 1 +#define VIRTIO_RDMA_DRIVER_VER "1.0" + +int init_device(struct virtio_rdma_info *dev); +void fini_device(struct virtio_rdma_info *dev); + +#endif diff --git a/drivers/infiniband/hw/virtio/virtio_rdma_ib.c b/drivers/infini= band/hw/virtio/virtio_rdma_ib.c new file mode 100644 index 000000000000..02bf4a332611 --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma_ib.c @@ -0,0 +1,711 @@ +/* + * Virtio RDMA device: IB related functions and data + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#include +#include +#include + +#include "virtio_rdma.h" +#include "virtio_rdma_device.h" +#include "virtio_rdma_ib.h" + +/* TODO: Move to uapi header file */ + +/* + * Control virtqueue data structures + * + * The control virtqueue expects a header in the first sg entry + * and an ack/status response in the last entry. Data for the + * command goes in between. + */ + +#define VIRTIO_RDMA_CTRL_OK 0 +#define VIRTIO_RDMA_CTRL_ERR 1 + +struct control_buf { + __u8 cmd; + __u8 status; +}; + +enum { + VIRTIO_CMD_QUERY_DEVICE =3D 10, + VIRTIO_CMD_QUERY_PORT, + VIRTIO_CMD_CREATE_CQ, + VIRTIO_CMD_DESTROY_CQ, + VIRTIO_CMD_CREATE_PD, + VIRTIO_CMD_DESTROY_PD, + VIRTIO_CMD_GET_DMA_MR, +}; + +struct cmd_query_port { + __u8 port; +}; + +struct cmd_create_cq { + __u32 cqe; +}; + +struct rsp_create_cq { + __u32 cqn; +}; + +struct cmd_destroy_cq { + __u32 cqn; +}; + +struct rsp_create_pd { + __u32 pdn; +}; + +struct cmd_destroy_pd { + __u32 pdn; +}; + +struct cmd_get_dma_mr { + __u32 pdn; + __u32 access_flags; +}; + +struct rsp_get_dma_mr { + __u32 mrn; + __u32 lkey; + __u32 rkey; +}; + +/* TODO: Move to uapi header file */ + +struct virtio_rdma_ib_cq { + struct ib_cq ibcq; + u32 cq_handle; +}; + +/* TODO: For the scope fof the RFC i'm utilizing ib*_*_attr structures */ + +static int virtio_rdma_exec_cmd(struct virtio_rdma_info *di, int cmd, + struct scatterlist *in, struct scatterlist *out) +{ + struct scatterlist *sgs[4], hdr, status; + struct control_buf *ctrl; + unsigned tmp; + int rc; + + ctrl =3D kmalloc(sizeof(*ctrl), GFP_ATOMIC); + ctrl->cmd =3D cmd; + ctrl->status =3D ~0; + + sg_init_one(&hdr, &ctrl->cmd, sizeof(ctrl->cmd)); + sgs[0] =3D &hdr; + sgs[1] =3D in; + sgs[2] =3D out; + sg_init_one(&status, &ctrl->status, sizeof(ctrl->status)); + sgs[3] =3D &status; + + rc =3D virtqueue_add_sgs(di->ctrl_vq, sgs, 2, 2, di, GFP_ATOMIC); + if (rc) + goto out; + + if (unlikely(!virtqueue_kick(di->ctrl_vq))) { + goto out_with_status; + } + + /* Spin for a response, the kick causes an ioport write, trapping + * into the hypervisor, so the request should be handled + * immediately */ + while (!virtqueue_get_buf(di->ctrl_vq, &tmp) && + !virtqueue_is_broken(di->ctrl_vq)) + cpu_relax(); + +out_with_status: + printk("%s: cmd %d, status %d\n", __func__, ctrl->cmd, ctrl->status); + rc =3D ctrl->status =3D=3D VIRTIO_RDMA_CTRL_OK ? 0 : 1; + +out: + kfree(ctrl); + + return rc; +} + +static int virtio_rdma_port_immutable(struct ib_device *ibdev, u8 port_num, + struct ib_port_immutable *immutable) +{ + struct ib_port_attr attr; + int rc; + + rc =3D ib_query_port(ibdev, port_num, &attr); + if (rc) + return rc; + + immutable->core_cap_flags |=3D RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP; + immutable->pkey_tbl_len =3D attr.pkey_tbl_len; + immutable->gid_tbl_len =3D attr.gid_tbl_len; + immutable->max_mad_size =3D IB_MGMT_MAD_SIZE; + + return 0; +} + +static int virtio_rdma_query_device(struct ib_device *ibdev, + struct ib_device_attr *props, + struct ib_udata *uhw) +{ + struct scatterlist data; + int offs; + int rc; + + if (uhw->inlen || uhw->outlen) + return -EINVAL; + + /* We start with sys_image_guid because of inconsistency beween ib_ + * and ibv_ */ + offs =3D offsetof(struct ib_device_attr, sys_image_guid); + sg_init_one(&data, (void *)props + offs, sizeof(*props) - offs); + + rc =3D virtio_rdma_exec_cmd(to_vdev(ibdev), VIRTIO_CMD_QUERY_DEVICE, NULL, + &data); + + printk("%s: sys_image_guid 0x%llx\n", __func__, + be64_to_cpu(props->sys_image_guid)); + + return rc; +} + +static int virtio_rdma_query_port(struct ib_device *ibdev, u8 port, + struct ib_port_attr *props) +{ + struct scatterlist in, out; + struct cmd_query_port *cmd; + int offs; + int rc; + + cmd =3D kmalloc(sizeof(*cmd), GFP_ATOMIC); + if (!cmd) + return -ENOMEM; + + /* We start with state because of inconsistency beween ib and ibv */ + offs =3D offsetof(struct ib_port_attr, state); + sg_init_one(&out, (void *)props + offs, sizeof(*props) - offs); + + cmd->port =3D port; + sg_init_one(&in, cmd, sizeof(*cmd)); + printk("%s: port %d\n", __func__, cmd->port); + + rc =3D virtio_rdma_exec_cmd(to_vdev(ibdev), VIRTIO_CMD_QUERY_PORT, &in, + &out); + + printk("%s: gid_tbl_len %d\n", __func__, props->gid_tbl_len); + + kfree(cmd); + + return rc; +} + +static struct net_device *virtio_rdma_get_netdev(struct ib_device *ibdev, + u8 port_num) +{ + struct virtio_rdma_info *ri =3D to_vdev(ibdev); + + printk("%s:\n", __func__); + + return ri->netdev; +} + +struct ib_cq *virtio_rdma_create_cq(struct ib_device *ibdev, + const struct ib_cq_init_attr *attr, + struct ib_ucontext *context, + struct ib_udata *udata) +{ + struct scatterlist in, out; + struct virtio_rdma_ib_cq *vcq; + struct cmd_create_cq *cmd; + struct rsp_create_cq *rsp; + struct ib_cq *cq =3D NULL; + int rc; + + /* TODO: Check MAX_CQ */ + + cmd =3D kmalloc(sizeof(*cmd), GFP_ATOMIC); + if (!cmd) + return ERR_PTR(-ENOMEM); + + rsp =3D kmalloc(sizeof(*rsp), GFP_ATOMIC); + if (!rsp) { + kfree(cmd); + return ERR_PTR(-ENOMEM); + } + + vcq =3D kzalloc(sizeof(*vcq), GFP_KERNEL); + if (!vcq) + goto out; + + cmd->cqe =3D attr->cqe; + sg_init_one(&in, cmd, sizeof(*cmd)); + printk("%s: cqe %d\n", __func__, cmd->cqe); + + sg_init_one(&out, rsp, sizeof(*rsp)); + + rc =3D virtio_rdma_exec_cmd(to_vdev(ibdev), VIRTIO_CMD_CREATE_CQ, &in, + &out); + if (rc) + goto out_err; + + printk("%s: cqn 0x%x\n", __func__, rsp->cqn); + vcq->cq_handle =3D rsp->cqn; + vcq->ibcq.cqe =3D attr->cqe; + cq =3D &vcq->ibcq; + + goto out; + +out_err: + kfree(vcq); + return ERR_PTR(rc); + +out: + kfree(rsp); + kfree(cmd); + return cq; +} + +int virtio_rdma_destroy_cq(struct ib_cq *cq) +{ + struct virtio_rdma_ib_cq *vcq; + struct scatterlist in; + struct cmd_destroy_cq *cmd; + int rc; + + cmd =3D kmalloc(sizeof(*cmd), GFP_ATOMIC); + if (!cmd) + return -ENOMEM; + + vcq =3D container_of(cq, struct virtio_rdma_ib_cq, ibcq); + + cmd->cqn =3D vcq->cq_handle; + sg_init_one(&in, cmd, sizeof(*cmd)); + + rc =3D virtio_rdma_exec_cmd(to_vdev(cq->device), VIRTIO_CMD_DESTROY_CQ, + &in, NULL); + + kfree(cmd); + + kfree(vcq); + + return rc; +} + +int virtio_rdma_alloc_pd(struct ib_pd *ibpd, struct ib_ucontext *context, + struct ib_udata *udata) +{ + struct virtio_rdma_pd *pd =3D to_vpd(ibpd); + struct ib_device *ibdev =3D ibpd->device; + struct rsp_create_pd *rsp; + struct scatterlist out; + int rc; + + /* TODO: Check MAX_PD */ + + rsp =3D kmalloc(sizeof(*rsp), GFP_ATOMIC); + if (!rsp) + return -ENOMEM; + + sg_init_one(&out, rsp, sizeof(*rsp)); + + rc =3D virtio_rdma_exec_cmd(to_vdev(ibdev), VIRTIO_CMD_CREATE_PD, NULL, + &out); + if (rc) + goto out; + + pd->pd_handle =3D rsp->pdn; + + printk("%s: pd_handle=3D%d\n", __func__, pd->pd_handle); + +out: + kfree(rsp); + + printk("%s: rc=3D%d\n", __func__, rc); + return rc; +} + +void virtio_rdma_dealloc_pd(struct ib_pd *pd) +{ + struct virtio_rdma_pd *vpd =3D to_vpd(pd); + struct ib_device *ibdev =3D pd->device; + struct cmd_destroy_pd *cmd; + struct scatterlist in; + + printk("%s:\n", __func__); + + cmd =3D kmalloc(sizeof(*cmd), GFP_ATOMIC); + if (!cmd) + return; + + cmd->pdn =3D vpd->pd_handle; + sg_init_one(&in, cmd, sizeof(*cmd)); + + virtio_rdma_exec_cmd(to_vdev(ibdev), VIRTIO_CMD_DESTROY_PD, &in, NULL); + + kfree(cmd); +} + +struct ib_mr *virtio_rdma_get_dma_mr(struct ib_pd *pd, int acc) + +{ + struct virtio_rdma_user_mr *mr; + struct scatterlist in, out; + struct cmd_get_dma_mr *cmd =3D NULL; + struct rsp_get_dma_mr *rsp =3D NULL; + int rc; + + mr =3D kzalloc(sizeof(*mr), GFP_KERNEL); + if (!mr) + return ERR_PTR(-ENOMEM); + + cmd =3D kmalloc(sizeof(*cmd), GFP_ATOMIC); + if (!cmd) { + kfree(mr); + return ERR_PTR(-ENOMEM); + } + + rsp =3D kmalloc(sizeof(*rsp), GFP_ATOMIC); + if (!cmd) { + kfree(mr); + kfree(cmd); + return ERR_PTR(-ENOMEM); + } + + cmd->pdn =3D to_vpd(pd)->pd_handle; + cmd->access_flags =3D acc; + sg_init_one(&in, cmd, sizeof(*cmd)); + + sg_init_one(&out, rsp, sizeof(*rsp)); + + rc =3D virtio_rdma_exec_cmd(to_vdev(pd->device), VIRTIO_CMD_GET_DMA_MR, + &in, &out); + if (rc) { + kfree(mr); + kfree(cmd); + return ERR_PTR(rc); + } + + mr->mr_handle =3D rsp->mrn; + mr->ibmr.lkey =3D rsp->lkey; + mr->ibmr.rkey =3D rsp->rkey; + + printk("%s: mr_handle=3D0x%x\n", __func__, mr->mr_handle); + + kfree(cmd); + kfree(rsp); + + return &mr->ibmr; +} + +struct ib_qp *virtio_rdma_create_qp(struct ib_pd *pd, + struct ib_qp_init_attr *init_attr, + struct ib_udata *udata) +{ + /* struct pvrdma_dev *dev =3D to_vdev(pd->device); */ + struct virtio_rdma_qp *qp; + + printk("%s:\n", __func__); + + qp =3D kzalloc(sizeof(*qp), GFP_KERNEL); + if (!qp) + return ERR_PTR(-ENOMEM); + + return &qp->ibqp; +} + +int virtio_rdma_query_gid(struct ib_device *ibdev, u8 port, int index, + union ib_gid *gid) +{ + memset(gid, 0, sizeof(union ib_gid)); + + printk("%s: port %d, index %d\n", __func__, port, index); + + return 0; +} + +static int virtio_rdma_add_gid(const struct ib_gid_attr *attr, void **cont= ext) +{ + printk("%s:\n", __func__); + + return 0; +} + +struct ib_mr *virtio_rdma_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_ty= pe, + u32 max_num_sg) +{ + printk("%s: mr_type %d, max_num_sg %d\n", __func__, mr_type, + max_num_sg); + + return NULL; +} + +int virtio_rdma_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *= udata) +{ + printk("%s:\n", __func__); + + return 0; +} + +struct ib_ah *virtio_rdma_create_ah(struct ib_pd *pd, + struct rdma_ah_attr *ah_attr, u32 flags, + struct ib_udata *udata) +{ + printk("%s:\n", __func__); + + return NULL; +} + +void virtio_rdma_dealloc_ucontext(struct ib_ucontext *ibcontext) + +{ +} + +static int virtio_rdma_del_gid(const struct ib_gid_attr *attr, void **cont= ext) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_dereg_mr(struct ib_mr *ibmr) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_destroy_ah(struct ib_ah *ah, u32 flags) +{ + printk("%s:\n", __func__); + + return 0; +} + +struct virtio_rdma_cq { + struct ib_cq ibcq; +}; + +int virtio_rdma_destroy_qp(struct ib_qp *qp) +{ + printk("%s:\n", __func__); + + return 0; +} + +static void virtio_rdma_get_fw_ver_str(struct ib_device *device, char *str) +{ + printk("%s:\n", __func__); +} + +enum rdma_link_layer virtio_rdma_port_link_layer(struct ib_device *ibdev, + u8 port) +{ + return IB_LINK_LAYER_ETHERNET; +} + +int virtio_rdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, + int sg_nents, unsigned int *sg_offset) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct = *vma) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_modify_port(struct ib_device *ibdev, u8 port, int mask, + struct ib_port_modify *props) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, + int attr_mask, struct ib_udata *udata) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc = *wc) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, + const struct ib_recv_wr **bad_wr) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, + const struct ib_send_wr **bad_wr) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_query_pkey(struct ib_device *ibdev, u8 port, u16 index, + u16 *pkey) +{ + printk("%s:\n", __func__); + + return 0; +} + +int virtio_rdma_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, + int attr_mask, struct ib_qp_init_attr *init_attr) +{ + printk("%s:\n", __func__); + + return 0; +} + +struct ib_mr *virtio_rdma_reg_user_mr(struct ib_pd *pd, u64 start, u64 len= gth, + u64 virt_addr, int access_flags, + struct ib_udata *udata) +{ + printk("%s:\n", __func__); + + return NULL; +} + +int virtio_rdma_req_notify_cq(struct ib_cq *ibcq, + enum ib_cq_notify_flags notify_flags) +{ + printk("%s:\n", __func__); + + return 0; +} + +static const struct ib_device_ops virtio_rdma_dev_ops =3D { + .get_port_immutable =3D virtio_rdma_port_immutable, + .query_device =3D virtio_rdma_query_device, + .query_port =3D virtio_rdma_query_port, + .get_netdev =3D virtio_rdma_get_netdev, + .create_cq =3D virtio_rdma_create_cq, + .destroy_cq =3D virtio_rdma_destroy_cq, + .alloc_pd =3D virtio_rdma_alloc_pd, + .dealloc_pd =3D virtio_rdma_dealloc_pd, + .get_dma_mr =3D virtio_rdma_get_dma_mr, + .create_qp =3D virtio_rdma_create_qp, + .query_gid =3D virtio_rdma_query_gid, + .add_gid =3D virtio_rdma_add_gid, + .alloc_mr =3D virtio_rdma_alloc_mr, + .alloc_ucontext =3D virtio_rdma_alloc_ucontext, + .create_ah =3D virtio_rdma_create_ah, + .dealloc_ucontext =3D virtio_rdma_dealloc_ucontext, + .del_gid =3D virtio_rdma_del_gid, + .dereg_mr =3D virtio_rdma_dereg_mr, + .destroy_ah =3D virtio_rdma_destroy_ah, + .destroy_qp =3D virtio_rdma_destroy_qp, + .get_dev_fw_str =3D virtio_rdma_get_fw_ver_str, + .get_link_layer =3D virtio_rdma_port_link_layer, + .get_port_immutable =3D virtio_rdma_port_immutable, + .map_mr_sg =3D virtio_rdma_map_mr_sg, + .mmap =3D virtio_rdma_mmap, + .modify_port =3D virtio_rdma_modify_port, + .modify_qp =3D virtio_rdma_modify_qp, + .poll_cq =3D virtio_rdma_poll_cq, + .post_recv =3D virtio_rdma_post_recv, + .post_send =3D virtio_rdma_post_send, + .query_device =3D virtio_rdma_query_device, + .query_pkey =3D virtio_rdma_query_pkey, + .query_port =3D virtio_rdma_query_port, + .query_qp =3D virtio_rdma_query_qp, + .reg_user_mr =3D virtio_rdma_reg_user_mr, + .req_notify_cq =3D virtio_rdma_req_notify_cq, + INIT_RDMA_OBJ_SIZE(ib_pd, virtio_rdma_pd, ibpd), +}; + +static ssize_t hca_type_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + return sprintf(buf, "%s-%s\n", VIRTIO_RDMA_HW_NAME, + VIRTIO_RDMA_DRIVER_VER); +} +static DEVICE_ATTR_RO(hca_type); + +static ssize_t hw_rev_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + return sprintf(buf, "%d\n", VIRTIO_RDMA_HW_REV); +} +static DEVICE_ATTR_RO(hw_rev); + +static ssize_t board_id_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + return sprintf(buf, "%d\n", VIRTIO_RDMA_BOARD_ID); +} +static DEVICE_ATTR_RO(board_id); + +static struct attribute *virtio_rdmaa_class_attributes[] =3D { + &dev_attr_hw_rev.attr, + &dev_attr_hca_type.attr, + &dev_attr_board_id.attr, + NULL, +}; + +static const struct attribute_group virtio_rdmaa_attr_group =3D { + .attrs =3D virtio_rdmaa_class_attributes, +}; + +int init_ib(struct virtio_rdma_info *ri) +{ + int rc; + + ri->ib_dev.owner =3D THIS_MODULE; + ri->ib_dev.num_comp_vectors =3D 1; + ri->ib_dev.dev.parent =3D &ri->vdev->dev; + ri->ib_dev.node_type =3D RDMA_NODE_IB_CA; + ri->ib_dev.phys_port_cnt =3D 1; + ri->ib_dev.uverbs_cmd_mask =3D + (1ull << IB_USER_VERBS_CMD_QUERY_DEVICE) | + (1ull << IB_USER_VERBS_CMD_QUERY_PORT) | + (1ull << IB_USER_VERBS_CMD_CREATE_CQ) | + (1ull << IB_USER_VERBS_CMD_DESTROY_CQ) | + (1ull << IB_USER_VERBS_CMD_ALLOC_PD) | + (1ull << IB_USER_VERBS_CMD_DEALLOC_PD); + + rdma_set_device_sysfs_group(&ri->ib_dev, &virtio_rdmaa_attr_group); + + ib_set_device_ops(&ri->ib_dev, &virtio_rdma_dev_ops); + + rc =3D ib_register_device(&ri->ib_dev, "virtio_rdma%d"); + + return rc; +} + +void fini_ib(struct virtio_rdma_info *ri) +{ + ib_unregister_device(&ri->ib_dev); +} diff --git a/drivers/infiniband/hw/virtio/virtio_rdma_ib.h b/drivers/infini= band/hw/virtio/virtio_rdma_ib.h new file mode 100644 index 000000000000..7b82a60581ff --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma_ib.h @@ -0,0 +1,48 @@ +/* + * Virtio RDMA device: IB related functions and data + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#ifndef __VIRTIO_RDMA_IB__ +#define __VIRTIO_RDMA_IB__ + +#include + +struct virtio_rdma_pd { + struct ib_pd ibpd; + u32 pd_handle; +}; + +struct virtio_rdma_user_mr { + struct ib_mr ibmr; + u32 mr_handle; +}; + +struct virtio_rdma_qp { + struct ib_qp ibqp; +}; + +static inline struct virtio_rdma_pd *to_vpd(struct ib_pd *ibpd) +{ + return container_of(ibpd, struct virtio_rdma_pd, ibpd); +} + +int init_ib(struct virtio_rdma_info *ri); +void fini_ib(struct virtio_rdma_info *ri); + +#endif diff --git a/drivers/infiniband/hw/virtio/virtio_rdma_main.c b/drivers/infi= niband/hw/virtio/virtio_rdma_main.c new file mode 100644 index 000000000000..811533d63160 --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma_main.c @@ -0,0 +1,149 @@ +/* + * Virtio RDMA device + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#include +#include +#include +#include +#include +#include + +#include "virtio_rdma.h" +#include "virtio_rdma_device.h" +#include "virtio_rdma_ib.h" +#include "virtio_rdma_netdev.h" + +/* TODO: + * - How to hook to unload driver, we need to undo all the stuff with did + * for all the devices that probed + * - + */ + +static int virtio_rdma_probe(struct virtio_device *vdev) +{ + struct virtio_rdma_info *ri; + int rc =3D -EIO; + + ri =3D ib_alloc_device(virtio_rdma_info, ib_dev); + if (!ri) { + pr_err("Fail to allocate IB device\n"); + rc =3D -ENOMEM; + goto out; + } + vdev->priv =3D ri; + + ri->vdev =3D vdev; + + rc =3D init_device(ri); + if (rc) { + pr_err("Fail to connect to device\n"); + goto out_dealloc_ib_device; + } + + rc =3D init_netdev(ri); + if (rc) { + pr_err("Fail to connect to NetDev layer\n"); + goto out_fini_device; + } + + rc =3D init_ib(ri); + if (rc) { + pr_err("Fail to connect to IB layer\n"); + goto out_fini_netdev; + } + + pr_info("VirtIO RDMA device %d probed\n", vdev->index); + + goto out; + +out_fini_netdev: + fini_netdev(ri); + +out_fini_device: + fini_device(ri); + +out_dealloc_ib_device: + ib_dealloc_device(&ri->ib_dev); + + vdev->priv =3D NULL; + +out: + return rc; +} + +static void virtio_rdma_remove(struct virtio_device *vdev) +{ + struct virtio_rdma_info *ri =3D vdev->priv; + + if (!ri) + return; + + vdev->priv =3D NULL; + + fini_ib(ri); + + fini_netdev(ri); + + fini_device(ri); + + ib_dealloc_device(&ri->ib_dev); + + pr_info("VirtIO RDMA device %d removed\n", vdev->index); +} + +static struct virtio_device_id id_table[] =3D { + { VIRTIO_ID_RDMA, VIRTIO_DEV_ANY_ID }, + { 0 }, +}; + +static struct virtio_driver virtio_rdma_driver =3D { + .driver.name =3D KBUILD_MODNAME, + .driver.owner =3D THIS_MODULE, + .id_table =3D id_table, + .probe =3D virtio_rdma_probe, + .remove =3D virtio_rdma_remove, +}; + +static int __init virtio_rdma_init(void) +{ + int rc; + + rc =3D register_virtio_driver(&virtio_rdma_driver); + if (rc) { + pr_err("%s: Fail to register virtio driver (%d)\n", __func__, + rc); + return rc; + } + + return 0; +} + +static void __exit virtio_rdma_fini(void) +{ + unregister_virtio_driver(&virtio_rdma_driver); +} + +module_init(virtio_rdma_init); +module_exit(virtio_rdma_fini); + +MODULE_DEVICE_TABLE(virtio, id_table); +MODULE_AUTHOR("Yuval Shaia"); +MODULE_DESCRIPTION("Virtio RDMA driver"); +MODULE_LICENSE("Dual BSD/GPL"); diff --git a/drivers/infiniband/hw/virtio/virtio_rdma_netdev.c b/drivers/in= finiband/hw/virtio/virtio_rdma_netdev.c new file mode 100644 index 000000000000..001f30b3e0b9 --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma_netdev.c @@ -0,0 +1,44 @@ +/* + * Virtio RDMA device + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#include "virtio_rdma_netdev.h" + +int init_netdev(struct virtio_rdma_info *ri) +{ + struct net_device *dev; + struct virtio_rdma_netdev_info *vrndi; + + dev =3D alloc_etherdev(sizeof(struct virtio_rdma_netdev_info)); + if (!dev) { + return -ENOMEM; + } + + SET_NETDEV_DEV(dev, &ri->vdev->dev); + vrndi =3D netdev_priv(dev); + vrndi->ri =3D ri; + ri->netdev =3D dev; + + return 0; +} + +void fini_netdev(struct virtio_rdma_info *ri) +{ + unregister_netdev(ri->netdev); +} diff --git a/drivers/infiniband/hw/virtio/virtio_rdma_netdev.h b/drivers/in= finiband/hw/virtio/virtio_rdma_netdev.h new file mode 100644 index 000000000000..e7e5d276d8ec --- /dev/null +++ b/drivers/infiniband/hw/virtio/virtio_rdma_netdev.h @@ -0,0 +1,33 @@ +/* + * Virtio RDMA device: Netdev related functions and data + * + * Copyright (C) 2019 Yuval Shaia Oracle Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 U= SA + */ + +#ifndef __VIRTIO_RDMA_NETDEV__ +#define __VIRTIO_RDMA_NETDEV__ + +#include "virtio_rdma.h" + +struct virtio_rdma_netdev_info { + struct virtio_rdma_info *ri; +}; + +int init_netdev(struct virtio_rdma_info *ri); +void fini_netdev(struct virtio_rdma_info *ri); + +#endif diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_id= s.h index 6d5c3b2d4f4d..288ee6fec8d3 100644 --- a/include/uapi/linux/virtio_ids.h +++ b/include/uapi/linux/virtio_ids.h @@ -43,5 +43,6 @@ #define VIRTIO_ID_INPUT 18 /* virtio input */ #define VIRTIO_ID_VSOCK 19 /* virtio vsock transport */ #define VIRTIO_ID_CRYPTO 20 /* virtio crypto */ +#define VIRTIO_ID_RDMA 26 /* RDMA */ =20 #endif /* _LINUX_VIRTIO_IDS_H */ --=20 2.17.0