From nobody Fri May 17 22:14:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604341341; cv=none; d=zohomail.com; s=zohoarc; b=TVVd8zPEjqWDhrsDASvpCGUAAlWtgE/w4mrfK+0TCNehz7oPf3wiieMBaNevx2i1hhVRbyzqZj4Jw9ZiuuNWPa1xSCEIDVmNEenq0dJxMXEbFdh9Jth90MzAyHfD8KTsW3S48gWuCWXeZro4rz0UVXJMVV66PBTFPjL4ZHQz68w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604341341; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=zZvFse3oqHhuPCkivJ8Tz4pHGWRjZvWGfQDergDm21A=; b=D3sSnF+ZpQj8KOBRYNS4SrGKwLcTyKB8VTHmtBr5tBFR/i4WYV4oB/5SEYg9GORXeddH97mUtyzEylQg2cTGI9jmN3Hb85tOkt3EjHzmMMt4dLhh1irRIg//yiGTG/Xk5mhDBI7Cnh1IE0teM1BffgmmTkaQU82fGmxppqdGEl0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1604341341030496.172871842837; Mon, 2 Nov 2020 10:22:21 -0800 (PST) Received: from localhost ([::1]:45636 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kZeTD-0003J1-Ov for importer@patchew.org; Mon, 02 Nov 2020 13:22:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40860) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kZeR0-0001Wi-GW for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:04 -0500 Received: from mail-lf1-x142.google.com ([2a00:1450:4864:20::142]:38909) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kZeQs-0006hc-84 for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:02 -0500 Received: by mail-lf1-x142.google.com with SMTP id 141so18658544lfn.5 for ; Mon, 02 Nov 2020 10:19:53 -0800 (PST) Received: from navi.cosmonova.net.ua ([95.67.24.131]) by smtp.gmail.com with ESMTPSA id c6sm2527007lfm.226.2020.11.02.10.19.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 10:19:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zZvFse3oqHhuPCkivJ8Tz4pHGWRjZvWGfQDergDm21A=; b=ERwKG+s3W00tHZ68lWQPt8K6vtXzZv9cAnR9J4haXllUqSsnGCT/VTztPYhqXv4z0A XI9cCNMPrLkXNoSo162e7jCeiCqXqHJFOZX5utcKVy1MWta2aSRs0AOwaK1hZydzF6j8 pF7xpscQ1OW1HNOZo6nNwFSTIbY9R/5U6RLsIbe6JtT5wknzw0Wzsf20xYC7scizqwn7 ktyVXAyqJRImfpnK3JPjFv/s1mvHGuhVe5q8yQv4CnOo/WyN0UFP5S9dE1JXcIrOSSx7 rh1UjGn2iKzW7uO1ezUdBjxVfqoanMyf4JiTv3bqF5H9bAYZFhfVQNs8tzoAXqpu9EZ7 wxcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zZvFse3oqHhuPCkivJ8Tz4pHGWRjZvWGfQDergDm21A=; b=HwN3oZTvgB9mgOYq3glcLIlUcqB+jrdo+9/uJE5BqBsUjiqibZwrHfg7mxxuMnexwc vVH3e/WPhXPfNaOC1OuwbMuxI5bx34eRanTb/EmnaO+4QhVasaM2vxOo8bWdWth7t37r yI1MOPHI0qqWDTBC7AraM6E7ZoOJZKXkzjkJP5Ag+CcgElGDW0LUYK162RfCkJ4VHjw/ QzK3xdBs/EsOsipY7wMZgy5hIckL30ro9D9bhYGohMFzfiBcnNXvZ1wzlCMexlsXPOIO YlEsSsnjynsk3wANIfTwbe4GjAgf2xmdZqbauQgmha8BJBU4j4fQxbnIORYZnakMamSj tC4A== X-Gm-Message-State: AOAM533dFUFXhs1LZVqax8HcW/3GKrOBAOTLROSf81ZLY3AZ34uMXxu8 TvCSNYCcV1glbMqOR9rdKr9uXg== X-Google-Smtp-Source: ABdhPJxTtWH4sA5v9GvGaINC4EwhEIae0lw+9gsTkNjf92azqomY4q7IyfERfnUJDrHkE4aoN4r0EQ== X-Received: by 2002:ac2:43ad:: with SMTP id t13mr3782788lfl.282.1604341191910; Mon, 02 Nov 2020 10:19:51 -0800 (PST) From: Andrew Melnychenko To: jasowang@redhat.com, mst@redhat.com Subject: [RFC PATCH 1/6] net: Added SetSteeringEBPF method for NetClientState. Date: Mon, 2 Nov 2020 20:51:11 +0200 Message-Id: <20201102185115.7425-2-andrew@daynix.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201102185115.7425-1-andrew@daynix.com> References: <20201102185115.7425-1-andrew@daynix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=2a00:1450:4864:20::142; envelope-from=andrew@daynix.com; helo=mail-lf1-x142.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yan@daynix.com, yuri.benditovich@daynix.com, Andrew , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrew For now, that method supported only by Linux TAP. Linux TAP uses TUNSETSTEERINGEBPF ioctl. TUNSETSTEERINGBPF was added 3 years ago. Qemu checks if it was defined before using. Signed-off-by: Andrew Melnychenko --- include/net/net.h | 2 ++ net/tap-bsd.c | 5 +++++ net/tap-linux.c | 19 +++++++++++++++++++ net/tap-solaris.c | 5 +++++ net/tap-stub.c | 5 +++++ net/tap.c | 9 +++++++++ net/tap_int.h | 1 + 7 files changed, 46 insertions(+) diff --git a/include/net/net.h b/include/net/net.h index 897b2d7595..d8a41fb010 100644 --- a/include/net/net.h +++ b/include/net/net.h @@ -60,6 +60,7 @@ typedef int (SetVnetBE)(NetClientState *, bool); typedef struct SocketReadState SocketReadState; typedef void (SocketReadStateFinalize)(SocketReadState *rs); typedef void (NetAnnounce)(NetClientState *); +typedef bool (SetSteeringEBPF)(NetClientState *, int); =20 typedef struct NetClientInfo { NetClientDriver type; @@ -81,6 +82,7 @@ typedef struct NetClientInfo { SetVnetLE *set_vnet_le; SetVnetBE *set_vnet_be; NetAnnounce *announce; + SetSteeringEBPF *set_steering_ebpf; } NetClientInfo; =20 struct NetClientState { diff --git a/net/tap-bsd.c b/net/tap-bsd.c index 77aaf674b1..4f64f31e98 100644 --- a/net/tap-bsd.c +++ b/net/tap-bsd.c @@ -259,3 +259,8 @@ int tap_fd_get_ifname(int fd, char *ifname) { return -1; } + +int tap_fd_set_steering_ebpf(int fd, int prog_fd) +{ + return -1; +} diff --git a/net/tap-linux.c b/net/tap-linux.c index b0635e9e32..196373019f 100644 --- a/net/tap-linux.c +++ b/net/tap-linux.c @@ -31,6 +31,7 @@ =20 #include #include +#include /* TUNSETSTEERINGEBPF */ =20 #include "qapi/error.h" #include "qemu/error-report.h" @@ -316,3 +317,21 @@ int tap_fd_get_ifname(int fd, char *ifname) pstrcpy(ifname, sizeof(ifr.ifr_name), ifr.ifr_name); return 0; } + +int tap_fd_set_steering_ebpf(int fd, int prog_fd) +{ +#ifdef TUNSETSTEERINGEBPF + if (ioctl(fd, TUNSETSTEERINGEBPF, (void *) &prog_fd) !=3D 0) { + error_report("Issue while setting TUNSETSTEERINGEBPF:" + " %s with fd: %d, prog_fd: %d", + strerror(errno), fd, prog_fd); + + return -1; + } + + return 0; +#else + error_report("TUNSETSTEERINGEBPF is not supported"); + return -1; +#endif +} diff --git a/net/tap-solaris.c b/net/tap-solaris.c index 0475a58207..d85224242b 100644 --- a/net/tap-solaris.c +++ b/net/tap-solaris.c @@ -255,3 +255,8 @@ int tap_fd_get_ifname(int fd, char *ifname) { return -1; } + +int tap_fd_set_steering_ebpf(int fd, int prog_fd) +{ + return -1; +} diff --git a/net/tap-stub.c b/net/tap-stub.c index de525a2e69..a0fa25804b 100644 --- a/net/tap-stub.c +++ b/net/tap-stub.c @@ -85,3 +85,8 @@ int tap_fd_get_ifname(int fd, char *ifname) { return -1; } + +int tap_fd_set_steering_ebpf(int fd, int prog_fd) +{ + return -1; +} diff --git a/net/tap.c b/net/tap.c index c46ff66184..81f50017bd 100644 --- a/net/tap.c +++ b/net/tap.c @@ -337,6 +337,14 @@ static void tap_poll(NetClientState *nc, bool enable) tap_write_poll(s, enable); } =20 +static bool tap_set_steering_ebpf(NetClientState *nc, int prog_fd) +{ + TAPState *s =3D DO_UPCAST(TAPState, nc, nc); + assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_TAP); + + return tap_fd_set_steering_ebpf(s->fd, prog_fd) =3D=3D 0; +} + int tap_get_fd(NetClientState *nc) { TAPState *s =3D DO_UPCAST(TAPState, nc, nc); @@ -362,6 +370,7 @@ static NetClientInfo net_tap_info =3D { .set_vnet_hdr_len =3D tap_set_vnet_hdr_len, .set_vnet_le =3D tap_set_vnet_le, .set_vnet_be =3D tap_set_vnet_be, + .set_steering_ebpf =3D tap_set_steering_ebpf, }; =20 static TAPState *net_tap_fd_init(NetClientState *peer, diff --git a/net/tap_int.h b/net/tap_int.h index 225a49ea48..547f8a5a28 100644 --- a/net/tap_int.h +++ b/net/tap_int.h @@ -44,5 +44,6 @@ int tap_fd_set_vnet_be(int fd, int vnet_is_be); int tap_fd_enable(int fd); int tap_fd_disable(int fd); int tap_fd_get_ifname(int fd, char *ifname); +int tap_fd_set_steering_ebpf(int fd, int prog_fd); =20 #endif /* NET_TAP_INT_H */ --=20 2.28.0 From nobody Fri May 17 22:14:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604341593; cv=none; d=zohomail.com; s=zohoarc; b=MnxejFQL7wtHnB1G0mT2O72NrmcBaSEVIU+j6268j5IhBGfX6yxM5yM5m3VPjLJEsefTDaotzxD9UfpRln6kebycoUYXPJjIr5fsN4RsnugOZ4wgJoTtocBvMSJL0+wdMzqqK2WpbQNOEBdE3QQbW9RiXgXXj9O42+dnC3GGyGY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604341593; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3+YyHTEOmoDlnxZwQCww4YXw4Lozy8Eon7UHJCT2EzM=; b=Dj4f1gVSgCrEGfpA1tHgAa/qIofv+mkEU2MARvESGgmIA7cMKIZskLgZAQFWoFsnotBWja6xmZNh0KmBzJUtnFe33ypmnbAU+qEiKDQ8ynx+lPrhzHxlYcIjImxO/utJkqjhWhn0e3CjICrmyicbqzC9fwdlkXfZZx2pZuymmwQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16043415932427.62164978627618; Mon, 2 Nov 2020 10:26:33 -0800 (PST) Received: from localhost ([::1]:60926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kZeXI-0001Dg-6N for importer@patchew.org; Mon, 02 Nov 2020 13:26:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40926) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kZeR7-0001fn-JC for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:09 -0500 Received: from mail-lf1-x141.google.com ([2a00:1450:4864:20::141]:42017) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kZeQu-0006hs-8u for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:09 -0500 Received: by mail-lf1-x141.google.com with SMTP id a7so18628526lfk.9 for ; Mon, 02 Nov 2020 10:19:54 -0800 (PST) Received: from navi.cosmonova.net.ua ([95.67.24.131]) by smtp.gmail.com with ESMTPSA id c6sm2527007lfm.226.2020.11.02.10.19.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 10:19:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3+YyHTEOmoDlnxZwQCww4YXw4Lozy8Eon7UHJCT2EzM=; b=NSz4/piZs7R5rtGK6iEwyUNc2Se7teWjllHhJvvkPZoOcbmA1wXHbX8aN2cxtWUqcT qSzXv0aH4cnfCjNf/JI74QcPE1eFrgi/B2KnjYvYMrnQh0Ybe0wj/BafbVvFvVxfR7YT 5ISxA4W/SeGiTYUwgv5EVCT4SsV6oon0IY1KQV39VHEaJt1/441oXOHu8bNmY3WO0twS 19GJx/VLt8aQWbaY0DbBw/KbICx04T0cozFXck2ddd3zaW0cpsVxvGB5A2p+nsdajEuE i3/nh3qVB0waw+tbWMcP455OjYidGMJzflvTzM9u5aQUuUuuILPGjBA5wuKZ7x5yvkOs /zxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3+YyHTEOmoDlnxZwQCww4YXw4Lozy8Eon7UHJCT2EzM=; b=YeWMDN9ZYukJ/WsAO8ur6KQTa+1ostZ9p8H1AMxdSXdQtRqplANLhYawcVUDgGpvsf MwERQXlu4HEfLOKyiwjOpgm1UZDrpAQJ6ypIhLGOHY1QS2bOMojfC7BSDCgMo4UkpuDn 0OzT7HU0rr0umplxJL4u3kwZtALzwg6njKDL9zNG32PORLcaxo0qAtfLIQkc7wKtDrJq mRz5bLQeJJysB1Iox+qrKHsUgjtE93rJdO03utcMoW337AEZkkublykyxtfuhVUBZKkw 7/MTlhSJz512h3FJjFiMeOr+ojBXewaLVkpjELF1dUIvezGCxYLA53tLZFJbX7hOk+1r Ap3g== X-Gm-Message-State: AOAM531xhdUsbI0tk0TFfJqY0OvivrLKjJaSyE7ZM8xUmCqoLtEp0csU Sa54/6+HIPnpiKelvabNfxna7w== X-Google-Smtp-Source: ABdhPJyRiaXqfwvJsq1jD9Ly3RknIiCLSt5abf9+ZSuAhO20/Ls5tn9ghUt3xXlBm+Il3nKjaUMYOw== X-Received: by 2002:ac2:47fc:: with SMTP id b28mr5969722lfp.454.1604341193176; Mon, 02 Nov 2020 10:19:53 -0800 (PST) From: Andrew Melnychenko To: jasowang@redhat.com, mst@redhat.com Subject: [RFC PATCH 2/6] ebpf: Added basic eBPF API. Date: Mon, 2 Nov 2020 20:51:12 +0200 Message-Id: <20201102185115.7425-3-andrew@daynix.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201102185115.7425-1-andrew@daynix.com> References: <20201102185115.7425-1-andrew@daynix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=2a00:1450:4864:20::141; envelope-from=andrew@daynix.com; helo=mail-lf1-x141.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yan@daynix.com, yuri.benditovich@daynix.com, Andrew , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrew Added basic functions for creating eBPF maps and loading programs. Also added helper function to 'fix' eBPF map descriptors in programs. During runtime, different values of eBPF map file descriptors created, and it required to place them into eBPF instructions for proper work. It's similar to ELF's relocation table section routine. Signed-off-by: Yuri Benditovich Signed-off-by: Andrew Melnychenko --- ebpf/ebpf.c | 107 ++++++++++++++++++++++++++++++++++++++++++++++ ebpf/ebpf.h | 35 +++++++++++++++ ebpf/trace-events | 4 ++ ebpf/trace.h | 2 + 4 files changed, 148 insertions(+) create mode 100644 ebpf/ebpf.c create mode 100644 ebpf/ebpf.h create mode 100644 ebpf/trace-events create mode 100644 ebpf/trace.h diff --git a/ebpf/ebpf.c b/ebpf/ebpf.c new file mode 100644 index 0000000000..cec35a484c --- /dev/null +++ b/ebpf/ebpf.c @@ -0,0 +1,107 @@ +#include "ebpf/ebpf.h" +#include +#include "trace.h" + +#define ptr_to_u64(x) ((uint64_t)(uintptr_t)x) + +static inline int ebpf(enum bpf_cmd cmd, union bpf_attr *attr, + unsigned int size) +{ + int ret =3D syscall(__NR_bpf, cmd, attr, size); + if (ret < 0) { + trace_ebpf_error("eBPF syscall error", strerror(errno)); + } + + return ret; +} + +int bpf_create_map(enum bpf_map_type map_type, + unsigned int key_size, + unsigned int value_size, + unsigned int max_entries) +{ + union bpf_attr attr =3D { + .map_type =3D map_type, + .key_size =3D key_size, + .value_size =3D value_size, + .max_entries =3D max_entries + }; + + return ebpf(BPF_MAP_CREATE, &attr, sizeof(attr)); +} + +int bpf_lookup_elem(int fd, const void *key, void *value) +{ + union bpf_attr attr =3D { + .map_fd =3D (uint32_t)fd, + .key =3D ptr_to_u64(key), + .value =3D ptr_to_u64(value), + }; + + return ebpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr)); +} + +int bpf_update_elem(int fd, const void *key, const void *value, + uint64_t flags) +{ + union bpf_attr attr =3D { + .map_fd =3D (uint32_t)fd, + .key =3D ptr_to_u64(key), + .value =3D ptr_to_u64(value), + .flags =3D flags, + }; + + return ebpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr)); +} + +int bpf_delete_elem(int fd, const void *key) +{ + union bpf_attr attr =3D { + .map_fd =3D (uint32_t)fd, + .key =3D ptr_to_u64(key), + }; + + return ebpf(BPF_MAP_DELETE_ELEM, &attr, sizeof(attr)); +} + +#define BPF_LOG_BUF_SIZE (UINT32_MAX >> 8) +static char bpf_log_buf[BPF_LOG_BUF_SIZE] =3D {}; + +int bpf_prog_load(enum bpf_prog_type type, + const struct bpf_insn *insns, int insn_cnt, + const char *license) +{ + int ret =3D 0; + union bpf_attr attr =3D {}; + attr.prog_type =3D type; + attr.insns =3D ptr_to_u64(insns); + attr.insn_cnt =3D (uint32_t)insn_cnt; + attr.license =3D ptr_to_u64(license); + attr.log_buf =3D ptr_to_u64(bpf_log_buf); + attr.log_size =3D BPF_LOG_BUF_SIZE; + attr.log_level =3D 1; + + ret =3D ebpf(BPF_PROG_LOAD, &attr, sizeof(attr)); + if (ret < 0) { + trace_ebpf_error("eBPF program load error:", bpf_log_buf); + } + + return ret; +} + +unsigned int bpf_fixup_mapfd(struct fixup_mapfd_t *table, + size_t table_size, struct bpf_insn *insn, + size_t insn_len, const char *map_name, int fd= ) { + unsigned int ret =3D 0; + int i =3D 0; + + for (; i < table_size; ++i) { + if (strcmp(table[i].map_name, map_name) =3D=3D 0) { + insn[table[i].instruction_num].src_reg =3D 1; + insn[table[i].instruction_num].imm =3D fd; + ++ret; + } + } + + return ret; +} diff --git a/ebpf/ebpf.h b/ebpf/ebpf.h new file mode 100644 index 0000000000..511ad0a06f --- /dev/null +++ b/ebpf/ebpf.h @@ -0,0 +1,35 @@ +#ifndef QEMU_EBPF_H +#define QEMU_EBPF_H + +#include "qemu/osdep.h" + +#ifdef CONFIG_EBPF +#include + +int bpf_create_map(enum bpf_map_type map_type, + unsigned int key_size, + unsigned int value_size, + unsigned int max_entries); + +int bpf_lookup_elem(int fd, const void *key, void *value); + +int bpf_update_elem(int fd, const void *key, const void *value, + uint64_t flags); + +int bpf_delete_elem(int fd, const void *key); + +int bpf_prog_load(enum bpf_prog_type type, + const struct bpf_insn *insns, int insn_cnt, + const char *license); + +struct fixup_mapfd_t { + const char *map_name; + size_t instruction_num; +}; + +unsigned int bpf_fixup_mapfd(struct fixup_mapfd_t *table, + size_t table_size, struct bpf_insn *insn, + size_t insn_len, const char *map_name, int fd= ); + +#endif /* CONFIG_EBPF */ +#endif /* QEMU_EBPF_H */ diff --git a/ebpf/trace-events b/ebpf/trace-events new file mode 100644 index 0000000000..3c189516e3 --- /dev/null +++ b/ebpf/trace-events @@ -0,0 +1,4 @@ +# See docs/devel/tracing.txt for syntax documentation. + +# ebpf.c +ebpf_error(const char *s1, const char *s2) "error in %s: %s" diff --git a/ebpf/trace.h b/ebpf/trace.h new file mode 100644 index 0000000000..ad570e6691 --- /dev/null +++ b/ebpf/trace.h @@ -0,0 +1,2 @@ +#include "trace/trace-ebpf.h" + --=20 2.28.0 From nobody Fri May 17 22:14:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604341469; cv=none; d=zohomail.com; s=zohoarc; b=B+kzdZUSImWK/cHrmRcNbzaROAmUnC6w+cGlyXcT9y7xq2IRUYj/OU/3R7O2geRTGm1t1CNyoM/+yT3cABqH9dCmk7wO03IXy1fAaxod0ZSRvGhmjw6MbTMr4Jnlq0bITohaAkPoESyW511dJ3FLv9+vUgnk5ql1LJzvon/PD70= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604341469; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Ww5EvrpJVdoc1slCZnMfPZ+2zfMTN1gD2SMi9IAZjPo=; b=VYLNl0f3XGzKvP0bUpa8VLebM7Zu7EsBNoO1mXHyK1LrpaIYp/5L+SdI8GXtIFh6ABKvD/wnDnv3R7zTA8T02T7VIdcfE76r/d8ZGOQUdE5kfBAgTevdW3YsiFb08JYV/F3uvU6H1amAGajrbeMaDvIhaRkYKNoNno9qDLeYrMk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1604341469155173.3711627434227; Mon, 2 Nov 2020 10:24:29 -0800 (PST) Received: from localhost ([::1]:54322 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kZeVG-0006pH-UB for importer@patchew.org; Mon, 02 Nov 2020 13:24:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kZeR3-0001Y7-IA for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:05 -0500 Received: from mail-lf1-x143.google.com ([2a00:1450:4864:20::143]:45967) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kZeQw-0006iB-86 for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:05 -0500 Received: by mail-lf1-x143.google.com with SMTP id y184so16693727lfa.12 for ; Mon, 02 Nov 2020 10:19:56 -0800 (PST) Received: from navi.cosmonova.net.ua ([95.67.24.131]) by smtp.gmail.com with ESMTPSA id c6sm2527007lfm.226.2020.11.02.10.19.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 10:19:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ww5EvrpJVdoc1slCZnMfPZ+2zfMTN1gD2SMi9IAZjPo=; b=pd/cuJS0E9TxPyU8TR8aaPllaMv6j/nEPBo2wos6hfZRHeYVNjvO2b8AwSV0lw1yTz wRXyZ9Eyn9btdmEBOl4+zVAVjNgFMs98mIMtdFPPzUvwgBcZ5QgQrw3seg+fGaGhCVuf l1LiV/f1VLM3TTVg8gFkWi+9O0O7jpPlqg+jDdZZ7wFXlSc2Vix2HEHHWcjuipeny2cJ xYdY8uJK1a9Yh1exeWJNTSHVYo+XZFpJhb8lmerbg9bQLQKjBUvsR5ad9MlMobyCrdCt hAVLFDMX0bkMH5QQYsTdMWdzwSWX7iYscifnWo1HY2nlO+BkJx6qvg7STjdR+6XaAVPh vGPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ww5EvrpJVdoc1slCZnMfPZ+2zfMTN1gD2SMi9IAZjPo=; b=JYYoGHpREmhpjO5ibgEzwMFx5U2skHlN31YV8YVCu1W6Ee7YIRrdsm62OqbgSXsSbZ 8dJQEDjB5quaccg4e5Un1b2chD63TgKKXX2Ipvd6NoKDTWKbqCgaqhoyikaP03WOwp7T Qz+OMP9PrJyvFDV6K2w2tOvdu190AkXbCrD2GNMBiK9qOMRT6NvAbtPH8jRawVRx1fyr ZSVZxK19XZvAPLifIaW/IGlPVAgS/hWxwPYgPqsWTB6lAzDb9lOgT63+gxg4JHgwlZZl ohD5y0WwZsNz4YlOs7McThG19EyUaDfjZ4XYa4D/MzT4bCGbEh41YwKyo1+RoN+RJ2lK spzQ== X-Gm-Message-State: AOAM531ZwAJjlwln7mR6neqvjqqi1+BE/9h9pd3RwP2oh1F9Mweyh/k5 wPj4NmXFhCdNsPaSZSixpLdxdA== X-Google-Smtp-Source: ABdhPJwa4UlIiRK8ddEfDwebMrox6H3d/ykkO28Xw4xqDKaO3FfOhp6l1OES1SxqUnsLmkGyOzP4bw== X-Received: by 2002:a05:6512:3496:: with SMTP id v22mr5725316lfr.230.1604341194619; Mon, 02 Nov 2020 10:19:54 -0800 (PST) From: Andrew Melnychenko To: jasowang@redhat.com, mst@redhat.com Subject: [RFC PATCH 3/6] ebpf: Added eBPF RSS program. Date: Mon, 2 Nov 2020 20:51:13 +0200 Message-Id: <20201102185115.7425-4-andrew@daynix.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201102185115.7425-1-andrew@daynix.com> References: <20201102185115.7425-1-andrew@daynix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=2a00:1450:4864:20::143; envelope-from=andrew@daynix.com; helo=mail-lf1-x143.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yan@daynix.com, yuri.benditovich@daynix.com, Andrew , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrew RSS program and Makefile to build it. Also, added a python script that would generate '.h' file. The data in that file may be loaded by eBPF API. EBPF compilation is not required for building qemu. You can use Makefile if you need to regenerate tun_rss_steering.h. Signed-off-by: Yuri Benditovich Signed-off-by: Andrew Melnychenko --- ebpf/EbpfElf_to_C.py | 67 +++++ ebpf/Makefile.ebpf | 38 +++ ebpf/rss.bpf.c | 470 +++++++++++++++++++++++++++++++++ ebpf/tun_rss_steering.h | 556 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 1131 insertions(+) create mode 100644 ebpf/EbpfElf_to_C.py create mode 100755 ebpf/Makefile.ebpf create mode 100644 ebpf/rss.bpf.c create mode 100644 ebpf/tun_rss_steering.h diff --git a/ebpf/EbpfElf_to_C.py b/ebpf/EbpfElf_to_C.py new file mode 100644 index 0000000000..a6e3476f2b --- /dev/null +++ b/ebpf/EbpfElf_to_C.py @@ -0,0 +1,67 @@ +#!/usr/bin/python3 +# pip install pyelftools + +import sys +import argparse + +from elftools.elf.elffile import ELFFile +from elftools.elf.relocation import RelocationSection +from elftools.elf.sections import Section +from elftools.elf.sections import SymbolTableSection + +def process_file(filename, prog_name): + print('Processing file:', filename) + with open(filename, 'rb') as f: + with open("%s.h" % prog_name, 'w') as w: + + elffile =3D ELFFile(f) + + symtab =3D elffile.get_section_by_name(".symtab") + if not isinstance(symtab, SymbolTableSection): + print(' The file has no %s section' % ".symtab") + return -1 + + prog_sec =3D elffile.get_section_by_name(prog_name); + if not isinstance(prog_sec, Section): + print(' The file has no %s section' % prog_name) + return -1 + + w.write('#ifndef %s\n' % prog_name.upper()) + w.write('#define %s\n\n' % prog_name.upper()) + + w.write("struct bpf_insn ins%s[] =3D {\n" % prog_name) + insns =3D [prog_sec.data()[i:i + 8] for i in range(0, prog_sec= .data_size, 8)] + for x in insns: + w.write( \ + ' {0x%02x, 0x%02x, 0x%02x, 0x%02x%02x, 0x%02x%02x%0= 2x%02x},\n' \ + % (x[0], x[1] & 0x0f, (x[1] >> 4) & 0x0f, \ + x[3], x[2], x[7], x[6], x[5], x[4])) + w.write('};\n\n') + + reladyn_name =3D '.rel' + prog_name + reladyn =3D elffile.get_section_by_name(reladyn_name) + + if isinstance(reladyn, RelocationSection): + w.write('struct fixup_mapfd_t rel%s[] =3D {\n' % prog_name) + for reloc in reladyn.iter_relocations(): + w.write(' {"%s", %i},\n' \ + % (symtab.get_symbol(reloc['r_info_sym']).name, \ + (reloc['r_offset']/8))) + w.write('};\n\n') + else: + print(' The file has no %s section' % reladyn_name) + + w.write('#endif /* %s */\n' % prog_name.upper()) + + return 0 + +if __name__ =3D=3D '__main__': + parser =3D argparse.ArgumentParser( + description=3D'Convert eBPF ELF to C header. ' + 'Section name will be used in C namings.') + parser.add_argument('--file', '-f', nargs=3D1, required=3DTrue, + help=3D'eBPF ELF file') + parser.add_argument('--section', '-s', nargs=3D1, required=3DTrue, + help=3D'section in ELF with eBPF program.') + args =3D parser.parse_args() + sys.exit(process_file(args.file[0], args.section[0])) diff --git a/ebpf/Makefile.ebpf b/ebpf/Makefile.ebpf new file mode 100755 index 0000000000..f7008d7d32 --- /dev/null +++ b/ebpf/Makefile.ebpf @@ -0,0 +1,38 @@ +OBJS =3D rss.bpf.o + +LLC ?=3D llc +CLANG ?=3D clang +INC_FLAGS =3D -nostdinc -isystem `$(CLANG) -print-file-name=3Dinclude` +EXTRA_CFLAGS ?=3D -O2 -emit-llvm + +linuxhdrs =3D ~/src/kernel/master + +LINUXINCLUDE =3D -I $(linuxhdrs)/arch/x86/include/uapi \ + -I $(linuxhdrs)/arch/x86/include/generated/uapi \ + -I $(linuxhdrs)/arch/x86/include/generated \ + -I $(linuxhdrs)/include/generated/uapi \ + -I $(linuxhdrs)/include/uapi \ + -I $(linuxhdrs)/include \ + -I $(linuxhdrs)/tools/lib + +all: $(OBJS) + +.PHONY: clean + +clean: + rm -f $(OBJS) + +INC_FLAGS =3D -nostdinc -isystem `$(CLANG) -print-file-name=3Dinclude` + +$(OBJS): %.o:%.c + $(CLANG) $(INC_FLAGS) \ + -D__KERNEL__ -D__ASM_SYSREG_H \ + -Wno-unused-value -Wno-pointer-sign \ + -Wno-compare-distinct-pointer-types \ + -Wno-gnu-variable-sized-type-not-at-end \ + -Wno-address-of-packed-member -Wno-tautological-compare \ + -Wno-unknown-warning-option \ + -I../include $(LINUXINCLUDE) \ + $(EXTRA_CFLAGS) -c $< -o -| $(LLC) -march=3Dbpf -filetype= =3Dobj -o $@ + python3 EbpfElf_to_C.py -f rss.bpf.o -s tun_rss_steering + diff --git a/ebpf/rss.bpf.c b/ebpf/rss.bpf.c new file mode 100644 index 0000000000..084fc33f96 --- /dev/null +++ b/ebpf/rss.bpf.c @@ -0,0 +1,470 @@ +#include +#include +#include + +#include +#include +#include +#include + +#include +#include + +#include +#include + +/* + * Prepare: + * Requires llvm, clang, python3 with pyelftools, linux kernel tree + * + * Build tun_rss_steering.h: + * make -f Mefile.ebpf clean all + */ + +#define INDIRECTION_TABLE_SIZE 128 +#define HASH_CALCULATION_BUFFER_SIZE 36 + +struct rss_config_t { + __u8 redirect; + __u8 populate_hash; + __u32 hash_types; + __u16 indirections_len; + __u16 default_queue; +}; + +struct toeplitz_key_data_t { + __u32 leftmost_32_bits; + __u8 next_byte[HASH_CALCULATION_BUFFER_SIZE]; +}; + +struct packet_hash_info_t { + __u8 is_ipv4; + __u8 is_ipv6; + __u8 is_udp; + __u8 is_tcp; + __u8 is_ipv6_ext_src; + __u8 is_ipv6_ext_dst; + + __u16 src_port; + __u16 dst_port; + + union { + struct { + __be32 in_src; + __be32 in_dst; + }; + + struct { + struct in6_addr in6_src; + struct in6_addr in6_dst; + struct in6_addr in6_ext_src; + struct in6_addr in6_ext_dst; + }; + }; +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, __u32); + __type(value, struct rss_config_t); + __uint(max_entries, 1); +} tap_rss_map_configurations SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, __u32); + __type(value, struct toeplitz_key_data_t); + __uint(max_entries, 1); +} tap_rss_map_toeplitz_key SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, __u32); + __type(value, __u16); + __uint(max_entries, INDIRECTION_TABLE_SIZE); +} tap_rss_map_indirection_table SEC(".maps"); + + +static inline void net_rx_rss_add_chunk(__u8 *rss_input, size_t *bytes_wri= tten, + const void *ptr, size_t size) { + __builtin_memcpy(&rss_input[*bytes_written], ptr, size); + *bytes_written +=3D size; +} + +static inline +void net_toeplitz_add(__u32 *result, + __u8 *input, + __u32 len + , struct toeplitz_key_data_t *key) { + + __u32 accumulator =3D *result; + __u32 leftmost_32_bits =3D key->leftmost_32_bits; + __u32 byte; + + for (byte =3D 0; byte < HASH_CALCULATION_BUFFER_SIZE; byte++) { + __u8 input_byte =3D input[byte]; + __u8 key_byte =3D key->next_byte[byte]; + __u8 bit; + + for (bit =3D 0; bit < 8; bit++) { + if (input_byte & (1 << 7)) { + accumulator ^=3D leftmost_32_bits; + } + + leftmost_32_bits =3D + (leftmost_32_bits << 1) | ((key_byte & (1 << 7)) >> 7); + + input_byte <<=3D 1; + key_byte <<=3D 1; + } + } + + *result =3D accumulator; +} + + +static inline int ip6_extension_header_type(__u8 hdr_type) +{ + switch (hdr_type) { + case IPPROTO_HOPOPTS: + case IPPROTO_ROUTING: + case IPPROTO_FRAGMENT: + case IPPROTO_ICMPV6: + case IPPROTO_NONE: + case IPPROTO_DSTOPTS: + case IPPROTO_MH: + return 1; + default: + return 0; + } +} +/* + * According to https://www.iana.org/assignments/ipv6-parameters/ipv6-para= meters.xhtml + * we suspect that there are would be no more than 11 extensions in IPv6 h= eader, + * also there is 27 TLV options for Destination and Hop-by-hop extensions. + * Need to choose reasonable amount of maximum extensions/options we may c= heck to find + * ext src/dst. + */ +#define IP6_EXTENSIONS_COUNT 11 +#define IP6_OPTIONS_COUNT 30 + +static inline void parse_ipv6_ext(struct __sk_buff *skb, + struct packet_hash_info_t *info, + __u8 *l4_protocol, size_t *l4_offset) +{ + if (!ip6_extension_header_type(*l4_protocol)) { + return; + } + + struct ipv6_opt_hdr ext_hdr =3D {}; + + for (unsigned int i =3D 0; i < IP6_EXTENSIONS_COUNT; ++i) { + + bpf_skb_load_bytes_relative(skb, *l4_offset, &ext_hdr, + sizeof(ext_hdr), BPF_HDR_START_NET); + + if (*l4_protocol =3D=3D IPPROTO_ROUTING) { + struct ipv6_rt_hdr ext_rt =3D {}; + + bpf_skb_load_bytes_relative(skb, *l4_offset, &ext_rt, + sizeof(ext_rt), BPF_HDR_START_NET); + + if ((ext_rt.type =3D=3D IPV6_SRCRT_TYPE_2) && + (ext_rt.hdrlen =3D=3D sizeof(struct in6_addr) / 8) && + (ext_rt.segments_left =3D=3D 1)) { + + bpf_skb_load_bytes_relative(skb, + *l4_offset + offsetof(struct rt2_hdr, addr), + &info->in6_ext_dst, sizeof(info->in6_ext_dst), + BPF_HDR_START_NET); + + info->is_ipv6_ext_dst =3D 1; + } + + } else if (*l4_protocol =3D=3D IPPROTO_DSTOPTS) { + struct ipv6_opt_t { + __u8 type; + __u8 length; + } __attribute__((packed)) opt =3D {}; + + size_t opt_offset =3D sizeof(ext_hdr); + + for (unsigned int j =3D 0; j < IP6_OPTIONS_COUNT; ++j) { + bpf_skb_load_bytes_relative(skb, *l4_offset + opt_offset, + &opt, sizeof(opt), BPF_HDR_START_N= ET); + + opt_offset +=3D (opt.type =3D=3D IPV6_TLV_PAD1) ? + 1 : opt.length + sizeof(opt); + + if (opt_offset + 1 >=3D ext_hdr.hdrlen * 8) { + break; + } + + if (opt.type =3D=3D IPV6_TLV_HAO) { + bpf_skb_load_bytes_relative(skb, + *l4_offset + opt_offset + offsetof(struct ipv6_des= topt_hao, addr), + &info->is_ipv6_ext_src, sizeof(info->is_ipv6_ext_s= rc), + BPF_HDR_START_NET); + + info->is_ipv6_ext_src =3D 1; + break; + } + } + } + + *l4_protocol =3D ext_hdr.nexthdr; + *l4_offset +=3D (ext_hdr.hdrlen + 1) * 8; + + if (!ip6_extension_header_type(ext_hdr.nexthdr)) { + return; + } + } +} + +static inline void parse_packet(struct __sk_buff *skb, + struct packet_hash_info_t *info) +{ + if (!info || !skb) { + return; + } + + size_t l4_offset =3D 0; + __u8 l4_protocol =3D 0; + __u16 l3_protocol =3D __be16_to_cpu(skb->protocol); + + if (l3_protocol =3D=3D ETH_P_IP) { + info->is_ipv4 =3D 1; + + struct iphdr ip =3D {}; + bpf_skb_load_bytes_relative(skb, 0, &ip, sizeof(ip), + BPF_HDR_START_NET); + + info->in_src =3D ip.saddr; + info->in_dst =3D ip.daddr; + + l4_protocol =3D ip.protocol; + l4_offset =3D ip.ihl * 4; + } else if (l3_protocol =3D=3D ETH_P_IPV6) { + info->is_ipv6 =3D 1; + + struct ipv6hdr ip6 =3D {}; + bpf_skb_load_bytes_relative(skb, 0, &ip6, sizeof(ip6), + BPF_HDR_START_NET); + + info->in6_src =3D ip6.saddr; + info->in6_dst =3D ip6.daddr; + + l4_protocol =3D ip6.nexthdr; + l4_offset =3D sizeof(ip6); + + parse_ipv6_ext(skb, info, &l4_protocol, &l4_offset); + } + + if (l4_protocol !=3D 0) { + if (l4_protocol =3D=3D IPPROTO_TCP) { + info->is_tcp =3D 1; + + struct tcphdr tcp =3D {}; + bpf_skb_load_bytes_relative(skb, l4_offset, &tcp, sizeof(tcp), + BPF_HDR_START_NET); + + info->src_port =3D tcp.source; + info->dst_port =3D tcp.dest; + } else if (l4_protocol =3D=3D IPPROTO_UDP) { /* TODO: add udplite?= */ + info->is_udp =3D 1; + + struct udphdr udp =3D {}; + bpf_skb_load_bytes_relative(skb, l4_offset, &udp, sizeof(udp), + BPF_HDR_START_NET); + + info->src_port =3D udp.source; + info->dst_port =3D udp.dest; + } + } +} + +static inline __u32 calculate_rss_hash(struct __sk_buff *skb, + struct rss_config_t *config, struct toeplitz_key_data_t *toe) +{ + __u8 rss_input[HASH_CALCULATION_BUFFER_SIZE] =3D {}; + size_t bytes_written =3D 0; + __u32 result =3D 0; + struct packet_hash_info_t packet_info =3D {}; + + parse_packet(skb, &packet_info); + + if (packet_info.is_ipv4) { + if (packet_info.is_tcp && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_TCPv4) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in_src, + sizeof(packet_info.in_src)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in_dst, + sizeof(packet_info.in_dst)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.src_port, + sizeof(packet_info.src_port)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.dst_port, + sizeof(packet_info.dst_port)); + } else if (packet_info.is_udp && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_UDPv4) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in_src, + sizeof(packet_info.in_src)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in_dst, + sizeof(packet_info.in_dst)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.src_port, + sizeof(packet_info.src_port)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.dst_port, + sizeof(packet_info.dst_port)); + } else if (config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_IPv4) { + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in_src, + sizeof(packet_info.in_src)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in_dst, + sizeof(packet_info.in_dst)); + } + } else if (packet_info.is_ipv6) { + if (packet_info.is_tcp && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_TCPv6) { + + if (packet_info.is_ipv6_ext_src && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_TCP_EX) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_ext_src, + sizeof(packet_info.in6_ext_src)); + } else { + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_src, + sizeof(packet_info.in6_src)); + } + if (packet_info.is_ipv6_ext_dst && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_TCP_EX) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_ext_dst, + sizeof(packet_info.in6_ext_dst)); + } else { + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_dst, + sizeof(packet_info.in6_dst)); + } + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.src_port, + sizeof(packet_info.src_port)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.dst_port, + sizeof(packet_info.dst_port)); + } else if (packet_info.is_udp && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_UDPv6) { + + if (packet_info.is_ipv6_ext_src && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_UDP_EX) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_ext_src, + sizeof(packet_info.in6_ext_src)); + } else { + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_src, + sizeof(packet_info.in6_src)); + } + if (packet_info.is_ipv6_ext_dst && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_UDP_EX) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_ext_dst, + sizeof(packet_info.in6_ext_dst)); + } else { + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_dst, + sizeof(packet_info.in6_dst)); + } + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.src_port, + sizeof(packet_info.src_port)); + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.dst_port, + sizeof(packet_info.dst_port)); + + } else if (config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_IPv6) { + if (packet_info.is_ipv6_ext_src && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_IP_EX) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_ext_src, + sizeof(packet_info.in6_ext_src)); + } else { + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_src, + sizeof(packet_info.in6_src)); + } + if (packet_info.is_ipv6_ext_dst && + config->hash_types & VIRTIO_NET_RSS_HASH_TYPE_IP_EX) { + + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_ext_dst, + sizeof(packet_info.in6_ext_dst)); + } else { + net_rx_rss_add_chunk(rss_input, &bytes_written, + &packet_info.in6_dst, + sizeof(packet_info.in6_dst)); + } + } + } + + if (bytes_written) { + net_toeplitz_add(&result, rss_input, bytes_written, toe); + } + + return result; +} + +SEC("tun_rss_steering") +int tun_rss_steering_prog(struct __sk_buff *skb) +{ + + struct rss_config_t *config =3D 0; + struct toeplitz_key_data_t *toe =3D 0; + + __u32 key =3D 0; + __u32 hash =3D 0; + + config =3D bpf_map_lookup_elem(&tap_rss_map_configurations, &key); + toe =3D bpf_map_lookup_elem(&tap_rss_map_toeplitz_key, &key); + + if (config && toe) { + if (!config->redirect) { + return config->default_queue; + } + + hash =3D calculate_rss_hash(skb, config, toe); + if (hash) { + __u32 table_idx =3D hash % config->indirections_len; + __u16 *queue =3D 0; + + queue =3D bpf_map_lookup_elem(&tap_rss_map_indirection_table, + &table_idx); + + if (queue) { + return *queue; + } + } + + return config->default_queue; + } + + return -1; +} + +char _license[] SEC("license") =3D "GPL"; diff --git a/ebpf/tun_rss_steering.h b/ebpf/tun_rss_steering.h new file mode 100644 index 0000000000..bbf63a109a --- /dev/null +++ b/ebpf/tun_rss_steering.h @@ -0,0 +1,556 @@ +#ifndef TUN_RSS_STEERING +#define TUN_RSS_STEERING + +struct bpf_insn instun_rss_steering[] =3D { + {0xbf, 0x09, 0x01, 0x0000, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff4c, 0x00000000}, + {0xbf, 0x06, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x06, 0x00, 0x0000, 0xffffff4c}, + {0x18, 0x01, 0x00, 0x0000, 0x00000000}, + {0x00, 0x00, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x02, 0x06, 0x0000, 0x00000000}, + {0x85, 0x00, 0x00, 0x0000, 0x00000001}, + {0xbf, 0x07, 0x00, 0x0000, 0x00000000}, + {0x18, 0x01, 0x00, 0x0000, 0x00000000}, + {0x00, 0x00, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x02, 0x06, 0x0000, 0x00000000}, + {0x85, 0x00, 0x00, 0x0000, 0x00000001}, + {0xbf, 0x08, 0x00, 0x0000, 0x00000000}, + {0x18, 0x00, 0x00, 0x0000, 0xffffffff}, + {0x00, 0x00, 0x00, 0x0000, 0x00000000}, + {0x15, 0x07, 0x00, 0x016e, 0x00000000}, + {0xbf, 0x05, 0x08, 0x0000, 0x00000000}, + {0x15, 0x05, 0x00, 0x016c, 0x00000000}, + {0x71, 0x01, 0x07, 0x0000, 0x00000000}, + {0x55, 0x01, 0x00, 0x0001, 0x00000000}, + {0x05, 0x00, 0x00, 0x0168, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x63, 0x0a, 0x01, 0xffc0, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffb8, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffb0, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffa8, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffa0, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff98, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff90, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff88, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff80, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff78, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff70, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff68, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff60, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff58, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff50, 0x00000000}, + {0x15, 0x09, 0x00, 0x007e, 0x00000000}, + {0x61, 0x01, 0x09, 0x0010, 0x00000000}, + {0xdc, 0x01, 0x00, 0x0000, 0x00000010}, + {0x15, 0x01, 0x00, 0x0030, 0x000086dd}, + {0x55, 0x01, 0x00, 0x007a, 0x00000800}, + {0x7b, 0x0a, 0x05, 0xff28, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000001}, + {0x73, 0x0a, 0x01, 0xff50, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x63, 0x0a, 0x01, 0xffe0, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffd8, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffd0, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffffd0}, + {0xbf, 0x01, 0x09, 0x0000, 0x00000000}, + {0xb7, 0x02, 0x00, 0x0000, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000014}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0x61, 0x01, 0x0a, 0xffdc, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff5c, 0x00000000}, + {0x61, 0x01, 0x0a, 0xffe0, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff60, 0x00000000}, + {0x71, 0x06, 0x0a, 0xffd9, 0x00000000}, + {0x71, 0x01, 0x0a, 0xffd0, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000002}, + {0x57, 0x01, 0x00, 0x0000, 0x0000003c}, + {0x7b, 0x0a, 0x01, 0xff40, 0x00000000}, + {0x57, 0x06, 0x00, 0x0000, 0x000000ff}, + {0x15, 0x06, 0x00, 0x0051, 0x00000011}, + {0x79, 0x05, 0x0a, 0xff28, 0x00000000}, + {0x55, 0x06, 0x00, 0x005f, 0x00000006}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000001}, + {0x73, 0x0a, 0x01, 0xff53, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x63, 0x0a, 0x01, 0xffe0, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffd8, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffd0, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffffd0}, + {0xbf, 0x01, 0x09, 0x0000, 0x00000000}, + {0x79, 0x02, 0x0a, 0xff40, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000014}, + {0xbf, 0x06, 0x05, 0x0000, 0x00000000}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0xbf, 0x05, 0x06, 0x0000, 0x00000000}, + {0x69, 0x01, 0x0a, 0xffd0, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xff56, 0x00000000}, + {0x69, 0x01, 0x0a, 0xffd2, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xff58, 0x00000000}, + {0x05, 0x00, 0x00, 0x004b, 0x00000000}, + {0x7b, 0x0a, 0x05, 0xff28, 0x00000000}, + {0x7b, 0x0a, 0x07, 0xff10, 0x00000000}, + {0xb7, 0x07, 0x00, 0x0000, 0x00000001}, + {0x73, 0x0a, 0x07, 0xff51, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xfff0, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffe8, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffe0, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffd8, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffd0, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffffd0}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000028}, + {0x7b, 0x0a, 0x01, 0xff40, 0x00000000}, + {0xbf, 0x01, 0x09, 0x0000, 0x00000000}, + {0xb7, 0x02, 0x00, 0x0000, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000028}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0x79, 0x01, 0x0a, 0xffd8, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff5c, 0x00000000}, + {0x77, 0x01, 0x00, 0x0000, 0x00000020}, + {0x63, 0x0a, 0x01, 0xff60, 0x00000000}, + {0x79, 0x01, 0x0a, 0xffe0, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff64, 0x00000000}, + {0x77, 0x01, 0x00, 0x0000, 0x00000020}, + {0x63, 0x0a, 0x01, 0xff68, 0x00000000}, + {0x79, 0x01, 0x0a, 0xffe8, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff6c, 0x00000000}, + {0x77, 0x01, 0x00, 0x0000, 0x00000020}, + {0x63, 0x0a, 0x01, 0xff70, 0x00000000}, + {0x79, 0x01, 0x0a, 0xfff0, 0x00000000}, + {0x63, 0x0a, 0x01, 0xff74, 0x00000000}, + {0x77, 0x01, 0x00, 0x0000, 0x00000020}, + {0x63, 0x0a, 0x01, 0xff78, 0x00000000}, + {0x71, 0x06, 0x0a, 0xffd6, 0x00000000}, + {0x25, 0x06, 0x00, 0x0126, 0x0000003c}, + {0x6f, 0x07, 0x06, 0x0000, 0x00000000}, + {0x18, 0x01, 0x00, 0x0000, 0x00000001}, + {0x00, 0x00, 0x00, 0x0000, 0x1c001800}, + {0x5f, 0x07, 0x01, 0x0000, 0x00000000}, + {0x55, 0x07, 0x00, 0x0001, 0x00000000}, + {0x05, 0x00, 0x00, 0x0120, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xfffe, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000028}, + {0x7b, 0x0a, 0x01, 0xff40, 0x00000000}, + {0xbf, 0x01, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x01, 0x00, 0x0000, 0xffffff8c}, + {0x7b, 0x0a, 0x01, 0xff20, 0x00000000}, + {0xbf, 0x01, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x01, 0x00, 0x0000, 0xffffff54}, + {0x7b, 0x0a, 0x01, 0xff18, 0x00000000}, + {0x18, 0x07, 0x00, 0x0000, 0x00000001}, + {0x00, 0x00, 0x00, 0x0000, 0x1c001800}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xff38, 0x00000000}, + {0x7b, 0x0a, 0x08, 0xff30, 0x00000000}, + {0x05, 0x00, 0x00, 0x0153, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000001}, + {0x73, 0x0a, 0x01, 0xff52, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffd0, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffffd0}, + {0xbf, 0x01, 0x09, 0x0000, 0x00000000}, + {0x79, 0x02, 0x0a, 0xff40, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000008}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0x69, 0x01, 0x0a, 0xffd0, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xff56, 0x00000000}, + {0x69, 0x01, 0x0a, 0xffd2, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xff58, 0x00000000}, + {0x79, 0x05, 0x0a, 0xff28, 0x00000000}, + {0x71, 0x01, 0x0a, 0xff50, 0x00000000}, + {0x15, 0x01, 0x00, 0x000f, 0x00000000}, + {0x61, 0x01, 0x07, 0x0004, 0x00000000}, + {0x71, 0x02, 0x0a, 0xff53, 0x00000000}, + {0x15, 0x02, 0x00, 0x002b, 0x00000000}, + {0xbf, 0x02, 0x01, 0x0000, 0x00000000}, + {0x57, 0x02, 0x00, 0x0000, 0x00000002}, + {0x15, 0x02, 0x00, 0x0028, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff5c, 0x00000000}, + {0x63, 0x0a, 0x01, 0xffa0, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff60, 0x00000000}, + {0x63, 0x0a, 0x01, 0xffa4, 0x00000000}, + {0x69, 0x01, 0x0a, 0xff56, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xffa8, 0x00000000}, + {0x69, 0x01, 0x0a, 0xff58, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xffaa, 0x00000000}, + {0x05, 0x00, 0x00, 0x005e, 0x00000000}, + {0x71, 0x01, 0x0a, 0xff51, 0x00000000}, + {0x15, 0x01, 0x00, 0x00c6, 0x00000000}, + {0x61, 0x01, 0x07, 0x0004, 0x00000000}, + {0x71, 0x02, 0x0a, 0xff53, 0x00000000}, + {0x15, 0x02, 0x00, 0x0027, 0x00000000}, + {0xbf, 0x02, 0x01, 0x0000, 0x00000000}, + {0x57, 0x02, 0x00, 0x0000, 0x00000010}, + {0x15, 0x02, 0x00, 0x0024, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffff5c}, + {0xbf, 0x02, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0xffffff7c}, + {0x71, 0x04, 0x0a, 0xff54, 0x00000000}, + {0x55, 0x04, 0x00, 0x0001, 0x00000000}, + {0xbf, 0x02, 0x03, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000038}, + {0xc7, 0x01, 0x00, 0x0000, 0x00000038}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000000}, + {0x6d, 0x04, 0x01, 0x0001, 0x00000000}, + {0xbf, 0x02, 0x03, 0x0000, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffff6c}, + {0xbf, 0x05, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x05, 0x00, 0x0000, 0xffffff8c}, + {0x6d, 0x04, 0x01, 0x0001, 0x00000000}, + {0xbf, 0x05, 0x03, 0x0000, 0x00000000}, + {0x71, 0x01, 0x0a, 0xff55, 0x00000000}, + {0x15, 0x01, 0x00, 0x0028, 0x00000000}, + {0xbf, 0x03, 0x05, 0x0000, 0x00000000}, + {0x05, 0x00, 0x00, 0x0026, 0x00000000}, + {0x71, 0x02, 0x0a, 0xff52, 0x00000000}, + {0x15, 0x02, 0x00, 0x0004, 0x00000000}, + {0xbf, 0x02, 0x01, 0x0000, 0x00000000}, + {0x57, 0x02, 0x00, 0x0000, 0x00000004}, + {0x15, 0x02, 0x00, 0x0001, 0x00000000}, + {0x05, 0x00, 0x00, 0xffd2, 0x00000000}, + {0x57, 0x01, 0x00, 0x0000, 0x00000001}, + {0x15, 0x01, 0x00, 0x00a1, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff5c, 0x00000000}, + {0x63, 0x0a, 0x01, 0xffa0, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff60, 0x00000000}, + {0x63, 0x0a, 0x01, 0xffa4, 0x00000000}, + {0x05, 0x00, 0x00, 0x0032, 0x00000000}, + {0x71, 0x02, 0x0a, 0xff52, 0x00000000}, + {0x15, 0x02, 0x00, 0x009e, 0x00000000}, + {0xbf, 0x02, 0x01, 0x0000, 0x00000000}, + {0x57, 0x02, 0x00, 0x0000, 0x00000020}, + {0x15, 0x02, 0x00, 0x009b, 0x00000000}, + {0xbf, 0x02, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0xffffff5c}, + {0x71, 0x04, 0x0a, 0xff54, 0x00000000}, + {0xbf, 0x03, 0x02, 0x0000, 0x00000000}, + {0x15, 0x04, 0x00, 0x0002, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffff7c}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x57, 0x01, 0x00, 0x0000, 0x00000100}, + {0x15, 0x01, 0x00, 0x0001, 0x00000000}, + {0xbf, 0x02, 0x03, 0x0000, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffff6c}, + {0x71, 0x05, 0x0a, 0xff55, 0x00000000}, + {0xbf, 0x04, 0x03, 0x0000, 0x00000000}, + {0x15, 0x05, 0x00, 0x0002, 0x00000000}, + {0xbf, 0x04, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x04, 0x00, 0x0000, 0xffffff8c}, + {0x15, 0x01, 0x00, 0x0001, 0x00000000}, + {0xbf, 0x03, 0x04, 0x0000, 0x00000000}, + {0x61, 0x01, 0x02, 0x0004, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x61, 0x04, 0x02, 0x0000, 0x00000000}, + {0x4f, 0x01, 0x04, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffa0, 0x00000000}, + {0x61, 0x01, 0x02, 0x0008, 0x00000000}, + {0x61, 0x02, 0x02, 0x000c, 0x00000000}, + {0x67, 0x02, 0x00, 0x0000, 0x00000020}, + {0x4f, 0x02, 0x01, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x02, 0xffa8, 0x00000000}, + {0x61, 0x01, 0x03, 0x0000, 0x00000000}, + {0x61, 0x02, 0x03, 0x0004, 0x00000000}, + {0x61, 0x04, 0x03, 0x0008, 0x00000000}, + {0x61, 0x03, 0x03, 0x000c, 0x00000000}, + {0x69, 0x05, 0x0a, 0xff58, 0x00000000}, + {0x6b, 0x0a, 0x05, 0xffc2, 0x00000000}, + {0x69, 0x05, 0x0a, 0xff56, 0x00000000}, + {0x6b, 0x0a, 0x05, 0xffc0, 0x00000000}, + {0x67, 0x03, 0x00, 0x0000, 0x00000020}, + {0x4f, 0x03, 0x04, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x03, 0xffb8, 0x00000000}, + {0x67, 0x02, 0x00, 0x0000, 0x00000020}, + {0x4f, 0x02, 0x01, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x02, 0xffb0, 0x00000000}, + {0xbf, 0x05, 0x00, 0x0000, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x07, 0x08, 0x00, 0x0000, 0x00000004}, + {0x61, 0x03, 0x05, 0x0000, 0x00000000}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x02, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0xffffffa0}, + {0x0f, 0x02, 0x01, 0x0000, 0x00000000}, + {0x71, 0x04, 0x02, 0x0000, 0x00000000}, + {0xbf, 0x02, 0x04, 0x0000, 0x00000000}, + {0x67, 0x02, 0x00, 0x0000, 0x00000038}, + {0xc7, 0x02, 0x00, 0x0000, 0x0000003f}, + {0x5f, 0x02, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x05, 0x0000, 0x00000000}, + {0xbf, 0x05, 0x08, 0x0000, 0x00000000}, + {0x0f, 0x05, 0x01, 0x0000, 0x00000000}, + {0x71, 0x05, 0x05, 0x0000, 0x00000000}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x77, 0x00, 0x00, 0x0000, 0x00000007}, + {0x4f, 0x03, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x04, 0x0000, 0x00000000}, + {0x67, 0x00, 0x00, 0x0000, 0x00000039}, + {0xc7, 0x00, 0x00, 0x0000, 0x0000003f}, + {0x5f, 0x00, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x77, 0x00, 0x00, 0x0000, 0x00000006}, + {0x57, 0x00, 0x00, 0x0000, 0x00000001}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0x4f, 0x03, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x04, 0x0000, 0x00000000}, + {0x67, 0x00, 0x00, 0x0000, 0x0000003a}, + {0xc7, 0x00, 0x00, 0x0000, 0x0000003f}, + {0x5f, 0x00, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x00, 0x0000, 0x00000000}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x77, 0x00, 0x00, 0x0000, 0x00000005}, + {0x57, 0x00, 0x00, 0x0000, 0x00000001}, + {0x4f, 0x03, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x04, 0x0000, 0x00000000}, + {0x67, 0x00, 0x00, 0x0000, 0x0000003b}, + {0xc7, 0x00, 0x00, 0x0000, 0x0000003f}, + {0x5f, 0x00, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x00, 0x0000, 0x00000000}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x77, 0x00, 0x00, 0x0000, 0x00000004}, + {0x57, 0x00, 0x00, 0x0000, 0x00000001}, + {0x4f, 0x03, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x04, 0x0000, 0x00000000}, + {0x67, 0x00, 0x00, 0x0000, 0x0000003c}, + {0xc7, 0x00, 0x00, 0x0000, 0x0000003f}, + {0x5f, 0x00, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x77, 0x00, 0x00, 0x0000, 0x00000003}, + {0x57, 0x00, 0x00, 0x0000, 0x00000001}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0x4f, 0x03, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x04, 0x0000, 0x00000000}, + {0x67, 0x00, 0x00, 0x0000, 0x0000003d}, + {0xc7, 0x00, 0x00, 0x0000, 0x0000003f}, + {0x5f, 0x00, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x77, 0x00, 0x00, 0x0000, 0x00000002}, + {0x57, 0x00, 0x00, 0x0000, 0x00000001}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0x4f, 0x03, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x04, 0x0000, 0x00000000}, + {0x67, 0x00, 0x00, 0x0000, 0x0000003e}, + {0xc7, 0x00, 0x00, 0x0000, 0x0000003f}, + {0x5f, 0x00, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x00, 0x0000, 0x00000000}, + {0xbf, 0x00, 0x05, 0x0000, 0x00000000}, + {0x77, 0x00, 0x00, 0x0000, 0x00000001}, + {0x57, 0x00, 0x00, 0x0000, 0x00000001}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0x4f, 0x03, 0x00, 0x0000, 0x00000000}, + {0x57, 0x04, 0x00, 0x0000, 0x00000001}, + {0x87, 0x04, 0x00, 0x0000, 0x00000000}, + {0x5f, 0x04, 0x03, 0x0000, 0x00000000}, + {0xaf, 0x02, 0x04, 0x0000, 0x00000000}, + {0x57, 0x05, 0x00, 0x0000, 0x00000001}, + {0x67, 0x03, 0x00, 0x0000, 0x00000001}, + {0x4f, 0x03, 0x05, 0x0000, 0x00000000}, + {0x07, 0x01, 0x00, 0x0000, 0x00000001}, + {0xbf, 0x05, 0x02, 0x0000, 0x00000000}, + {0x15, 0x01, 0x00, 0x0001, 0x00000024}, + {0x05, 0x00, 0x00, 0xffa9, 0x00000000}, + {0xbf, 0x01, 0x02, 0x0000, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x77, 0x01, 0x00, 0x0000, 0x00000020}, + {0x15, 0x01, 0x00, 0x000b, 0x00000000}, + {0x69, 0x03, 0x07, 0x0008, 0x00000000}, + {0x3f, 0x01, 0x03, 0x0000, 0x00000000}, + {0x2f, 0x01, 0x03, 0x0000, 0x00000000}, + {0x1f, 0x02, 0x01, 0x0000, 0x00000000}, + {0x63, 0x0a, 0x02, 0xff50, 0x00000000}, + {0xbf, 0x02, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0xffffff50}, + {0x18, 0x01, 0x00, 0x0000, 0x00000000}, + {0x00, 0x00, 0x00, 0x0000, 0x00000000}, + {0x85, 0x00, 0x00, 0x0000, 0x00000001}, + {0x55, 0x00, 0x00, 0x0002, 0x00000000}, + {0x69, 0x00, 0x07, 0x000a, 0x00000000}, + {0x95, 0x00, 0x00, 0x0000, 0x00000000}, + {0x69, 0x00, 0x00, 0x0000, 0x00000000}, + {0x05, 0x00, 0x00, 0xfffd, 0x00000000}, + {0xbf, 0x02, 0x01, 0x0000, 0x00000000}, + {0x57, 0x02, 0x00, 0x0000, 0x00000008}, + {0x15, 0x02, 0x00, 0xfff9, 0x00000000}, + {0xbf, 0x02, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0xffffff5c}, + {0x71, 0x04, 0x0a, 0xff54, 0x00000000}, + {0xbf, 0x03, 0x02, 0x0000, 0x00000000}, + {0x15, 0x04, 0x00, 0x0002, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xffffff7c}, + {0x57, 0x01, 0x00, 0x0000, 0x00000040}, + {0x15, 0x01, 0x00, 0x0001, 0x00000000}, + {0xbf, 0x02, 0x03, 0x0000, 0x00000000}, + {0x61, 0x03, 0x02, 0x0004, 0x00000000}, + {0x67, 0x03, 0x00, 0x0000, 0x00000020}, + {0x61, 0x04, 0x02, 0x0000, 0x00000000}, + {0x4f, 0x03, 0x04, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x03, 0xffa0, 0x00000000}, + {0x61, 0x03, 0x02, 0x0008, 0x00000000}, + {0x61, 0x02, 0x02, 0x000c, 0x00000000}, + {0x67, 0x02, 0x00, 0x0000, 0x00000020}, + {0x4f, 0x02, 0x03, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x02, 0xffa8, 0x00000000}, + {0x15, 0x01, 0x00, 0x0079, 0x00000000}, + {0x71, 0x01, 0x0a, 0xff55, 0x00000000}, + {0x15, 0x01, 0x00, 0x0077, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff98, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x61, 0x02, 0x0a, 0xff94, 0x00000000}, + {0x4f, 0x01, 0x02, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffb8, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff90, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x61, 0x02, 0x0a, 0xff8c, 0x00000000}, + {0x05, 0x00, 0x00, 0x0076, 0x00000000}, + {0x15, 0x06, 0x00, 0xfedf, 0x00000087}, + {0x05, 0x00, 0x00, 0x003f, 0x00000000}, + {0x0f, 0x06, 0x09, 0x0000, 0x00000000}, + {0xbf, 0x02, 0x06, 0x0000, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0x00000001}, + {0x71, 0x03, 0x0a, 0xffff, 0x00000000}, + {0x67, 0x03, 0x00, 0x0000, 0x00000003}, + {0x3d, 0x02, 0x03, 0x0022, 0x00000000}, + {0x55, 0x01, 0x00, 0x000c, 0x000000c9}, + {0x79, 0x01, 0x0a, 0xff40, 0x00000000}, + {0x0f, 0x06, 0x01, 0x0000, 0x00000000}, + {0x07, 0x06, 0x00, 0x0000, 0x00000002}, + {0xbf, 0x01, 0x07, 0x0000, 0x00000000}, + {0xbf, 0x02, 0x06, 0x0000, 0x00000000}, + {0x79, 0x03, 0x0a, 0xff18, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000001}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000001}, + {0x73, 0x0a, 0x01, 0xff54, 0x00000000}, + {0x05, 0x00, 0x00, 0x0015, 0x00000000}, + {0x07, 0x08, 0x00, 0x0000, 0xffffffff}, + {0xbf, 0x01, 0x08, 0x0000, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x77, 0x01, 0x00, 0x0000, 0x00000020}, + {0xbf, 0x09, 0x06, 0x0000, 0x00000000}, + {0x15, 0x01, 0x00, 0x000f, 0x00000000}, + {0xbf, 0x02, 0x09, 0x0000, 0x00000000}, + {0x79, 0x01, 0x0a, 0xff40, 0x00000000}, + {0x0f, 0x02, 0x01, 0x0000, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xfffffff8}, + {0xb7, 0x06, 0x00, 0x0000, 0x00000001}, + {0xbf, 0x01, 0x07, 0x0000, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000002}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0x71, 0x01, 0x0a, 0xfff8, 0x00000000}, + {0x15, 0x01, 0x00, 0xffdb, 0x00000000}, + {0x71, 0x06, 0x0a, 0xfff9, 0x00000000}, + {0x07, 0x06, 0x00, 0x0000, 0x00000002}, + {0x05, 0x00, 0x00, 0xffd8, 0x00000000}, + {0x79, 0x08, 0x0a, 0xff30, 0x00000000}, + {0xbf, 0x09, 0x07, 0x0000, 0x00000000}, + {0x18, 0x07, 0x00, 0x0000, 0x00000001}, + {0x00, 0x00, 0x00, 0x0000, 0x1c001800}, + {0x71, 0x01, 0x0a, 0xffff, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000003}, + {0x79, 0x02, 0x0a, 0xff40, 0x00000000}, + {0x0f, 0x02, 0x01, 0x0000, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0x00000008}, + {0x7b, 0x0a, 0x02, 0xff40, 0x00000000}, + {0x71, 0x06, 0x0a, 0xfffe, 0x00000000}, + {0x25, 0x06, 0x00, 0x0036, 0x0000003c}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000001}, + {0x6f, 0x01, 0x06, 0x0000, 0x00000000}, + {0x5f, 0x01, 0x07, 0x0000, 0x00000000}, + {0x55, 0x01, 0x00, 0x0001, 0x00000000}, + {0x05, 0x00, 0x00, 0x0031, 0x00000000}, + {0x79, 0x01, 0x0a, 0xff38, 0x00000000}, + {0x07, 0x01, 0x00, 0x0000, 0x00000001}, + {0x7b, 0x0a, 0x01, 0xff38, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x77, 0x01, 0x00, 0x0000, 0x00000020}, + {0x55, 0x01, 0x00, 0x0002, 0x0000000b}, + {0x79, 0x07, 0x0a, 0xff10, 0x00000000}, + {0x05, 0x00, 0x00, 0xfe5a, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xfffffffe}, + {0xbf, 0x01, 0x09, 0x0000, 0x00000000}, + {0x79, 0x02, 0x0a, 0xff40, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000002}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0xbf, 0x01, 0x06, 0x0000, 0x00000000}, + {0x15, 0x01, 0x00, 0x001a, 0x0000003c}, + {0x55, 0x01, 0x00, 0xffe1, 0x0000002b}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x63, 0x0a, 0x01, 0xfff8, 0x00000000}, + {0xbf, 0x03, 0x0a, 0x0000, 0x00000000}, + {0x07, 0x03, 0x00, 0x0000, 0xfffffff8}, + {0xbf, 0x01, 0x09, 0x0000, 0x00000000}, + {0x79, 0x02, 0x0a, 0xff40, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000004}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0x71, 0x01, 0x0a, 0xfffa, 0x00000000}, + {0x55, 0x01, 0x00, 0xffd6, 0x00000002}, + {0x71, 0x01, 0x0a, 0xfff9, 0x00000000}, + {0x55, 0x01, 0x00, 0xffd4, 0x00000002}, + {0x71, 0x01, 0x0a, 0xfffb, 0x00000000}, + {0x55, 0x01, 0x00, 0xffd2, 0x00000001}, + {0x79, 0x02, 0x0a, 0xff40, 0x00000000}, + {0x07, 0x02, 0x00, 0x0000, 0x00000008}, + {0xbf, 0x01, 0x09, 0x0000, 0x00000000}, + {0x79, 0x03, 0x0a, 0xff20, 0x00000000}, + {0xb7, 0x04, 0x00, 0x0000, 0x00000010}, + {0xb7, 0x05, 0x00, 0x0000, 0x00000001}, + {0x85, 0x00, 0x00, 0x0000, 0x00000044}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000001}, + {0x73, 0x0a, 0x01, 0xff55, 0x00000000}, + {0x05, 0x00, 0x00, 0xffc8, 0x00000000}, + {0xbf, 0x07, 0x09, 0x0000, 0x00000000}, + {0xb7, 0x01, 0x00, 0x0000, 0x00000000}, + {0x6b, 0x0a, 0x01, 0xfff8, 0x00000000}, + {0xb7, 0x09, 0x00, 0x0000, 0x00000002}, + {0xb7, 0x08, 0x00, 0x0000, 0x0000001e}, + {0x05, 0x00, 0x00, 0xffaf, 0x00000000}, + {0x15, 0x06, 0x00, 0xffce, 0x00000087}, + {0x05, 0x00, 0x00, 0xffd3, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff78, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x61, 0x02, 0x0a, 0xff74, 0x00000000}, + {0x4f, 0x01, 0x02, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffb8, 0x00000000}, + {0x61, 0x01, 0x0a, 0xff70, 0x00000000}, + {0x67, 0x01, 0x00, 0x0000, 0x00000020}, + {0x61, 0x02, 0x0a, 0xff6c, 0x00000000}, + {0x4f, 0x01, 0x02, 0x0000, 0x00000000}, + {0x7b, 0x0a, 0x01, 0xffb0, 0x00000000}, + {0x05, 0x00, 0x00, 0xfef6, 0x00000000}, +}; + +struct fixup_mapfd_t reltun_rss_steering[] =3D { + {"tap_rss_map_configurations", 5}, + {"tap_rss_map_toeplitz_key", 10}, + {"tap_rss_map_indirection_table", 379}, +}; + +#endif /* TUN_RSS_STEERING */ --=20 2.28.0 From nobody Fri May 17 22:14:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604341462; cv=none; d=zohomail.com; s=zohoarc; b=ej5Qdcc1fHF9pngQi2lZY1wep8RgnDfcR7Aivl9jADKOgOBYQLIzUEcoapm77PUQT7q+phVLdh5uPf+rZwM79Wgi61blv5SlJIping9Zwqx0rmrjGwCjWzhxoSh1Zufuw6k3iK+dcxMk1P+37I2m/fjVJRUQWWV5q0imBjiHuik= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604341462; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=WUVUKv4PYWLLTid/U5M6cN4MwS3ke7uD6rvIMKkcOa8=; b=OF1j7EqEwtKQ0LTI934bhdQqwiV94PgAvFYknFv+WPqbCuzF8xd3h7Ltjcj/mdxyNKn3eFs7qXVxT7nXWDXGE3qhBaU0lVVQQ1Y62veIJ+uKOg766NewgAI+F1eh3yt2L4gxOEBaBFki3Os5X/4B8lAFnrZBM3JBULJZ2WM7krE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 160434146231748.54059675245321; Mon, 2 Nov 2020 10:24:22 -0800 (PST) Received: from localhost ([::1]:53816 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kZeVB-0006d1-4t for importer@patchew.org; Mon, 02 Nov 2020 13:24:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kZeR3-0001YT-TF for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:05 -0500 Received: from mail-lf1-x144.google.com ([2a00:1450:4864:20::144]:37762) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kZeQw-0006iJ-8j for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:05 -0500 Received: by mail-lf1-x144.google.com with SMTP id j30so18671363lfp.4 for ; Mon, 02 Nov 2020 10:19:57 -0800 (PST) Received: from navi.cosmonova.net.ua ([95.67.24.131]) by smtp.gmail.com with ESMTPSA id c6sm2527007lfm.226.2020.11.02.10.19.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 10:19:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WUVUKv4PYWLLTid/U5M6cN4MwS3ke7uD6rvIMKkcOa8=; b=am3c7YU6Zc6q27BtqVlk9sOu7yFbX37F41tlp/oGYRUw1ULx/NXqDtrr+1NRdKK+Cp sPWS0rl/+TpycG/vbHKZjtQ7aYF5mscdWTDjl99m0V+EA0IhicM7EvDhEuQBEBQrLgtx ZdKlVp16NwVA/CvmInU5cAgvxq/XcBxSB+HxJbmoDrABderaSj1uqsDlVB0dXmS3Z92Z T6hup70vA60BXsJkB2T3a8+UEoDsXpCKk/yA24nA7BJys8PDoq23m55y5zKJ+uW/TNPM zgTvoO6BvkeSOpBA1VRTV5r0hymMiOsxKJi7BWUpH6bP3FL99d97K1ktBQVe8TXYmPec GdOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WUVUKv4PYWLLTid/U5M6cN4MwS3ke7uD6rvIMKkcOa8=; b=iDh1FNxrGUFuWtKXUjLPQMmXj/g80QqhSbeOhXK7KgRknzEEPVwIlmICI4zqK28sCz hBIiTIf7n1JCOyRgqQLltAOccgn/jM+goV6Iih5BjwCx6zBU9QLsHFyhC9vf1EyX1D76 hBG/yRHA0AEkEkhyM/Nplya7ZKm1k1xVtftB1cBbsFEB1PDbM9LyxE2gZQU/bsxK2C7b RYRJjlJMcg4DJxJtDKjY/KdyTdoChVKDhHAEXZfVN6kqF09RM/p8UHD+EHD2mti+VYgn tQAItg55Od3kw6g+QWqR0jxHTH4QPsqLPPgPh7zWg48C2mAgprg4ylHxQucpc/GpBjmR AbYQ== X-Gm-Message-State: AOAM530SXD9bpwAwi1O7YnXxFv074J+U2v9cYH2PppjojLbeIXjhgvmK f4vqT8xN54QAImEtdHuzJty3FA== X-Google-Smtp-Source: ABdhPJym5PpktOhe1HYFmkMhYeEYy6d77PYukIjPHukWIXViZyS/mlocpiy0CaNs5onurd+DNuzROA== X-Received: by 2002:a05:6512:210f:: with SMTP id q15mr5756726lfr.78.1604341195879; Mon, 02 Nov 2020 10:19:55 -0800 (PST) From: Andrew Melnychenko To: jasowang@redhat.com, mst@redhat.com Subject: [RFC PATCH 4/6] ebpf: Added eBPF RSS loader. Date: Mon, 2 Nov 2020 20:51:14 +0200 Message-Id: <20201102185115.7425-5-andrew@daynix.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201102185115.7425-1-andrew@daynix.com> References: <20201102185115.7425-1-andrew@daynix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=2a00:1450:4864:20::144; envelope-from=andrew@daynix.com; helo=mail-lf1-x144.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yan@daynix.com, yuri.benditovich@daynix.com, Andrew , Sameeh Jubran , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrew Added function that loads RSS eBPF program. Added stub functions for RSS eBPF. Added meson and configuration options. Signed-off-by: Sameeh Jubran Signed-off-by: Yuri Benditovich Signed-off-by: Andrew Melnychenko --- configure | 36 ++++++++++ ebpf/ebpf-stub.c | 28 ++++++++ ebpf/ebpf_rss.c | 178 +++++++++++++++++++++++++++++++++++++++++++++++ ebpf/ebpf_rss.h | 30 ++++++++ ebpf/meson.build | 1 + meson.build | 3 + 6 files changed, 276 insertions(+) create mode 100644 ebpf/ebpf-stub.c create mode 100644 ebpf/ebpf_rss.c create mode 100644 ebpf/ebpf_rss.h create mode 100644 ebpf/meson.build diff --git a/configure b/configure index 6df4306c88..bae4ea54f8 100755 --- a/configure +++ b/configure @@ -330,6 +330,7 @@ vhost_scsi=3D"" vhost_vsock=3D"" vhost_user=3D"" vhost_user_fs=3D"" +bpf=3D"" kvm=3D"auto" hax=3D"auto" hvf=3D"auto" @@ -1210,6 +1211,10 @@ for opt do ;; --enable-membarrier) membarrier=3D"yes" ;; + --disable-bpf) bpf=3D"no" + ;; + --enable-bpf) bpf=3D"yes" + ;; --disable-blobs) blobs=3D"false" ;; --with-pkgversion=3D*) pkgversion=3D"$optarg" @@ -1792,6 +1797,7 @@ disabled with --disable-FEATURE, default is enabled i= f available: vhost-kernel vhost kernel backend support vhost-user vhost-user backend support vhost-vdpa vhost-vdpa kernel backend support + bpf BPF kernel support spice spice rbd rados block device (rbd) libiscsi iscsi support @@ -5347,6 +5353,33 @@ else membarrier=3Dno fi =20 +########################################## +# check for usable bpf system call +if test "$bpf" =3D ""; then + have_bpf=3Dno + if test "$linux" =3D "yes" ; then + cat > $TMPC << EOF + #include + #include + #include + #include + #include + int main(void) { + union bpf_attr * attr =3D NULL; + syscall(__NR_bpf, BPF_PROG_LOAD, attr, sizeof(attr)); + exit(0); + } +EOF + if compile_prog "" "" ; then + have_bpf=3Dyes + bpf=3Dyes + fi + fi + if test "$have_bpf" =3D "no"; then + feature_not_found "bpf" "the bpf system call is not available" + fi +fi + ########################################## # check if rtnetlink.h exists and is useful have_rtnetlink=3Dno @@ -6279,6 +6312,9 @@ fi if test "$membarrier" =3D "yes" ; then echo "CONFIG_MEMBARRIER=3Dy" >> $config_host_mak fi +if test "$bpf" =3D "yes" -a "$bigendian" !=3D "yes"; then + echo "CONFIG_EBPF=3Dy" >> $config_host_mak +fi if test "$signalfd" =3D "yes" ; then echo "CONFIG_SIGNALFD=3Dy" >> $config_host_mak fi diff --git a/ebpf/ebpf-stub.c b/ebpf/ebpf-stub.c new file mode 100644 index 0000000000..281dc039d3 --- /dev/null +++ b/ebpf/ebpf-stub.c @@ -0,0 +1,28 @@ +#include "qemu/osdep.h" +#include "ebpf/ebpf_rss.h" + +void ebpf_rss_init(struct EBPFRSSContext *ctx) +{ + +} + +bool ebpf_rss_is_loaded(struct EBPFRSSContext *ctx) +{ + return false; +} + +bool ebpf_rss_load(struct EBPFRSSContext *ctx) +{ + return false; +} + +bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *co= nfig, + uint16_t *indirections_table, uint8_t *toeplitz_key) +{ + return false; +} + +void ebpf_rss_unload(struct EBPFRSSContext *ctx) +{ + +} diff --git a/ebpf/ebpf_rss.c b/ebpf/ebpf_rss.c new file mode 100644 index 0000000000..f3c948a7a0 --- /dev/null +++ b/ebpf/ebpf_rss.c @@ -0,0 +1,178 @@ +#include "qemu/osdep.h" +#include "qemu/error-report.h" + +#include "hw/virtio/virtio-net.h" /* VIRTIO_NET_RSS_MAX_TABLE_LEN */ + +#include "ebpf/ebpf_rss.h" +#include "ebpf/ebpf.h" +#include "ebpf/tun_rss_steering.h" +#include "trace.h" + +void ebpf_rss_init(struct EBPFRSSContext *ctx) +{ + if (ctx !=3D NULL) { + ctx->program_fd =3D -1; + } +} + +bool ebpf_rss_is_loaded(struct EBPFRSSContext *ctx) +{ + return ctx !=3D NULL && ctx->program_fd >=3D 0; +} + +bool ebpf_rss_load(struct EBPFRSSContext *ctx) +{ + if (ctx =3D=3D NULL) { + return false; + } + + ctx->map_configuration =3D + bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(uint32_t), + sizeof(struct EBPFRSSConfig), 1); + if (ctx->map_configuration < 0) { + trace_ebpf_error("eBPF RSS", "can not create MAP for configuration= s"); + goto l_conf_create; + } + ctx->map_toeplitz_key =3D + bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(uint32_t), + VIRTIO_NET_RSS_MAX_KEY_SIZE, 1); + if (ctx->map_toeplitz_key < 0) { + trace_ebpf_error("eBPF RSS", "can not create MAP for toeplitz key"= ); + goto l_toe_create; + } + + ctx->map_indirections_table =3D + bpf_create_map(BPF_MAP_TYPE_ARRAY, sizeof(uint32_t), + sizeof(uint16_t), VIRTIO_NET_RSS_MAX_TABLE_LEN); + if (ctx->map_indirections_table < 0) { + trace_ebpf_error("eBPF RSS", "can not create MAP for indirections = table"); + goto l_table_create; + } + + bpf_fixup_mapfd(reltun_rss_steering, + sizeof(reltun_rss_steering) / sizeof(struct fixup_mapfd_t), + instun_rss_steering, + sizeof(instun_rss_steering) / sizeof(struct bpf_insn), + "tap_rss_map_configurations", ctx->map_configuration); + + bpf_fixup_mapfd(reltun_rss_steering, + sizeof(reltun_rss_steering) / sizeof(struct fixup_mapfd_t), + instun_rss_steering, + sizeof(instun_rss_steering) / sizeof(struct bpf_insn), + "tap_rss_map_toeplitz_key", ctx->map_toeplitz_key); + + bpf_fixup_mapfd(reltun_rss_steering, + sizeof(reltun_rss_steering) / sizeof(struct fixup_mapfd_t), + instun_rss_steering, + sizeof(instun_rss_steering) / sizeof(struct bpf_insn), + "tap_rss_map_indirection_table", ctx->map_indirections_table); + + ctx->program_fd =3D + bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, instun_rss_steering, + sizeof(instun_rss_steering) / sizeof(struct bpf_i= nsn), + "GPL"); + if (ctx->program_fd < 0) { + trace_ebpf_error("eBPF RSS", "can not load eBPF program"); + goto l_prog_load; + } + + return true; +l_prog_load: + close(ctx->map_indirections_table); +l_table_create: + close(ctx->map_toeplitz_key); +l_toe_create: + close(ctx->map_configuration); +l_conf_create: + return false; +} + +static bool ebpf_rss_set_config(struct EBPFRSSContext *ctx, + struct EBPFRSSConfig *config) +{ + if (!ebpf_rss_is_loaded(ctx)) { + return false; + } + uint32_t map_key =3D 0; + if (bpf_update_elem(ctx->map_configuration, + &map_key, config, BPF_ANY) < 0) { + return false; + } + return true; +} + +static bool ebpf_rss_set_indirections_table(struct EBPFRSSContext *ctx, + uint16_t *indirections_table, + size_t len) +{ + if (!ebpf_rss_is_loaded(ctx) || indirections_table =3D=3D NULL || + len > VIRTIO_NET_RSS_MAX_TABLE_LEN) { + return false; + } + uint32_t i =3D 0; + + for (; i < len; ++i) { + if (bpf_update_elem(ctx->map_indirections_table, &i, + indirections_table + i, BPF_ANY) < 0) { + return false; + } + } + return true; +} + +static bool ebpf_rss_set_toepliz_key(struct EBPFRSSContext *ctx, + uint8_t *toeplitz_key) +{ + if (!ebpf_rss_is_loaded(ctx) || toeplitz_key =3D=3D NULL) { + return false; + } + uint32_t map_key =3D 0; + + /* prepare toeplitz key */ + uint8_t toe[VIRTIO_NET_RSS_MAX_KEY_SIZE] =3D {}; + memcpy(toe, toeplitz_key, VIRTIO_NET_RSS_MAX_KEY_SIZE); + *(uint32_t *)toe =3D ntohl(*(uint32_t *)toe); + + if (bpf_update_elem(ctx->map_toeplitz_key, &map_key, toe, + BPF_ANY) < 0) { + return false; + } + return true; +} + +bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *co= nfig, + uint16_t *indirections_table, uint8_t *toeplitz_key) +{ + if (!ebpf_rss_is_loaded(ctx) || config =3D=3D NULL || + indirections_table =3D=3D NULL || toeplitz_key =3D=3D NULL) { + return false; + } + + if (!ebpf_rss_set_config(ctx, config)) { + return false; + } + + if (!ebpf_rss_set_indirections_table(ctx, indirections_table, + config->indirections_len)) { + return false; + } + + if (!ebpf_rss_set_toepliz_key(ctx, toeplitz_key)) { + return false; + } + + return true; +} + +void ebpf_rss_unload(struct EBPFRSSContext *ctx) +{ + if (!ebpf_rss_is_loaded(ctx)) { + return; + } + + close(ctx->program_fd); + close(ctx->map_configuration); + close(ctx->map_toeplitz_key); + close(ctx->map_indirections_table); + ctx->program_fd =3D -1; +} diff --git a/ebpf/ebpf_rss.h b/ebpf/ebpf_rss.h new file mode 100644 index 0000000000..ffed7b571a --- /dev/null +++ b/ebpf/ebpf_rss.h @@ -0,0 +1,30 @@ +#ifndef QEMU_EBPF_RSS_H +#define QEMU_EBPF_RSS_H + +struct EBPFRSSContext { + int program_fd; + int map_configuration; + int map_toeplitz_key; + int map_indirections_table; +}; + +struct EBPFRSSConfig { + uint8_t redirect; + uint8_t populate_hash; + uint32_t hash_types; + uint16_t indirections_len; + uint16_t default_queue; +}; + +void ebpf_rss_init(struct EBPFRSSContext *ctx); + +bool ebpf_rss_is_loaded(struct EBPFRSSContext *ctx); + +bool ebpf_rss_load(struct EBPFRSSContext *ctx); + +bool ebpf_rss_set_all(struct EBPFRSSContext *ctx, struct EBPFRSSConfig *co= nfig, + uint16_t *indirections_table, uint8_t *toeplitz_key); + +void ebpf_rss_unload(struct EBPFRSSContext *ctx); + +#endif /* QEMU_EBPF_RSS_H */ diff --git a/ebpf/meson.build b/ebpf/meson.build new file mode 100644 index 0000000000..10f4bc9ca8 --- /dev/null +++ b/ebpf/meson.build @@ -0,0 +1 @@ +specific_ss.add(when: 'CONFIG_EBPF', if_true: files('ebpf_rss.c', 'ebpf.c'= ), if_false: files('ebpf-stub.c')) diff --git a/meson.build b/meson.build index 47e32e1fcb..d0ea1a0e9d 100644 --- a/meson.build +++ b/meson.build @@ -1368,6 +1368,7 @@ if have_system 'backends', 'backends/tpm', 'chardev', + 'ebpf', 'hw/9pfs', 'hw/acpi', 'hw/alpha', @@ -1530,6 +1531,7 @@ subdir('accel') subdir('plugins') subdir('bsd-user') subdir('linux-user') +subdir('ebpf') =20 bsd_user_ss.add(files('gdbstub.c')) specific_ss.add_all(when: 'CONFIG_BSD_USER', if_true: bsd_user_ss) @@ -2093,6 +2095,7 @@ summary_info +=3D {'vhost-vsock support': config_host= .has_key('CONFIG_VHOST_VSOCK' summary_info +=3D {'vhost-user support': config_host.has_key('CONFIG_VHOST= _KERNEL')} summary_info +=3D {'vhost-user-fs support': config_host.has_key('CONFIG_VH= OST_USER_FS')} summary_info +=3D {'vhost-vdpa support': config_host.has_key('CONFIG_VHOST= _VDPA')} +summary_info +=3D {'bpf support': config_host.has_key('CONFIG_EBPF')} summary_info +=3D {'Trace backends': config_host['TRACE_BACKENDS']} if config_host['TRACE_BACKENDS'].split().contains('simple') summary_info +=3D {'Trace output file': config_host['CONFIG_TRACE_FILE']= + '-'} --=20 2.28.0 From nobody Fri May 17 22:14:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604341344; cv=none; d=zohomail.com; s=zohoarc; b=BesoAVRy3S1Klw6Es2T8ANLnFGeK6TNHzKrqRDzQoHEblhmqFdjj1YZQRTq50b4pjCy53AidBmdO5Pe6OdNU6Uga7F6ucnQGu/QFLMn69XTVxMT/sbwYEG/uFW/BNUqtZPjxsWGSJiXBfdiL7pFptb4iYs0cW9Vn4tdRs9St3W8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604341344; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EFtkV8vk9mojAwXkn433NVuLfi5G+E0yJnsXG3+z/Cg=; b=WC2+yIitZOEMN4shUrGmDYJpgktt2wfR4jEXhTv32g3DROSkLEZ9ZwmePrwMIPftUkdEC5AlLS+fqEPRYqFo0lDWt1KF3/k/GtB0hzIO3tjRXGWoGheS1uHljGiMlw9a3OF32J0fQMt7CMHjdaJ+7rzo6YpFPL/P/zm/D0JyNwI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1604341344050564.3729865878178; Mon, 2 Nov 2020 10:22:24 -0800 (PST) Received: from localhost ([::1]:45940 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kZeTG-0003QZ-SV for importer@patchew.org; Mon, 02 Nov 2020 13:22:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40910) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kZeR4-0001Z6-8J for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:06 -0500 Received: from mail-lf1-x144.google.com ([2a00:1450:4864:20::144]:39298) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kZeQy-0006ic-7W for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:05 -0500 Received: by mail-lf1-x144.google.com with SMTP id 184so18647048lfd.6 for ; Mon, 02 Nov 2020 10:19:58 -0800 (PST) Received: from navi.cosmonova.net.ua ([95.67.24.131]) by smtp.gmail.com with ESMTPSA id c6sm2527007lfm.226.2020.11.02.10.19.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 10:19:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EFtkV8vk9mojAwXkn433NVuLfi5G+E0yJnsXG3+z/Cg=; b=xUkk56RaLggvLvpA4i/rbOe3nBZt+mgqu5QHEZ/3QKs26JSMzF8OX0EgTakeFolGAD yIePVz3XNvU+5WgJhGsM+8tiE03pU13ZbLan0KXas70beYh5LRHTOldaZaQIQN12TEsj tZJ47h2Pd7/EKECiGz1Q/mheiYzJlHipSsKIpjxnvRJPkWzdWiUvvCO7HXoXid32AfQe Awinnvi+gqH1g4cpGlRCa+w2RLdBvftZmU+szd7kFOqv7lTLyXvYkWk7TYHW0s36pUXA DiKzLodj1t4E9tPYfJbzKJse8amjxdJgje5cQ9keAire3rgKYblX0Q0KosmBERpbz6dT 4DOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EFtkV8vk9mojAwXkn433NVuLfi5G+E0yJnsXG3+z/Cg=; b=Z7v+pDpL9UpAeIsdKf3jZ0fe5vQJ29XIBMqD2Sa4JdZ2UDzuDlgDbezfy9ueEQCnBk Syz+v+oVsq4RHMC18nO4o6Pt6K/fMy1WtssMJBFAz95YqwJ4fMQjFbP6yAskEICv86jc 1camp1eDpAoMLu8qnhXT+okMMELFB/s88Fadv1Sw8pBdFZgLBw2Q1tK2ZS9TnToHrc36 TSSB72gv1rRM1slPSjP2JFN0oYRUF7jfjzz0mRjUZ8irxR+ubTCCzY31lMa8dpkpB5NO sybxwJhzFxwH3e5oEbGFO3j5ELkQT8S5NBEvsEHzYAbpF0O9Tp3wIyfEQkO8gcmXjjyG eFPA== X-Gm-Message-State: AOAM532QSm0m4+gPc1mue1Vls0+w0OCKFiFUFw/LM780W3/0CeRBFyaK onBrUDYr5RB4fa+7pt3039Bxwg== X-Google-Smtp-Source: ABdhPJyILR/U8PvhT/LO24ZoIKzdnGYff+Gj5TxAQybWNZEtqicLTx/EeL/2AR5FrvqT3q3bkfU0Nw== X-Received: by 2002:ac2:5973:: with SMTP id h19mr5835910lfp.458.1604341197079; Mon, 02 Nov 2020 10:19:57 -0800 (PST) From: Andrew Melnychenko To: jasowang@redhat.com, mst@redhat.com Subject: [RFC PATCH 5/6] virtio-net: Added eBPF RSS to virtio-net. Date: Mon, 2 Nov 2020 20:51:15 +0200 Message-Id: <20201102185115.7425-6-andrew@daynix.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201102185115.7425-1-andrew@daynix.com> References: <20201102185115.7425-1-andrew@daynix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=2a00:1450:4864:20::144; envelope-from=andrew@daynix.com; helo=mail-lf1-x144.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yan@daynix.com, yuri.benditovich@daynix.com, Andrew , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrew When RSS is enabled the device tries to load the eBPF program to select RX virtqueue in the TUN. If eBPF can be loaded the RSS will function also with vhost (works with kernel 5.8 and later). Software RSS is used as a fallback with vhost=3Doff when eBPF can't be load= ed or when hash population requested by the guest. Signed-off-by: Yuri Benditovich Signed-off-by: Andrew Melnychenko --- hw/net/vhost_net.c | 2 + hw/net/virtio-net.c | 120 +++++++++++++++++++++++++++++++-- include/hw/virtio/virtio-net.h | 4 ++ net/vhost-vdpa.c | 2 + 4 files changed, 124 insertions(+), 4 deletions(-) diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 24d555e764..16124f99c3 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -71,6 +71,8 @@ static const int user_feature_bits[] =3D { VIRTIO_NET_F_MTU, VIRTIO_F_IOMMU_PLATFORM, VIRTIO_F_RING_PACKED, + VIRTIO_NET_F_RSS, + VIRTIO_NET_F_HASH_REPORT, =20 /* This bit implies RARP isn't sent by QEMU out of band */ VIRTIO_NET_F_GUEST_ANNOUNCE, diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 277289d56e..afcc3032ec 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -698,6 +698,19 @@ static void virtio_net_set_queues(VirtIONet *n) =20 static void virtio_net_set_multiqueue(VirtIONet *n, int multiqueue); =20 +static uint64_t fix_ebpf_vhost_features(uint64_t features) +{ + /* If vhost=3Don & CONFIG_EBPF doesn't set - disable RSS feature */ + uint64_t ret =3D features; +#ifndef CONFIG_EBPF + virtio_clear_feature(&ret, VIRTIO_NET_F_RSS); +#endif + /* for now, there is no solution for populating the hash from eBPF */ + virtio_clear_feature(&ret, VIRTIO_NET_F_HASH_REPORT); + + return ret; +} + static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t featu= res, Error **errp) { @@ -732,9 +745,9 @@ static uint64_t virtio_net_get_features(VirtIODevice *v= dev, uint64_t features, return features; } =20 - virtio_clear_feature(&features, VIRTIO_NET_F_RSS); - virtio_clear_feature(&features, VIRTIO_NET_F_HASH_REPORT); - features =3D vhost_net_get_features(get_vhost_net(nc->peer), features); + features =3D fix_ebpf_vhost_features( + vhost_net_get_features(get_vhost_net(nc->peer), features)); + vdev->backend_features =3D features; =20 if (n->mtu_bypass_backend && @@ -1169,12 +1182,75 @@ static int virtio_net_handle_announce(VirtIONet *n,= uint8_t cmd, } } =20 +static void virtio_net_unload_epbf_rss(VirtIONet *n); + static void virtio_net_disable_rss(VirtIONet *n) { if (n->rss_data.enabled) { trace_virtio_net_rss_disable(); } n->rss_data.enabled =3D false; + + if (!n->rss_data.enabled_software_rss && ebpf_rss_is_loaded(&n->ebpf_r= ss)) { + virtio_net_unload_epbf_rss(n); + } +} + +static bool virtio_net_attach_steering_ebpf(NICState *nic, int prog_fd) +{ + NetClientState *nc =3D qemu_get_peer(qemu_get_queue(nic), 0); + if (nc =3D=3D NULL || nc->info->set_steering_ebpf =3D=3D NULL) { + return false; + } + + return nc->info->set_steering_ebpf(nc, prog_fd); +} + +static void rss_data_to_rss_config(struct VirtioNetRssData *data, + struct EBPFRSSConfig *config) +{ + config->redirect =3D data->redirect; + config->populate_hash =3D data->populate_hash; + config->hash_types =3D data->hash_types; + config->indirections_len =3D data->indirections_len; + config->default_queue =3D data->default_queue; +} + +static bool virtio_net_load_epbf_rss(VirtIONet *n) +{ + struct EBPFRSSConfig config =3D {}; + + if (!n->rss_data.enabled) { + if (ebpf_rss_is_loaded(&n->ebpf_rss)) { + ebpf_rss_unload(&n->ebpf_rss); + } + return true; + } + + if (!ebpf_rss_is_loaded(&n->ebpf_rss) && !ebpf_rss_load(&n->ebpf_rss))= { + return false; + } + + rss_data_to_rss_config(&n->rss_data, &config); + + if (!ebpf_rss_set_all(&n->ebpf_rss, &config, + n->rss_data.indirections_table, n->rss_data.key)= ) { + ebpf_rss_unload(&n->ebpf_rss); + return false; + } + + if (!virtio_net_attach_steering_ebpf(n->nic, n->ebpf_rss.program_fd)) { + ebpf_rss_unload(&n->ebpf_rss); + return false; + } + + return true; +} + +static void virtio_net_unload_epbf_rss(VirtIONet *n) +{ + virtio_net_attach_steering_ebpf(n->nic, -1); + ebpf_rss_unload(&n->ebpf_rss); } =20 static uint16_t virtio_net_handle_rss(VirtIONet *n, @@ -1208,6 +1284,7 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n, err_value =3D (uint32_t)s; goto error; } + n->rss_data.enabled_software_rss =3D false; n->rss_data.hash_types =3D virtio_ldl_p(vdev, &cfg.hash_types); n->rss_data.indirections_len =3D virtio_lduw_p(vdev, &cfg.indirection_table_mask); @@ -1289,9 +1366,30 @@ static uint16_t virtio_net_handle_rss(VirtIONet *n, goto error; } n->rss_data.enabled =3D true; + + if (!n->rss_data.populate_hash) { + /* load EBPF RSS */ + if (!virtio_net_load_epbf_rss(n)) { + /* EBPF mast be loaded for vhost */ + if (get_vhost_net(qemu_get_queue(n->nic)->peer)) { + warn_report("Can't load eBPF RSS for vhost"); + goto error; + } + /* fallback to software RSS */ + warn_report("Can't load eBPF RSS - fallback to software RSS"); + n->rss_data.enabled_software_rss =3D true; + } + } else { + /* use software RSS for hash populating */ + /* and unload eBPF if was loaded before */ + virtio_net_unload_epbf_rss(n); + n->rss_data.enabled_software_rss =3D true; + } + trace_virtio_net_rss_enable(n->rss_data.hash_types, n->rss_data.indirections_len, temp.b); + return queues; error: trace_virtio_net_rss_error(err_msg, err_value); @@ -1674,7 +1772,7 @@ static ssize_t virtio_net_receive_rcu(NetClientState = *nc, const uint8_t *buf, return -1; } =20 - if (!no_rss && n->rss_data.enabled) { + if (!no_rss && n->rss_data.enabled && n->rss_data.enabled_software_rss= ) { int index =3D virtio_net_process_rss(nc, buf, size); if (index >=3D 0) { NetClientState *nc2 =3D qemu_get_subqueue(n->nic, index); @@ -2780,6 +2878,18 @@ static int virtio_net_post_load_device(void *opaque,= int version_id) } =20 if (n->rss_data.enabled) { + n->rss_data.enabled_software_rss =3D n->rss_data.populate_hash; + if (!n->rss_data.populate_hash) { + if (!virtio_net_load_epbf_rss(n)) { + if (get_vhost_net(qemu_get_queue(n->nic)->peer)) { + error_report("Can't post-load eBPF RSS for vhost"); + } else { + warn_report("Can't post-load eBPF RSS - fallback to so= ftware RSS"); + n->rss_data.enabled_software_rss =3D true; + } + } + } + trace_virtio_net_rss_enable(n->rss_data.hash_types, n->rss_data.indirections_len, sizeof(n->rss_data.key)); @@ -3453,6 +3563,8 @@ static void virtio_net_instance_init(Object *obj) device_add_bootindex_property(obj, &n->nic_conf.bootindex, "bootindex", "/ethernet-phy@0", DEVICE(n)); + + ebpf_rss_init(&n->ebpf_rss); } =20 static int virtio_net_pre_save(void *opaque) diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h index f4852ac27b..4d29a577eb 100644 --- a/include/hw/virtio/virtio-net.h +++ b/include/hw/virtio/virtio-net.h @@ -21,6 +21,8 @@ #include "qemu/option_int.h" #include "qom/object.h" =20 +#include "ebpf/ebpf_rss.h" + #define TYPE_VIRTIO_NET "virtio-net-device" OBJECT_DECLARE_SIMPLE_TYPE(VirtIONet, VIRTIO_NET) =20 @@ -130,6 +132,7 @@ typedef struct VirtioNetRscChain { =20 typedef struct VirtioNetRssData { bool enabled; + bool enabled_software_rss; bool redirect; bool populate_hash; uint32_t hash_types; @@ -214,6 +217,7 @@ struct VirtIONet { Notifier migration_state; VirtioNetRssData rss_data; struct NetRxPkt *rx_pkt; + struct EBPFRSSContext ebpf_rss; }; =20 void virtio_net_set_netclient_name(VirtIONet *n, const char *name, diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 99c476db8c..feb5fa8624 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -54,6 +54,8 @@ const int vdpa_feature_bits[] =3D { VIRTIO_NET_F_MTU, VIRTIO_F_IOMMU_PLATFORM, VIRTIO_F_RING_PACKED, + VIRTIO_NET_F_RSS, + VIRTIO_NET_F_HASH_REPORT, VIRTIO_NET_F_GUEST_ANNOUNCE, VIRTIO_NET_F_STATUS, VHOST_INVALID_FEATURE_BIT --=20 2.28.0 From nobody Fri May 17 22:14:50 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604341468; cv=none; d=zohomail.com; s=zohoarc; b=HBwQ1Ciw4lbAy13iRCrvZ8ScBVtPSucSriR2N/wRFgTreptPhgraN9XIaevwrvjrmaAO/A/8vHip7mzUnJeej6rOwPxclbItXjQNsbYXm7V1VoSI2dlnGrWI6EBE4cS7FXJ17XyLzZ2jrzHHV8s7zikr2aW5hpIUu4NeRZ4o1/g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604341468; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=G3YrJVgX998qKuhMvPXP2IEY2yBy5bIH1EzHgRLGBYE=; b=Nx5LkgbL25dNR9WqWgCYqutDSsEckPwQhr3iEQDpNfhOwOfMomEUV9v/o87kRcLH0HJRkWn1w1Gp4byhepgZ4Qn2pK0bBU2mr8nYapVru8dNGhJ4c4xUw8rOZA9GVNVTayxZpqAPtXlgseBZSmNdw+0y4KRrwJYwt9LAHM9vwvY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1604341468714847.3671090803758; Mon, 2 Nov 2020 10:24:28 -0800 (PST) Received: from localhost ([::1]:54436 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kZeVG-0006s8-F6 for importer@patchew.org; Mon, 02 Nov 2020 13:24:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40920) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kZeR6-0001dI-GK for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:08 -0500 Received: from mail-lj1-x242.google.com ([2a00:1450:4864:20::242]:34289) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kZeQy-0006ix-9Z for qemu-devel@nongnu.org; Mon, 02 Nov 2020 13:20:08 -0500 Received: by mail-lj1-x242.google.com with SMTP id y16so16117928ljk.1 for ; Mon, 02 Nov 2020 10:19:59 -0800 (PST) Received: from navi.cosmonova.net.ua ([95.67.24.131]) by smtp.gmail.com with ESMTPSA id c6sm2527007lfm.226.2020.11.02.10.19.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2020 10:19:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G3YrJVgX998qKuhMvPXP2IEY2yBy5bIH1EzHgRLGBYE=; b=BjPycm7D1uUxofL/ILn6p9n2tItlEAGmYonYawVDoe48j9DkDcEwxhPvTOkvsgXBSP 1Nb68AJCiNObWQuDa5DO8MWJoyAZcu6bjKtshY6QooRg5CNFtoB0dyiXlEwjmv8jlWyE vYLnCXdzjB7OUS4T+sAvQBS+uQPMuTxBbAGL4tLrscMFRdJA8hzBnmwu/yrXG45zT3ip tnQacb7GNhCKhJk13gtDEpvAtT2i+S9/5GrzJU9Bd1tVsLRcmLJC1JN+75GsXWf9p8TM yqNIzmHMBKuDVqO1g+vq78s7/NiyZhtm6BdiQFjmeE8/E3LggzEOolwP0sUFdoDHN0F2 Qn+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G3YrJVgX998qKuhMvPXP2IEY2yBy5bIH1EzHgRLGBYE=; b=V5aZlfbY47EYssMIDVieqaBvCB65cCk15QrqJpd3xbMCEU1EQB+TfAVPeRdArY+u2P Z/mArKUeVVPkzeRFXjm3Jpw3c6UsAOrztIhxqdJABmh33BjwktNNVETra0dI69PvqLzp zl1VykL+EvexYr45cUmrKgXHjMRVstNZVnqZ2AxyAXcWuUEPbtjAJCPjdhgGxHKK79kb 1WyTypOIJXdRALnyamFLFUSbSrhnt7x7FW/kbCfBuZWUIMq7I/PMEiv3jvNHX77vxAf3 1UpWEpsjFkzdhFT3JDCSjsz9q1PnXh1ddsQoy4MB2mCyXe2L00R7+ZUSSajHt2M/PdcH tL+g== X-Gm-Message-State: AOAM533QTuXuHsGaFHfuc+sFc479V4nw3iiYxBo72M0DA75bpwxvuofd mFD9nzdeRZXJzRL4CXgf/VV14Q== X-Google-Smtp-Source: ABdhPJxDljeBM6U+uC5luHm6WgUGUIDcqLOO7W2UTKJ3BylzInvH0piwBMoBjQeQ/SXxDj2AIf2p/g== X-Received: by 2002:a2e:7007:: with SMTP id l7mr7533678ljc.185.1604341198473; Mon, 02 Nov 2020 10:19:58 -0800 (PST) From: Andrew Melnychenko To: jasowang@redhat.com, mst@redhat.com Subject: [RFC PATCH 6/6] docs: Added eBPF documentation. Date: Mon, 2 Nov 2020 20:51:16 +0200 Message-Id: <20201102185115.7425-7-andrew@daynix.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201102185115.7425-1-andrew@daynix.com> References: <20201102185115.7425-1-andrew@daynix.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=2a00:1450:4864:20::242; envelope-from=andrew@daynix.com; helo=mail-lj1-x242.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yan@daynix.com, yuri.benditovich@daynix.com, Andrew , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" From: Andrew Also, added maintainers information. Signed-off-by: Yuri Benditovich Signed-off-by: Andrew Melnychenko --- MAINTAINERS | 6 +++ docs/ebpf.rst | 29 +++++++++++ docs/ebpf_rss.rst | 129 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 164 insertions(+) create mode 100644 docs/ebpf.rst create mode 100644 docs/ebpf_rss.rst diff --git a/MAINTAINERS b/MAINTAINERS index 2c22bbca5a..464b3f3c95 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3111,6 +3111,12 @@ S: Maintained F: hw/semihosting/ F: include/hw/semihosting/ =20 +EBPF: +M: Andrew Melnychenko +M: Yuri Benditovich +S: Maintained +F: ebpf/* + Build and test automation ------------------------- Build and test automation diff --git a/docs/ebpf.rst b/docs/ebpf.rst new file mode 100644 index 0000000000..e45d085432 --- /dev/null +++ b/docs/ebpf.rst @@ -0,0 +1,29 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +eBPF qemu support +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + +eBPF support (CONFIG_EBPF) is enabled automatically by 'configure' script +if 'bpf' system call is available. +To disable eBPF support use './configure --disable-bpf' + +Basic eBPF functionality is located in ebpf/ebpf.c and ebpf/ebpf.h. +There are basic functions to load the eBPF program into the kernel. +Mostly, functions name are self-explanatory: + +- `bpf_create_map()`, `bpf_lookup_element()`, `bpf_update_element()`, `bpf= _delete_element()` - manages eBPF maps. On error, a basic error message wou= ld be reported and returned -1. On success, 0 would be returned(`bpf_create= _map()` returns map's file descriptor). +- `bpf_prog_load()` - load the program. The program has to have proper map= file descriptors if there are used. On error - the log eBPF would be repor= ted. On success, the program file descriptor returned. +- `bpf_fixup_mapfd()` - would place map file descriptor into the program a= ccording to 'relocate array' of 'struct fixup_mapfd_t'. The function would = return how many instructions were 'fixed' aka how many relocations was occu= rred. + +Simplified workflow would look like this: + +.. code:: C + + int map1 =3D bpf_create_map(...); + int map2 =3D bpf_create_map(...); + + bpf_fixup_mapfd(, ARRAY_SIZE(), , ARRAY_SIZE(), , map1); + bpf_fixup_mapfd(, ARRAY_SIZE(), , ARRAY_SIZE(), , map2); + + int prog =3D bpf_prog_load(, , ARR= AY_SIZE(), "GPL"); + +See the bpf(2) for details. diff --git a/docs/ebpf_rss.rst b/docs/ebpf_rss.rst new file mode 100644 index 0000000000..96fee391b8 --- /dev/null +++ b/docs/ebpf_rss.rst @@ -0,0 +1,129 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +eBPF RSS virtio-net support +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + +RSS(Receive Side Scaling) is used to distribute network packets to guest v= irtqueues +by calculating packet hash. Usually every queue is processed then by a spe= cific guest CPU core. + +For now there are 2 RSS implementations in qemu: +- 'software' RSS (functions if qemu receives network packets, i.e. vhost= =3Doff) +- eBPF RSS (can function with also with vhost=3Don) + +If steering BPF is not set for kernel's TUN module, the TUN uses automatic= selection +of rx virtqueue based on lookup table built according to calculated symmet= ric hash +of transmitted packets. +If steering BPF is set for TUN the BPF code calculates the hash of packet = header and +returns the virtqueue number to place the packet to. + +Simplified decision formula: + +.. code:: C + + queue_index =3D indirection_table[hash()%] + + +Not for all packets, the hash can/should be calculated. + +Note: currently, eBPF RSS does not support hash reporting. + +eBPF RSS turned on by different combinations of vhost-net, vitrio-net and = tap configurations: + +- eBPF is used: + + tap,vhost=3Doff & virtio-net-pci,rss=3Don,hash=3Doff + +- eBPF is used: + + tap,vhost=3Don & virtio-net-pci,rss=3Don,hash=3Doff + +- 'software' RSS is used: + + tap,vhost=3Doff & virtio-net-pci,rss=3Don,hash=3Don + +- eBPF is used, hash population feature is not reported to the guest: + + tap,vhost=3Don & virtio-net-pci,rss=3Don,hash=3Don + +If CONFIG_EBPF is not set then only 'software' RSS is supported. +Also 'software' RSS, as a fallback, is used if the eBPF program failed to = load or set to TUN. + +RSS eBPF program +---------------- + +RSS program located in ebpf/tun_rss_steering.h as an array of 'struct bpf_= insn'. +So the program is part of the qemu binary. +Initially, the eBPF program was compiled by clang and source code located = at ebpf/rss.bpf.c. +Prerequisites to recompile the eBPF program (regenerate ebpf/tun_rss_steer= ing.h): + + llvm, clang, kernel source tree, python3 + (pip3 pyelftools) + Adjust 'linuxhdrs' in Makefile.ebpf to reflect the location of the= kernel source tree + + $ cd ebpf + $ make -f Makefile.ebpf + +Note the python script for convertation from eBPF ELF object to '.h' file = - Ebpf_to_C.py: + + $ python EbpfElf_to_C.py rss.bpf.o tun_rss_steering + +The first argument of the script is ELF object, second - section name wher= e the eBPF program located. +The script would generate
.h file with eBPF instructions and= 'relocate array'. +'relocate array' is an array of 'struct fixup_mapfd_t' with the name of th= e eBPF map and instruction offset where the file descriptor of the map shou= ld be placed. + +Current eBPF RSS implementation uses 'bounded loops' with 'backward jump i= nstructions' which present in the last kernels. +Overall eBPF RSS works on kernels 5.8+. + +eBPF RSS implementation +----------------------- + +eBPF RSS loading functionality located in ebpf/ebpf_rss.c and ebpf/ebpf_rs= s.h. + +The `struct EBPFRSSContext` structure that holds 4 file descriptors: + +- program_fd - file descriptor of the eBPF RSS program. +- map_configuration - file descriptor of the 'configuration' map. This map= contains one element of 'struct EBPFRSSConfig'. This configuration determi= nes eBPF program behavior. +- map_toeplitz_key - file descriptor of the 'Toeplitz key' map. One elemen= t of the 40byte key prepared for the hashing algorithm. +- map_indirections_table - 128 elements of queue indexes. + +`struct EBPFRSSConfig` fields: + +- redirect - "boolean" value, should the hash be calculated, on false - `= default_queue` would be used as the final decision. +- populate_hash - for now, not used. eBPF RSS doesn't support hash reporti= ng. +- hash_types - binary mask of different hash types. See `VIRTIO_NET_RSS_HA= SH_TYPE_*` defines. If for packet hash should not be calculated - `default_= queue` would be used. +- indirections_len - length of the indirections table, maximum 128. +- default_queue - the queue index that used for packet that shouldn't be h= ashed. For some packets, the hash can't be calculated(g.e ARP). + +Functions: + +- `ebpf_rss_init()` - sets program_fd to -1, which indicates that EBPFRSSC= ontext is not loaded. +- `ebpf_rss_load()` - creates 3 maps and loads eBPF program from tun_rss_s= teering.h. Returns 'true' on success. After that, program_fd can be used to= set steering for TAP. +- `ebpf_rss_set_all()` - sets values for eBPF maps. `indirections_table` l= ength is in EBPFRSSConfig. `toeplitz_key` is VIRTIO_NET_RSS_MAX_KEY_SIZE ak= a 40 bytes array. +- `ebpf_rss_unload()` - close all file descriptors and set program_fd to -= 1. + +Simplified eBPF RSS workflow: + +.. code:: C + + struct EBPFRSSConfig config; + config.redirect =3D 1; + config.hash_types =3D VIRTIO_NET_RSS_HASH_TYPE_UDPv4 | VIRTIO_NET_RSS_= HASH_TYPE_TCPv4; + config.indirections_len =3D VIRTIO_NET_RSS_MAX_TABLE_LEN; + config.default_queue =3D 0; + + uint16_t table[VIRTIO_NET_RSS_MAX_TABLE_LEN] =3D {...}; + uint8_t key[VIRTIO_NET_RSS_MAX_KEY_SIZE] =3D {...}; + + struct EBPFRSSContext ctx; + ebpf_rss_init(&ctx); + ebpf_rss_load(&ctx); + ebpf_rss_set_all(&ctx, &config, table, key); + if (net_client->info->set_steering_ebpf !=3D NULL) { + net_client->info->set_steering_ebpf(net_client, ctx->program_fd); + } + ... + ebpf_unload(&ctx); + + +NetClientState SetSteeringEBPF() +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For now, `set_steering_ebpf()` method supported by Linux TAP NetClientStat= e. The method requires an eBPF program file descriptor as an argument. --=20 2.28.0