From nobody Mon Nov 25 02:01:18 2024 Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 593AD132122 for ; Fri, 1 Nov 2024 00:49:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730422175; cv=none; b=sG2msYH97TfDUE06TuHYXlypTrRgoiFGXZayB+zFNL9qqzdSw0joFweNXphFtaOinW8f92n67vi6zk0sXwdL2lsw3/xwrL0Fz9rl77aB6UbY06hhmAOvoww0ctrSPs/jyDDAM0BQ1aHdT0CgE4e/dxPDSnUHHkQkZVnU9VebK28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730422175; c=relaxed/simple; bh=TFXbJW6wVeRX+arBDsjadpw5E4y4OvAphmOQrEKaPIM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BPKLv6MZEFGSFiskAtWxACDUFoiq+Jq0C1a1+fgyDk0/7XgowSim1SYHweNwOGs0PVyIolbnfyvjweQReMEHQfDPfPATgKU+fGNKszQglrfBxknqQoOesH9h5vzdkTFsY5cYGgLevRV6nYvz+/c1SFdvN/GBrRP/1ZGvVnXBteo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com; spf=pass smtp.mailfrom=fastly.com; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b=nCCg3sz5; arc=none smtp.client-ip=209.85.210.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fastly.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fastly.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fastly.com header.i=@fastly.com header.b="nCCg3sz5" Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-715716974baso819904a34.1 for ; Thu, 31 Oct 2024 17:49:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; t=1730422165; x=1731026965; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XkoSEaVvEKME4ZqF6oQuraqIgV/sbYh4Q6GVty3dPs8=; b=nCCg3sz5XtQ1E3mqfbb+LCBMl1KeYtMH2zFXKz8JAtW7aeB7i5oa1DOzkczXhXleU4 8IRZN/hOsTKhGoxZel3eSMDI2yTgDWPo74xsCwUVFTezkKTi9Ja39uzMzV4L0TuRaFA1 Ad6QJNeP5xKG/cdzEiHCgeSauuTupDRuMa7ks= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730422165; x=1731026965; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XkoSEaVvEKME4ZqF6oQuraqIgV/sbYh4Q6GVty3dPs8=; b=W6aG6JHkBszGmrCdkEER4uLCUrXu4d6xgcYZdYJyFPNgHPo30rnwmXtRVfCwijaG6m +OWHSx3LdTQHtD14aj+Dpk2YCRc7OHdGiku6kSUVrVLyYAukh7d6Yek7r3eBpYmx23Zk ddnpWzclo4mZ0SX97H6ME5hkLJtSKfvu2r1PTqJCk4W1y2D7JUppe8n+MZ+uena4yI9G 9cGlAHuWOq+IsqK/DN9RPu8eS0WSBi4tbtk0aOymHBhLG9ZPBntDhWJoatk9ckEr2ryf hJFSAUeu+vJ920FLtfI77wVgdoG6s5niBE2dQ7Hw9Y2mx7g92dZMcjuC5QDcG77dVhOk 5sHA== X-Forwarded-Encrypted: i=1; AJvYcCXfmTto+064G9XxtfnSwlMyImJGswtuGyMc5BcvfiZ9Oaq96IIWDaFo42dHcWUozaD6y/9DBLNa20ir82Y=@vger.kernel.org X-Gm-Message-State: AOJu0YwWupoXy5A7ruyeN+stUGn5oZpXUYQBl+iyO2ulkktVkm8OePuz IGC9skiREI78BnWWYk+B2F0gD+CCGwwQVLi+NDMW0l3PFMmKfNpW5ARsKwlWy2Q= X-Google-Smtp-Source: AGHT+IEDVwLwzF5d/8qYC9UrmWwdMTMdr3jym2akk77xaWJwrb7upupKTbx/Un21wVI9DTh6SWf04g== X-Received: by 2002:a05:6830:3152:b0:718:99a0:2168 with SMTP id 46e09a7af769-719ca174bd6mr2051826a34.4.1730422164808; Thu, 31 Oct 2024 17:49:24 -0700 (PDT) Received: from localhost.localdomain ([2620:11a:c019:0:65e:3115:2f58:c5fd]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7ee45a12c27sm1585365a12.93.2024.10.31.17.49.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Oct 2024 17:49:24 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org Cc: pabeni@redhat.com, namangulati@google.com, edumazet@google.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, sdf@fomichev.me, peter@typeblog.net, m2shafiei@uwaterloo.ca, bjorn@rivosinc.com, hch@infradead.org, willy@infradead.org, willemdebruijn.kernel@gmail.com, skhawaja@google.com, kuba@kernel.org, Joe Damato , Martin Karsten , "David S. Miller" , Simon Horman , Shuah Khan , linux-kernel@vger.kernel.org (open list), linux-kselftest@vger.kernel.org (open list:KERNEL SELFTEST FRAMEWORK) Subject: [PATCH net-next v3 6/7] selftests: net: Add busy_poll_test Date: Fri, 1 Nov 2024 00:48:33 +0000 Message-Id: <20241101004846.32532-7-jdamato@fastly.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241101004846.32532-1-jdamato@fastly.com> References: <20241101004846.32532-1-jdamato@fastly.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an epoll busy poll test using netdevsim. This test is comprised of: - busy_poller (via busy_poller.c) - busy_poll_test.sh which loads netdevsim, sets up network namespaces, and runs busy_poller to receive data and netcat to send data. The selftest tests two different scenarios: - busy poll (the pre-existing version in the kernel) - busy poll with suspend enabled (what this series adds) The data transmit is a 1MiB temporary file generated from /dev/urandom and the test is considered passing if the md5sum of the input file to netcat matches the md5sum of the output file from busy_poller. netdevsim was chosen instead of veth due to netdevsim's support for netdev-genl. For now, this test uses the functionality that netdevim provides. In the future, perhaps netdevsim can be extended to emulate device IRQs to more thoroughly test all pre-existing kernel options (like defer_hard_irqs) and suspend. Signed-off-by: Joe Damato Co-developed-by: Martin Karsten Signed-off-by: Martin Karsten --- v3: - New in v3 tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 2 + tools/testing/selftests/net/busy_poll_test.sh | 164 ++++++++++++ tools/testing/selftests/net/busy_poller.c | 245 ++++++++++++++++++ 4 files changed, 412 insertions(+) create mode 100755 tools/testing/selftests/net/busy_poll_test.sh create mode 100644 tools/testing/selftests/net/busy_poller.c diff --git a/tools/testing/selftests/net/.gitignore b/tools/testing/selftes= ts/net/.gitignore index 217d8b7a7365..85b0c4a2179f 100644 --- a/tools/testing/selftests/net/.gitignore +++ b/tools/testing/selftests/net/.gitignore @@ -2,6 +2,7 @@ bind_bhash bind_timewait bind_wildcard +busy_poller cmsg_sender diag_uid epoll_busy_poll diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests= /net/Makefile index 26a4883a65c9..4b5f3d7d900b 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -69,6 +69,7 @@ TEST_GEN_FILES +=3D hwtstamp_config rxtimestamp timestamp= ing txtimestamp TEST_GEN_FILES +=3D ipsec TEST_GEN_FILES +=3D ioam6_parser TEST_GEN_FILES +=3D gro +TEST_GEN_FILES +=3D busy_poller TEST_GEN_PROGS =3D reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa TEST_GEN_PROGS +=3D reuseport_dualstack reuseaddr_conflict tls tun tap epo= ll_busy_poll TEST_GEN_FILES +=3D toeplitz @@ -96,6 +97,7 @@ TEST_PROGS +=3D fdb_flush.sh TEST_PROGS +=3D fq_band_pktlimit.sh TEST_PROGS +=3D vlan_hw_filter.sh TEST_PROGS +=3D bpf_offload.py +TEST_PROGS +=3D busy_poll_test.sh =20 # YNL files, must be before "include ..lib.mk" YNL_GEN_FILES :=3D ncdevmem diff --git a/tools/testing/selftests/net/busy_poll_test.sh b/tools/testing/= selftests/net/busy_poll_test.sh new file mode 100755 index 000000000000..bf33737691a4 --- /dev/null +++ b/tools/testing/selftests/net/busy_poll_test.sh @@ -0,0 +1,164 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0-only +source net_helper.sh + +NSIM_DEV_1_ID=3D$((256 + RANDOM % 256)) +NSIM_DEV_1_SYS=3D/sys/bus/netdevsim/devices/netdevsim$NSIM_DEV_1_ID +NSIM_DEV_2_ID=3D$((512 + RANDOM % 256)) +NSIM_DEV_2_SYS=3D/sys/bus/netdevsim/devices/netdevsim$NSIM_DEV_2_ID + +NSIM_DEV_SYS_NEW=3D/sys/bus/netdevsim/new_device +NSIM_DEV_SYS_DEL=3D/sys/bus/netdevsim/del_device +NSIM_DEV_SYS_LINK=3D/sys/bus/netdevsim/link_device +NSIM_DEV_SYS_UNLINK=3D/sys/bus/netdevsim/unlink_device + +YNL_PATH=3D$(realpath $(dirname ${0}))/../../../net/ynl/cli.py +SPEC_PATH=3D$(realpath $(dirname ${0}))/../../../../Documentation/netlink/= specs/netdev.yaml + +check_ynl() +{ + if [ ! -f ${YNL_PATH} ]; then + echo "YNL path invalid: ${YNL_PATH}" + exit 1 + fi + + if [ ! -f ${SPEC_PATH} ]; then + echo "spec path invalid: ${SPEC_PATH}" + exit 1 + fi +} + +setup_ns() +{ + set -e + ip netns add nssv + ip netns add nscl + + NSIM_DEV_1_NAME=3D$(find $NSIM_DEV_1_SYS/net -maxdepth 1 -type d ! \ + -path $NSIM_DEV_1_SYS/net -exec basename {} \;) + NSIM_DEV_2_NAME=3D$(find $NSIM_DEV_2_SYS/net -maxdepth 1 -type d ! \ + -path $NSIM_DEV_2_SYS/net -exec basename {} \;) + + # ensure the server has 1 queue + ethtool -L $NSIM_DEV_1_NAME combined 1 2>/dev/null + + ip link set $NSIM_DEV_1_NAME netns nssv + ip link set $NSIM_DEV_2_NAME netns nscl + + ip netns exec nssv ip addr add '192.168.1.1/24' dev $NSIM_DEV_1_NAME + ip netns exec nscl ip addr add '192.168.1.2/24' dev $NSIM_DEV_2_NAME + + ip netns exec nssv ip link set dev $NSIM_DEV_1_NAME up + ip netns exec nscl ip link set dev $NSIM_DEV_2_NAME up + + set +e +} + +cleanup_ns() +{ + ip netns del nscl + ip netns del nssv +} + +test_busypoll() +{ + tmp_file=3D$(mktemp) + out_file=3D$(mktemp) + + # fill a test file with random data + dd if=3D/dev/urandom of=3D${tmp_file} bs=3D1M count=3D1 2> /dev/null + + timeout -k 60s 60s ip netns exec nssv ./busy_poller -p48675 -b192.168.1.1= -m8 -u0 -P1 -g16 -o${out_file}& + + wait_local_port_listen nssv 48675 tcp + + ip netns exec nscl nc -N 192.168.1.1 48675 < $tmp_file + + wait + + tmp_file_md5sum=3D$(md5sum $tmp_file | cut -f1 -d' ') + out_file_md5sum=3D$(md5sum $out_file | cut -f1 -d' ') + + if [ "$tmp_file_md5sum" =3D "$out_file_md5sum" ]; then + res=3D0 + else + echo "md5sum mismatch" + echo "input file md5sum: ${tmp_file_md5sum}"; + echo "output file md5sum: ${out_file_md5sum}"; + res=3D1 + fi + + rm $out_file $tmp_file + + return $res +} + +test_busypoll_with_suspend() +{ + # set the suspend parameter for the server via its IFIDX + + DUMP_CMD=3D"${YNL_PATH} --spec ${SPEC_PATH} --dump napi-get --json=3D\"{\= \\"ifindex\\\": ${NSIM_DEV_1_IFIDX}}\" --output-json" + NSIM_DEV_1_NAPIID=3D$(ip netns exec nssv bash -c "$DUMP_CMD") + NSIM_DEV_1_NAPIID=3D$(echo $NSIM_DEV_1_NAPIID | jq '.[] | .id') + + SUSPEND_CMD=3D"${YNL_PATH} --spec ${SPEC_PATH} --do napi-set --json=3D\"{= \\\"id\\\": ${NSIM_DEV_1_NAPIID}, \\\"irq-suspend-timeout\\\": 20000000, \\= \"gro-flush-timeout\\\": 50000, \\\"defer-hard-irqs\\\": 100}\"" + NSIM_DEV_1_SETCONFIG=3D$(ip netns exec nssv bash -c "$SUSPEND_CMD") + + test_busypoll + return $? +} + +### +### Code start +### + +check_ynl + +modprobe netdevsim + +# linking + +echo $NSIM_DEV_1_ID > $NSIM_DEV_SYS_NEW +echo $NSIM_DEV_2_ID > $NSIM_DEV_SYS_NEW +udevadm settle + +setup_ns + +NSIM_DEV_1_FD=3D$((256 + RANDOM % 256)) +exec {NSIM_DEV_1_FD}= $NSIM_DEV_SYS_LINK +if [ $? -ne 0 ]; then + echo "linking netdevsim1 with netdevsim2 should succeed" + cleanup_ns + exit 1 +fi + +test_busypoll +if [ $? -ne 0 ]; then + echo "test_busypoll failed" + cleanup_ns + exit 1 +fi + +test_busypoll_with_suspend +if [ $? -ne 0 ]; then + echo "test_busypoll_with_suspend failed" + cleanup_ns + exit 1 +fi + +echo "$NSIM_DEV_1_FD:$NSIM_DEV_1_IFIDX" > $NSIM_DEV_SYS_UNLINK + +echo $NSIM_DEV_2_ID > $NSIM_DEV_SYS_DEL + +cleanup_ns + +modprobe -r netdevsim + +exit 0 diff --git a/tools/testing/selftests/net/busy_poller.c b/tools/testing/self= tests/net/busy_poller.c new file mode 100644 index 000000000000..b49d67ce0ca8 --- /dev/null +++ b/tools/testing/selftests/net/busy_poller.c @@ -0,0 +1,245 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include + +/* if the headers haven't been updated, we need to define some things */ +#if !defined(EPOLL_IOC_TYPE) +struct epoll_params { + uint32_t busy_poll_usecs; + uint16_t busy_poll_budget; + uint8_t prefer_busy_poll; + + /* pad the struct to a multiple of 64bits */ + uint8_t __pad; +}; + +#define EPOLL_IOC_TYPE 0x8A +#define EPIOCSPARAMS _IOW(EPOLL_IOC_TYPE, 0x01, struct epoll_params) +#define EPIOCGPARAMS _IOR(EPOLL_IOC_TYPE, 0x02, struct epoll_params) +#endif + +static uint32_t cfg_port =3D 8000; +static struct in_addr cfg_bind_addr =3D { .s_addr =3D INADDR_ANY }; +static char *cfg_outfile; +static int cfg_max_events =3D 8; + +/* busy poll params */ +static uint32_t cfg_busy_poll_usecs; +static uint16_t cfg_busy_poll_budget; +static uint8_t cfg_prefer_busy_poll; + +static void usage(const char *filepath) +{ + error(1, 0, + "Usage: %s -p -b -m -u -P<= prefer_busy_poll> -g -o", + filepath); +} + +static void parse_opts(int argc, char **argv) +{ + int ret; + int c; + + if (argc <=3D 1) + usage(argv[0]); + + while ((c =3D getopt(argc, argv, "p:m:b:u:P:g:o:")) !=3D -1) { + switch (c) { + case 'u': + cfg_busy_poll_usecs =3D strtoul(optarg, NULL, 0); + if (cfg_busy_poll_usecs =3D=3D ULONG_MAX || + cfg_busy_poll_usecs > UINT32_MAX) + error(1, ERANGE, "busy_poll_usecs too large"); + break; + case 'P': + cfg_prefer_busy_poll =3D strtoul(optarg, NULL, 0); + if (cfg_prefer_busy_poll =3D=3D ULONG_MAX || + cfg_prefer_busy_poll > 1) + error(1, ERANGE, + "prefer busy poll should be 0 or 1"); + break; + case 'g': + cfg_busy_poll_budget =3D strtoul(optarg, NULL, 0); + if (cfg_busy_poll_budget =3D=3D ULONG_MAX || + cfg_busy_poll_budget > UINT16_MAX) + error(1, ERANGE, + "busy poll budget must be [0, UINT16_MAX]"); + break; + case 'p': + cfg_port =3D strtoul(optarg, NULL, 0); + if (cfg_port > UINT16_MAX) + error(1, ERANGE, "port must be <=3D 65535"); + break; + case 'b': + ret =3D inet_aton(optarg, &cfg_bind_addr); + if (ret =3D=3D 0) + error(1, errno, + "bind address %s invalid", optarg); + break; + case 'o': + cfg_outfile =3D strdup(optarg); + if (!cfg_outfile) + error(1, 0, "outfile invalid"); + break; + case 'm': + cfg_max_events =3D strtol(optarg, NULL, 0); + + if (cfg_max_events =3D=3D LONG_MIN || + cfg_max_events =3D=3D LONG_MAX || + cfg_max_events <=3D 0) + error(1, ERANGE, + "max events must be > 0 and < LONG_MAX"); + break; + } + } + + if (optind !=3D argc) + usage(argv[0]); +} + +static void epoll_ctl_add(int epfd, int fd, uint32_t events) +{ + struct epoll_event ev; + + ev.events =3D events; + ev.data.fd =3D fd; + if (epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev) =3D=3D -1) + error(1, errno, "epoll_ctl add fd: %d", fd); +} + +static void setnonblock(int sockfd) +{ + int flags; + + flags =3D fcntl(sockfd, F_GETFL, 0); + + if (fcntl(sockfd, F_SETFL, flags | O_NONBLOCK) =3D=3D -1) + error(1, errno, "unable to set socket to nonblocking mode"); +} + +static void write_chunk(int fd, char *buf, ssize_t buflen) +{ + ssize_t remaining =3D buflen; + char *buf_offset =3D buf; + ssize_t writelen =3D 0; + ssize_t write_result; + + while (writelen < buflen) { + write_result =3D write(fd, buf_offset, remaining); + if (write_result =3D=3D -1) + error(1, errno, "unable to write data to outfile"); + + writelen +=3D write_result; + remaining -=3D write_result; + buf_offset +=3D write_result; + } +} + +static void run_poller(void) +{ + struct epoll_event events[cfg_max_events]; + struct epoll_params epoll_params =3D {0}; + struct sockaddr_in server_addr; + int i, epfd, nfds; + ssize_t readlen; + int outfile_fd; + char buf[1024]; + int sockfd; + int conn; + int val; + + outfile_fd =3D open(cfg_outfile, O_WRONLY | O_CREAT, 0644); + if (outfile_fd =3D=3D -1) + error(1, errno, "unable to open outfile: %s", cfg_outfile); + + sockfd =3D socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); + if (sockfd =3D=3D -1) + error(1, errno, "unable to create listen socket"); + + server_addr.sin_family =3D AF_INET; + server_addr.sin_port =3D htons(cfg_port); + server_addr.sin_addr =3D cfg_bind_addr; + + epoll_params.busy_poll_usecs =3D cfg_busy_poll_usecs; + epoll_params.busy_poll_budget =3D cfg_busy_poll_budget; + epoll_params.prefer_busy_poll =3D cfg_prefer_busy_poll; + epoll_params.__pad =3D 0; + + val =3D 1; + if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val))) + error(1, errno, "poller setsockopt reuseaddr"); + + setnonblock(sockfd); + + if (bind(sockfd, (struct sockaddr *)&server_addr, + sizeof(struct sockaddr_in))) + error(0, errno, "poller bind to port: %d\n", cfg_port); + + if (listen(sockfd, 1)) + error(1, errno, "poller listen"); + + epfd =3D epoll_create1(0); + if (ioctl(epfd, EPIOCSPARAMS, &epoll_params) =3D=3D -1) + error(1, errno, "unable to set busy poll params"); + + epoll_ctl_add(epfd, sockfd, EPOLLIN | EPOLLOUT | EPOLLET); + + for (;;) { + nfds =3D epoll_wait(epfd, events, cfg_max_events, -1); + for (i =3D 0; i < nfds; i++) { + if (events[i].data.fd =3D=3D sockfd) { + conn =3D accept(sockfd, NULL, NULL); + if (conn =3D=3D -1) + error(1, errno, + "accepting incoming connection failed"); + + setnonblock(conn); + epoll_ctl_add(epfd, conn, + EPOLLIN | EPOLLET | EPOLLRDHUP | + EPOLLHUP); + } else if (events[i].events & EPOLLIN) { + for (;;) { + readlen =3D read(events[i].data.fd, buf, + sizeof(buf)); + if (readlen > 0) + write_chunk(outfile_fd, buf, + readlen); + else + break; + } + } else { + /* spurious event ? */ + } + if (events[i].events & (EPOLLRDHUP | EPOLLHUP)) { + epoll_ctl(epfd, EPOLL_CTL_DEL, + events[i].data.fd, NULL); + close(events[i].data.fd); + close(outfile_fd); + return; + } + } + } +} + +int main(int argc, char *argv[]) +{ + parse_opts(argc, argv); + run_poller(); + return 0; +} --=20 2.25.1