From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-04.galae.net (smtpout-04.galae.net [185.171.202.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06E6D3002A6; Wed, 24 Sep 2025 14:50:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.171.202.116 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725412; cv=none; b=k8FKvH8JZxb40YuZ8tRcX/nRoV2wOb3SKpUPVOg/d++uiHqkJBm2/11L1maqOM9ucXYLCDwZrJJDRs4f1gY7sNTuzOdaQbGDItMRerbLdLvok6YsLnmF1u7lr225ODlDmQXzU2tqCn2G9ulgX9fasw4uWd7FiFJiTq2evwzwLW0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725412; c=relaxed/simple; bh=4BHoRlJa0gLnpeYoqS7ybM82HJmqERxRMEpOgmu8MDE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Wjiek0TAaK3iTTtKcd1v1UlCAU3SMGNd4peVJoN3dEyG3zNFItueiSgU4dfMSIZ1nZfVaiP8lpaY8MIh68eLPEkWdQ/jlhD8Tv/mOn3Pq15k17+OPGwtLJbPYrWDHEwmpUHXbePC6/zSJNl0J9S5/CDMC0M8CgtWLB6mGsJth9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=xKMqCpah; arc=none smtp.client-ip=185.171.202.116 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="xKMqCpah" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-04.galae.net (Postfix) with ESMTPS id D1273C011C4; Wed, 24 Sep 2025 14:49:45 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 0AEB760634; Wed, 24 Sep 2025 14:50:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 3B5AD102F1916; Wed, 24 Sep 2025 16:49:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725401; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=PmxxP/rWLXk7Sp/yWeDgg5zpw5hdDiyehTe7fwo5vn0=; b=xKMqCpah/BWGtXxcqnu0Izb1ucOgsJ5HGMroRkso+Vk9wh8gAfA3KpClnBxr2UzL2S8ZwC eKKyCdhXr2TBZlLOBNzMRGNRThbgz1sQ2JKX2elXj/cXlhhK5SkUpIiIRdmBl3sUF3/RHJ OI+qt7W0xrGzs48pvlIaIpQm7Hn1lwFq+eOiXskhFcpBhz2fsmPOB8iFFnYgZ+ZK6zJ2ce yTaTUD0q/sqfxKJwto1++uUiR6btmrZRhpJ3gDPR5kL/lLwVOnsr+p622gS38ZJmCRP/5y B7UxDuUWC1bcqqRyXIHcj+gvBvkHlj11Dhyc8rlMo7RtZk1PmdfN4JyH9xD5eQ== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:36 +0200 Subject: [PATCH bpf-next v4 01/15] selftests/bpf: test_xsk: Split xskxceiver Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-1-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 AF_XDP features are tested by the test_xsk.sh script but not by the test_progs framework. The tests used by the script are defined in xksxceiver.c which can't be integrated in the test_progs framework as is. Extract these test definitions from xskxceiver{.c/.h} to put them in new test_xsk{.c/.h} files. Keep the main() function and its unshared dependencies in xksxceiver to avoid impacting the test_xsk.sh script which is often used to test real hardware. Move ksft_test_result_*() calls to xskxceiver.c to keep the kselftest's report valid Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/Makefile | 2 +- tools/testing/selftests/bpf/test_xsk.c | 2420 ++++++++++++++++++++++++++= ++ tools/testing/selftests/bpf/test_xsk.h | 297 ++++ tools/testing/selftests/bpf/xskxceiver.c | 2545 +-------------------------= ---- tools/testing/selftests/bpf/xskxceiver.h | 156 -- 5 files changed, 2762 insertions(+), 2658 deletions(-) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests= /bpf/Makefile index 0b6ee902bce512944dbff0b559b3c418134a97b3..ecd6f6fb540d968473227c770c6= 617f56257c7d8 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -804,7 +804,7 @@ $(OUTPUT)/test_verifier: test_verifier.c verifier/tests= .h $(BPFOBJ) | $(OUTPUT) =20 # Include find_bit.c to compile xskxceiver. EXTRA_SRC :=3D $(TOOLSDIR)/lib/find_bit.c -$(OUTPUT)/xskxceiver: $(EXTRA_SRC) xskxceiver.c xskxceiver.h $(OUTPUT)/net= work_helpers.o $(OUTPUT)/xsk.o $(OUTPUT)/xsk_xdp_progs.skel.h $(BPFOBJ) | $= (OUTPUT) +$(OUTPUT)/xskxceiver: $(EXTRA_SRC) test_xsk.c test_xsk.h xskxceiver.c xskx= ceiver.h $(OUTPUT)/network_helpers.o $(OUTPUT)/xsk.o $(OUTPUT)/xsk_xdp_prog= s.skel.h $(BPFOBJ) | $(OUTPUT) $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(filter %.a %.o %.c,$^) $(LDLIBS) -o $@ =20 diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c new file mode 100644 index 0000000000000000000000000000000000000000..02250f29f9946d1ca701e30a900= 617056d91587b --- /dev/null +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -0,0 +1,2420 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "network_helpers.h" +#include "test_xsk.h" +#include "xsk_xdp_common.h" +#include "xsk_xdp_progs.skel.h" + +#define DEFAULT_BATCH_SIZE 64 +#define MIN_PKT_SIZE 64 +#define MAX_ETH_JUMBO_SIZE 9000 +#define MAX_INTERFACES 2 +#define MAX_TEARDOWN_ITER 10 +#define MAX_TX_BUDGET_DEFAULT 32 +#define PKT_DUMP_NB_TO_PRINT 16 +/* Just to align the data in the packet */ +#define PKT_HDR_SIZE (sizeof(struct ethhdr) + 2) +#define POLL_TMOUT 1000 +#define THREAD_TMOUT 3 +#define UMEM_HEADROOM_TEST_SIZE 128 +#define XSK_DESC__INVALID_OPTION (0xffff) +#define XSK_UMEM__INVALID_FRAME_SIZE (MAX_ETH_JUMBO_SIZE + 1) +#define XSK_UMEM__LARGE_FRAME_SIZE (3 * 1024) +#define XSK_UMEM__MAX_FRAME_SIZE (4 * 1024) + +static const u8 g_mac[ETH_ALEN] =3D {0x55, 0x44, 0x33, 0x22, 0x11, 0x00}; + +bool opt_verbose; +pthread_barrier_t barr; +pthread_mutex_t pacing_mutex =3D PTHREAD_MUTEX_INITIALIZER; + +int pkts_in_flight; + +/* The payload is a word consisting of a packet sequence number in the upp= er + * 16-bits and a intra packet data sequence number in the lower 16 bits. S= o the 3rd packet's + * 5th word of data will contain the number (2<<16) | 4 as they are number= ed from 0. + */ +static void write_payload(void *dest, u32 pkt_nb, u32 start, u32 size) +{ + u32 *ptr =3D (u32 *)dest, i; + + start /=3D sizeof(*ptr); + size /=3D sizeof(*ptr); + for (i =3D 0; i < size; i++) + ptr[i] =3D htonl(pkt_nb << 16 | (i + start)); +} + +static void gen_eth_hdr(struct xsk_socket_info *xsk, struct ethhdr *eth_hd= r) +{ + memcpy(eth_hdr->h_dest, xsk->dst_mac, ETH_ALEN); + memcpy(eth_hdr->h_source, xsk->src_mac, ETH_ALEN); + eth_hdr->h_proto =3D htons(ETH_P_LOOPBACK); +} + +static bool is_umem_valid(struct ifobject *ifobj) +{ + return !!ifobj->umem->umem; +} + +static u32 mode_to_xdp_flags(enum test_mode mode) +{ + return (mode =3D=3D TEST_MODE_SKB) ? XDP_FLAGS_SKB_MODE : XDP_FLAGS_DRV_M= ODE; +} + +static u64 umem_size(struct xsk_umem_info *umem) +{ + return umem->num_frames * umem->frame_size; +} + +int xsk_configure_umem(struct ifobject *ifobj, struct xsk_umem_info *umem,= void *buffer, + u64 size) +{ + struct xsk_umem_config cfg =3D { + .fill_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS, + .comp_size =3D XSK_RING_CONS__DEFAULT_NUM_DESCS, + .frame_size =3D umem->frame_size, + .frame_headroom =3D umem->frame_headroom, + .flags =3D XSK_UMEM__DEFAULT_FLAGS + }; + int ret; + + if (umem->fill_size) + cfg.fill_size =3D umem->fill_size; + + if (umem->comp_size) + cfg.comp_size =3D umem->comp_size; + + if (umem->unaligned_mode) + cfg.flags |=3D XDP_UMEM_UNALIGNED_CHUNK_FLAG; + + ret =3D xsk_umem__create(&umem->umem, buffer, size, + &umem->fq, &umem->cq, &cfg); + if (ret) + return ret; + + umem->buffer =3D buffer; + if (ifobj->shared_umem && ifobj->rx_on) { + umem->base_addr =3D umem_size(umem); + umem->next_buffer =3D umem_size(umem); + } + + return 0; +} + +static u64 umem_alloc_buffer(struct xsk_umem_info *umem) +{ + u64 addr; + + addr =3D umem->next_buffer; + umem->next_buffer +=3D umem->frame_size; + if (umem->next_buffer >=3D umem->base_addr + umem_size(umem)) + umem->next_buffer =3D umem->base_addr; + + return addr; +} + +static void umem_reset_alloc(struct xsk_umem_info *umem) +{ + umem->next_buffer =3D 0; +} + +static void enable_busy_poll(struct xsk_socket_info *xsk) +{ + int sock_opt; + + sock_opt =3D 1; + if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_PREFER_BUSY_POLL, + (void *)&sock_opt, sizeof(sock_opt)) < 0) + exit_with_error(errno); + + sock_opt =3D 20; + if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL, + (void *)&sock_opt, sizeof(sock_opt)) < 0) + exit_with_error(errno); + + sock_opt =3D xsk->batch_size; + if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL_BUDGET, + (void *)&sock_opt, sizeof(sock_opt)) < 0) + exit_with_error(errno); +} + +int xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info= *umem, + struct ifobject *ifobject, bool shared) +{ + struct xsk_socket_config cfg =3D {}; + struct xsk_ring_cons *rxr; + struct xsk_ring_prod *txr; + + xsk->umem =3D umem; + cfg.rx_size =3D xsk->rxqsize; + cfg.tx_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS; + cfg.bind_flags =3D ifobject->bind_flags; + if (shared) + cfg.bind_flags |=3D XDP_SHARED_UMEM; + if (ifobject->mtu > MAX_ETH_PKT_SIZE) + cfg.bind_flags |=3D XDP_USE_SG; + if (umem->comp_size) + cfg.tx_size =3D umem->comp_size; + if (umem->fill_size) + cfg.rx_size =3D umem->fill_size; + + txr =3D ifobject->tx_on ? &xsk->tx : NULL; + rxr =3D ifobject->rx_on ? &xsk->rx : NULL; + return xsk_socket__create(&xsk->xsk, ifobject->ifindex, 0, umem->umem, rx= r, txr, &cfg); +} + +#define MAX_SKB_FRAGS_PATH "/proc/sys/net/core/max_skb_frags" +static unsigned int get_max_skb_frags(void) +{ + unsigned int max_skb_frags =3D 0; + FILE *file; + + file =3D fopen(MAX_SKB_FRAGS_PATH, "r"); + if (!file) { + ksft_print_msg("Error opening %s\n", MAX_SKB_FRAGS_PATH); + return 0; + } + + if (fscanf(file, "%u", &max_skb_frags) !=3D 1) + ksft_print_msg("Error reading %s\n", MAX_SKB_FRAGS_PATH); + + fclose(file); + return max_skb_frags; +} + +static int set_ring_size(struct ifobject *ifobj) +{ + int ret; + u32 ctr =3D 0; + + while (ctr++ < SOCK_RECONF_CTR) { + ret =3D set_hw_ring_size(ifobj->ifname, &ifobj->ring); + if (!ret) + break; + + /* Retry if it fails */ + if (ctr >=3D SOCK_RECONF_CTR || errno !=3D EBUSY) + return -errno; + + usleep(USLEEP_MAX); + } + + return ret; +} + +int hw_ring_size_reset(struct ifobject *ifobj) +{ + ifobj->ring.tx_pending =3D ifobj->set_ring.default_tx; + ifobj->ring.rx_pending =3D ifobj->set_ring.default_rx; + return set_ring_size(ifobj); +} + +static void __test_spec_init(struct test_spec *test, struct ifobject *ifob= j_tx, + struct ifobject *ifobj_rx) +{ + u32 i, j; + + for (i =3D 0; i < MAX_INTERFACES; i++) { + struct ifobject *ifobj =3D i ? ifobj_rx : ifobj_tx; + + ifobj->xsk =3D &ifobj->xsk_arr[0]; + ifobj->use_poll =3D false; + ifobj->use_fill_ring =3D true; + ifobj->release_rx =3D true; + ifobj->validation_func =3D NULL; + ifobj->use_metadata =3D false; + + if (i =3D=3D 0) { + ifobj->rx_on =3D false; + ifobj->tx_on =3D true; + } else { + ifobj->rx_on =3D true; + ifobj->tx_on =3D false; + } + + memset(ifobj->umem, 0, sizeof(*ifobj->umem)); + ifobj->umem->num_frames =3D DEFAULT_UMEM_BUFFERS; + ifobj->umem->frame_size =3D XSK_UMEM__DEFAULT_FRAME_SIZE; + + for (j =3D 0; j < MAX_SOCKETS; j++) { + memset(&ifobj->xsk_arr[j], 0, sizeof(ifobj->xsk_arr[j])); + ifobj->xsk_arr[j].rxqsize =3D XSK_RING_CONS__DEFAULT_NUM_DESCS; + ifobj->xsk_arr[j].batch_size =3D DEFAULT_BATCH_SIZE; + if (i =3D=3D 0) + ifobj->xsk_arr[j].pkt_stream =3D test->tx_pkt_stream_default; + else + ifobj->xsk_arr[j].pkt_stream =3D test->rx_pkt_stream_default; + + memcpy(ifobj->xsk_arr[j].src_mac, g_mac, ETH_ALEN); + memcpy(ifobj->xsk_arr[j].dst_mac, g_mac, ETH_ALEN); + ifobj->xsk_arr[j].src_mac[5] +=3D ((j * 2) + 0); + ifobj->xsk_arr[j].dst_mac[5] +=3D ((j * 2) + 1); + } + } + + if (ifobj_tx->hw_ring_size_supp) + hw_ring_size_reset(ifobj_tx); + + test->ifobj_tx =3D ifobj_tx; + test->ifobj_rx =3D ifobj_rx; + test->current_step =3D 0; + test->total_steps =3D 1; + test->nb_sockets =3D 1; + test->fail =3D false; + test->set_ring =3D false; + test->adjust_tail =3D false; + test->adjust_tail_support =3D false; + test->mtu =3D MAX_ETH_PKT_SIZE; + test->xdp_prog_rx =3D ifobj_rx->xdp_progs->progs.xsk_def_prog; + test->xskmap_rx =3D ifobj_rx->xdp_progs->maps.xsk; + test->xdp_prog_tx =3D ifobj_tx->xdp_progs->progs.xsk_def_prog; + test->xskmap_tx =3D ifobj_tx->xdp_progs->maps.xsk; +} + +void test_init(struct test_spec *test, struct ifobject *ifobj_tx, + struct ifobject *ifobj_rx, enum test_mode mode, + const struct test_spec *test_to_run) +{ + struct pkt_stream *tx_pkt_stream; + struct pkt_stream *rx_pkt_stream; + u32 i; + + tx_pkt_stream =3D test->tx_pkt_stream_default; + rx_pkt_stream =3D test->rx_pkt_stream_default; + memset(test, 0, sizeof(*test)); + test->tx_pkt_stream_default =3D tx_pkt_stream; + test->rx_pkt_stream_default =3D rx_pkt_stream; + + for (i =3D 0; i < MAX_INTERFACES; i++) { + struct ifobject *ifobj =3D i ? ifobj_rx : ifobj_tx; + + ifobj->bind_flags =3D XDP_USE_NEED_WAKEUP; + if (mode =3D=3D TEST_MODE_ZC) + ifobj->bind_flags |=3D XDP_ZEROCOPY; + else + ifobj->bind_flags |=3D XDP_COPY; + } + + memcpy(test->name, test_to_run->name, MAX_TEST_NAME_SIZE); + test->test_func =3D test_to_run->test_func; + test->mode =3D mode; + __test_spec_init(test, ifobj_tx, ifobj_rx); +} + +static void test_spec_reset(struct test_spec *test) +{ + __test_spec_init(test, test->ifobj_tx, test->ifobj_rx); +} + +static void test_spec_set_xdp_prog(struct test_spec *test, struct bpf_prog= ram *xdp_prog_rx, + struct bpf_program *xdp_prog_tx, struct bpf_map *xskmap_rx, + struct bpf_map *xskmap_tx) +{ + test->xdp_prog_rx =3D xdp_prog_rx; + test->xdp_prog_tx =3D xdp_prog_tx; + test->xskmap_rx =3D xskmap_rx; + test->xskmap_tx =3D xskmap_tx; +} + +static int test_spec_set_mtu(struct test_spec *test, int mtu) +{ + int err; + + if (test->ifobj_rx->mtu !=3D mtu) { + err =3D xsk_set_mtu(test->ifobj_rx->ifindex, mtu); + if (err) + return err; + test->ifobj_rx->mtu =3D mtu; + } + if (test->ifobj_tx->mtu !=3D mtu) { + err =3D xsk_set_mtu(test->ifobj_tx->ifindex, mtu); + if (err) + return err; + test->ifobj_tx->mtu =3D mtu; + } + + return 0; +} + +void pkt_stream_reset(struct pkt_stream *pkt_stream) +{ + if (pkt_stream) { + pkt_stream->current_pkt_nb =3D 0; + pkt_stream->nb_rx_pkts =3D 0; + } +} + +static struct pkt *pkt_stream_get_next_tx_pkt(struct pkt_stream *pkt_strea= m) +{ + if (pkt_stream->current_pkt_nb >=3D pkt_stream->nb_pkts) + return NULL; + + return &pkt_stream->pkts[pkt_stream->current_pkt_nb++]; +} + +static struct pkt *pkt_stream_get_next_rx_pkt(struct pkt_stream *pkt_strea= m, u32 *pkts_sent) +{ + while (pkt_stream->current_pkt_nb < pkt_stream->nb_pkts) { + (*pkts_sent)++; + if (pkt_stream->pkts[pkt_stream->current_pkt_nb].valid) + return &pkt_stream->pkts[pkt_stream->current_pkt_nb++]; + pkt_stream->current_pkt_nb++; + } + return NULL; +} + +void pkt_stream_delete(struct pkt_stream *pkt_stream) +{ + free(pkt_stream->pkts); + free(pkt_stream); +} + +void pkt_stream_restore_default(struct test_spec *test) +{ + struct pkt_stream *tx_pkt_stream =3D test->ifobj_tx->xsk->pkt_stream; + struct pkt_stream *rx_pkt_stream =3D test->ifobj_rx->xsk->pkt_stream; + + if (tx_pkt_stream !=3D test->tx_pkt_stream_default) { + pkt_stream_delete(test->ifobj_tx->xsk->pkt_stream); + test->ifobj_tx->xsk->pkt_stream =3D test->tx_pkt_stream_default; + } + + if (rx_pkt_stream !=3D test->rx_pkt_stream_default) { + pkt_stream_delete(test->ifobj_rx->xsk->pkt_stream); + test->ifobj_rx->xsk->pkt_stream =3D test->rx_pkt_stream_default; + } +} + +static struct pkt_stream *__pkt_stream_alloc(u32 nb_pkts) +{ + struct pkt_stream *pkt_stream; + + pkt_stream =3D calloc(1, sizeof(*pkt_stream)); + if (!pkt_stream) + return NULL; + + pkt_stream->pkts =3D calloc(nb_pkts, sizeof(*pkt_stream->pkts)); + if (!pkt_stream->pkts) { + free(pkt_stream); + return NULL; + } + + pkt_stream->nb_pkts =3D nb_pkts; + return pkt_stream; +} + +static u32 pkt_nb_frags(u32 frame_size, struct pkt_stream *pkt_stream, str= uct pkt *pkt) +{ + u32 nb_frags =3D 1, next_frag; + + if (!pkt) + return 1; + + if (!pkt_stream->verbatim) { + if (!pkt->valid || !pkt->len) + return 1; + return ceil_u32(pkt->len, frame_size); + } + + /* Search for the end of the packet in verbatim mode */ + if (!pkt_continues(pkt->options)) + return nb_frags; + + next_frag =3D pkt_stream->current_pkt_nb; + pkt++; + while (next_frag++ < pkt_stream->nb_pkts) { + nb_frags++; + if (!pkt_continues(pkt->options) || !pkt->valid) + break; + pkt++; + } + return nb_frags; +} + +static bool set_pkt_valid(int offset, u32 len) +{ + return len <=3D MAX_ETH_JUMBO_SIZE; +} + +static void pkt_set(struct pkt_stream *pkt_stream, struct pkt *pkt, int of= fset, u32 len) +{ + pkt->offset =3D offset; + pkt->len =3D len; + pkt->valid =3D set_pkt_valid(offset, len); +} + +static void pkt_stream_pkt_set(struct pkt_stream *pkt_stream, struct pkt *= pkt, int offset, u32 len) +{ + bool prev_pkt_valid =3D pkt->valid; + + pkt_set(pkt_stream, pkt, offset, len); + pkt_stream->nb_valid_entries +=3D pkt->valid - prev_pkt_valid; +} + +static u32 pkt_get_buffer_len(struct xsk_umem_info *umem, u32 len) +{ + return ceil_u32(len, umem->frame_size) * umem->frame_size; +} + +static struct pkt_stream *__pkt_stream_generate(u32 nb_pkts, u32 pkt_len, = u32 nb_start, u32 nb_off) +{ + struct pkt_stream *pkt_stream; + u32 i; + + pkt_stream =3D __pkt_stream_alloc(nb_pkts); + if (!pkt_stream) + exit_with_error(ENOMEM); + + pkt_stream->nb_pkts =3D nb_pkts; + pkt_stream->max_pkt_len =3D pkt_len; + for (i =3D 0; i < nb_pkts; i++) { + struct pkt *pkt =3D &pkt_stream->pkts[i]; + + pkt_stream_pkt_set(pkt_stream, pkt, 0, pkt_len); + pkt->pkt_nb =3D nb_start + i * nb_off; + } + + return pkt_stream; +} + +struct pkt_stream *pkt_stream_generate(u32 nb_pkts, u32 pkt_len) +{ + return __pkt_stream_generate(nb_pkts, pkt_len, 0, 1); +} + +static struct pkt_stream *pkt_stream_clone(struct pkt_stream *pkt_stream) +{ + return pkt_stream_generate(pkt_stream->nb_pkts, pkt_stream->pkts[0].len); +} + +static void pkt_stream_replace_ifobject(struct ifobject *ifobj, u32 nb_pkt= s, u32 pkt_len) +{ + ifobj->xsk->pkt_stream =3D pkt_stream_generate(nb_pkts, pkt_len); +} + +static void pkt_stream_replace(struct test_spec *test, u32 nb_pkts, u32 pk= t_len) +{ + pkt_stream_replace_ifobject(test->ifobj_tx, nb_pkts, pkt_len); + pkt_stream_replace_ifobject(test->ifobj_rx, nb_pkts, pkt_len); +} + +static void __pkt_stream_replace_half(struct ifobject *ifobj, u32 pkt_len, + int offset) +{ + struct pkt_stream *pkt_stream; + u32 i; + + pkt_stream =3D pkt_stream_clone(ifobj->xsk->pkt_stream); + for (i =3D 1; i < ifobj->xsk->pkt_stream->nb_pkts; i +=3D 2) + pkt_stream_pkt_set(pkt_stream, &pkt_stream->pkts[i], offset, pkt_len); + + ifobj->xsk->pkt_stream =3D pkt_stream; +} + +static void pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, i= nt offset) +{ + __pkt_stream_replace_half(test->ifobj_tx, pkt_len, offset); + __pkt_stream_replace_half(test->ifobj_rx, pkt_len, offset); +} + +static void pkt_stream_receive_half(struct test_spec *test) +{ + struct pkt_stream *pkt_stream =3D test->ifobj_tx->xsk->pkt_stream; + u32 i; + + test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(pkt_stream->nb_pk= ts, + pkt_stream->pkts[0].len); + pkt_stream =3D test->ifobj_rx->xsk->pkt_stream; + for (i =3D 1; i < pkt_stream->nb_pkts; i +=3D 2) + pkt_stream->pkts[i].valid =3D false; + + pkt_stream->nb_valid_entries /=3D 2; +} + +static void pkt_stream_even_odd_sequence(struct test_spec *test) +{ + struct pkt_stream *pkt_stream; + u32 i; + + for (i =3D 0; i < test->nb_sockets; i++) { + pkt_stream =3D test->ifobj_tx->xsk_arr[i].pkt_stream; + pkt_stream =3D __pkt_stream_generate(pkt_stream->nb_pkts / 2, + pkt_stream->pkts[0].len, i, 2); + test->ifobj_tx->xsk_arr[i].pkt_stream =3D pkt_stream; + + pkt_stream =3D test->ifobj_rx->xsk_arr[i].pkt_stream; + pkt_stream =3D __pkt_stream_generate(pkt_stream->nb_pkts / 2, + pkt_stream->pkts[0].len, i, 2); + test->ifobj_rx->xsk_arr[i].pkt_stream =3D pkt_stream; + } +} + +static u64 pkt_get_addr(struct pkt *pkt, struct xsk_umem_info *umem) +{ + if (!pkt->valid) + return pkt->offset; + return pkt->offset + umem_alloc_buffer(umem); +} + +static void pkt_stream_cancel(struct pkt_stream *pkt_stream) +{ + pkt_stream->current_pkt_nb--; +} + +static void pkt_generate(struct xsk_socket_info *xsk, struct xsk_umem_info= *umem, u64 addr, u32 len, + u32 pkt_nb, u32 bytes_written) +{ + void *data =3D xsk_umem__get_data(umem->buffer, addr); + + if (len < MIN_PKT_SIZE) + return; + + if (!bytes_written) { + gen_eth_hdr(xsk, data); + + len -=3D PKT_HDR_SIZE; + data +=3D PKT_HDR_SIZE; + } else { + bytes_written -=3D PKT_HDR_SIZE; + } + + write_payload(data, pkt_nb, bytes_written, len); +} + +static struct pkt_stream *__pkt_stream_generate_custom(struct ifobject *if= obj, struct pkt *frames, + u32 nb_frames, bool verbatim) +{ + u32 i, len =3D 0, pkt_nb =3D 0, payload =3D 0; + struct pkt_stream *pkt_stream; + + pkt_stream =3D __pkt_stream_alloc(nb_frames); + if (!pkt_stream) + exit_with_error(ENOMEM); + + for (i =3D 0; i < nb_frames; i++) { + struct pkt *pkt =3D &pkt_stream->pkts[pkt_nb]; + struct pkt *frame =3D &frames[i]; + + pkt->offset =3D frame->offset; + if (verbatim) { + *pkt =3D *frame; + pkt->pkt_nb =3D payload; + if (!frame->valid || !pkt_continues(frame->options)) + payload++; + } else { + if (frame->valid) + len +=3D frame->len; + if (frame->valid && pkt_continues(frame->options)) + continue; + + pkt->pkt_nb =3D pkt_nb; + pkt->len =3D len; + pkt->valid =3D frame->valid; + pkt->options =3D 0; + + len =3D 0; + } + + print_verbose("offset: %d len: %u valid: %u options: %u pkt_nb: %u\n", + pkt->offset, pkt->len, pkt->valid, pkt->options, pkt->pkt_nb); + + if (pkt->valid && pkt->len > pkt_stream->max_pkt_len) + pkt_stream->max_pkt_len =3D pkt->len; + + if (pkt->valid) + pkt_stream->nb_valid_entries++; + + pkt_nb++; + } + + pkt_stream->nb_pkts =3D pkt_nb; + pkt_stream->verbatim =3D verbatim; + return pkt_stream; +} + +static void pkt_stream_generate_custom(struct test_spec *test, struct pkt = *pkts, u32 nb_pkts) +{ + struct pkt_stream *pkt_stream; + + pkt_stream =3D __pkt_stream_generate_custom(test->ifobj_tx, pkts, nb_pkts= , true); + test->ifobj_tx->xsk->pkt_stream =3D pkt_stream; + + pkt_stream =3D __pkt_stream_generate_custom(test->ifobj_rx, pkts, nb_pkts= , false); + test->ifobj_rx->xsk->pkt_stream =3D pkt_stream; +} + +static void pkt_print_data(u32 *data, u32 cnt) +{ + u32 i; + + for (i =3D 0; i < cnt; i++) { + u32 seqnum, pkt_nb; + + seqnum =3D ntohl(*data) & 0xffff; + pkt_nb =3D ntohl(*data) >> 16; + ksft_print_msg("%u:%u ", pkt_nb, seqnum); + data++; + } +} + +static void pkt_dump(void *pkt, u32 len, bool eth_header) +{ + struct ethhdr *ethhdr =3D pkt; + u32 i, *data; + + if (eth_header) { + /*extract L2 frame */ + ksft_print_msg("DEBUG>> L2: dst mac: "); + for (i =3D 0; i < ETH_ALEN; i++) + ksft_print_msg("%02X", ethhdr->h_dest[i]); + + ksft_print_msg("\nDEBUG>> L2: src mac: "); + for (i =3D 0; i < ETH_ALEN; i++) + ksft_print_msg("%02X", ethhdr->h_source[i]); + + data =3D pkt + PKT_HDR_SIZE; + } else { + data =3D pkt; + } + + /*extract L5 frame */ + ksft_print_msg("\nDEBUG>> L5: seqnum: "); + pkt_print_data(data, PKT_DUMP_NB_TO_PRINT); + ksft_print_msg("...."); + if (len > PKT_DUMP_NB_TO_PRINT * sizeof(u32)) { + ksft_print_msg("\n.... "); + pkt_print_data(data + len / sizeof(u32) - PKT_DUMP_NB_TO_PRINT, + PKT_DUMP_NB_TO_PRINT); + } + ksft_print_msg("\n---------------------------------------\n"); +} + +static bool is_offset_correct(struct xsk_umem_info *umem, struct pkt *pkt,= u64 addr) +{ + u32 headroom =3D umem->unaligned_mode ? 0 : umem->frame_headroom; + u32 offset =3D addr % umem->frame_size, expected_offset; + int pkt_offset =3D pkt->valid ? pkt->offset : 0; + + if (!umem->unaligned_mode) + pkt_offset =3D 0; + + expected_offset =3D (pkt_offset + headroom + XDP_PACKET_HEADROOM) % umem-= >frame_size; + + if (offset =3D=3D expected_offset) + return true; + + ksft_print_msg("[%s] expected [%u], got [%u]\n", __func__, expected_offse= t, offset); + return false; +} + +static bool is_metadata_correct(struct pkt *pkt, void *buffer, u64 addr) +{ + void *data =3D xsk_umem__get_data(buffer, addr); + struct xdp_info *meta =3D data - sizeof(struct xdp_info); + + if (meta->count !=3D pkt->pkt_nb) { + ksft_print_msg("[%s] expected meta_count [%d], got meta_count [%llu]\n", + __func__, pkt->pkt_nb, + (unsigned long long)meta->count); + return false; + } + + return true; +} + +static bool is_adjust_tail_supported(struct xsk_xdp_progs *skel_rx) +{ + struct bpf_map *data_map; + int adjust_value =3D 0; + int key =3D 0; + int ret; + + data_map =3D bpf_object__find_map_by_name(skel_rx->obj, "xsk_xdp_.bss"); + if (!data_map || !bpf_map__is_internal(data_map)) { + ksft_print_msg("Error: could not find bss section of XDP program\n"); + exit_with_error(errno); + } + + ret =3D bpf_map_lookup_elem(bpf_map__fd(data_map), &key, &adjust_value); + if (ret) { + ksft_print_msg("Error: bpf_map_lookup_elem failed with error %d\n", ret); + exit_with_error(errno); + } + + /* Set the 'adjust_value' variable to -EOPNOTSUPP in the XDP program if t= he adjust_tail + * helper is not supported. Skip the adjust_tail test case in this scenar= io. + */ + return adjust_value !=3D -EOPNOTSUPP; +} + +static bool is_frag_valid(struct xsk_umem_info *umem, u64 addr, u32 len, u= 32 expected_pkt_nb, + u32 bytes_processed) +{ + u32 seqnum, pkt_nb, *pkt_data, words_to_end, expected_seqnum; + void *data =3D xsk_umem__get_data(umem->buffer, addr); + + addr -=3D umem->base_addr; + + if (addr >=3D umem->num_frames * umem->frame_size || + addr + len > umem->num_frames * umem->frame_size) { + ksft_print_msg("Frag invalid addr: %llx len: %u\n", + (unsigned long long)addr, len); + return false; + } + if (!umem->unaligned_mode && addr % umem->frame_size + len > umem->frame_= size) { + ksft_print_msg("Frag crosses frame boundary addr: %llx len: %u\n", + (unsigned long long)addr, len); + return false; + } + + pkt_data =3D data; + if (!bytes_processed) { + pkt_data +=3D PKT_HDR_SIZE / sizeof(*pkt_data); + len -=3D PKT_HDR_SIZE; + } else { + bytes_processed -=3D PKT_HDR_SIZE; + } + + expected_seqnum =3D bytes_processed / sizeof(*pkt_data); + seqnum =3D ntohl(*pkt_data) & 0xffff; + pkt_nb =3D ntohl(*pkt_data) >> 16; + + if (expected_pkt_nb !=3D pkt_nb) { + ksft_print_msg("[%s] expected pkt_nb [%u], got pkt_nb [%u]\n", + __func__, expected_pkt_nb, pkt_nb); + goto error; + } + if (expected_seqnum !=3D seqnum) { + ksft_print_msg("[%s] expected seqnum at start [%u], got seqnum [%u]\n", + __func__, expected_seqnum, seqnum); + goto error; + } + + words_to_end =3D len / sizeof(*pkt_data) - 1; + pkt_data +=3D words_to_end; + seqnum =3D ntohl(*pkt_data) & 0xffff; + expected_seqnum +=3D words_to_end; + if (expected_seqnum !=3D seqnum) { + ksft_print_msg("[%s] expected seqnum at end [%u], got seqnum [%u]\n", + __func__, expected_seqnum, seqnum); + goto error; + } + + return true; + +error: + pkt_dump(data, len, !bytes_processed); + return false; +} + +static bool is_pkt_valid(struct pkt *pkt, void *buffer, u64 addr, u32 len) +{ + if (pkt->len !=3D len) { + ksft_print_msg("[%s] expected packet length [%d], got length [%d]\n", + __func__, pkt->len, len); + pkt_dump(xsk_umem__get_data(buffer, addr), len, true); + return false; + } + + return true; +} + +static u32 load_value(u32 *counter) +{ + return __atomic_load_n(counter, __ATOMIC_ACQUIRE); +} + +static bool kick_tx_with_check(struct xsk_socket_info *xsk, int *ret) +{ + u32 max_budget =3D MAX_TX_BUDGET_DEFAULT; + u32 cons, ready_to_send; + int delta; + + cons =3D load_value(xsk->tx.consumer); + ready_to_send =3D load_value(xsk->tx.producer) - cons; + *ret =3D sendto(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, 0); + + delta =3D load_value(xsk->tx.consumer) - cons; + /* By default, xsk should consume exact @max_budget descs at one + * send in this case where hitting the max budget limit in while + * loop is triggered in __xsk_generic_xmit(). Please make sure that + * the number of descs to be sent is larger than @max_budget, or + * else the tx.consumer will be updated in xskq_cons_peek_desc() + * in time which hides the issue we try to verify. + */ + if (ready_to_send > max_budget && delta !=3D max_budget) + return false; + + return true; +} + +int kick_tx(struct xsk_socket_info *xsk) +{ + int ret; + + if (xsk->check_consumer) { + if (!kick_tx_with_check(xsk, &ret)) + return TEST_FAILURE; + } else { + ret =3D sendto(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, 0); + } + if (ret >=3D 0) + return TEST_PASS; + if (errno =3D=3D ENOBUFS || errno =3D=3D EAGAIN || errno =3D=3D EBUSY || = errno =3D=3D ENETDOWN) { + usleep(100); + return TEST_PASS; + } + return TEST_FAILURE; +} + +int kick_rx(struct xsk_socket_info *xsk) +{ + int ret; + + ret =3D recvfrom(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, N= ULL); + if (ret < 0) + return TEST_FAILURE; + + return TEST_PASS; +} + +static int complete_pkts(struct xsk_socket_info *xsk, int batch_size) +{ + unsigned int rcvd; + u32 idx; + int ret; + + if (xsk_ring_prod__needs_wakeup(&xsk->tx)) { + ret =3D kick_tx(xsk); + if (ret) + return TEST_FAILURE; + } + + rcvd =3D xsk_ring_cons__peek(&xsk->umem->cq, batch_size, &idx); + if (rcvd) { + if (rcvd > xsk->outstanding_tx) { + u64 addr =3D *xsk_ring_cons__comp_addr(&xsk->umem->cq, idx + rcvd - 1); + + ksft_print_msg("[%s] Too many packets completed\n", __func__); + ksft_print_msg("Last completion address: %llx\n", + (unsigned long long)addr); + return TEST_FAILURE; + } + + xsk_ring_cons__release(&xsk->umem->cq, rcvd); + xsk->outstanding_tx -=3D rcvd; + } + + return TEST_PASS; +} + +static int __receive_pkts(struct test_spec *test, struct xsk_socket_info *= xsk) +{ + u32 frags_processed =3D 0, nb_frags =3D 0, pkt_len =3D 0; + u32 idx_rx =3D 0, idx_fq =3D 0, rcvd, pkts_sent =3D 0; + struct pkt_stream *pkt_stream =3D xsk->pkt_stream; + struct ifobject *ifobj =3D test->ifobj_rx; + struct xsk_umem_info *umem =3D xsk->umem; + struct pollfd fds =3D { }; + struct pkt *pkt; + u64 first_addr =3D 0; + int ret; + + fds.fd =3D xsk_socket__fd(xsk->xsk); + fds.events =3D POLLIN; + + ret =3D kick_rx(xsk); + if (ret) + return TEST_FAILURE; + + if (ifobj->use_poll) { + ret =3D poll(&fds, 1, POLL_TMOUT); + if (ret < 0) + return TEST_FAILURE; + + if (!ret) { + if (!is_umem_valid(test->ifobj_tx)) + return TEST_PASS; + + ksft_print_msg("ERROR: [%s] Poll timed out\n", __func__); + return TEST_CONTINUE; + } + + if (!(fds.revents & POLLIN)) + return TEST_CONTINUE; + } + + rcvd =3D xsk_ring_cons__peek(&xsk->rx, xsk->batch_size, &idx_rx); + if (!rcvd) + return TEST_CONTINUE; + + if (ifobj->use_fill_ring) { + ret =3D xsk_ring_prod__reserve(&umem->fq, rcvd, &idx_fq); + while (ret !=3D rcvd) { + if (xsk_ring_prod__needs_wakeup(&umem->fq)) { + ret =3D poll(&fds, 1, POLL_TMOUT); + if (ret < 0) + return TEST_FAILURE; + } + ret =3D xsk_ring_prod__reserve(&umem->fq, rcvd, &idx_fq); + } + } + + while (frags_processed < rcvd) { + const struct xdp_desc *desc =3D xsk_ring_cons__rx_desc(&xsk->rx, idx_rx+= +); + u64 addr =3D desc->addr, orig; + + orig =3D xsk_umem__extract_addr(addr); + addr =3D xsk_umem__add_offset_to_addr(addr); + + if (!nb_frags) { + pkt =3D pkt_stream_get_next_rx_pkt(pkt_stream, &pkts_sent); + if (!pkt) { + ksft_print_msg("[%s] received too many packets addr: %lx len %u\n", + __func__, addr, desc->len); + return TEST_FAILURE; + } + } + + print_verbose("Rx: addr: %lx len: %u options: %u pkt_nb: %u valid: %u\n", + addr, desc->len, desc->options, pkt->pkt_nb, pkt->valid); + + if (!is_frag_valid(umem, addr, desc->len, pkt->pkt_nb, pkt_len) || + !is_offset_correct(umem, pkt, addr) || (ifobj->use_metadata && + !is_metadata_correct(pkt, umem->buffer, addr))) + return TEST_FAILURE; + + if (!nb_frags++) + first_addr =3D addr; + frags_processed++; + pkt_len +=3D desc->len; + if (ifobj->use_fill_ring) + *xsk_ring_prod__fill_addr(&umem->fq, idx_fq++) =3D orig; + + if (pkt_continues(desc->options)) + continue; + + /* The complete packet has been received */ + if (!is_pkt_valid(pkt, umem->buffer, first_addr, pkt_len) || + !is_offset_correct(umem, pkt, addr)) + return TEST_FAILURE; + + pkt_stream->nb_rx_pkts++; + nb_frags =3D 0; + pkt_len =3D 0; + } + + if (nb_frags) { + /* In the middle of a packet. Start over from beginning of packet. */ + idx_rx -=3D nb_frags; + xsk_ring_cons__cancel(&xsk->rx, nb_frags); + if (ifobj->use_fill_ring) { + idx_fq -=3D nb_frags; + xsk_ring_prod__cancel(&umem->fq, nb_frags); + } + frags_processed -=3D nb_frags; + } + + if (ifobj->use_fill_ring) + xsk_ring_prod__submit(&umem->fq, frags_processed); + if (ifobj->release_rx) + xsk_ring_cons__release(&xsk->rx, frags_processed); + + pthread_mutex_lock(&pacing_mutex); + pkts_in_flight -=3D pkts_sent; + pthread_mutex_unlock(&pacing_mutex); + pkts_sent =3D 0; + + return TEST_CONTINUE; +} + +bool all_packets_received(struct test_spec *test, struct xsk_socket_info *= xsk, u32 sock_num, + unsigned long *bitmap) +{ + struct pkt_stream *pkt_stream =3D xsk->pkt_stream; + + if (!pkt_stream) { + __set_bit(sock_num, bitmap); + return false; + } + + if (pkt_stream->nb_rx_pkts =3D=3D pkt_stream->nb_valid_entries) { + __set_bit(sock_num, bitmap); + if (bitmap_full(bitmap, test->nb_sockets)) + return true; + } + + return false; +} + +static int receive_pkts(struct test_spec *test) +{ + struct timeval tv_end, tv_now, tv_timeout =3D {THREAD_TMOUT, 0}; + DECLARE_BITMAP(bitmap, test->nb_sockets); + struct xsk_socket_info *xsk; + u32 sock_num =3D 0; + int res, ret; + + ret =3D gettimeofday(&tv_now, NULL); + if (ret) + exit_with_error(errno); + + timeradd(&tv_now, &tv_timeout, &tv_end); + + while (1) { + xsk =3D &test->ifobj_rx->xsk_arr[sock_num]; + + if ((all_packets_received(test, xsk, sock_num, bitmap))) + break; + + res =3D __receive_pkts(test, xsk); + if (!(res =3D=3D TEST_PASS || res =3D=3D TEST_CONTINUE)) + return res; + + ret =3D gettimeofday(&tv_now, NULL); + if (ret) + exit_with_error(errno); + + if (timercmp(&tv_now, &tv_end, >)) { + ksft_print_msg("ERROR: [%s] Receive loop timed out\n", __func__); + return TEST_FAILURE; + } + sock_num =3D (sock_num + 1) % test->nb_sockets; + } + + return TEST_PASS; +} + +static int __send_pkts(struct ifobject *ifobject, struct xsk_socket_info *= xsk, bool timeout) +{ + u32 i, idx =3D 0, valid_pkts =3D 0, valid_frags =3D 0, buffer_len; + struct pkt_stream *pkt_stream =3D xsk->pkt_stream; + struct xsk_umem_info *umem =3D ifobject->umem; + bool use_poll =3D ifobject->use_poll; + struct pollfd fds =3D { }; + int ret; + + buffer_len =3D pkt_get_buffer_len(umem, pkt_stream->max_pkt_len); + /* pkts_in_flight might be negative if many invalid packets are sent */ + if (pkts_in_flight >=3D (int)((umem_size(umem) - xsk->batch_size * buffer= _len) / + buffer_len)) { + ret =3D kick_tx(xsk); + if (ret) + return TEST_FAILURE; + return TEST_CONTINUE; + } + + fds.fd =3D xsk_socket__fd(xsk->xsk); + fds.events =3D POLLOUT; + + while (xsk_ring_prod__reserve(&xsk->tx, xsk->batch_size, &idx) < xsk->bat= ch_size) { + if (use_poll) { + ret =3D poll(&fds, 1, POLL_TMOUT); + if (timeout) { + if (ret < 0) { + ksft_print_msg("ERROR: [%s] Poll error %d\n", + __func__, errno); + return TEST_FAILURE; + } + if (ret =3D=3D 0) + return TEST_PASS; + break; + } + if (ret <=3D 0) { + ksft_print_msg("ERROR: [%s] Poll error %d\n", + __func__, errno); + return TEST_FAILURE; + } + } + + complete_pkts(xsk, xsk->batch_size); + } + + for (i =3D 0; i < xsk->batch_size; i++) { + struct pkt *pkt =3D pkt_stream_get_next_tx_pkt(pkt_stream); + u32 nb_frags_left, nb_frags, bytes_written =3D 0; + + if (!pkt) + break; + + nb_frags =3D pkt_nb_frags(umem->frame_size, pkt_stream, pkt); + if (nb_frags > xsk->batch_size - i) { + pkt_stream_cancel(pkt_stream); + xsk_ring_prod__cancel(&xsk->tx, xsk->batch_size - i); + break; + } + nb_frags_left =3D nb_frags; + + while (nb_frags_left--) { + struct xdp_desc *tx_desc =3D xsk_ring_prod__tx_desc(&xsk->tx, idx + i); + + tx_desc->addr =3D pkt_get_addr(pkt, ifobject->umem); + if (pkt_stream->verbatim) { + tx_desc->len =3D pkt->len; + tx_desc->options =3D pkt->options; + } else if (nb_frags_left) { + tx_desc->len =3D umem->frame_size; + tx_desc->options =3D XDP_PKT_CONTD; + } else { + tx_desc->len =3D pkt->len - bytes_written; + tx_desc->options =3D 0; + } + if (pkt->valid) + pkt_generate(xsk, umem, tx_desc->addr, tx_desc->len, pkt->pkt_nb, + bytes_written); + bytes_written +=3D tx_desc->len; + + print_verbose("Tx addr: %llx len: %u options: %u pkt_nb: %u\n", + tx_desc->addr, tx_desc->len, tx_desc->options, pkt->pkt_nb); + + if (nb_frags_left) { + i++; + if (pkt_stream->verbatim) + pkt =3D pkt_stream_get_next_tx_pkt(pkt_stream); + } + } + + if (pkt && pkt->valid) { + valid_pkts++; + valid_frags +=3D nb_frags; + } + } + + pthread_mutex_lock(&pacing_mutex); + pkts_in_flight +=3D valid_pkts; + pthread_mutex_unlock(&pacing_mutex); + + xsk_ring_prod__submit(&xsk->tx, i); + xsk->outstanding_tx +=3D valid_frags; + + if (use_poll) { + ret =3D poll(&fds, 1, POLL_TMOUT); + if (ret <=3D 0) { + if (ret =3D=3D 0 && timeout) + return TEST_PASS; + + ksft_print_msg("ERROR: [%s] Poll error %d\n", __func__, ret); + return TEST_FAILURE; + } + } + + if (!timeout) { + if (complete_pkts(xsk, i)) + return TEST_FAILURE; + + usleep(10); + return TEST_PASS; + } + + return TEST_CONTINUE; +} + +static int wait_for_tx_completion(struct xsk_socket_info *xsk) +{ + struct timeval tv_end, tv_now, tv_timeout =3D {THREAD_TMOUT, 0}; + int ret; + + ret =3D gettimeofday(&tv_now, NULL); + if (ret) + exit_with_error(errno); + timeradd(&tv_now, &tv_timeout, &tv_end); + + while (xsk->outstanding_tx) { + ret =3D gettimeofday(&tv_now, NULL); + if (ret) + exit_with_error(errno); + if (timercmp(&tv_now, &tv_end, >)) { + ksft_print_msg("ERROR: [%s] Transmission loop timed out\n", __func__); + return TEST_FAILURE; + } + + complete_pkts(xsk, xsk->batch_size); + } + + return TEST_PASS; +} + +bool all_packets_sent(struct test_spec *test, unsigned long *bitmap) +{ + return bitmap_full(bitmap, test->nb_sockets); +} + +static int send_pkts(struct test_spec *test, struct ifobject *ifobject) +{ + bool timeout =3D !is_umem_valid(test->ifobj_rx); + DECLARE_BITMAP(bitmap, test->nb_sockets); + u32 i, ret; + + while (!(all_packets_sent(test, bitmap))) { + for (i =3D 0; i < test->nb_sockets; i++) { + struct pkt_stream *pkt_stream; + + pkt_stream =3D ifobject->xsk_arr[i].pkt_stream; + if (!pkt_stream || pkt_stream->current_pkt_nb >=3D pkt_stream->nb_pkts)= { + __set_bit(i, bitmap); + continue; + } + ret =3D __send_pkts(ifobject, &ifobject->xsk_arr[i], timeout); + if (ret =3D=3D TEST_CONTINUE && !test->fail) + continue; + + if ((ret || test->fail) && !timeout) + return TEST_FAILURE; + + if (ret =3D=3D TEST_PASS && timeout) + return ret; + + ret =3D wait_for_tx_completion(&ifobject->xsk_arr[i]); + if (ret) + return TEST_FAILURE; + } + } + + return TEST_PASS; +} + +static int get_xsk_stats(struct xsk_socket *xsk, struct xdp_statistics *st= ats) +{ + int fd =3D xsk_socket__fd(xsk), err; + socklen_t optlen, expected_len; + + optlen =3D sizeof(*stats); + err =3D getsockopt(fd, SOL_XDP, XDP_STATISTICS, stats, &optlen); + if (err) { + ksft_print_msg("[%s] getsockopt(XDP_STATISTICS) error %u %s\n", + __func__, -err, strerror(-err)); + return TEST_FAILURE; + } + + expected_len =3D sizeof(struct xdp_statistics); + if (optlen !=3D expected_len) { + ksft_print_msg("[%s] getsockopt optlen error. Expected: %u got: %u\n", + __func__, expected_len, optlen); + return TEST_FAILURE; + } + + return TEST_PASS; +} + +static int validate_rx_dropped(struct ifobject *ifobject) +{ + struct xsk_socket *xsk =3D ifobject->xsk->xsk; + struct xdp_statistics stats; + int err; + + err =3D kick_rx(ifobject->xsk); + if (err) + return TEST_FAILURE; + + err =3D get_xsk_stats(xsk, &stats); + if (err) + return TEST_FAILURE; + + /* The receiver calls getsockopt after receiving the last (valid) + * packet which is not the final packet sent in this test (valid and + * invalid packets are sent in alternating fashion with the final + * packet being invalid). Since the last packet may or may not have + * been dropped already, both outcomes must be allowed. + */ + if (stats.rx_dropped =3D=3D ifobject->xsk->pkt_stream->nb_pkts / 2 || + stats.rx_dropped =3D=3D ifobject->xsk->pkt_stream->nb_pkts / 2 - 1) + return TEST_PASS; + + return TEST_FAILURE; +} + +static int validate_rx_full(struct ifobject *ifobject) +{ + struct xsk_socket *xsk =3D ifobject->xsk->xsk; + struct xdp_statistics stats; + int err; + + usleep(1000); + err =3D kick_rx(ifobject->xsk); + if (err) + return TEST_FAILURE; + + err =3D get_xsk_stats(xsk, &stats); + if (err) + return TEST_FAILURE; + + if (stats.rx_ring_full) + return TEST_PASS; + + return TEST_FAILURE; +} + +static int validate_fill_empty(struct ifobject *ifobject) +{ + struct xsk_socket *xsk =3D ifobject->xsk->xsk; + struct xdp_statistics stats; + int err; + + usleep(1000); + err =3D kick_rx(ifobject->xsk); + if (err) + return TEST_FAILURE; + + err =3D get_xsk_stats(xsk, &stats); + if (err) + return TEST_FAILURE; + + if (stats.rx_fill_ring_empty_descs) + return TEST_PASS; + + return TEST_FAILURE; +} + +static int validate_tx_invalid_descs(struct ifobject *ifobject) +{ + struct xsk_socket *xsk =3D ifobject->xsk->xsk; + int fd =3D xsk_socket__fd(xsk); + struct xdp_statistics stats; + socklen_t optlen; + int err; + + optlen =3D sizeof(stats); + err =3D getsockopt(fd, SOL_XDP, XDP_STATISTICS, &stats, &optlen); + if (err) { + ksft_print_msg("[%s] getsockopt(XDP_STATISTICS) error %u %s\n", + __func__, -err, strerror(-err)); + return TEST_FAILURE; + } + + if (stats.tx_invalid_descs !=3D ifobject->xsk->pkt_stream->nb_pkts / 2) { + ksft_print_msg("[%s] tx_invalid_descs incorrect. Got [%llu] expected [%u= ]\n", + __func__, + (unsigned long long)stats.tx_invalid_descs, + ifobject->xsk->pkt_stream->nb_pkts); + return TEST_FAILURE; + } + + return TEST_PASS; +} + +static void xsk_configure(struct test_spec *test, struct ifobject *ifobjec= t, + struct xsk_umem_info *umem, bool tx) +{ + int i, ret; + + for (i =3D 0; i < test->nb_sockets; i++) { + bool shared =3D (ifobject->shared_umem && tx) ? true : !!i; + u32 ctr =3D 0; + + while (ctr++ < SOCK_RECONF_CTR) { + ret =3D xsk_configure_socket(&ifobject->xsk_arr[i], umem, + ifobject, shared); + if (!ret) + break; + + /* Retry if it fails as xsk_socket__create() is asynchronous */ + if (ctr >=3D SOCK_RECONF_CTR) + exit_with_error(-ret); + usleep(USLEEP_MAX); + } + if (ifobject->busy_poll) + enable_busy_poll(&ifobject->xsk_arr[i]); + } +} + +static void thread_common_ops_tx(struct test_spec *test, struct ifobject *= ifobject) +{ + xsk_configure(test, ifobject, test->ifobj_rx->umem, true); + ifobject->xsk =3D &ifobject->xsk_arr[0]; + ifobject->xskmap =3D test->ifobj_rx->xskmap; + memcpy(ifobject->umem, test->ifobj_rx->umem, sizeof(struct xsk_umem_info)= ); + ifobject->umem->base_addr =3D 0; +} + +static void xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_= stream *pkt_stream, + bool fill_up) +{ + u32 rx_frame_size =3D umem->frame_size - XDP_PACKET_HEADROOM; + u32 idx =3D 0, filled =3D 0, buffers_to_fill, nb_pkts; + int ret; + + if (umem->num_frames < XSK_RING_PROD__DEFAULT_NUM_DESCS) + buffers_to_fill =3D umem->num_frames; + else + buffers_to_fill =3D umem->fill_size; + + ret =3D xsk_ring_prod__reserve(&umem->fq, buffers_to_fill, &idx); + if (ret !=3D buffers_to_fill) + exit_with_error(ENOSPC); + + while (filled < buffers_to_fill) { + struct pkt *pkt =3D pkt_stream_get_next_rx_pkt(pkt_stream, &nb_pkts); + u64 addr; + u32 i; + + for (i =3D 0; i < pkt_nb_frags(rx_frame_size, pkt_stream, pkt); i++) { + if (!pkt) { + if (!fill_up) + break; + addr =3D filled * umem->frame_size + umem->base_addr; + } else if (pkt->offset >=3D 0) { + addr =3D pkt->offset % umem->frame_size + umem_alloc_buffer(umem); + } else { + addr =3D pkt->offset + umem_alloc_buffer(umem); + } + + *xsk_ring_prod__fill_addr(&umem->fq, idx++) =3D addr; + if (++filled >=3D buffers_to_fill) + break; + } + } + xsk_ring_prod__submit(&umem->fq, filled); + xsk_ring_prod__cancel(&umem->fq, buffers_to_fill - filled); + + pkt_stream_reset(pkt_stream); + umem_reset_alloc(umem); +} + +static void thread_common_ops(struct test_spec *test, struct ifobject *ifo= bject) +{ + LIBBPF_OPTS(bpf_xdp_query_opts, opts); + int mmap_flags; + u64 umem_sz; + void *bufs; + int ret; + u32 i; + + umem_sz =3D ifobject->umem->num_frames * ifobject->umem->frame_size; + mmap_flags =3D MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE; + + if (ifobject->umem->unaligned_mode) + mmap_flags |=3D MAP_HUGETLB | MAP_HUGE_2MB; + + if (ifobject->shared_umem) + umem_sz *=3D 2; + + bufs =3D mmap(NULL, umem_sz, PROT_READ | PROT_WRITE, mmap_flags, -1, 0); + if (bufs =3D=3D MAP_FAILED) + exit_with_error(errno); + + ret =3D xsk_configure_umem(ifobject, ifobject->umem, bufs, umem_sz); + if (ret) + exit_with_error(-ret); + + xsk_configure(test, ifobject, ifobject->umem, false); + + ifobject->xsk =3D &ifobject->xsk_arr[0]; + + if (!ifobject->rx_on) + return; + + xsk_populate_fill_ring(ifobject->umem, ifobject->xsk->pkt_stream, ifobjec= t->use_fill_ring); + + for (i =3D 0; i < test->nb_sockets; i++) { + ifobject->xsk =3D &ifobject->xsk_arr[i]; + ret =3D xsk_update_xskmap(ifobject->xskmap, ifobject->xsk->xsk, i); + if (ret) + exit_with_error(errno); + } +} + +void *worker_testapp_validate_tx(void *arg) +{ + struct test_spec *test =3D (struct test_spec *)arg; + struct ifobject *ifobject =3D test->ifobj_tx; + int err; + + if (test->current_step =3D=3D 1) { + if (!ifobject->shared_umem) + thread_common_ops(test, ifobject); + else + thread_common_ops_tx(test, ifobject); + } + + err =3D send_pkts(test, ifobject); + + if (!err && ifobject->validation_func) + err =3D ifobject->validation_func(ifobject); + if (err) + test->fail =3D true; + + pthread_exit(NULL); +} + +void *worker_testapp_validate_rx(void *arg) +{ + struct test_spec *test =3D (struct test_spec *)arg; + struct ifobject *ifobject =3D test->ifobj_rx; + int err; + + if (test->current_step =3D=3D 1) { + thread_common_ops(test, ifobject); + } else { + xsk_clear_xskmap(ifobject->xskmap); + err =3D xsk_update_xskmap(ifobject->xskmap, ifobject->xsk->xsk, 0); + if (err) { + ksft_print_msg("Error: Failed to update xskmap, error %s\n", + strerror(-err)); + exit_with_error(-err); + } + } + + pthread_barrier_wait(&barr); + + err =3D receive_pkts(test); + + if (!err && ifobject->validation_func) + err =3D ifobject->validation_func(ifobject); + + if (err) { + if (test->adjust_tail && !is_adjust_tail_supported(ifobject->xdp_progs)) + test->adjust_tail_support =3D false; + else + test->fail =3D true; + } + + pthread_exit(NULL); +} + +static void testapp_clean_xsk_umem(struct ifobject *ifobj) +{ + u64 umem_sz =3D ifobj->umem->num_frames * ifobj->umem->frame_size; + + if (ifobj->shared_umem) + umem_sz *=3D 2; + + umem_sz =3D ceil_u64(umem_sz, HUGEPAGE_SIZE) * HUGEPAGE_SIZE; + xsk_umem__delete(ifobj->umem->umem); + munmap(ifobj->umem->buffer, umem_sz); +} + +static void handler(int signum) +{ + pthread_exit(NULL); +} + +static bool xdp_prog_changed_rx(struct test_spec *test) +{ + struct ifobject *ifobj =3D test->ifobj_rx; + + return ifobj->xdp_prog !=3D test->xdp_prog_rx || ifobj->mode !=3D test->m= ode; +} + +static bool xdp_prog_changed_tx(struct test_spec *test) +{ + struct ifobject *ifobj =3D test->ifobj_tx; + + return ifobj->xdp_prog !=3D test->xdp_prog_tx || ifobj->mode !=3D test->m= ode; +} + +static void xsk_reattach_xdp(struct ifobject *ifobj, struct bpf_program *x= dp_prog, + struct bpf_map *xskmap, enum test_mode mode) +{ + int err; + + xsk_detach_xdp_program(ifobj->ifindex, mode_to_xdp_flags(ifobj->mode)); + err =3D xsk_attach_xdp_program(xdp_prog, ifobj->ifindex, mode_to_xdp_flag= s(mode)); + if (err) { + ksft_print_msg("Error attaching XDP program\n"); + exit_with_error(-err); + } + + if (ifobj->mode !=3D mode && (mode =3D=3D TEST_MODE_DRV || mode =3D=3D TE= ST_MODE_ZC)) + if (!xsk_is_in_mode(ifobj->ifindex, XDP_FLAGS_DRV_MODE)) { + ksft_print_msg("ERROR: XDP prog not in DRV mode\n"); + exit_with_error(EINVAL); + } + + ifobj->xdp_prog =3D xdp_prog; + ifobj->xskmap =3D xskmap; + ifobj->mode =3D mode; +} + +static void xsk_attach_xdp_progs(struct test_spec *test, struct ifobject *= ifobj_rx, + struct ifobject *ifobj_tx) +{ + if (xdp_prog_changed_rx(test)) + xsk_reattach_xdp(ifobj_rx, test->xdp_prog_rx, test->xskmap_rx, test->mod= e); + + if (!ifobj_tx || ifobj_tx->shared_umem) + return; + + if (xdp_prog_changed_tx(test)) + xsk_reattach_xdp(ifobj_tx, test->xdp_prog_tx, test->xskmap_tx, test->mod= e); +} + +static int __testapp_validate_traffic(struct test_spec *test, struct ifobj= ect *ifobj1, + struct ifobject *ifobj2) +{ + pthread_t t0, t1; + int err; + + if (test->mtu > MAX_ETH_PKT_SIZE) { + if (test->mode =3D=3D TEST_MODE_ZC && (!ifobj1->multi_buff_zc_supp || + (ifobj2 && !ifobj2->multi_buff_zc_supp))) { + ksft_print_msg("Multi buffer for zero-copy not supported.\n"); + return TEST_SKIP; + } + if (test->mode !=3D TEST_MODE_ZC && (!ifobj1->multi_buff_supp || + (ifobj2 && !ifobj2->multi_buff_supp))) { + ksft_print_msg("Multi buffer not supported.\n"); + return TEST_SKIP; + } + } + err =3D test_spec_set_mtu(test, test->mtu); + if (err) { + ksft_print_msg("Error, could not set mtu.\n"); + exit_with_error(err); + } + + if (ifobj2) { + if (pthread_barrier_init(&barr, NULL, 2)) + exit_with_error(errno); + pkt_stream_reset(ifobj2->xsk->pkt_stream); + } + + test->current_step++; + pkt_stream_reset(ifobj1->xsk->pkt_stream); + pkts_in_flight =3D 0; + + signal(SIGUSR1, handler); + /*Spawn RX thread */ + pthread_create(&t0, NULL, ifobj1->func_ptr, test); + + if (ifobj2) { + pthread_barrier_wait(&barr); + if (pthread_barrier_destroy(&barr)) + exit_with_error(errno); + + /*Spawn TX thread */ + pthread_create(&t1, NULL, ifobj2->func_ptr, test); + + pthread_join(t1, NULL); + } + + if (!ifobj2) + pthread_kill(t0, SIGUSR1); + else + pthread_join(t0, NULL); + + if (test->total_steps =3D=3D test->current_step || test->fail) { + u32 i; + + if (ifobj2) + for (i =3D 0; i < test->nb_sockets; i++) + xsk_socket__delete(ifobj2->xsk_arr[i].xsk); + + for (i =3D 0; i < test->nb_sockets; i++) + xsk_socket__delete(ifobj1->xsk_arr[i].xsk); + + testapp_clean_xsk_umem(ifobj1); + if (ifobj2 && !ifobj2->shared_umem) + testapp_clean_xsk_umem(ifobj2); + } + + return !!test->fail; +} + +static int testapp_validate_traffic(struct test_spec *test) +{ + struct ifobject *ifobj_rx =3D test->ifobj_rx; + struct ifobject *ifobj_tx =3D test->ifobj_tx; + + if ((ifobj_rx->umem->unaligned_mode && !ifobj_rx->unaligned_supp) || + (ifobj_tx->umem->unaligned_mode && !ifobj_tx->unaligned_supp)) { + ksft_print_msg("No huge pages present.\n"); + return TEST_SKIP; + } + + if (test->set_ring) { + if (ifobj_tx->hw_ring_size_supp) { + if (set_ring_size(ifobj_tx)) { + ksft_print_msg("Failed to change HW ring size.\n"); + return TEST_FAILURE; + } + } else { + ksft_print_msg("Changing HW ring size not supported.\n"); + return TEST_SKIP; + } + } + + xsk_attach_xdp_progs(test, ifobj_rx, ifobj_tx); + return __testapp_validate_traffic(test, ifobj_rx, ifobj_tx); +} + +static int testapp_validate_traffic_single_thread(struct test_spec *test, = struct ifobject *ifobj) +{ + return __testapp_validate_traffic(test, ifobj, NULL); +} + +int testapp_teardown(struct test_spec *test) +{ + int i; + + for (i =3D 0; i < MAX_TEARDOWN_ITER; i++) { + if (testapp_validate_traffic(test)) + return TEST_FAILURE; + test_spec_reset(test); + } + + return TEST_PASS; +} + +static void swap_directions(struct ifobject **ifobj1, struct ifobject **if= obj2) +{ + thread_func_t tmp_func_ptr =3D (*ifobj1)->func_ptr; + struct ifobject *tmp_ifobj =3D (*ifobj1); + + (*ifobj1)->func_ptr =3D (*ifobj2)->func_ptr; + (*ifobj2)->func_ptr =3D tmp_func_ptr; + + *ifobj1 =3D *ifobj2; + *ifobj2 =3D tmp_ifobj; +} + +int testapp_bidirectional(struct test_spec *test) +{ + int res; + + test->ifobj_tx->rx_on =3D true; + test->ifobj_rx->tx_on =3D true; + test->total_steps =3D 2; + if (testapp_validate_traffic(test)) + return TEST_FAILURE; + + print_verbose("Switching Tx/Rx direction\n"); + swap_directions(&test->ifobj_rx, &test->ifobj_tx); + res =3D __testapp_validate_traffic(test, test->ifobj_rx, test->ifobj_tx); + + swap_directions(&test->ifobj_rx, &test->ifobj_tx); + return res; +} + +static int swap_xsk_resources(struct test_spec *test) +{ + int ret; + + test->ifobj_tx->xsk_arr[0].pkt_stream =3D NULL; + test->ifobj_rx->xsk_arr[0].pkt_stream =3D NULL; + test->ifobj_tx->xsk_arr[1].pkt_stream =3D test->tx_pkt_stream_default; + test->ifobj_rx->xsk_arr[1].pkt_stream =3D test->rx_pkt_stream_default; + test->ifobj_tx->xsk =3D &test->ifobj_tx->xsk_arr[1]; + test->ifobj_rx->xsk =3D &test->ifobj_rx->xsk_arr[1]; + + ret =3D xsk_update_xskmap(test->ifobj_rx->xskmap, test->ifobj_rx->xsk->xs= k, 0); + if (ret) + return TEST_FAILURE; + + return TEST_PASS; +} + +int testapp_xdp_prog_cleanup(struct test_spec *test) +{ + test->total_steps =3D 2; + test->nb_sockets =3D 2; + if (testapp_validate_traffic(test)) + return TEST_FAILURE; + + if (swap_xsk_resources(test)) + return TEST_FAILURE; + return testapp_validate_traffic(test); +} + +int testapp_headroom(struct test_spec *test) +{ + test->ifobj_rx->umem->frame_headroom =3D UMEM_HEADROOM_TEST_SIZE; + return testapp_validate_traffic(test); +} + +int testapp_stats_rx_dropped(struct test_spec *test) +{ + if (test->mode =3D=3D TEST_MODE_ZC) { + ksft_print_msg("Can not run RX_DROPPED test for ZC mode\n"); + return TEST_SKIP; + } + + pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0); + test->ifobj_rx->umem->frame_headroom =3D test->ifobj_rx->umem->frame_size= - + XDP_PACKET_HEADROOM - MIN_PKT_SIZE * 3; + pkt_stream_receive_half(test); + test->ifobj_rx->validation_func =3D validate_rx_dropped; + return testapp_validate_traffic(test); +} + +int testapp_stats_tx_invalid_descs(struct test_spec *test) +{ + pkt_stream_replace_half(test, XSK_UMEM__INVALID_FRAME_SIZE, 0); + test->ifobj_tx->validation_func =3D validate_tx_invalid_descs; + return testapp_validate_traffic(test); +} + +int testapp_stats_rx_full(struct test_spec *test) +{ + pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS / 2,= MIN_PKT_SIZE); + test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(DEFAULT_UMEM_BUFF= ERS, MIN_PKT_SIZE); + + test->ifobj_rx->xsk->rxqsize =3D DEFAULT_UMEM_BUFFERS; + test->ifobj_rx->release_rx =3D false; + test->ifobj_rx->validation_func =3D validate_rx_full; + return testapp_validate_traffic(test); +} + +int testapp_stats_fill_empty(struct test_spec *test) +{ + pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS / 2,= MIN_PKT_SIZE); + test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(DEFAULT_UMEM_BUFF= ERS, MIN_PKT_SIZE); + + test->ifobj_rx->use_fill_ring =3D false; + test->ifobj_rx->validation_func =3D validate_fill_empty; + return testapp_validate_traffic(test); +} + +int testapp_send_receive_unaligned(struct test_spec *test) +{ + test->ifobj_tx->umem->unaligned_mode =3D true; + test->ifobj_rx->umem->unaligned_mode =3D true; + /* Let half of the packets straddle a 4K buffer boundary */ + pkt_stream_replace_half(test, MIN_PKT_SIZE, -MIN_PKT_SIZE / 2); + + return testapp_validate_traffic(test); +} + +int testapp_send_receive_unaligned_mb(struct test_spec *test) +{ + test->mtu =3D MAX_ETH_JUMBO_SIZE; + test->ifobj_tx->umem->unaligned_mode =3D true; + test->ifobj_rx->umem->unaligned_mode =3D true; + pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE); + return testapp_validate_traffic(test); +} + +int testapp_single_pkt(struct test_spec *test) +{ + struct pkt pkts[] =3D {{0, MIN_PKT_SIZE, 0, true}}; + + pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); + return testapp_validate_traffic(test); +} + +int testapp_send_receive_mb(struct test_spec *test) +{ + test->mtu =3D MAX_ETH_JUMBO_SIZE; + pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE); + + return testapp_validate_traffic(test); +} + +int testapp_invalid_desc_mb(struct test_spec *test) +{ + struct xsk_umem_info *umem =3D test->ifobj_tx->umem; + u64 umem_size =3D umem->num_frames * umem->frame_size; + struct pkt pkts[] =3D { + /* Valid packet for synch to start with */ + {0, MIN_PKT_SIZE, 0, true, 0}, + /* Zero frame len is not legal */ + {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + {0, 0, 0, false, 0}, + /* Invalid address in the second frame */ + {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + {umem_size, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + /* Invalid len in the middle */ + {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + {0, XSK_UMEM__INVALID_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + /* Invalid options in the middle */ + {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XSK_DESC__INVALID_OPTION}, + /* Transmit 2 frags, receive 3 */ + {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, XDP_PKT_CONTD}, + {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, 0}, + /* Middle frame crosses chunk boundary with small length */ + {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, + {-MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, false, 0}, + /* Valid packet for synch so that something is received */ + {0, MIN_PKT_SIZE, 0, true, 0}}; + + if (umem->unaligned_mode) { + /* Crossing a chunk boundary allowed */ + pkts[12].valid =3D true; + pkts[13].valid =3D true; + } + + test->mtu =3D MAX_ETH_JUMBO_SIZE; + pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); + return testapp_validate_traffic(test); +} + +int testapp_invalid_desc(struct test_spec *test) +{ + struct xsk_umem_info *umem =3D test->ifobj_tx->umem; + u64 umem_size =3D umem->num_frames * umem->frame_size; + struct pkt pkts[] =3D { + /* Zero packet address allowed */ + {0, MIN_PKT_SIZE, 0, true}, + /* Allowed packet */ + {0, MIN_PKT_SIZE, 0, true}, + /* Straddling the start of umem */ + {-2, MIN_PKT_SIZE, 0, false}, + /* Packet too large */ + {0, XSK_UMEM__INVALID_FRAME_SIZE, 0, false}, + /* Up to end of umem allowed */ + {umem_size - MIN_PKT_SIZE - 2 * umem->frame_size, MIN_PKT_SIZE, 0, true}, + /* After umem ends */ + {umem_size, MIN_PKT_SIZE, 0, false}, + /* Straddle the end of umem */ + {umem_size - MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, false}, + /* Straddle a 4K boundary */ + {0x1000 - MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, false}, + /* Straddle a 2K boundary */ + {0x800 - MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, true}, + /* Valid packet for synch so that something is received */ + {0, MIN_PKT_SIZE, 0, true}}; + + if (umem->unaligned_mode) { + /* Crossing a page boundary allowed */ + pkts[7].valid =3D true; + } + if (umem->frame_size =3D=3D XSK_UMEM__DEFAULT_FRAME_SIZE / 2) { + /* Crossing a 2K frame size boundary not allowed */ + pkts[8].valid =3D false; + } + + if (test->ifobj_tx->shared_umem) { + pkts[4].offset +=3D umem_size; + pkts[5].offset +=3D umem_size; + pkts[6].offset +=3D umem_size; + } + + pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); + return testapp_validate_traffic(test); +} + +int testapp_xdp_drop(struct test_spec *test) +{ + struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; + struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; + + test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_drop, skel_tx->progs.= xsk_xdp_drop, + skel_rx->maps.xsk, skel_tx->maps.xsk); + + pkt_stream_receive_half(test); + return testapp_validate_traffic(test); +} + +int testapp_xdp_metadata_copy(struct test_spec *test) +{ + struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; + struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; + + test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_populate_metadata, + skel_tx->progs.xsk_xdp_populate_metadata, + skel_rx->maps.xsk, skel_tx->maps.xsk); + test->ifobj_rx->use_metadata =3D true; + + skel_rx->bss->count =3D 0; + + return testapp_validate_traffic(test); +} + +int testapp_xdp_shared_umem(struct test_spec *test) +{ + struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; + struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; + + test->total_steps =3D 1; + test->nb_sockets =3D 2; + + test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_shared_umem, + skel_tx->progs.xsk_xdp_shared_umem, + skel_rx->maps.xsk, skel_tx->maps.xsk); + + pkt_stream_even_odd_sequence(test); + + return testapp_validate_traffic(test); +} + +int testapp_poll_txq_tmout(struct test_spec *test) +{ + test->ifobj_tx->use_poll =3D true; + /* create invalid frame by set umem frame_size and pkt length equal to 20= 48 */ + test->ifobj_tx->umem->frame_size =3D 2048; + pkt_stream_replace(test, 2 * DEFAULT_PKT_CNT, 2048); + return testapp_validate_traffic_single_thread(test, test->ifobj_tx); +} + +int testapp_poll_rxq_tmout(struct test_spec *test) +{ + test->ifobj_rx->use_poll =3D true; + return testapp_validate_traffic_single_thread(test, test->ifobj_rx); +} + +int testapp_too_many_frags(struct test_spec *test) +{ + struct pkt *pkts; + u32 max_frags, i; + int ret; + + if (test->mode =3D=3D TEST_MODE_ZC) { + max_frags =3D test->ifobj_tx->xdp_zc_max_segs; + } else { + max_frags =3D get_max_skb_frags(); + if (!max_frags) { + ksft_print_msg("Can't get MAX_SKB_FRAGS from system, using default (17)= \n"); + max_frags =3D 17; + } + max_frags +=3D 1; + } + + pkts =3D calloc(2 * max_frags + 2, sizeof(struct pkt)); + if (!pkts) + return TEST_FAILURE; + + test->mtu =3D MAX_ETH_JUMBO_SIZE; + + /* Valid packet for synch */ + pkts[0].len =3D MIN_PKT_SIZE; + pkts[0].valid =3D true; + + /* One valid packet with the max amount of frags */ + for (i =3D 1; i < max_frags + 1; i++) { + pkts[i].len =3D MIN_PKT_SIZE; + pkts[i].options =3D XDP_PKT_CONTD; + pkts[i].valid =3D true; + } + pkts[max_frags].options =3D 0; + + /* An invalid packet with the max amount of frags but signals packet + * continues on the last frag + */ + for (i =3D max_frags + 1; i < 2 * max_frags + 1; i++) { + pkts[i].len =3D MIN_PKT_SIZE; + pkts[i].options =3D XDP_PKT_CONTD; + pkts[i].valid =3D false; + } + + /* Valid packet for synch */ + pkts[2 * max_frags + 1].len =3D MIN_PKT_SIZE; + pkts[2 * max_frags + 1].valid =3D true; + + pkt_stream_generate_custom(test, pkts, 2 * max_frags + 2); + ret =3D testapp_validate_traffic(test); + + free(pkts); + return ret; +} + +static int xsk_load_xdp_programs(struct ifobject *ifobj) +{ + ifobj->xdp_progs =3D xsk_xdp_progs__open_and_load(); + if (libbpf_get_error(ifobj->xdp_progs)) + return libbpf_get_error(ifobj->xdp_progs); + + return 0; +} + +/* Simple test */ +static bool hugepages_present(void) +{ + size_t mmap_sz =3D 2 * DEFAULT_UMEM_BUFFERS * XSK_UMEM__DEFAULT_FRAME_SIZ= E; + void *bufs; + + bufs =3D mmap(NULL, mmap_sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, MAP_HUGE_2MB); + if (bufs =3D=3D MAP_FAILED) + return false; + + mmap_sz =3D ceil_u64(mmap_sz, HUGEPAGE_SIZE) * HUGEPAGE_SIZE; + munmap(bufs, mmap_sz); + return true; +} + +void init_iface(struct ifobject *ifobj, thread_func_t func_ptr) +{ + LIBBPF_OPTS(bpf_xdp_query_opts, query_opts); + int err; + + ifobj->func_ptr =3D func_ptr; + + err =3D xsk_load_xdp_programs(ifobj); + if (err) { + ksft_print_msg("Error loading XDP program\n"); + exit_with_error(err); + } + + if (hugepages_present()) + ifobj->unaligned_supp =3D true; + + err =3D bpf_xdp_query(ifobj->ifindex, XDP_FLAGS_DRV_MODE, &query_opts); + if (err) { + ksft_print_msg("Error querying XDP capabilities\n"); + exit_with_error(-err); + } + if (query_opts.feature_flags & NETDEV_XDP_ACT_RX_SG) + ifobj->multi_buff_supp =3D true; + if (query_opts.feature_flags & NETDEV_XDP_ACT_XSK_ZEROCOPY) { + if (query_opts.xdp_zc_max_segs > 1) { + ifobj->multi_buff_zc_supp =3D true; + ifobj->xdp_zc_max_segs =3D query_opts.xdp_zc_max_segs; + } else { + ifobj->xdp_zc_max_segs =3D 0; + } + } +} + +int testapp_send_receive(struct test_spec *test) +{ + return testapp_validate_traffic(test); +} + +int testapp_send_receive_2k_frame(struct test_spec *test) +{ + test->ifobj_tx->umem->frame_size =3D 2048; + test->ifobj_rx->umem->frame_size =3D 2048; + pkt_stream_replace(test, DEFAULT_PKT_CNT, MIN_PKT_SIZE); + return testapp_validate_traffic(test); +} + +int testapp_poll_rx(struct test_spec *test) +{ + test->ifobj_rx->use_poll =3D true; + return testapp_validate_traffic(test); +} + +int testapp_poll_tx(struct test_spec *test) +{ + test->ifobj_tx->use_poll =3D true; + return testapp_validate_traffic(test); +} + +int testapp_aligned_inv_desc(struct test_spec *test) +{ + return testapp_invalid_desc(test); +} + +int testapp_aligned_inv_desc_2k_frame(struct test_spec *test) +{ + test->ifobj_tx->umem->frame_size =3D 2048; + test->ifobj_rx->umem->frame_size =3D 2048; + return testapp_invalid_desc(test); +} + +int testapp_unaligned_inv_desc(struct test_spec *test) +{ + test->ifobj_tx->umem->unaligned_mode =3D true; + test->ifobj_rx->umem->unaligned_mode =3D true; + return testapp_invalid_desc(test); +} + +int testapp_unaligned_inv_desc_4001_frame(struct test_spec *test) +{ + u64 page_size, umem_size; + + /* Odd frame size so the UMEM doesn't end near a page boundary. */ + test->ifobj_tx->umem->frame_size =3D 4001; + test->ifobj_rx->umem->frame_size =3D 4001; + test->ifobj_tx->umem->unaligned_mode =3D true; + test->ifobj_rx->umem->unaligned_mode =3D true; + /* This test exists to test descriptors that staddle the end of + * the UMEM but not a page. + */ + page_size =3D sysconf(_SC_PAGESIZE); + umem_size =3D test->ifobj_tx->umem->num_frames * test->ifobj_tx->umem->fr= ame_size; + assert(umem_size % page_size > MIN_PKT_SIZE); + assert(umem_size % page_size < page_size - MIN_PKT_SIZE); + + return testapp_invalid_desc(test); +} + +int testapp_aligned_inv_desc_mb(struct test_spec *test) +{ + return testapp_invalid_desc_mb(test); +} + +int testapp_unaligned_inv_desc_mb(struct test_spec *test) +{ + test->ifobj_tx->umem->unaligned_mode =3D true; + test->ifobj_rx->umem->unaligned_mode =3D true; + return testapp_invalid_desc_mb(test); +} + +int testapp_xdp_metadata(struct test_spec *test) +{ + return testapp_xdp_metadata_copy(test); +} + +int testapp_xdp_metadata_mb(struct test_spec *test) +{ + test->mtu =3D MAX_ETH_JUMBO_SIZE; + return testapp_xdp_metadata_copy(test); +} + +int testapp_hw_sw_min_ring_size(struct test_spec *test) +{ + int ret; + + test->set_ring =3D true; + test->total_steps =3D 2; + test->ifobj_tx->ring.tx_pending =3D DEFAULT_BATCH_SIZE; + test->ifobj_tx->ring.rx_pending =3D DEFAULT_BATCH_SIZE * 2; + test->ifobj_tx->xsk->batch_size =3D 1; + test->ifobj_rx->xsk->batch_size =3D 1; + ret =3D testapp_validate_traffic(test); + if (ret) + return ret; + + /* Set batch size to hw_ring_size - 1 */ + test->ifobj_tx->xsk->batch_size =3D DEFAULT_BATCH_SIZE - 1; + test->ifobj_rx->xsk->batch_size =3D DEFAULT_BATCH_SIZE - 1; + return testapp_validate_traffic(test); +} + +int testapp_hw_sw_max_ring_size(struct test_spec *test) +{ + u32 max_descs =3D XSK_RING_PROD__DEFAULT_NUM_DESCS * 4; + int ret; + + test->set_ring =3D true; + test->total_steps =3D 2; + test->ifobj_tx->ring.tx_pending =3D test->ifobj_tx->ring.tx_max_pending; + test->ifobj_tx->ring.rx_pending =3D test->ifobj_tx->ring.rx_max_pending; + test->ifobj_rx->umem->num_frames =3D max_descs; + test->ifobj_rx->umem->fill_size =3D max_descs; + test->ifobj_rx->umem->comp_size =3D max_descs; + test->ifobj_tx->xsk->batch_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS; + test->ifobj_rx->xsk->batch_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS; + + ret =3D testapp_validate_traffic(test); + if (ret) + return ret; + + /* Set batch_size to 8152 for testing, as the ice HW ignores the 3 lowest= bits when + * updating the Rx HW tail register. + */ + test->ifobj_tx->xsk->batch_size =3D test->ifobj_tx->ring.tx_max_pending -= 8; + test->ifobj_rx->xsk->batch_size =3D test->ifobj_tx->ring.tx_max_pending -= 8; + pkt_stream_replace(test, max_descs, MIN_PKT_SIZE); + return testapp_validate_traffic(test); +} + +static int testapp_xdp_adjust_tail(struct test_spec *test, int adjust_valu= e) +{ + struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; + struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; + + test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_adjust_tail, + skel_tx->progs.xsk_xdp_adjust_tail, + skel_rx->maps.xsk, skel_tx->maps.xsk); + + skel_rx->bss->adjust_value =3D adjust_value; + + return testapp_validate_traffic(test); +} + +static int testapp_adjust_tail(struct test_spec *test, u32 value, u32 pkt_= len) +{ + int ret; + + test->adjust_tail_support =3D true; + test->adjust_tail =3D true; + test->total_steps =3D 1; + + pkt_stream_replace_ifobject(test->ifobj_tx, DEFAULT_BATCH_SIZE, pkt_len); + pkt_stream_replace_ifobject(test->ifobj_rx, DEFAULT_BATCH_SIZE, pkt_len += value); + + ret =3D testapp_xdp_adjust_tail(test, value); + if (ret) + return ret; + + if (!test->adjust_tail_support) { + ksft_print_msg("%s %sResize pkt with bpf_xdp_adjust_tail() not supported= \n", + mode_string(test), busy_poll_string(test)); + return TEST_SKIP; + } + + return 0; +} + +int testapp_adjust_tail_shrink(struct test_spec *test) +{ + /* Shrink by 4 bytes for testing purpose */ + return testapp_adjust_tail(test, -4, MIN_PKT_SIZE * 2); +} + +int testapp_adjust_tail_shrink_mb(struct test_spec *test) +{ + test->mtu =3D MAX_ETH_JUMBO_SIZE; + /* Shrink by the frag size */ + return testapp_adjust_tail(test, -XSK_UMEM__MAX_FRAME_SIZE, XSK_UMEM__LAR= GE_FRAME_SIZE * 2); +} + +int testapp_adjust_tail_grow(struct test_spec *test) +{ + /* Grow by 4 bytes for testing purpose */ + return testapp_adjust_tail(test, 4, MIN_PKT_SIZE * 2); +} + +int testapp_adjust_tail_grow_mb(struct test_spec *test) +{ + test->mtu =3D MAX_ETH_JUMBO_SIZE; + /* Grow by (frag_size - last_frag_Size) - 1 to stay inside the last fragm= ent */ + return testapp_adjust_tail(test, (XSK_UMEM__MAX_FRAME_SIZE / 2) - 1, + XSK_UMEM__LARGE_FRAME_SIZE * 2); +} + +int testapp_tx_queue_consumer(struct test_spec *test) +{ + int nr_packets; + + if (test->mode =3D=3D TEST_MODE_ZC) { + ksft_print_msg("Can not run TX_QUEUE_CONSUMER test for ZC mode\n"); + return TEST_SKIP; + } + + nr_packets =3D MAX_TX_BUDGET_DEFAULT + 1; + pkt_stream_replace(test, nr_packets, MIN_PKT_SIZE); + test->ifobj_tx->xsk->batch_size =3D nr_packets; + test->ifobj_tx->xsk->check_consumer =3D true; + + return testapp_validate_traffic(test); +} + +struct ifobject *ifobject_create(void) +{ + struct ifobject *ifobj; + + ifobj =3D calloc(1, sizeof(struct ifobject)); + if (!ifobj) + return NULL; + + ifobj->xsk_arr =3D calloc(MAX_SOCKETS, sizeof(*ifobj->xsk_arr)); + if (!ifobj->xsk_arr) + goto out_xsk_arr; + + ifobj->umem =3D calloc(1, sizeof(*ifobj->umem)); + if (!ifobj->umem) + goto out_umem; + + return ifobj; + +out_umem: + free(ifobj->xsk_arr); +out_xsk_arr: + free(ifobj); + return NULL; +} + +void ifobject_delete(struct ifobject *ifobj) +{ + free(ifobj->umem); + free(ifobj->xsk_arr); + free(ifobj); +} diff --git a/tools/testing/selftests/bpf/test_xsk.h b/tools/testing/selftes= ts/bpf/test_xsk.h new file mode 100644 index 0000000000000000000000000000000000000000..fb546cab39fdfbd22dcb352784a= 7c5ef383f8ac6 --- /dev/null +++ b/tools/testing/selftests/bpf/test_xsk.h @@ -0,0 +1,297 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef TEST_XSK_H_ +#define TEST_XSK_H_ + +#include +#include + +#include "../kselftest.h" +#include "xsk.h" + +#ifndef SO_PREFER_BUSY_POLL +#define SO_PREFER_BUSY_POLL 69 +#endif + +#ifndef SO_BUSY_POLL_BUDGET +#define SO_BUSY_POLL_BUDGET 70 +#endif + +#define TEST_PASS 0 +#define TEST_FAILURE -1 +#define TEST_CONTINUE 1 +#define TEST_SKIP 2 + +#define DEFAULT_PKT_CNT (4 * 1024) +#define DEFAULT_UMEM_BUFFERS (DEFAULT_PKT_CNT / 4) +#define HUGEPAGE_SIZE (2 * 1024 * 1024) +#define MIN_PKT_SIZE 64 +#define MAX_ETH_PKT_SIZE 1518 +#define MAX_INTERFACE_NAME_CHARS 16 +#define MAX_TEST_NAME_SIZE 48 +#define SOCK_RECONF_CTR 10 +#define USLEEP_MAX 10000 + +extern bool opt_verbose; +#define print_verbose(x...) do { if (opt_verbose) ksft_print_msg(x); } whi= le (0) + +static void __exit_with_error(int error, const char *file, const char *fun= c, int line) +{ + ksft_test_result_fail("[%s:%s:%i]: ERROR: %d/\"%s\"\n", file, func, line,= error, + strerror(error)); + ksft_exit_xfail(); +} +#define exit_with_error(error) __exit_with_error(error, __FILE__, __func__= , __LINE__) + +static inline u32 ceil_u32(u32 a, u32 b) +{ + return (a + b - 1) / b; +} + +static inline u64 ceil_u64(u64 a, u64 b) +{ + return (a + b - 1) / b; +} + +/* Simple test */ +enum test_mode { + TEST_MODE_SKB, + TEST_MODE_DRV, + TEST_MODE_ZC, + TEST_MODE_ALL +}; + +struct ifobject; +struct test_spec; +typedef int (*validation_func_t)(struct ifobject *ifobj); +typedef void *(*thread_func_t)(void *arg); +typedef int (*test_func_t)(struct test_spec *test); + +struct xsk_socket_info { + struct xsk_ring_cons rx; + struct xsk_ring_prod tx; + struct xsk_umem_info *umem; + struct xsk_socket *xsk; + struct pkt_stream *pkt_stream; + u32 outstanding_tx; + u32 rxqsize; + u32 batch_size; + u8 dst_mac[ETH_ALEN]; + u8 src_mac[ETH_ALEN]; + bool check_consumer; +}; + +int kick_rx(struct xsk_socket_info *xsk); +int kick_tx(struct xsk_socket_info *xsk); + +struct xsk_umem_info { + struct xsk_ring_prod fq; + struct xsk_ring_cons cq; + struct xsk_umem *umem; + u64 next_buffer; + u32 num_frames; + u32 frame_headroom; + void *buffer; + u32 frame_size; + u32 base_addr; + u32 fill_size; + u32 comp_size; + bool unaligned_mode; +}; + +struct set_hw_ring { + u32 default_tx; + u32 default_rx; +}; + +int hw_ring_size_reset(struct ifobject *ifobj); + +struct ifobject { + char ifname[MAX_INTERFACE_NAME_CHARS]; + struct xsk_socket_info *xsk; + struct xsk_socket_info *xsk_arr; + struct xsk_umem_info *umem; + thread_func_t func_ptr; + validation_func_t validation_func; + struct xsk_xdp_progs *xdp_progs; + struct bpf_map *xskmap; + struct bpf_program *xdp_prog; + struct ethtool_ringparam ring; + struct set_hw_ring set_ring; + enum test_mode mode; + int ifindex; + int mtu; + u32 bind_flags; + u32 xdp_zc_max_segs; + bool tx_on; + bool rx_on; + bool use_poll; + bool busy_poll; + bool use_fill_ring; + bool release_rx; + bool shared_umem; + bool use_metadata; + bool unaligned_supp; + bool multi_buff_supp; + bool multi_buff_zc_supp; + bool hw_ring_size_supp; +}; +struct ifobject *ifobject_create(void); +void ifobject_delete(struct ifobject *ifobj); +void init_iface(struct ifobject *ifobj, thread_func_t func_ptr); + +int xsk_configure_umem(struct ifobject *ifobj, struct xsk_umem_info *umem,= void *buffer, u64 size); +int xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info= *umem, + struct ifobject *ifobject, bool shared); + + +struct pkt { + int offset; + u32 len; + u32 pkt_nb; + bool valid; + u16 options; +}; + +struct pkt_stream { + u32 nb_pkts; + u32 current_pkt_nb; + struct pkt *pkts; + u32 max_pkt_len; + u32 nb_rx_pkts; + u32 nb_valid_entries; + bool verbatim; +}; + +static inline bool pkt_continues(u32 options) +{ + return options & XDP_PKT_CONTD; +} + +struct pkt_stream *pkt_stream_generate(u32 nb_pkts, u32 pkt_len); +void pkt_stream_delete(struct pkt_stream *pkt_stream); +void pkt_stream_reset(struct pkt_stream *pkt_stream); +void pkt_stream_restore_default(struct test_spec *test); + +struct test_spec { + struct ifobject *ifobj_tx; + struct ifobject *ifobj_rx; + struct pkt_stream *tx_pkt_stream_default; + struct pkt_stream *rx_pkt_stream_default; + struct bpf_program *xdp_prog_rx; + struct bpf_program *xdp_prog_tx; + struct bpf_map *xskmap_rx; + struct bpf_map *xskmap_tx; + test_func_t test_func; + int mtu; + u16 total_steps; + u16 current_step; + u16 nb_sockets; + bool fail; + bool set_ring; + bool adjust_tail; + bool adjust_tail_support; + enum test_mode mode; + char name[MAX_TEST_NAME_SIZE]; +}; + +#define busy_poll_string(test) (test)->ifobj_tx->busy_poll ? "BUSY-POLL " = : "" +static inline char *mode_string(struct test_spec *test) +{ + switch (test->mode) { + case TEST_MODE_SKB: + return "SKB"; + case TEST_MODE_DRV: + return "DRV"; + case TEST_MODE_ZC: + return "ZC"; + default: + return "BOGUS"; + } +} + +void test_init(struct test_spec *test, struct ifobject *ifobj_tx, + struct ifobject *ifobj_rx, enum test_mode mode, + const struct test_spec *test_to_run); + +int testapp_adjust_tail_grow(struct test_spec *test); +int testapp_adjust_tail_grow_mb(struct test_spec *test); +int testapp_adjust_tail_shrink(struct test_spec *test); +int testapp_adjust_tail_shrink_mb(struct test_spec *test); +int testapp_aligned_inv_desc(struct test_spec *test); +int testapp_aligned_inv_desc_2k_frame(struct test_spec *test); +int testapp_aligned_inv_desc_mb(struct test_spec *test); +int testapp_bidirectional(struct test_spec *test); +int testapp_headroom(struct test_spec *test); +int testapp_hw_sw_max_ring_size(struct test_spec *test); +int testapp_hw_sw_min_ring_size(struct test_spec *test); +int testapp_poll_rx(struct test_spec *test); +int testapp_poll_rxq_tmout(struct test_spec *test); +int testapp_poll_tx(struct test_spec *test); +int testapp_poll_txq_tmout(struct test_spec *test); +int testapp_send_receive(struct test_spec *test); +int testapp_send_receive_2k_frame(struct test_spec *test); +int testapp_send_receive_mb(struct test_spec *test); +int testapp_send_receive_unaligned(struct test_spec *test); +int testapp_send_receive_unaligned_mb(struct test_spec *test); +int testapp_single_pkt(struct test_spec *test); +int testapp_stats_fill_empty(struct test_spec *test); +int testapp_stats_rx_dropped(struct test_spec *test); +int testapp_stats_tx_invalid_descs(struct test_spec *test); +int testapp_stats_rx_full(struct test_spec *test); +int testapp_teardown(struct test_spec *test); +int testapp_too_many_frags(struct test_spec *test); +int testapp_tx_queue_consumer(struct test_spec *test); +int testapp_unaligned_inv_desc(struct test_spec *test); +int testapp_unaligned_inv_desc_4001_frame(struct test_spec *test); +int testapp_unaligned_inv_desc_mb(struct test_spec *test); +int testapp_xdp_drop(struct test_spec *test); +int testapp_xdp_metadata(struct test_spec *test); +int testapp_xdp_metadata_mb(struct test_spec *test); +int testapp_xdp_prog_cleanup(struct test_spec *test); +int testapp_xdp_shared_umem(struct test_spec *test); + +void *worker_testapp_validate_rx(void *arg); +void *worker_testapp_validate_tx(void *arg); + +static const struct test_spec tests[] =3D { + {.name =3D "SEND_RECEIVE", .test_func =3D testapp_send_receive}, + {.name =3D "SEND_RECEIVE_2K_FRAME", .test_func =3D testapp_send_receive_2= k_frame}, + {.name =3D "SEND_RECEIVE_SINGLE_PKT", .test_func =3D testapp_single_pkt}, + {.name =3D "POLL_RX", .test_func =3D testapp_poll_rx}, + {.name =3D "POLL_TX", .test_func =3D testapp_poll_tx}, + {.name =3D "POLL_RXQ_FULL", .test_func =3D testapp_poll_rxq_tmout}, + {.name =3D "POLL_TXQ_FULL", .test_func =3D testapp_poll_txq_tmout}, + {.name =3D "SEND_RECEIVE_UNALIGNED", .test_func =3D testapp_send_receive_= unaligned}, + {.name =3D "ALIGNED_INV_DESC", .test_func =3D testapp_aligned_inv_desc}, + {.name =3D "ALIGNED_INV_DESC_2K_FRAME_SIZE", .test_func =3D testapp_align= ed_inv_desc_2k_frame}, + {.name =3D "UNALIGNED_INV_DESC", .test_func =3D testapp_unaligned_inv_des= c}, + {.name =3D "UNALIGNED_INV_DESC_4001_FRAME_SIZE", + .test_func =3D testapp_unaligned_inv_desc_4001_frame}, + {.name =3D "UMEM_HEADROOM", .test_func =3D testapp_headroom}, + {.name =3D "TEARDOWN", .test_func =3D testapp_teardown}, + {.name =3D "BIDIRECTIONAL", .test_func =3D testapp_bidirectional}, + {.name =3D "STAT_RX_DROPPED", .test_func =3D testapp_stats_rx_dropped}, + {.name =3D "STAT_TX_INVALID", .test_func =3D testapp_stats_tx_invalid_des= cs}, + {.name =3D "STAT_RX_FULL", .test_func =3D testapp_stats_rx_full}, + {.name =3D "STAT_FILL_EMPTY", .test_func =3D testapp_stats_fill_empty}, + {.name =3D "XDP_PROG_CLEANUP", .test_func =3D testapp_xdp_prog_cleanup}, + {.name =3D "XDP_DROP_HALF", .test_func =3D testapp_xdp_drop}, + {.name =3D "XDP_SHARED_UMEM", .test_func =3D testapp_xdp_shared_umem}, + {.name =3D "XDP_METADATA_COPY", .test_func =3D testapp_xdp_metadata}, + {.name =3D "XDP_METADATA_COPY_MULTI_BUFF", .test_func =3D testapp_xdp_met= adata_mb}, + {.name =3D "SEND_RECEIVE_9K_PACKETS", .test_func =3D testapp_send_receive= _mb}, + {.name =3D "SEND_RECEIVE_UNALIGNED_9K_PACKETS", + .test_func =3D testapp_send_receive_unaligned_mb}, + {.name =3D "ALIGNED_INV_DESC_MULTI_BUFF", .test_func =3D testapp_aligned_= inv_desc_mb}, + {.name =3D "UNALIGNED_INV_DESC_MULTI_BUFF", .test_func =3D testapp_unalig= ned_inv_desc_mb}, + {.name =3D "TOO_MANY_FRAGS", .test_func =3D testapp_too_many_frags}, + {.name =3D "HW_SW_MIN_RING_SIZE", .test_func =3D testapp_hw_sw_min_ring_s= ize}, + {.name =3D "HW_SW_MAX_RING_SIZE", .test_func =3D testapp_hw_sw_max_ring_s= ize}, + {.name =3D "XDP_ADJUST_TAIL_SHRINK", .test_func =3D testapp_adjust_tail_s= hrink}, + {.name =3D "XDP_ADJUST_TAIL_SHRINK_MULTI_BUFF", .test_func =3D testapp_ad= just_tail_shrink_mb}, + {.name =3D "XDP_ADJUST_TAIL_GROW", .test_func =3D testapp_adjust_tail_gro= w}, + {.name =3D "XDP_ADJUST_TAIL_GROW_MULTI_BUFF", .test_func =3D testapp_adju= st_tail_grow_mb}, + {.name =3D "TX_QUEUE_CONSUMER", .test_func =3D testapp_tx_queue_consumer}, + }; + +#endif /* TEST_XSK_H_ */ diff --git a/tools/testing/selftests/bpf/xskxceiver.c b/tools/testing/selft= ests/bpf/xskxceiver.c index 352adc8df2d1cd777c823c5a89f1720ee043f342..8e108e3162695d5d50b3e380567= 2601024e385e2 100644 --- a/tools/testing/selftests/bpf/xskxceiver.c +++ b/tools/testing/selftests/bpf/xskxceiver.c @@ -74,31 +74,23 @@ #define _GNU_SOURCE #include #include -#include #include #include #include #include #include -#include #include #include #include #include -#include -#include -#include #include #include #include -#include #include #include -#include -#include #include -#include =20 +#include "test_xsk.h" #include "xsk_xdp_progs.skel.h" #include "xsk.h" #include "xskxceiver.h" @@ -109,181 +101,12 @@ =20 #include =20 -#define MAX_TX_BUDGET_DEFAULT 32 - -static bool opt_verbose; static bool opt_print_tests; static enum test_mode opt_mode =3D TEST_MODE_ALL; static u32 opt_run_test =3D RUN_ALL_TESTS; =20 void test__fail(void) { /* for network_helpers.c */ } =20 -static void __exit_with_error(int error, const char *file, const char *fun= c, int line) -{ - ksft_test_result_fail("[%s:%s:%i]: ERROR: %d/\"%s\"\n", file, func, line,= error, - strerror(error)); - ksft_exit_xfail(); -} - -#define exit_with_error(error) __exit_with_error(error, __FILE__, __func__= , __LINE__) -#define busy_poll_string(test) (test)->ifobj_tx->busy_poll ? "BUSY-POLL " = : "" -static char *mode_string(struct test_spec *test) -{ - switch (test->mode) { - case TEST_MODE_SKB: - return "SKB"; - case TEST_MODE_DRV: - return "DRV"; - case TEST_MODE_ZC: - return "ZC"; - default: - return "BOGUS"; - } -} - -static void report_failure(struct test_spec *test) -{ - if (test->fail) - return; - - ksft_test_result_fail("FAIL: %s %s%s\n", mode_string(test), busy_poll_str= ing(test), - test->name); - test->fail =3D true; -} - -/* The payload is a word consisting of a packet sequence number in the upp= er - * 16-bits and a intra packet data sequence number in the lower 16 bits. S= o the 3rd packet's - * 5th word of data will contain the number (2<<16) | 4 as they are number= ed from 0. - */ -static void write_payload(void *dest, u32 pkt_nb, u32 start, u32 size) -{ - u32 *ptr =3D (u32 *)dest, i; - - start /=3D sizeof(*ptr); - size /=3D sizeof(*ptr); - for (i =3D 0; i < size; i++) - ptr[i] =3D htonl(pkt_nb << 16 | (i + start)); -} - -static void gen_eth_hdr(struct xsk_socket_info *xsk, struct ethhdr *eth_hd= r) -{ - memcpy(eth_hdr->h_dest, xsk->dst_mac, ETH_ALEN); - memcpy(eth_hdr->h_source, xsk->src_mac, ETH_ALEN); - eth_hdr->h_proto =3D htons(ETH_P_LOOPBACK); -} - -static bool is_umem_valid(struct ifobject *ifobj) -{ - return !!ifobj->umem->umem; -} - -static u32 mode_to_xdp_flags(enum test_mode mode) -{ - return (mode =3D=3D TEST_MODE_SKB) ? XDP_FLAGS_SKB_MODE : XDP_FLAGS_DRV_M= ODE; -} - -static u64 umem_size(struct xsk_umem_info *umem) -{ - return umem->num_frames * umem->frame_size; -} - -static int xsk_configure_umem(struct ifobject *ifobj, struct xsk_umem_info= *umem, void *buffer, - u64 size) -{ - struct xsk_umem_config cfg =3D { - .fill_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS, - .comp_size =3D XSK_RING_CONS__DEFAULT_NUM_DESCS, - .frame_size =3D umem->frame_size, - .frame_headroom =3D umem->frame_headroom, - .flags =3D XSK_UMEM__DEFAULT_FLAGS - }; - int ret; - - if (umem->fill_size) - cfg.fill_size =3D umem->fill_size; - - if (umem->comp_size) - cfg.comp_size =3D umem->comp_size; - - if (umem->unaligned_mode) - cfg.flags |=3D XDP_UMEM_UNALIGNED_CHUNK_FLAG; - - ret =3D xsk_umem__create(&umem->umem, buffer, size, - &umem->fq, &umem->cq, &cfg); - if (ret) - return ret; - - umem->buffer =3D buffer; - if (ifobj->shared_umem && ifobj->rx_on) { - umem->base_addr =3D umem_size(umem); - umem->next_buffer =3D umem_size(umem); - } - - return 0; -} - -static u64 umem_alloc_buffer(struct xsk_umem_info *umem) -{ - u64 addr; - - addr =3D umem->next_buffer; - umem->next_buffer +=3D umem->frame_size; - if (umem->next_buffer >=3D umem->base_addr + umem_size(umem)) - umem->next_buffer =3D umem->base_addr; - - return addr; -} - -static void umem_reset_alloc(struct xsk_umem_info *umem) -{ - umem->next_buffer =3D 0; -} - -static void enable_busy_poll(struct xsk_socket_info *xsk) -{ - int sock_opt; - - sock_opt =3D 1; - if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_PREFER_BUSY_POLL, - (void *)&sock_opt, sizeof(sock_opt)) < 0) - exit_with_error(errno); - - sock_opt =3D 20; - if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL, - (void *)&sock_opt, sizeof(sock_opt)) < 0) - exit_with_error(errno); - - sock_opt =3D xsk->batch_size; - if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL_BUDGET, - (void *)&sock_opt, sizeof(sock_opt)) < 0) - exit_with_error(errno); -} - -static int __xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_= umem_info *umem, - struct ifobject *ifobject, bool shared) -{ - struct xsk_socket_config cfg =3D {}; - struct xsk_ring_cons *rxr; - struct xsk_ring_prod *txr; - - xsk->umem =3D umem; - cfg.rx_size =3D xsk->rxqsize; - cfg.tx_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS; - cfg.bind_flags =3D ifobject->bind_flags; - if (shared) - cfg.bind_flags |=3D XDP_SHARED_UMEM; - if (ifobject->mtu > MAX_ETH_PKT_SIZE) - cfg.bind_flags |=3D XDP_USE_SG; - if (umem->comp_size) - cfg.tx_size =3D umem->comp_size; - if (umem->fill_size) - cfg.rx_size =3D umem->fill_size; - - txr =3D ifobject->tx_on ? &xsk->tx : NULL; - rxr =3D ifobject->rx_on ? &xsk->rx : NULL; - return xsk_socket__create(&xsk->xsk, ifobject->ifindex, 0, umem->umem, rx= r, txr, &cfg); -} - static bool ifobj_zc_avail(struct ifobject *ifobject) { size_t umem_sz =3D DEFAULT_UMEM_BUFFERS * XSK_UMEM__DEFAULT_FRAME_SIZE; @@ -314,7 +137,7 @@ static bool ifobj_zc_avail(struct ifobject *ifobject) ifobject->bind_flags =3D XDP_USE_NEED_WAKEUP | XDP_ZEROCOPY; ifobject->rx_on =3D true; xsk->rxqsize =3D XSK_RING_CONS__DEFAULT_NUM_DESCS; - ret =3D __xsk_configure_socket(xsk, umem, ifobject, false); + ret =3D xsk_configure_socket(xsk, umem, ifobject, false); if (!ret) zc_avail =3D true; =20 @@ -327,25 +150,6 @@ static bool ifobj_zc_avail(struct ifobject *ifobject) return zc_avail; } =20 -#define MAX_SKB_FRAGS_PATH "/proc/sys/net/core/max_skb_frags" -static unsigned int get_max_skb_frags(void) -{ - unsigned int max_skb_frags =3D 0; - FILE *file; - - file =3D fopen(MAX_SKB_FRAGS_PATH, "r"); - if (!file) { - ksft_print_msg("Error opening %s\n", MAX_SKB_FRAGS_PATH); - return 0; - } - - if (fscanf(file, "%u", &max_skb_frags) !=3D 1) - ksft_print_msg("Error reading %s\n", MAX_SKB_FRAGS_PATH); - - fclose(file); - return max_skb_frags; -} - static struct option long_options[] =3D { {"interface", required_argument, 0, 'i'}, {"busy-poll", no_argument, 0, 'b'}, @@ -446,2327 +250,66 @@ static void parse_command_line(struct ifobject *ifo= bj_tx, struct ifobject *ifobj } } =20 -static int set_ring_size(struct ifobject *ifobj) -{ - int ret; - u32 ctr =3D 0; - - while (ctr++ < SOCK_RECONF_CTR) { - ret =3D set_hw_ring_size(ifobj->ifname, &ifobj->ring); - if (!ret) - break; - - /* Retry if it fails */ - if (ctr >=3D SOCK_RECONF_CTR || errno !=3D EBUSY) - return -errno; - - usleep(USLEEP_MAX); - } - - return ret; -} - -static int hw_ring_size_reset(struct ifobject *ifobj) -{ - ifobj->ring.tx_pending =3D ifobj->set_ring.default_tx; - ifobj->ring.rx_pending =3D ifobj->set_ring.default_rx; - return set_ring_size(ifobj); -} - -static void __test_spec_init(struct test_spec *test, struct ifobject *ifob= j_tx, - struct ifobject *ifobj_rx) +static void xsk_unload_xdp_programs(struct ifobject *ifobj) { - u32 i, j; - - for (i =3D 0; i < MAX_INTERFACES; i++) { - struct ifobject *ifobj =3D i ? ifobj_rx : ifobj_tx; - - ifobj->xsk =3D &ifobj->xsk_arr[0]; - ifobj->use_poll =3D false; - ifobj->use_fill_ring =3D true; - ifobj->release_rx =3D true; - ifobj->validation_func =3D NULL; - ifobj->use_metadata =3D false; - - if (i =3D=3D 0) { - ifobj->rx_on =3D false; - ifobj->tx_on =3D true; - } else { - ifobj->rx_on =3D true; - ifobj->tx_on =3D false; - } - - memset(ifobj->umem, 0, sizeof(*ifobj->umem)); - ifobj->umem->num_frames =3D DEFAULT_UMEM_BUFFERS; - ifobj->umem->frame_size =3D XSK_UMEM__DEFAULT_FRAME_SIZE; - - for (j =3D 0; j < MAX_SOCKETS; j++) { - memset(&ifobj->xsk_arr[j], 0, sizeof(ifobj->xsk_arr[j])); - ifobj->xsk_arr[j].rxqsize =3D XSK_RING_CONS__DEFAULT_NUM_DESCS; - ifobj->xsk_arr[j].batch_size =3D DEFAULT_BATCH_SIZE; - if (i =3D=3D 0) - ifobj->xsk_arr[j].pkt_stream =3D test->tx_pkt_stream_default; - else - ifobj->xsk_arr[j].pkt_stream =3D test->rx_pkt_stream_default; - - memcpy(ifobj->xsk_arr[j].src_mac, g_mac, ETH_ALEN); - memcpy(ifobj->xsk_arr[j].dst_mac, g_mac, ETH_ALEN); - ifobj->xsk_arr[j].src_mac[5] +=3D ((j * 2) + 0); - ifobj->xsk_arr[j].dst_mac[5] +=3D ((j * 2) + 1); - } - } - - if (ifobj_tx->hw_ring_size_supp) - hw_ring_size_reset(ifobj_tx); - - test->ifobj_tx =3D ifobj_tx; - test->ifobj_rx =3D ifobj_rx; - test->current_step =3D 0; - test->total_steps =3D 1; - test->nb_sockets =3D 1; - test->fail =3D false; - test->set_ring =3D false; - test->adjust_tail =3D false; - test->adjust_tail_support =3D false; - test->mtu =3D MAX_ETH_PKT_SIZE; - test->xdp_prog_rx =3D ifobj_rx->xdp_progs->progs.xsk_def_prog; - test->xskmap_rx =3D ifobj_rx->xdp_progs->maps.xsk; - test->xdp_prog_tx =3D ifobj_tx->xdp_progs->progs.xsk_def_prog; - test->xskmap_tx =3D ifobj_tx->xdp_progs->maps.xsk; + xsk_xdp_progs__destroy(ifobj->xdp_progs); } =20 -static void test_spec_init(struct test_spec *test, struct ifobject *ifobj_= tx, - struct ifobject *ifobj_rx, enum test_mode mode, - const struct test_spec *test_to_run) +static void run_pkt_test(struct test_spec *test) { - struct pkt_stream *tx_pkt_stream; - struct pkt_stream *rx_pkt_stream; - u32 i; - - tx_pkt_stream =3D test->tx_pkt_stream_default; - rx_pkt_stream =3D test->rx_pkt_stream_default; - memset(test, 0, sizeof(*test)); - test->tx_pkt_stream_default =3D tx_pkt_stream; - test->rx_pkt_stream_default =3D rx_pkt_stream; + int ret; =20 - for (i =3D 0; i < MAX_INTERFACES; i++) { - struct ifobject *ifobj =3D i ? ifobj_rx : ifobj_tx; + ret =3D test->test_func(test); =20 - ifobj->bind_flags =3D XDP_USE_NEED_WAKEUP; - if (mode =3D=3D TEST_MODE_ZC) - ifobj->bind_flags |=3D XDP_ZEROCOPY; - else - ifobj->bind_flags |=3D XDP_COPY; + switch (ret) { + case TEST_PASS: + ksft_test_result_pass("PASS: %s %s%s\n", mode_string(test), busy_poll_st= ring(test), + test->name); + break; + case TEST_SKIP: + ksft_test_result_skip("SKIP: %s %s%s\n", mode_string(test), busy_poll_st= ring(test), + test->name); + break; + case TEST_FAILURE: + ksft_test_result_fail("FAIL: %s %s%s\n", mode_string(test), busy_poll_st= ring(test), + test->name); + break; + default: + ksft_test_result_fail("FAIL: %s %s%s -- Unexpected returned value (%d)\n= ", + mode_string(test), busy_poll_string(test), test->name, ret); } =20 - strncpy(test->name, test_to_run->name, MAX_TEST_NAME_SIZE); - test->test_func =3D test_to_run->test_func; - test->mode =3D mode; - __test_spec_init(test, ifobj_tx, ifobj_rx); -} - -static void test_spec_reset(struct test_spec *test) -{ - __test_spec_init(test, test->ifobj_tx, test->ifobj_rx); + pkt_stream_restore_default(test); } =20 -static void test_spec_set_xdp_prog(struct test_spec *test, struct bpf_prog= ram *xdp_prog_rx, - struct bpf_program *xdp_prog_tx, struct bpf_map *xskmap_rx, - struct bpf_map *xskmap_tx) +static bool is_xdp_supported(int ifindex) { - test->xdp_prog_rx =3D xdp_prog_rx; - test->xdp_prog_tx =3D xdp_prog_tx; - test->xskmap_rx =3D xskmap_rx; - test->xskmap_tx =3D xskmap_tx; -} + int flags =3D XDP_FLAGS_DRV_MODE; =20 -static int test_spec_set_mtu(struct test_spec *test, int mtu) -{ + LIBBPF_OPTS(bpf_link_create_opts, opts, .flags =3D flags); + struct bpf_insn insns[2] =3D { + BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), + BPF_EXIT_INSN() + }; + int prog_fd, insn_cnt =3D ARRAY_SIZE(insns); int err; =20 - if (test->ifobj_rx->mtu !=3D mtu) { - err =3D xsk_set_mtu(test->ifobj_rx->ifindex, mtu); - if (err) - return err; - test->ifobj_rx->mtu =3D mtu; - } - if (test->ifobj_tx->mtu !=3D mtu) { - err =3D xsk_set_mtu(test->ifobj_tx->ifindex, mtu); - if (err) - return err; - test->ifobj_tx->mtu =3D mtu; - } - - return 0; -} - -static void pkt_stream_reset(struct pkt_stream *pkt_stream) -{ - if (pkt_stream) { - pkt_stream->current_pkt_nb =3D 0; - pkt_stream->nb_rx_pkts =3D 0; - } -} - -static struct pkt *pkt_stream_get_next_tx_pkt(struct pkt_stream *pkt_strea= m) -{ - if (pkt_stream->current_pkt_nb >=3D pkt_stream->nb_pkts) - return NULL; - - return &pkt_stream->pkts[pkt_stream->current_pkt_nb++]; -} - -static struct pkt *pkt_stream_get_next_rx_pkt(struct pkt_stream *pkt_strea= m, u32 *pkts_sent) -{ - while (pkt_stream->current_pkt_nb < pkt_stream->nb_pkts) { - (*pkts_sent)++; - if (pkt_stream->pkts[pkt_stream->current_pkt_nb].valid) - return &pkt_stream->pkts[pkt_stream->current_pkt_nb++]; - pkt_stream->current_pkt_nb++; - } - return NULL; -} - -static void pkt_stream_delete(struct pkt_stream *pkt_stream) -{ - free(pkt_stream->pkts); - free(pkt_stream); -} - -static void pkt_stream_restore_default(struct test_spec *test) -{ - struct pkt_stream *tx_pkt_stream =3D test->ifobj_tx->xsk->pkt_stream; - struct pkt_stream *rx_pkt_stream =3D test->ifobj_rx->xsk->pkt_stream; - - if (tx_pkt_stream !=3D test->tx_pkt_stream_default) { - pkt_stream_delete(test->ifobj_tx->xsk->pkt_stream); - test->ifobj_tx->xsk->pkt_stream =3D test->tx_pkt_stream_default; - } - - if (rx_pkt_stream !=3D test->rx_pkt_stream_default) { - pkt_stream_delete(test->ifobj_rx->xsk->pkt_stream); - test->ifobj_rx->xsk->pkt_stream =3D test->rx_pkt_stream_default; - } -} - -static struct pkt_stream *__pkt_stream_alloc(u32 nb_pkts) -{ - struct pkt_stream *pkt_stream; - - pkt_stream =3D calloc(1, sizeof(*pkt_stream)); - if (!pkt_stream) - return NULL; - - pkt_stream->pkts =3D calloc(nb_pkts, sizeof(*pkt_stream->pkts)); - if (!pkt_stream->pkts) { - free(pkt_stream); - return NULL; - } - - pkt_stream->nb_pkts =3D nb_pkts; - return pkt_stream; -} - -static bool pkt_continues(u32 options) -{ - return options & XDP_PKT_CONTD; -} - -static u32 ceil_u32(u32 a, u32 b) -{ - return (a + b - 1) / b; -} - -static u32 pkt_nb_frags(u32 frame_size, struct pkt_stream *pkt_stream, str= uct pkt *pkt) -{ - u32 nb_frags =3D 1, next_frag; - - if (!pkt) - return 1; - - if (!pkt_stream->verbatim) { - if (!pkt->valid || !pkt->len) - return 1; - return ceil_u32(pkt->len, frame_size); - } - - /* Search for the end of the packet in verbatim mode */ - if (!pkt_continues(pkt->options)) - return nb_frags; - - next_frag =3D pkt_stream->current_pkt_nb; - pkt++; - while (next_frag++ < pkt_stream->nb_pkts) { - nb_frags++; - if (!pkt_continues(pkt->options) || !pkt->valid) - break; - pkt++; - } - return nb_frags; -} - -static bool set_pkt_valid(int offset, u32 len) -{ - return len <=3D MAX_ETH_JUMBO_SIZE; -} - -static void pkt_set(struct pkt_stream *pkt_stream, struct pkt *pkt, int of= fset, u32 len) -{ - pkt->offset =3D offset; - pkt->len =3D len; - pkt->valid =3D set_pkt_valid(offset, len); -} - -static void pkt_stream_pkt_set(struct pkt_stream *pkt_stream, struct pkt *= pkt, int offset, u32 len) -{ - bool prev_pkt_valid =3D pkt->valid; - - pkt_set(pkt_stream, pkt, offset, len); - pkt_stream->nb_valid_entries +=3D pkt->valid - prev_pkt_valid; -} - -static u32 pkt_get_buffer_len(struct xsk_umem_info *umem, u32 len) -{ - return ceil_u32(len, umem->frame_size) * umem->frame_size; -} - -static struct pkt_stream *__pkt_stream_generate(u32 nb_pkts, u32 pkt_len, = u32 nb_start, u32 nb_off) -{ - struct pkt_stream *pkt_stream; - u32 i; - - pkt_stream =3D __pkt_stream_alloc(nb_pkts); - if (!pkt_stream) - exit_with_error(ENOMEM); - - pkt_stream->nb_pkts =3D nb_pkts; - pkt_stream->max_pkt_len =3D pkt_len; - for (i =3D 0; i < nb_pkts; i++) { - struct pkt *pkt =3D &pkt_stream->pkts[i]; - - pkt_stream_pkt_set(pkt_stream, pkt, 0, pkt_len); - pkt->pkt_nb =3D nb_start + i * nb_off; - } - - return pkt_stream; -} - -static struct pkt_stream *pkt_stream_generate(u32 nb_pkts, u32 pkt_len) -{ - return __pkt_stream_generate(nb_pkts, pkt_len, 0, 1); -} - -static struct pkt_stream *pkt_stream_clone(struct pkt_stream *pkt_stream) -{ - return pkt_stream_generate(pkt_stream->nb_pkts, pkt_stream->pkts[0].len); -} - -static void pkt_stream_replace_ifobject(struct ifobject *ifobj, u32 nb_pkt= s, u32 pkt_len) -{ - ifobj->xsk->pkt_stream =3D pkt_stream_generate(nb_pkts, pkt_len); -} - -static void pkt_stream_replace(struct test_spec *test, u32 nb_pkts, u32 pk= t_len) -{ - pkt_stream_replace_ifobject(test->ifobj_tx, nb_pkts, pkt_len); - pkt_stream_replace_ifobject(test->ifobj_rx, nb_pkts, pkt_len); -} - -static void __pkt_stream_replace_half(struct ifobject *ifobj, u32 pkt_len, - int offset) -{ - struct pkt_stream *pkt_stream; - u32 i; - - pkt_stream =3D pkt_stream_clone(ifobj->xsk->pkt_stream); - for (i =3D 1; i < ifobj->xsk->pkt_stream->nb_pkts; i +=3D 2) - pkt_stream_pkt_set(pkt_stream, &pkt_stream->pkts[i], offset, pkt_len); - - ifobj->xsk->pkt_stream =3D pkt_stream; -} - -static void pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, i= nt offset) -{ - __pkt_stream_replace_half(test->ifobj_tx, pkt_len, offset); - __pkt_stream_replace_half(test->ifobj_rx, pkt_len, offset); -} - -static void pkt_stream_receive_half(struct test_spec *test) -{ - struct pkt_stream *pkt_stream =3D test->ifobj_tx->xsk->pkt_stream; - u32 i; - - test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(pkt_stream->nb_pk= ts, - pkt_stream->pkts[0].len); - pkt_stream =3D test->ifobj_rx->xsk->pkt_stream; - for (i =3D 1; i < pkt_stream->nb_pkts; i +=3D 2) - pkt_stream->pkts[i].valid =3D false; - - pkt_stream->nb_valid_entries /=3D 2; -} - -static void pkt_stream_even_odd_sequence(struct test_spec *test) -{ - struct pkt_stream *pkt_stream; - u32 i; - - for (i =3D 0; i < test->nb_sockets; i++) { - pkt_stream =3D test->ifobj_tx->xsk_arr[i].pkt_stream; - pkt_stream =3D __pkt_stream_generate(pkt_stream->nb_pkts / 2, - pkt_stream->pkts[0].len, i, 2); - test->ifobj_tx->xsk_arr[i].pkt_stream =3D pkt_stream; + prog_fd =3D bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt= , NULL); + if (prog_fd < 0) + return false; =20 - pkt_stream =3D test->ifobj_rx->xsk_arr[i].pkt_stream; - pkt_stream =3D __pkt_stream_generate(pkt_stream->nb_pkts / 2, - pkt_stream->pkts[0].len, i, 2); - test->ifobj_rx->xsk_arr[i].pkt_stream =3D pkt_stream; + err =3D bpf_xdp_attach(ifindex, prog_fd, flags, NULL); + if (err) { + close(prog_fd); + return false; } -} - -static u64 pkt_get_addr(struct pkt *pkt, struct xsk_umem_info *umem) -{ - if (!pkt->valid) - return pkt->offset; - return pkt->offset + umem_alloc_buffer(umem); -} =20 -static void pkt_stream_cancel(struct pkt_stream *pkt_stream) -{ - pkt_stream->current_pkt_nb--; -} - -static void pkt_generate(struct xsk_socket_info *xsk, struct xsk_umem_info= *umem, u64 addr, u32 len, - u32 pkt_nb, u32 bytes_written) -{ - void *data =3D xsk_umem__get_data(umem->buffer, addr); - - if (len < MIN_PKT_SIZE) - return; - - if (!bytes_written) { - gen_eth_hdr(xsk, data); - - len -=3D PKT_HDR_SIZE; - data +=3D PKT_HDR_SIZE; - } else { - bytes_written -=3D PKT_HDR_SIZE; - } + bpf_xdp_detach(ifindex, flags, NULL); + close(prog_fd); =20 - write_payload(data, pkt_nb, bytes_written, len); + return true; } =20 -static struct pkt_stream *__pkt_stream_generate_custom(struct ifobject *if= obj, struct pkt *frames, - u32 nb_frames, bool verbatim) -{ - u32 i, len =3D 0, pkt_nb =3D 0, payload =3D 0; - struct pkt_stream *pkt_stream; - - pkt_stream =3D __pkt_stream_alloc(nb_frames); - if (!pkt_stream) - exit_with_error(ENOMEM); - - for (i =3D 0; i < nb_frames; i++) { - struct pkt *pkt =3D &pkt_stream->pkts[pkt_nb]; - struct pkt *frame =3D &frames[i]; - - pkt->offset =3D frame->offset; - if (verbatim) { - *pkt =3D *frame; - pkt->pkt_nb =3D payload; - if (!frame->valid || !pkt_continues(frame->options)) - payload++; - } else { - if (frame->valid) - len +=3D frame->len; - if (frame->valid && pkt_continues(frame->options)) - continue; - - pkt->pkt_nb =3D pkt_nb; - pkt->len =3D len; - pkt->valid =3D frame->valid; - pkt->options =3D 0; - - len =3D 0; - } - - print_verbose("offset: %d len: %u valid: %u options: %u pkt_nb: %u\n", - pkt->offset, pkt->len, pkt->valid, pkt->options, pkt->pkt_nb); - - if (pkt->valid && pkt->len > pkt_stream->max_pkt_len) - pkt_stream->max_pkt_len =3D pkt->len; - - if (pkt->valid) - pkt_stream->nb_valid_entries++; - - pkt_nb++; - } - - pkt_stream->nb_pkts =3D pkt_nb; - pkt_stream->verbatim =3D verbatim; - return pkt_stream; -} - -static void pkt_stream_generate_custom(struct test_spec *test, struct pkt = *pkts, u32 nb_pkts) -{ - struct pkt_stream *pkt_stream; - - pkt_stream =3D __pkt_stream_generate_custom(test->ifobj_tx, pkts, nb_pkts= , true); - test->ifobj_tx->xsk->pkt_stream =3D pkt_stream; - - pkt_stream =3D __pkt_stream_generate_custom(test->ifobj_rx, pkts, nb_pkts= , false); - test->ifobj_rx->xsk->pkt_stream =3D pkt_stream; -} - -static void pkt_print_data(u32 *data, u32 cnt) -{ - u32 i; - - for (i =3D 0; i < cnt; i++) { - u32 seqnum, pkt_nb; - - seqnum =3D ntohl(*data) & 0xffff; - pkt_nb =3D ntohl(*data) >> 16; - ksft_print_msg("%u:%u ", pkt_nb, seqnum); - data++; - } -} - -static void pkt_dump(void *pkt, u32 len, bool eth_header) -{ - struct ethhdr *ethhdr =3D pkt; - u32 i, *data; - - if (eth_header) { - /*extract L2 frame */ - ksft_print_msg("DEBUG>> L2: dst mac: "); - for (i =3D 0; i < ETH_ALEN; i++) - ksft_print_msg("%02X", ethhdr->h_dest[i]); - - ksft_print_msg("\nDEBUG>> L2: src mac: "); - for (i =3D 0; i < ETH_ALEN; i++) - ksft_print_msg("%02X", ethhdr->h_source[i]); - - data =3D pkt + PKT_HDR_SIZE; - } else { - data =3D pkt; - } - - /*extract L5 frame */ - ksft_print_msg("\nDEBUG>> L5: seqnum: "); - pkt_print_data(data, PKT_DUMP_NB_TO_PRINT); - ksft_print_msg("...."); - if (len > PKT_DUMP_NB_TO_PRINT * sizeof(u32)) { - ksft_print_msg("\n.... "); - pkt_print_data(data + len / sizeof(u32) - PKT_DUMP_NB_TO_PRINT, - PKT_DUMP_NB_TO_PRINT); - } - ksft_print_msg("\n---------------------------------------\n"); -} - -static bool is_offset_correct(struct xsk_umem_info *umem, struct pkt *pkt,= u64 addr) -{ - u32 headroom =3D umem->unaligned_mode ? 0 : umem->frame_headroom; - u32 offset =3D addr % umem->frame_size, expected_offset; - int pkt_offset =3D pkt->valid ? pkt->offset : 0; - - if (!umem->unaligned_mode) - pkt_offset =3D 0; - - expected_offset =3D (pkt_offset + headroom + XDP_PACKET_HEADROOM) % umem-= >frame_size; - - if (offset =3D=3D expected_offset) - return true; - - ksft_print_msg("[%s] expected [%u], got [%u]\n", __func__, expected_offse= t, offset); - return false; -} - -static bool is_metadata_correct(struct pkt *pkt, void *buffer, u64 addr) -{ - void *data =3D xsk_umem__get_data(buffer, addr); - struct xdp_info *meta =3D data - sizeof(struct xdp_info); - - if (meta->count !=3D pkt->pkt_nb) { - ksft_print_msg("[%s] expected meta_count [%d], got meta_count [%llu]\n", - __func__, pkt->pkt_nb, - (unsigned long long)meta->count); - return false; - } - - return true; -} - -static bool is_adjust_tail_supported(struct xsk_xdp_progs *skel_rx) -{ - struct bpf_map *data_map; - int adjust_value =3D 0; - int key =3D 0; - int ret; - - data_map =3D bpf_object__find_map_by_name(skel_rx->obj, "xsk_xdp_.bss"); - if (!data_map || !bpf_map__is_internal(data_map)) { - ksft_print_msg("Error: could not find bss section of XDP program\n"); - exit_with_error(errno); - } - - ret =3D bpf_map_lookup_elem(bpf_map__fd(data_map), &key, &adjust_value); - if (ret) { - ksft_print_msg("Error: bpf_map_lookup_elem failed with error %d\n", ret); - exit_with_error(errno); - } - - /* Set the 'adjust_value' variable to -EOPNOTSUPP in the XDP program if t= he adjust_tail - * helper is not supported. Skip the adjust_tail test case in this scenar= io. - */ - return adjust_value !=3D -EOPNOTSUPP; -} - -static bool is_frag_valid(struct xsk_umem_info *umem, u64 addr, u32 len, u= 32 expected_pkt_nb, - u32 bytes_processed) -{ - u32 seqnum, pkt_nb, *pkt_data, words_to_end, expected_seqnum; - void *data =3D xsk_umem__get_data(umem->buffer, addr); - - addr -=3D umem->base_addr; - - if (addr >=3D umem->num_frames * umem->frame_size || - addr + len > umem->num_frames * umem->frame_size) { - ksft_print_msg("Frag invalid addr: %llx len: %u\n", - (unsigned long long)addr, len); - return false; - } - if (!umem->unaligned_mode && addr % umem->frame_size + len > umem->frame_= size) { - ksft_print_msg("Frag crosses frame boundary addr: %llx len: %u\n", - (unsigned long long)addr, len); - return false; - } - - pkt_data =3D data; - if (!bytes_processed) { - pkt_data +=3D PKT_HDR_SIZE / sizeof(*pkt_data); - len -=3D PKT_HDR_SIZE; - } else { - bytes_processed -=3D PKT_HDR_SIZE; - } - - expected_seqnum =3D bytes_processed / sizeof(*pkt_data); - seqnum =3D ntohl(*pkt_data) & 0xffff; - pkt_nb =3D ntohl(*pkt_data) >> 16; - - if (expected_pkt_nb !=3D pkt_nb) { - ksft_print_msg("[%s] expected pkt_nb [%u], got pkt_nb [%u]\n", - __func__, expected_pkt_nb, pkt_nb); - goto error; - } - if (expected_seqnum !=3D seqnum) { - ksft_print_msg("[%s] expected seqnum at start [%u], got seqnum [%u]\n", - __func__, expected_seqnum, seqnum); - goto error; - } - - words_to_end =3D len / sizeof(*pkt_data) - 1; - pkt_data +=3D words_to_end; - seqnum =3D ntohl(*pkt_data) & 0xffff; - expected_seqnum +=3D words_to_end; - if (expected_seqnum !=3D seqnum) { - ksft_print_msg("[%s] expected seqnum at end [%u], got seqnum [%u]\n", - __func__, expected_seqnum, seqnum); - goto error; - } - - return true; - -error: - pkt_dump(data, len, !bytes_processed); - return false; -} - -static bool is_pkt_valid(struct pkt *pkt, void *buffer, u64 addr, u32 len) -{ - if (pkt->len !=3D len) { - ksft_print_msg("[%s] expected packet length [%d], got length [%d]\n", - __func__, pkt->len, len); - pkt_dump(xsk_umem__get_data(buffer, addr), len, true); - return false; - } - - return true; -} - -static u32 load_value(u32 *counter) -{ - return __atomic_load_n(counter, __ATOMIC_ACQUIRE); -} - -static bool kick_tx_with_check(struct xsk_socket_info *xsk, int *ret) -{ - u32 max_budget =3D MAX_TX_BUDGET_DEFAULT; - u32 cons, ready_to_send; - int delta; - - cons =3D load_value(xsk->tx.consumer); - ready_to_send =3D load_value(xsk->tx.producer) - cons; - *ret =3D sendto(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, 0); - - delta =3D load_value(xsk->tx.consumer) - cons; - /* By default, xsk should consume exact @max_budget descs at one - * send in this case where hitting the max budget limit in while - * loop is triggered in __xsk_generic_xmit(). Please make sure that - * the number of descs to be sent is larger than @max_budget, or - * else the tx.consumer will be updated in xskq_cons_peek_desc() - * in time which hides the issue we try to verify. - */ - if (ready_to_send > max_budget && delta !=3D max_budget) - return false; - - return true; -} - -static int kick_tx(struct xsk_socket_info *xsk) -{ - int ret; - - if (xsk->check_consumer) { - if (!kick_tx_with_check(xsk, &ret)) - return TEST_FAILURE; - } else { - ret =3D sendto(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, 0); - } - if (ret >=3D 0) - return TEST_PASS; - if (errno =3D=3D ENOBUFS || errno =3D=3D EAGAIN || errno =3D=3D EBUSY || = errno =3D=3D ENETDOWN) { - usleep(100); - return TEST_PASS; - } - return TEST_FAILURE; -} - -static int kick_rx(struct xsk_socket_info *xsk) -{ - int ret; - - ret =3D recvfrom(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, N= ULL); - if (ret < 0) - return TEST_FAILURE; - - return TEST_PASS; -} - -static int complete_pkts(struct xsk_socket_info *xsk, int batch_size) -{ - unsigned int rcvd; - u32 idx; - int ret; - - if (xsk_ring_prod__needs_wakeup(&xsk->tx)) { - ret =3D kick_tx(xsk); - if (ret) - return TEST_FAILURE; - } - - rcvd =3D xsk_ring_cons__peek(&xsk->umem->cq, batch_size, &idx); - if (rcvd) { - if (rcvd > xsk->outstanding_tx) { - u64 addr =3D *xsk_ring_cons__comp_addr(&xsk->umem->cq, idx + rcvd - 1); - - ksft_print_msg("[%s] Too many packets completed\n", __func__); - ksft_print_msg("Last completion address: %llx\n", - (unsigned long long)addr); - return TEST_FAILURE; - } - - xsk_ring_cons__release(&xsk->umem->cq, rcvd); - xsk->outstanding_tx -=3D rcvd; - } - - return TEST_PASS; -} - -static int __receive_pkts(struct test_spec *test, struct xsk_socket_info *= xsk) -{ - u32 frags_processed =3D 0, nb_frags =3D 0, pkt_len =3D 0; - u32 idx_rx =3D 0, idx_fq =3D 0, rcvd, pkts_sent =3D 0; - struct pkt_stream *pkt_stream =3D xsk->pkt_stream; - struct ifobject *ifobj =3D test->ifobj_rx; - struct xsk_umem_info *umem =3D xsk->umem; - struct pollfd fds =3D { }; - struct pkt *pkt; - u64 first_addr =3D 0; - int ret; - - fds.fd =3D xsk_socket__fd(xsk->xsk); - fds.events =3D POLLIN; - - ret =3D kick_rx(xsk); - if (ret) - return TEST_FAILURE; - - if (ifobj->use_poll) { - ret =3D poll(&fds, 1, POLL_TMOUT); - if (ret < 0) - return TEST_FAILURE; - - if (!ret) { - if (!is_umem_valid(test->ifobj_tx)) - return TEST_PASS; - - ksft_print_msg("ERROR: [%s] Poll timed out\n", __func__); - return TEST_CONTINUE; - } - - if (!(fds.revents & POLLIN)) - return TEST_CONTINUE; - } - - rcvd =3D xsk_ring_cons__peek(&xsk->rx, xsk->batch_size, &idx_rx); - if (!rcvd) - return TEST_CONTINUE; - - if (ifobj->use_fill_ring) { - ret =3D xsk_ring_prod__reserve(&umem->fq, rcvd, &idx_fq); - while (ret !=3D rcvd) { - if (xsk_ring_prod__needs_wakeup(&umem->fq)) { - ret =3D poll(&fds, 1, POLL_TMOUT); - if (ret < 0) - return TEST_FAILURE; - } - ret =3D xsk_ring_prod__reserve(&umem->fq, rcvd, &idx_fq); - } - } - - while (frags_processed < rcvd) { - const struct xdp_desc *desc =3D xsk_ring_cons__rx_desc(&xsk->rx, idx_rx+= +); - u64 addr =3D desc->addr, orig; - - orig =3D xsk_umem__extract_addr(addr); - addr =3D xsk_umem__add_offset_to_addr(addr); - - if (!nb_frags) { - pkt =3D pkt_stream_get_next_rx_pkt(pkt_stream, &pkts_sent); - if (!pkt) { - ksft_print_msg("[%s] received too many packets addr: %lx len %u\n", - __func__, addr, desc->len); - return TEST_FAILURE; - } - } - - print_verbose("Rx: addr: %lx len: %u options: %u pkt_nb: %u valid: %u\n", - addr, desc->len, desc->options, pkt->pkt_nb, pkt->valid); - - if (!is_frag_valid(umem, addr, desc->len, pkt->pkt_nb, pkt_len) || - !is_offset_correct(umem, pkt, addr) || (ifobj->use_metadata && - !is_metadata_correct(pkt, umem->buffer, addr))) - return TEST_FAILURE; - - if (!nb_frags++) - first_addr =3D addr; - frags_processed++; - pkt_len +=3D desc->len; - if (ifobj->use_fill_ring) - *xsk_ring_prod__fill_addr(&umem->fq, idx_fq++) =3D orig; - - if (pkt_continues(desc->options)) - continue; - - /* The complete packet has been received */ - if (!is_pkt_valid(pkt, umem->buffer, first_addr, pkt_len) || - !is_offset_correct(umem, pkt, addr)) - return TEST_FAILURE; - - pkt_stream->nb_rx_pkts++; - nb_frags =3D 0; - pkt_len =3D 0; - } - - if (nb_frags) { - /* In the middle of a packet. Start over from beginning of packet. */ - idx_rx -=3D nb_frags; - xsk_ring_cons__cancel(&xsk->rx, nb_frags); - if (ifobj->use_fill_ring) { - idx_fq -=3D nb_frags; - xsk_ring_prod__cancel(&umem->fq, nb_frags); - } - frags_processed -=3D nb_frags; - } - - if (ifobj->use_fill_ring) - xsk_ring_prod__submit(&umem->fq, frags_processed); - if (ifobj->release_rx) - xsk_ring_cons__release(&xsk->rx, frags_processed); - - pthread_mutex_lock(&pacing_mutex); - pkts_in_flight -=3D pkts_sent; - pthread_mutex_unlock(&pacing_mutex); - pkts_sent =3D 0; - -return TEST_CONTINUE; -} - -bool all_packets_received(struct test_spec *test, struct xsk_socket_info *= xsk, u32 sock_num, - unsigned long *bitmap) -{ - struct pkt_stream *pkt_stream =3D xsk->pkt_stream; - - if (!pkt_stream) { - __set_bit(sock_num, bitmap); - return false; - } - - if (pkt_stream->nb_rx_pkts =3D=3D pkt_stream->nb_valid_entries) { - __set_bit(sock_num, bitmap); - if (bitmap_full(bitmap, test->nb_sockets)) - return true; - } - - return false; -} - -static int receive_pkts(struct test_spec *test) -{ - struct timeval tv_end, tv_now, tv_timeout =3D {THREAD_TMOUT, 0}; - DECLARE_BITMAP(bitmap, test->nb_sockets); - struct xsk_socket_info *xsk; - u32 sock_num =3D 0; - int res, ret; - - ret =3D gettimeofday(&tv_now, NULL); - if (ret) - exit_with_error(errno); - - timeradd(&tv_now, &tv_timeout, &tv_end); - - while (1) { - xsk =3D &test->ifobj_rx->xsk_arr[sock_num]; - - if ((all_packets_received(test, xsk, sock_num, bitmap))) - break; - - res =3D __receive_pkts(test, xsk); - if (!(res =3D=3D TEST_PASS || res =3D=3D TEST_CONTINUE)) - return res; - - ret =3D gettimeofday(&tv_now, NULL); - if (ret) - exit_with_error(errno); - - if (timercmp(&tv_now, &tv_end, >)) { - ksft_print_msg("ERROR: [%s] Receive loop timed out\n", __func__); - return TEST_FAILURE; - } - sock_num =3D (sock_num + 1) % test->nb_sockets; - } - - return TEST_PASS; -} - -static int __send_pkts(struct ifobject *ifobject, struct xsk_socket_info *= xsk, bool timeout) -{ - u32 i, idx =3D 0, valid_pkts =3D 0, valid_frags =3D 0, buffer_len; - struct pkt_stream *pkt_stream =3D xsk->pkt_stream; - struct xsk_umem_info *umem =3D ifobject->umem; - bool use_poll =3D ifobject->use_poll; - struct pollfd fds =3D { }; - int ret; - - buffer_len =3D pkt_get_buffer_len(umem, pkt_stream->max_pkt_len); - /* pkts_in_flight might be negative if many invalid packets are sent */ - if (pkts_in_flight >=3D (int)((umem_size(umem) - xsk->batch_size * buffer= _len) / - buffer_len)) { - ret =3D kick_tx(xsk); - if (ret) - return TEST_FAILURE; - return TEST_CONTINUE; - } - - fds.fd =3D xsk_socket__fd(xsk->xsk); - fds.events =3D POLLOUT; - - while (xsk_ring_prod__reserve(&xsk->tx, xsk->batch_size, &idx) < xsk->bat= ch_size) { - if (use_poll) { - ret =3D poll(&fds, 1, POLL_TMOUT); - if (timeout) { - if (ret < 0) { - ksft_print_msg("ERROR: [%s] Poll error %d\n", - __func__, errno); - return TEST_FAILURE; - } - if (ret =3D=3D 0) - return TEST_PASS; - break; - } - if (ret <=3D 0) { - ksft_print_msg("ERROR: [%s] Poll error %d\n", - __func__, errno); - return TEST_FAILURE; - } - } - - complete_pkts(xsk, xsk->batch_size); - } - - for (i =3D 0; i < xsk->batch_size; i++) { - struct pkt *pkt =3D pkt_stream_get_next_tx_pkt(pkt_stream); - u32 nb_frags_left, nb_frags, bytes_written =3D 0; - - if (!pkt) - break; - - nb_frags =3D pkt_nb_frags(umem->frame_size, pkt_stream, pkt); - if (nb_frags > xsk->batch_size - i) { - pkt_stream_cancel(pkt_stream); - xsk_ring_prod__cancel(&xsk->tx, xsk->batch_size - i); - break; - } - nb_frags_left =3D nb_frags; - - while (nb_frags_left--) { - struct xdp_desc *tx_desc =3D xsk_ring_prod__tx_desc(&xsk->tx, idx + i); - - tx_desc->addr =3D pkt_get_addr(pkt, ifobject->umem); - if (pkt_stream->verbatim) { - tx_desc->len =3D pkt->len; - tx_desc->options =3D pkt->options; - } else if (nb_frags_left) { - tx_desc->len =3D umem->frame_size; - tx_desc->options =3D XDP_PKT_CONTD; - } else { - tx_desc->len =3D pkt->len - bytes_written; - tx_desc->options =3D 0; - } - if (pkt->valid) - pkt_generate(xsk, umem, tx_desc->addr, tx_desc->len, pkt->pkt_nb, - bytes_written); - bytes_written +=3D tx_desc->len; - - print_verbose("Tx addr: %llx len: %u options: %u pkt_nb: %u\n", - tx_desc->addr, tx_desc->len, tx_desc->options, pkt->pkt_nb); - - if (nb_frags_left) { - i++; - if (pkt_stream->verbatim) - pkt =3D pkt_stream_get_next_tx_pkt(pkt_stream); - } - } - - if (pkt && pkt->valid) { - valid_pkts++; - valid_frags +=3D nb_frags; - } - } - - pthread_mutex_lock(&pacing_mutex); - pkts_in_flight +=3D valid_pkts; - pthread_mutex_unlock(&pacing_mutex); - - xsk_ring_prod__submit(&xsk->tx, i); - xsk->outstanding_tx +=3D valid_frags; - - if (use_poll) { - ret =3D poll(&fds, 1, POLL_TMOUT); - if (ret <=3D 0) { - if (ret =3D=3D 0 && timeout) - return TEST_PASS; - - ksft_print_msg("ERROR: [%s] Poll error %d\n", __func__, ret); - return TEST_FAILURE; - } - } - - if (!timeout) { - if (complete_pkts(xsk, i)) - return TEST_FAILURE; - - usleep(10); - return TEST_PASS; - } - - return TEST_CONTINUE; -} - -static int wait_for_tx_completion(struct xsk_socket_info *xsk) -{ - struct timeval tv_end, tv_now, tv_timeout =3D {THREAD_TMOUT, 0}; - int ret; - - ret =3D gettimeofday(&tv_now, NULL); - if (ret) - exit_with_error(errno); - timeradd(&tv_now, &tv_timeout, &tv_end); - - while (xsk->outstanding_tx) { - ret =3D gettimeofday(&tv_now, NULL); - if (ret) - exit_with_error(errno); - if (timercmp(&tv_now, &tv_end, >)) { - ksft_print_msg("ERROR: [%s] Transmission loop timed out\n", __func__); - return TEST_FAILURE; - } - - complete_pkts(xsk, xsk->batch_size); - } - - return TEST_PASS; -} - -bool all_packets_sent(struct test_spec *test, unsigned long *bitmap) -{ - return bitmap_full(bitmap, test->nb_sockets); -} - -static int send_pkts(struct test_spec *test, struct ifobject *ifobject) -{ - bool timeout =3D !is_umem_valid(test->ifobj_rx); - DECLARE_BITMAP(bitmap, test->nb_sockets); - u32 i, ret; - - while (!(all_packets_sent(test, bitmap))) { - for (i =3D 0; i < test->nb_sockets; i++) { - struct pkt_stream *pkt_stream; - - pkt_stream =3D ifobject->xsk_arr[i].pkt_stream; - if (!pkt_stream || pkt_stream->current_pkt_nb >=3D pkt_stream->nb_pkts)= { - __set_bit(i, bitmap); - continue; - } - ret =3D __send_pkts(ifobject, &ifobject->xsk_arr[i], timeout); - if (ret =3D=3D TEST_CONTINUE && !test->fail) - continue; - - if ((ret || test->fail) && !timeout) - return TEST_FAILURE; - - if (ret =3D=3D TEST_PASS && timeout) - return ret; - - ret =3D wait_for_tx_completion(&ifobject->xsk_arr[i]); - if (ret) - return TEST_FAILURE; - } - } - - return TEST_PASS; -} - -static int get_xsk_stats(struct xsk_socket *xsk, struct xdp_statistics *st= ats) -{ - int fd =3D xsk_socket__fd(xsk), err; - socklen_t optlen, expected_len; - - optlen =3D sizeof(*stats); - err =3D getsockopt(fd, SOL_XDP, XDP_STATISTICS, stats, &optlen); - if (err) { - ksft_print_msg("[%s] getsockopt(XDP_STATISTICS) error %u %s\n", - __func__, -err, strerror(-err)); - return TEST_FAILURE; - } - - expected_len =3D sizeof(struct xdp_statistics); - if (optlen !=3D expected_len) { - ksft_print_msg("[%s] getsockopt optlen error. Expected: %u got: %u\n", - __func__, expected_len, optlen); - return TEST_FAILURE; - } - - return TEST_PASS; -} - -static int validate_rx_dropped(struct ifobject *ifobject) -{ - struct xsk_socket *xsk =3D ifobject->xsk->xsk; - struct xdp_statistics stats; - int err; - - err =3D kick_rx(ifobject->xsk); - if (err) - return TEST_FAILURE; - - err =3D get_xsk_stats(xsk, &stats); - if (err) - return TEST_FAILURE; - - /* The receiver calls getsockopt after receiving the last (valid) - * packet which is not the final packet sent in this test (valid and - * invalid packets are sent in alternating fashion with the final - * packet being invalid). Since the last packet may or may not have - * been dropped already, both outcomes must be allowed. - */ - if (stats.rx_dropped =3D=3D ifobject->xsk->pkt_stream->nb_pkts / 2 || - stats.rx_dropped =3D=3D ifobject->xsk->pkt_stream->nb_pkts / 2 - 1) - return TEST_PASS; - - return TEST_FAILURE; -} - -static int validate_rx_full(struct ifobject *ifobject) -{ - struct xsk_socket *xsk =3D ifobject->xsk->xsk; - struct xdp_statistics stats; - int err; - - usleep(1000); - err =3D kick_rx(ifobject->xsk); - if (err) - return TEST_FAILURE; - - err =3D get_xsk_stats(xsk, &stats); - if (err) - return TEST_FAILURE; - - if (stats.rx_ring_full) - return TEST_PASS; - - return TEST_FAILURE; -} - -static int validate_fill_empty(struct ifobject *ifobject) -{ - struct xsk_socket *xsk =3D ifobject->xsk->xsk; - struct xdp_statistics stats; - int err; - - usleep(1000); - err =3D kick_rx(ifobject->xsk); - if (err) - return TEST_FAILURE; - - err =3D get_xsk_stats(xsk, &stats); - if (err) - return TEST_FAILURE; - - if (stats.rx_fill_ring_empty_descs) - return TEST_PASS; - - return TEST_FAILURE; -} - -static int validate_tx_invalid_descs(struct ifobject *ifobject) -{ - struct xsk_socket *xsk =3D ifobject->xsk->xsk; - int fd =3D xsk_socket__fd(xsk); - struct xdp_statistics stats; - socklen_t optlen; - int err; - - optlen =3D sizeof(stats); - err =3D getsockopt(fd, SOL_XDP, XDP_STATISTICS, &stats, &optlen); - if (err) { - ksft_print_msg("[%s] getsockopt(XDP_STATISTICS) error %u %s\n", - __func__, -err, strerror(-err)); - return TEST_FAILURE; - } - - if (stats.tx_invalid_descs !=3D ifobject->xsk->pkt_stream->nb_pkts / 2) { - ksft_print_msg("[%s] tx_invalid_descs incorrect. Got [%llu] expected [%u= ]\n", - __func__, - (unsigned long long)stats.tx_invalid_descs, - ifobject->xsk->pkt_stream->nb_pkts); - return TEST_FAILURE; - } - - return TEST_PASS; -} - -static void xsk_configure_socket(struct test_spec *test, struct ifobject *= ifobject, - struct xsk_umem_info *umem, bool tx) -{ - int i, ret; - - for (i =3D 0; i < test->nb_sockets; i++) { - bool shared =3D (ifobject->shared_umem && tx) ? true : !!i; - u32 ctr =3D 0; - - while (ctr++ < SOCK_RECONF_CTR) { - ret =3D __xsk_configure_socket(&ifobject->xsk_arr[i], umem, - ifobject, shared); - if (!ret) - break; - - /* Retry if it fails as xsk_socket__create() is asynchronous */ - if (ctr >=3D SOCK_RECONF_CTR) - exit_with_error(-ret); - usleep(USLEEP_MAX); - } - if (ifobject->busy_poll) - enable_busy_poll(&ifobject->xsk_arr[i]); - } -} - -static void thread_common_ops_tx(struct test_spec *test, struct ifobject *= ifobject) -{ - xsk_configure_socket(test, ifobject, test->ifobj_rx->umem, true); - ifobject->xsk =3D &ifobject->xsk_arr[0]; - ifobject->xskmap =3D test->ifobj_rx->xskmap; - memcpy(ifobject->umem, test->ifobj_rx->umem, sizeof(struct xsk_umem_info)= ); - ifobject->umem->base_addr =3D 0; -} - -static void xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_= stream *pkt_stream, - bool fill_up) -{ - u32 rx_frame_size =3D umem->frame_size - XDP_PACKET_HEADROOM; - u32 idx =3D 0, filled =3D 0, buffers_to_fill, nb_pkts; - int ret; - - if (umem->num_frames < XSK_RING_PROD__DEFAULT_NUM_DESCS) - buffers_to_fill =3D umem->num_frames; - else - buffers_to_fill =3D umem->fill_size; - - ret =3D xsk_ring_prod__reserve(&umem->fq, buffers_to_fill, &idx); - if (ret !=3D buffers_to_fill) - exit_with_error(ENOSPC); - - while (filled < buffers_to_fill) { - struct pkt *pkt =3D pkt_stream_get_next_rx_pkt(pkt_stream, &nb_pkts); - u64 addr; - u32 i; - - for (i =3D 0; i < pkt_nb_frags(rx_frame_size, pkt_stream, pkt); i++) { - if (!pkt) { - if (!fill_up) - break; - addr =3D filled * umem->frame_size + umem->base_addr; - } else if (pkt->offset >=3D 0) { - addr =3D pkt->offset % umem->frame_size + umem_alloc_buffer(umem); - } else { - addr =3D pkt->offset + umem_alloc_buffer(umem); - } - - *xsk_ring_prod__fill_addr(&umem->fq, idx++) =3D addr; - if (++filled >=3D buffers_to_fill) - break; - } - } - xsk_ring_prod__submit(&umem->fq, filled); - xsk_ring_prod__cancel(&umem->fq, buffers_to_fill - filled); - - pkt_stream_reset(pkt_stream); - umem_reset_alloc(umem); -} - -static void thread_common_ops(struct test_spec *test, struct ifobject *ifo= bject) -{ - u64 umem_sz =3D ifobject->umem->num_frames * ifobject->umem->frame_size; - int mmap_flags =3D MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE; - LIBBPF_OPTS(bpf_xdp_query_opts, opts); - void *bufs; - int ret; - u32 i; - - if (ifobject->umem->unaligned_mode) - mmap_flags |=3D MAP_HUGETLB | MAP_HUGE_2MB; - - if (ifobject->shared_umem) - umem_sz *=3D 2; - - bufs =3D mmap(NULL, umem_sz, PROT_READ | PROT_WRITE, mmap_flags, -1, 0); - if (bufs =3D=3D MAP_FAILED) - exit_with_error(errno); - - ret =3D xsk_configure_umem(ifobject, ifobject->umem, bufs, umem_sz); - if (ret) - exit_with_error(-ret); - - xsk_configure_socket(test, ifobject, ifobject->umem, false); - - ifobject->xsk =3D &ifobject->xsk_arr[0]; - - if (!ifobject->rx_on) - return; - - xsk_populate_fill_ring(ifobject->umem, ifobject->xsk->pkt_stream, ifobjec= t->use_fill_ring); - - for (i =3D 0; i < test->nb_sockets; i++) { - ifobject->xsk =3D &ifobject->xsk_arr[i]; - ret =3D xsk_update_xskmap(ifobject->xskmap, ifobject->xsk->xsk, i); - if (ret) - exit_with_error(errno); - } -} - -static void *worker_testapp_validate_tx(void *arg) -{ - struct test_spec *test =3D (struct test_spec *)arg; - struct ifobject *ifobject =3D test->ifobj_tx; - int err; - - if (test->current_step =3D=3D 1) { - if (!ifobject->shared_umem) - thread_common_ops(test, ifobject); - else - thread_common_ops_tx(test, ifobject); - } - - err =3D send_pkts(test, ifobject); - - if (!err && ifobject->validation_func) - err =3D ifobject->validation_func(ifobject); - if (err) - report_failure(test); - - pthread_exit(NULL); -} - -static void *worker_testapp_validate_rx(void *arg) -{ - struct test_spec *test =3D (struct test_spec *)arg; - struct ifobject *ifobject =3D test->ifobj_rx; - int err; - - if (test->current_step =3D=3D 1) { - thread_common_ops(test, ifobject); - } else { - xsk_clear_xskmap(ifobject->xskmap); - err =3D xsk_update_xskmap(ifobject->xskmap, ifobject->xsk->xsk, 0); - if (err) { - ksft_print_msg("Error: Failed to update xskmap, error %s\n", - strerror(-err)); - exit_with_error(-err); - } - } - - pthread_barrier_wait(&barr); - - err =3D receive_pkts(test); - - if (!err && ifobject->validation_func) - err =3D ifobject->validation_func(ifobject); - - if (err) { - if (test->adjust_tail && !is_adjust_tail_supported(ifobject->xdp_progs)) - test->adjust_tail_support =3D false; - else - report_failure(test); - } - - pthread_exit(NULL); -} - -static u64 ceil_u64(u64 a, u64 b) -{ - return (a + b - 1) / b; -} - -static void testapp_clean_xsk_umem(struct ifobject *ifobj) -{ - u64 umem_sz =3D ifobj->umem->num_frames * ifobj->umem->frame_size; - - if (ifobj->shared_umem) - umem_sz *=3D 2; - - umem_sz =3D ceil_u64(umem_sz, HUGEPAGE_SIZE) * HUGEPAGE_SIZE; - xsk_umem__delete(ifobj->umem->umem); - munmap(ifobj->umem->buffer, umem_sz); -} - -static void handler(int signum) -{ - pthread_exit(NULL); -} - -static bool xdp_prog_changed_rx(struct test_spec *test) -{ - struct ifobject *ifobj =3D test->ifobj_rx; - - return ifobj->xdp_prog !=3D test->xdp_prog_rx || ifobj->mode !=3D test->m= ode; -} - -static bool xdp_prog_changed_tx(struct test_spec *test) -{ - struct ifobject *ifobj =3D test->ifobj_tx; - - return ifobj->xdp_prog !=3D test->xdp_prog_tx || ifobj->mode !=3D test->m= ode; -} - -static void xsk_reattach_xdp(struct ifobject *ifobj, struct bpf_program *x= dp_prog, - struct bpf_map *xskmap, enum test_mode mode) -{ - int err; - - xsk_detach_xdp_program(ifobj->ifindex, mode_to_xdp_flags(ifobj->mode)); - err =3D xsk_attach_xdp_program(xdp_prog, ifobj->ifindex, mode_to_xdp_flag= s(mode)); - if (err) { - ksft_print_msg("Error attaching XDP program\n"); - exit_with_error(-err); - } - - if (ifobj->mode !=3D mode && (mode =3D=3D TEST_MODE_DRV || mode =3D=3D TE= ST_MODE_ZC)) - if (!xsk_is_in_mode(ifobj->ifindex, XDP_FLAGS_DRV_MODE)) { - ksft_print_msg("ERROR: XDP prog not in DRV mode\n"); - exit_with_error(EINVAL); - } - - ifobj->xdp_prog =3D xdp_prog; - ifobj->xskmap =3D xskmap; - ifobj->mode =3D mode; -} - -static void xsk_attach_xdp_progs(struct test_spec *test, struct ifobject *= ifobj_rx, - struct ifobject *ifobj_tx) -{ - if (xdp_prog_changed_rx(test)) - xsk_reattach_xdp(ifobj_rx, test->xdp_prog_rx, test->xskmap_rx, test->mod= e); - - if (!ifobj_tx || ifobj_tx->shared_umem) - return; - - if (xdp_prog_changed_tx(test)) - xsk_reattach_xdp(ifobj_tx, test->xdp_prog_tx, test->xskmap_tx, test->mod= e); -} - -static int __testapp_validate_traffic(struct test_spec *test, struct ifobj= ect *ifobj1, - struct ifobject *ifobj2) -{ - pthread_t t0, t1; - int err; - - if (test->mtu > MAX_ETH_PKT_SIZE) { - if (test->mode =3D=3D TEST_MODE_ZC && (!ifobj1->multi_buff_zc_supp || - (ifobj2 && !ifobj2->multi_buff_zc_supp))) { - ksft_test_result_skip("Multi buffer for zero-copy not supported.\n"); - return TEST_SKIP; - } - if (test->mode !=3D TEST_MODE_ZC && (!ifobj1->multi_buff_supp || - (ifobj2 && !ifobj2->multi_buff_supp))) { - ksft_test_result_skip("Multi buffer not supported.\n"); - return TEST_SKIP; - } - } - err =3D test_spec_set_mtu(test, test->mtu); - if (err) { - ksft_print_msg("Error, could not set mtu.\n"); - exit_with_error(err); - } - - if (ifobj2) { - if (pthread_barrier_init(&barr, NULL, 2)) - exit_with_error(errno); - pkt_stream_reset(ifobj2->xsk->pkt_stream); - } - - test->current_step++; - pkt_stream_reset(ifobj1->xsk->pkt_stream); - pkts_in_flight =3D 0; - - signal(SIGUSR1, handler); - /*Spawn RX thread */ - pthread_create(&t0, NULL, ifobj1->func_ptr, test); - - if (ifobj2) { - pthread_barrier_wait(&barr); - if (pthread_barrier_destroy(&barr)) - exit_with_error(errno); - - /*Spawn TX thread */ - pthread_create(&t1, NULL, ifobj2->func_ptr, test); - - pthread_join(t1, NULL); - } - - if (!ifobj2) - pthread_kill(t0, SIGUSR1); - else - pthread_join(t0, NULL); - - if (test->total_steps =3D=3D test->current_step || test->fail) { - u32 i; - - if (ifobj2) - for (i =3D 0; i < test->nb_sockets; i++) - xsk_socket__delete(ifobj2->xsk_arr[i].xsk); - - for (i =3D 0; i < test->nb_sockets; i++) - xsk_socket__delete(ifobj1->xsk_arr[i].xsk); - - testapp_clean_xsk_umem(ifobj1); - if (ifobj2 && !ifobj2->shared_umem) - testapp_clean_xsk_umem(ifobj2); - } - - return !!test->fail; -} - -static int testapp_validate_traffic(struct test_spec *test) -{ - struct ifobject *ifobj_rx =3D test->ifobj_rx; - struct ifobject *ifobj_tx =3D test->ifobj_tx; - - if ((ifobj_rx->umem->unaligned_mode && !ifobj_rx->unaligned_supp) || - (ifobj_tx->umem->unaligned_mode && !ifobj_tx->unaligned_supp)) { - ksft_test_result_skip("No huge pages present.\n"); - return TEST_SKIP; - } - - if (test->set_ring) { - if (ifobj_tx->hw_ring_size_supp) { - if (set_ring_size(ifobj_tx)) { - ksft_test_result_skip("Failed to change HW ring size.\n"); - return TEST_FAILURE; - } - } else { - ksft_test_result_skip("Changing HW ring size not supported.\n"); - return TEST_SKIP; - } - } - - xsk_attach_xdp_progs(test, ifobj_rx, ifobj_tx); - return __testapp_validate_traffic(test, ifobj_rx, ifobj_tx); -} - -static int testapp_validate_traffic_single_thread(struct test_spec *test, = struct ifobject *ifobj) -{ - return __testapp_validate_traffic(test, ifobj, NULL); -} - -static int testapp_teardown(struct test_spec *test) -{ - int i; - - for (i =3D 0; i < MAX_TEARDOWN_ITER; i++) { - if (testapp_validate_traffic(test)) - return TEST_FAILURE; - test_spec_reset(test); - } - - return TEST_PASS; -} - -static void swap_directions(struct ifobject **ifobj1, struct ifobject **if= obj2) -{ - thread_func_t tmp_func_ptr =3D (*ifobj1)->func_ptr; - struct ifobject *tmp_ifobj =3D (*ifobj1); - - (*ifobj1)->func_ptr =3D (*ifobj2)->func_ptr; - (*ifobj2)->func_ptr =3D tmp_func_ptr; - - *ifobj1 =3D *ifobj2; - *ifobj2 =3D tmp_ifobj; -} - -static int testapp_bidirectional(struct test_spec *test) -{ - int res; - - test->ifobj_tx->rx_on =3D true; - test->ifobj_rx->tx_on =3D true; - test->total_steps =3D 2; - if (testapp_validate_traffic(test)) - return TEST_FAILURE; - - print_verbose("Switching Tx/Rx direction\n"); - swap_directions(&test->ifobj_rx, &test->ifobj_tx); - res =3D __testapp_validate_traffic(test, test->ifobj_rx, test->ifobj_tx); - - swap_directions(&test->ifobj_rx, &test->ifobj_tx); - return res; -} - -static int swap_xsk_resources(struct test_spec *test) -{ - int ret; - - test->ifobj_tx->xsk_arr[0].pkt_stream =3D NULL; - test->ifobj_rx->xsk_arr[0].pkt_stream =3D NULL; - test->ifobj_tx->xsk_arr[1].pkt_stream =3D test->tx_pkt_stream_default; - test->ifobj_rx->xsk_arr[1].pkt_stream =3D test->rx_pkt_stream_default; - test->ifobj_tx->xsk =3D &test->ifobj_tx->xsk_arr[1]; - test->ifobj_rx->xsk =3D &test->ifobj_rx->xsk_arr[1]; - - ret =3D xsk_update_xskmap(test->ifobj_rx->xskmap, test->ifobj_rx->xsk->xs= k, 0); - if (ret) - return TEST_FAILURE; - - return TEST_PASS; -} - -static int testapp_xdp_prog_cleanup(struct test_spec *test) -{ - test->total_steps =3D 2; - test->nb_sockets =3D 2; - if (testapp_validate_traffic(test)) - return TEST_FAILURE; - - if (swap_xsk_resources(test)) - return TEST_FAILURE; - return testapp_validate_traffic(test); -} - -static int testapp_headroom(struct test_spec *test) -{ - test->ifobj_rx->umem->frame_headroom =3D UMEM_HEADROOM_TEST_SIZE; - return testapp_validate_traffic(test); -} - -static int testapp_stats_rx_dropped(struct test_spec *test) -{ - if (test->mode =3D=3D TEST_MODE_ZC) { - ksft_test_result_skip("Can not run RX_DROPPED test for ZC mode\n"); - return TEST_SKIP; - } - - pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0); - test->ifobj_rx->umem->frame_headroom =3D test->ifobj_rx->umem->frame_size= - - XDP_PACKET_HEADROOM - MIN_PKT_SIZE * 3; - pkt_stream_receive_half(test); - test->ifobj_rx->validation_func =3D validate_rx_dropped; - return testapp_validate_traffic(test); -} - -static int testapp_stats_tx_invalid_descs(struct test_spec *test) -{ - pkt_stream_replace_half(test, XSK_UMEM__INVALID_FRAME_SIZE, 0); - test->ifobj_tx->validation_func =3D validate_tx_invalid_descs; - return testapp_validate_traffic(test); -} - -static int testapp_stats_rx_full(struct test_spec *test) -{ - pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS / 2,= MIN_PKT_SIZE); - test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(DEFAULT_UMEM_BUFF= ERS, MIN_PKT_SIZE); - - test->ifobj_rx->xsk->rxqsize =3D DEFAULT_UMEM_BUFFERS; - test->ifobj_rx->release_rx =3D false; - test->ifobj_rx->validation_func =3D validate_rx_full; - return testapp_validate_traffic(test); -} - -static int testapp_stats_fill_empty(struct test_spec *test) -{ - pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS / 2,= MIN_PKT_SIZE); - test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(DEFAULT_UMEM_BUFF= ERS, MIN_PKT_SIZE); - - test->ifobj_rx->use_fill_ring =3D false; - test->ifobj_rx->validation_func =3D validate_fill_empty; - return testapp_validate_traffic(test); -} - -static int testapp_send_receive_unaligned(struct test_spec *test) -{ - test->ifobj_tx->umem->unaligned_mode =3D true; - test->ifobj_rx->umem->unaligned_mode =3D true; - /* Let half of the packets straddle a 4K buffer boundary */ - pkt_stream_replace_half(test, MIN_PKT_SIZE, -MIN_PKT_SIZE / 2); - - return testapp_validate_traffic(test); -} - -static int testapp_send_receive_unaligned_mb(struct test_spec *test) -{ - test->mtu =3D MAX_ETH_JUMBO_SIZE; - test->ifobj_tx->umem->unaligned_mode =3D true; - test->ifobj_rx->umem->unaligned_mode =3D true; - pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE); - return testapp_validate_traffic(test); -} - -static int testapp_single_pkt(struct test_spec *test) -{ - struct pkt pkts[] =3D {{0, MIN_PKT_SIZE, 0, true}}; - - pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); - return testapp_validate_traffic(test); -} - -static int testapp_send_receive_mb(struct test_spec *test) -{ - test->mtu =3D MAX_ETH_JUMBO_SIZE; - pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE); - - return testapp_validate_traffic(test); -} - -static int testapp_invalid_desc_mb(struct test_spec *test) -{ - struct xsk_umem_info *umem =3D test->ifobj_tx->umem; - u64 umem_size =3D umem->num_frames * umem->frame_size; - struct pkt pkts[] =3D { - /* Valid packet for synch to start with */ - {0, MIN_PKT_SIZE, 0, true, 0}, - /* Zero frame len is not legal */ - {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - {0, 0, 0, false, 0}, - /* Invalid address in the second frame */ - {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - {umem_size, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - /* Invalid len in the middle */ - {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - {0, XSK_UMEM__INVALID_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - /* Invalid options in the middle */ - {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XSK_DESC__INVALID_OPTION}, - /* Transmit 2 frags, receive 3 */ - {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, XDP_PKT_CONTD}, - {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, 0}, - /* Middle frame crosses chunk boundary with small length */ - {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, - {-MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, false, 0}, - /* Valid packet for synch so that something is received */ - {0, MIN_PKT_SIZE, 0, true, 0}}; - - if (umem->unaligned_mode) { - /* Crossing a chunk boundary allowed */ - pkts[12].valid =3D true; - pkts[13].valid =3D true; - } - - test->mtu =3D MAX_ETH_JUMBO_SIZE; - pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); - return testapp_validate_traffic(test); -} - -static int testapp_invalid_desc(struct test_spec *test) -{ - struct xsk_umem_info *umem =3D test->ifobj_tx->umem; - u64 umem_size =3D umem->num_frames * umem->frame_size; - struct pkt pkts[] =3D { - /* Zero packet address allowed */ - {0, MIN_PKT_SIZE, 0, true}, - /* Allowed packet */ - {0, MIN_PKT_SIZE, 0, true}, - /* Straddling the start of umem */ - {-2, MIN_PKT_SIZE, 0, false}, - /* Packet too large */ - {0, XSK_UMEM__INVALID_FRAME_SIZE, 0, false}, - /* Up to end of umem allowed */ - {umem_size - MIN_PKT_SIZE - 2 * umem->frame_size, MIN_PKT_SIZE, 0, true}, - /* After umem ends */ - {umem_size, MIN_PKT_SIZE, 0, false}, - /* Straddle the end of umem */ - {umem_size - MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, false}, - /* Straddle a 4K boundary */ - {0x1000 - MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, false}, - /* Straddle a 2K boundary */ - {0x800 - MIN_PKT_SIZE / 2, MIN_PKT_SIZE, 0, true}, - /* Valid packet for synch so that something is received */ - {0, MIN_PKT_SIZE, 0, true}}; - - if (umem->unaligned_mode) { - /* Crossing a page boundary allowed */ - pkts[7].valid =3D true; - } - if (umem->frame_size =3D=3D XSK_UMEM__DEFAULT_FRAME_SIZE / 2) { - /* Crossing a 2K frame size boundary not allowed */ - pkts[8].valid =3D false; - } - - if (test->ifobj_tx->shared_umem) { - pkts[4].offset +=3D umem_size; - pkts[5].offset +=3D umem_size; - pkts[6].offset +=3D umem_size; - } - - pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); - return testapp_validate_traffic(test); -} - -static int testapp_xdp_drop(struct test_spec *test) -{ - struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; - struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; - - test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_drop, skel_tx->progs.= xsk_xdp_drop, - skel_rx->maps.xsk, skel_tx->maps.xsk); - - pkt_stream_receive_half(test); - return testapp_validate_traffic(test); -} - -static int testapp_xdp_metadata_copy(struct test_spec *test) -{ - struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; - struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; - - test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_populate_metadata, - skel_tx->progs.xsk_xdp_populate_metadata, - skel_rx->maps.xsk, skel_tx->maps.xsk); - test->ifobj_rx->use_metadata =3D true; - - skel_rx->bss->count =3D 0; - - return testapp_validate_traffic(test); -} - -static int testapp_xdp_shared_umem(struct test_spec *test) -{ - struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; - struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; - - test->total_steps =3D 1; - test->nb_sockets =3D 2; - - test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_shared_umem, - skel_tx->progs.xsk_xdp_shared_umem, - skel_rx->maps.xsk, skel_tx->maps.xsk); - - pkt_stream_even_odd_sequence(test); - - return testapp_validate_traffic(test); -} - -static int testapp_poll_txq_tmout(struct test_spec *test) -{ - test->ifobj_tx->use_poll =3D true; - /* create invalid frame by set umem frame_size and pkt length equal to 20= 48 */ - test->ifobj_tx->umem->frame_size =3D 2048; - pkt_stream_replace(test, 2 * DEFAULT_PKT_CNT, 2048); - return testapp_validate_traffic_single_thread(test, test->ifobj_tx); -} - -static int testapp_poll_rxq_tmout(struct test_spec *test) -{ - test->ifobj_rx->use_poll =3D true; - return testapp_validate_traffic_single_thread(test, test->ifobj_rx); -} - -static int testapp_too_many_frags(struct test_spec *test) -{ - struct pkt *pkts; - u32 max_frags, i; - int ret; - - if (test->mode =3D=3D TEST_MODE_ZC) { - max_frags =3D test->ifobj_tx->xdp_zc_max_segs; - } else { - max_frags =3D get_max_skb_frags(); - if (!max_frags) { - ksft_print_msg("Couldn't retrieve MAX_SKB_FRAGS from system, using defa= ult (17) value\n"); - max_frags =3D 17; - } - max_frags +=3D 1; - } - - pkts =3D calloc(2 * max_frags + 2, sizeof(struct pkt)); - if (!pkts) - return TEST_FAILURE; - - test->mtu =3D MAX_ETH_JUMBO_SIZE; - - /* Valid packet for synch */ - pkts[0].len =3D MIN_PKT_SIZE; - pkts[0].valid =3D true; - - /* One valid packet with the max amount of frags */ - for (i =3D 1; i < max_frags + 1; i++) { - pkts[i].len =3D MIN_PKT_SIZE; - pkts[i].options =3D XDP_PKT_CONTD; - pkts[i].valid =3D true; - } - pkts[max_frags].options =3D 0; - - /* An invalid packet with the max amount of frags but signals packet - * continues on the last frag - */ - for (i =3D max_frags + 1; i < 2 * max_frags + 1; i++) { - pkts[i].len =3D MIN_PKT_SIZE; - pkts[i].options =3D XDP_PKT_CONTD; - pkts[i].valid =3D false; - } - - /* Valid packet for synch */ - pkts[2 * max_frags + 1].len =3D MIN_PKT_SIZE; - pkts[2 * max_frags + 1].valid =3D true; - - pkt_stream_generate_custom(test, pkts, 2 * max_frags + 2); - ret =3D testapp_validate_traffic(test); - - free(pkts); - return ret; -} - -static int xsk_load_xdp_programs(struct ifobject *ifobj) -{ - ifobj->xdp_progs =3D xsk_xdp_progs__open_and_load(); - if (libbpf_get_error(ifobj->xdp_progs)) - return libbpf_get_error(ifobj->xdp_progs); - - return 0; -} - -static void xsk_unload_xdp_programs(struct ifobject *ifobj) -{ - xsk_xdp_progs__destroy(ifobj->xdp_progs); -} - -/* Simple test */ -static bool hugepages_present(void) -{ - size_t mmap_sz =3D 2 * DEFAULT_UMEM_BUFFERS * XSK_UMEM__DEFAULT_FRAME_SIZ= E; - void *bufs; - - bufs =3D mmap(NULL, mmap_sz, PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, MAP_HUGE_2MB); - if (bufs =3D=3D MAP_FAILED) - return false; - - mmap_sz =3D ceil_u64(mmap_sz, HUGEPAGE_SIZE) * HUGEPAGE_SIZE; - munmap(bufs, mmap_sz); - return true; -} - -static void init_iface(struct ifobject *ifobj, thread_func_t func_ptr) -{ - LIBBPF_OPTS(bpf_xdp_query_opts, query_opts); - int err; - - ifobj->func_ptr =3D func_ptr; - - err =3D xsk_load_xdp_programs(ifobj); - if (err) { - ksft_print_msg("Error loading XDP program\n"); - exit_with_error(err); - } - - if (hugepages_present()) - ifobj->unaligned_supp =3D true; - - err =3D bpf_xdp_query(ifobj->ifindex, XDP_FLAGS_DRV_MODE, &query_opts); - if (err) { - ksft_print_msg("Error querying XDP capabilities\n"); - exit_with_error(-err); - } - if (query_opts.feature_flags & NETDEV_XDP_ACT_RX_SG) - ifobj->multi_buff_supp =3D true; - if (query_opts.feature_flags & NETDEV_XDP_ACT_XSK_ZEROCOPY) { - if (query_opts.xdp_zc_max_segs > 1) { - ifobj->multi_buff_zc_supp =3D true; - ifobj->xdp_zc_max_segs =3D query_opts.xdp_zc_max_segs; - } else { - ifobj->xdp_zc_max_segs =3D 0; - } - } -} - -static int testapp_send_receive(struct test_spec *test) -{ - return testapp_validate_traffic(test); -} - -static int testapp_send_receive_2k_frame(struct test_spec *test) -{ - test->ifobj_tx->umem->frame_size =3D 2048; - test->ifobj_rx->umem->frame_size =3D 2048; - pkt_stream_replace(test, DEFAULT_PKT_CNT, MIN_PKT_SIZE); - return testapp_validate_traffic(test); -} - -static int testapp_poll_rx(struct test_spec *test) -{ - test->ifobj_rx->use_poll =3D true; - return testapp_validate_traffic(test); -} - -static int testapp_poll_tx(struct test_spec *test) -{ - test->ifobj_tx->use_poll =3D true; - return testapp_validate_traffic(test); -} - -static int testapp_aligned_inv_desc(struct test_spec *test) -{ - return testapp_invalid_desc(test); -} - -static int testapp_aligned_inv_desc_2k_frame(struct test_spec *test) -{ - test->ifobj_tx->umem->frame_size =3D 2048; - test->ifobj_rx->umem->frame_size =3D 2048; - return testapp_invalid_desc(test); -} - -static int testapp_unaligned_inv_desc(struct test_spec *test) -{ - test->ifobj_tx->umem->unaligned_mode =3D true; - test->ifobj_rx->umem->unaligned_mode =3D true; - return testapp_invalid_desc(test); -} - -static int testapp_unaligned_inv_desc_4001_frame(struct test_spec *test) -{ - u64 page_size, umem_size; - - /* Odd frame size so the UMEM doesn't end near a page boundary. */ - test->ifobj_tx->umem->frame_size =3D 4001; - test->ifobj_rx->umem->frame_size =3D 4001; - test->ifobj_tx->umem->unaligned_mode =3D true; - test->ifobj_rx->umem->unaligned_mode =3D true; - /* This test exists to test descriptors that staddle the end of - * the UMEM but not a page. - */ - page_size =3D sysconf(_SC_PAGESIZE); - umem_size =3D test->ifobj_tx->umem->num_frames * test->ifobj_tx->umem->fr= ame_size; - assert(umem_size % page_size > MIN_PKT_SIZE); - assert(umem_size % page_size < page_size - MIN_PKT_SIZE); - - return testapp_invalid_desc(test); -} - -static int testapp_aligned_inv_desc_mb(struct test_spec *test) -{ - return testapp_invalid_desc_mb(test); -} - -static int testapp_unaligned_inv_desc_mb(struct test_spec *test) -{ - test->ifobj_tx->umem->unaligned_mode =3D true; - test->ifobj_rx->umem->unaligned_mode =3D true; - return testapp_invalid_desc_mb(test); -} - -static int testapp_xdp_metadata(struct test_spec *test) -{ - return testapp_xdp_metadata_copy(test); -} - -static int testapp_xdp_metadata_mb(struct test_spec *test) -{ - test->mtu =3D MAX_ETH_JUMBO_SIZE; - return testapp_xdp_metadata_copy(test); -} - -static int testapp_hw_sw_min_ring_size(struct test_spec *test) -{ - int ret; - - test->set_ring =3D true; - test->total_steps =3D 2; - test->ifobj_tx->ring.tx_pending =3D DEFAULT_BATCH_SIZE; - test->ifobj_tx->ring.rx_pending =3D DEFAULT_BATCH_SIZE * 2; - test->ifobj_tx->xsk->batch_size =3D 1; - test->ifobj_rx->xsk->batch_size =3D 1; - ret =3D testapp_validate_traffic(test); - if (ret) - return ret; - - /* Set batch size to hw_ring_size - 1 */ - test->ifobj_tx->xsk->batch_size =3D DEFAULT_BATCH_SIZE - 1; - test->ifobj_rx->xsk->batch_size =3D DEFAULT_BATCH_SIZE - 1; - return testapp_validate_traffic(test); -} - -static int testapp_hw_sw_max_ring_size(struct test_spec *test) -{ - u32 max_descs =3D XSK_RING_PROD__DEFAULT_NUM_DESCS * 4; - int ret; - - test->set_ring =3D true; - test->total_steps =3D 2; - test->ifobj_tx->ring.tx_pending =3D test->ifobj_tx->ring.tx_max_pending; - test->ifobj_tx->ring.rx_pending =3D test->ifobj_tx->ring.rx_max_pending; - test->ifobj_rx->umem->num_frames =3D max_descs; - test->ifobj_rx->umem->fill_size =3D max_descs; - test->ifobj_rx->umem->comp_size =3D max_descs; - test->ifobj_tx->xsk->batch_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS; - test->ifobj_rx->xsk->batch_size =3D XSK_RING_PROD__DEFAULT_NUM_DESCS; - - ret =3D testapp_validate_traffic(test); - if (ret) - return ret; - - /* Set batch_size to 8152 for testing, as the ice HW ignores the 3 lowest= bits when - * updating the Rx HW tail register. - */ - test->ifobj_tx->xsk->batch_size =3D test->ifobj_tx->ring.tx_max_pending -= 8; - test->ifobj_rx->xsk->batch_size =3D test->ifobj_tx->ring.tx_max_pending -= 8; - pkt_stream_replace(test, max_descs, MIN_PKT_SIZE); - return testapp_validate_traffic(test); -} - -static int testapp_xdp_adjust_tail(struct test_spec *test, int adjust_valu= e) -{ - struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; - struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; - - test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_adjust_tail, - skel_tx->progs.xsk_xdp_adjust_tail, - skel_rx->maps.xsk, skel_tx->maps.xsk); - - skel_rx->bss->adjust_value =3D adjust_value; - - return testapp_validate_traffic(test); -} - -static int testapp_adjust_tail(struct test_spec *test, u32 value, u32 pkt_= len) -{ - int ret; - - test->adjust_tail_support =3D true; - test->adjust_tail =3D true; - test->total_steps =3D 1; - - pkt_stream_replace_ifobject(test->ifobj_tx, DEFAULT_BATCH_SIZE, pkt_len); - pkt_stream_replace_ifobject(test->ifobj_rx, DEFAULT_BATCH_SIZE, pkt_len += value); - - ret =3D testapp_xdp_adjust_tail(test, value); - if (ret) - return ret; - - if (!test->adjust_tail_support) { - ksft_test_result_skip("%s %sResize pkt with bpf_xdp_adjust_tail() not su= pported\n", - mode_string(test), busy_poll_string(test)); - return TEST_SKIP; - } - - return 0; -} - -static int testapp_adjust_tail_shrink(struct test_spec *test) -{ - /* Shrink by 4 bytes for testing purpose */ - return testapp_adjust_tail(test, -4, MIN_PKT_SIZE * 2); -} - -static int testapp_adjust_tail_shrink_mb(struct test_spec *test) -{ - test->mtu =3D MAX_ETH_JUMBO_SIZE; - /* Shrink by the frag size */ - return testapp_adjust_tail(test, -XSK_UMEM__MAX_FRAME_SIZE, XSK_UMEM__LAR= GE_FRAME_SIZE * 2); -} - -static int testapp_adjust_tail_grow(struct test_spec *test) -{ - /* Grow by 4 bytes for testing purpose */ - return testapp_adjust_tail(test, 4, MIN_PKT_SIZE * 2); -} - -static int testapp_adjust_tail_grow_mb(struct test_spec *test) -{ - test->mtu =3D MAX_ETH_JUMBO_SIZE; - /* Grow by (frag_size - last_frag_Size) - 1 to stay inside the last fragm= ent */ - return testapp_adjust_tail(test, (XSK_UMEM__MAX_FRAME_SIZE / 2) - 1, - XSK_UMEM__LARGE_FRAME_SIZE * 2); -} - -static int testapp_tx_queue_consumer(struct test_spec *test) -{ - int nr_packets; - - if (test->mode =3D=3D TEST_MODE_ZC) { - ksft_test_result_skip("Can not run TX_QUEUE_CONSUMER test for ZC mode\n"= ); - return TEST_SKIP; - } - - nr_packets =3D MAX_TX_BUDGET_DEFAULT + 1; - pkt_stream_replace(test, nr_packets, MIN_PKT_SIZE); - test->ifobj_tx->xsk->batch_size =3D nr_packets; - test->ifobj_tx->xsk->check_consumer =3D true; - - return testapp_validate_traffic(test); -} - -static void run_pkt_test(struct test_spec *test) -{ - int ret; - - ret =3D test->test_func(test); - - if (ret =3D=3D TEST_PASS) - ksft_test_result_pass("PASS: %s %s%s\n", mode_string(test), busy_poll_st= ring(test), - test->name); - pkt_stream_restore_default(test); -} - -static struct ifobject *ifobject_create(void) -{ - struct ifobject *ifobj; - - ifobj =3D calloc(1, sizeof(struct ifobject)); - if (!ifobj) - return NULL; - - ifobj->xsk_arr =3D calloc(MAX_SOCKETS, sizeof(*ifobj->xsk_arr)); - if (!ifobj->xsk_arr) - goto out_xsk_arr; - - ifobj->umem =3D calloc(1, sizeof(*ifobj->umem)); - if (!ifobj->umem) - goto out_umem; - - return ifobj; - -out_umem: - free(ifobj->xsk_arr); -out_xsk_arr: - free(ifobj); - return NULL; -} - -static void ifobject_delete(struct ifobject *ifobj) -{ - free(ifobj->umem); - free(ifobj->xsk_arr); - free(ifobj); -} - -static bool is_xdp_supported(int ifindex) -{ - int flags =3D XDP_FLAGS_DRV_MODE; - - LIBBPF_OPTS(bpf_link_create_opts, opts, .flags =3D flags); - struct bpf_insn insns[2] =3D { - BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), - BPF_EXIT_INSN() - }; - int prog_fd, insn_cnt =3D ARRAY_SIZE(insns); - int err; - - prog_fd =3D bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt= , NULL); - if (prog_fd < 0) - return false; - - err =3D bpf_xdp_attach(ifindex, prog_fd, flags, NULL); - if (err) { - close(prog_fd); - return false; - } - - bpf_xdp_detach(ifindex, flags, NULL); - close(prog_fd); - - return true; -} - -static const struct test_spec tests[] =3D { - {.name =3D "SEND_RECEIVE", .test_func =3D testapp_send_receive}, - {.name =3D "SEND_RECEIVE_2K_FRAME", .test_func =3D testapp_send_receive_2= k_frame}, - {.name =3D "SEND_RECEIVE_SINGLE_PKT", .test_func =3D testapp_single_pkt}, - {.name =3D "POLL_RX", .test_func =3D testapp_poll_rx}, - {.name =3D "POLL_TX", .test_func =3D testapp_poll_tx}, - {.name =3D "POLL_RXQ_FULL", .test_func =3D testapp_poll_rxq_tmout}, - {.name =3D "POLL_TXQ_FULL", .test_func =3D testapp_poll_txq_tmout}, - {.name =3D "SEND_RECEIVE_UNALIGNED", .test_func =3D testapp_send_receive_= unaligned}, - {.name =3D "ALIGNED_INV_DESC", .test_func =3D testapp_aligned_inv_desc}, - {.name =3D "ALIGNED_INV_DESC_2K_FRAME_SIZE", .test_func =3D testapp_align= ed_inv_desc_2k_frame}, - {.name =3D "UNALIGNED_INV_DESC", .test_func =3D testapp_unaligned_inv_des= c}, - {.name =3D "UNALIGNED_INV_DESC_4001_FRAME_SIZE", - .test_func =3D testapp_unaligned_inv_desc_4001_frame}, - {.name =3D "UMEM_HEADROOM", .test_func =3D testapp_headroom}, - {.name =3D "TEARDOWN", .test_func =3D testapp_teardown}, - {.name =3D "BIDIRECTIONAL", .test_func =3D testapp_bidirectional}, - {.name =3D "STAT_RX_DROPPED", .test_func =3D testapp_stats_rx_dropped}, - {.name =3D "STAT_TX_INVALID", .test_func =3D testapp_stats_tx_invalid_des= cs}, - {.name =3D "STAT_RX_FULL", .test_func =3D testapp_stats_rx_full}, - {.name =3D "STAT_FILL_EMPTY", .test_func =3D testapp_stats_fill_empty}, - {.name =3D "XDP_PROG_CLEANUP", .test_func =3D testapp_xdp_prog_cleanup}, - {.name =3D "XDP_DROP_HALF", .test_func =3D testapp_xdp_drop}, - {.name =3D "XDP_SHARED_UMEM", .test_func =3D testapp_xdp_shared_umem}, - {.name =3D "XDP_METADATA_COPY", .test_func =3D testapp_xdp_metadata}, - {.name =3D "XDP_METADATA_COPY_MULTI_BUFF", .test_func =3D testapp_xdp_met= adata_mb}, - {.name =3D "SEND_RECEIVE_9K_PACKETS", .test_func =3D testapp_send_receive= _mb}, - {.name =3D "SEND_RECEIVE_UNALIGNED_9K_PACKETS", - .test_func =3D testapp_send_receive_unaligned_mb}, - {.name =3D "ALIGNED_INV_DESC_MULTI_BUFF", .test_func =3D testapp_aligned_= inv_desc_mb}, - {.name =3D "UNALIGNED_INV_DESC_MULTI_BUFF", .test_func =3D testapp_unalig= ned_inv_desc_mb}, - {.name =3D "TOO_MANY_FRAGS", .test_func =3D testapp_too_many_frags}, - {.name =3D "HW_SW_MIN_RING_SIZE", .test_func =3D testapp_hw_sw_min_ring_s= ize}, - {.name =3D "HW_SW_MAX_RING_SIZE", .test_func =3D testapp_hw_sw_max_ring_s= ize}, - {.name =3D "XDP_ADJUST_TAIL_SHRINK", .test_func =3D testapp_adjust_tail_s= hrink}, - {.name =3D "XDP_ADJUST_TAIL_SHRINK_MULTI_BUFF", .test_func =3D testapp_ad= just_tail_shrink_mb}, - {.name =3D "XDP_ADJUST_TAIL_GROW", .test_func =3D testapp_adjust_tail_gro= w}, - {.name =3D "XDP_ADJUST_TAIL_GROW_MULTI_BUFF", .test_func =3D testapp_adju= st_tail_grow_mb}, - {.name =3D "TX_QUEUE_CONSUMER", .test_func =3D testapp_tx_queue_consumer}, - }; - static void print_tests(void) { u32 i; @@ -2833,7 +376,7 @@ int main(int argc, char **argv) init_iface(ifobj_rx, worker_testapp_validate_rx); init_iface(ifobj_tx, worker_testapp_validate_tx); =20 - test_spec_init(&test, ifobj_tx, ifobj_rx, 0, &tests[0]); + test_init(&test, ifobj_tx, ifobj_rx, 0, &tests[0]); tx_pkt_stream_default =3D pkt_stream_generate(DEFAULT_PKT_CNT, MIN_PKT_SI= ZE); rx_pkt_stream_default =3D pkt_stream_generate(DEFAULT_PKT_CNT, MIN_PKT_SI= ZE); if (!tx_pkt_stream_default || !rx_pkt_stream_default) @@ -2868,7 +411,7 @@ int main(int argc, char **argv) if (opt_run_test !=3D RUN_ALL_TESTS && j !=3D opt_run_test) continue; =20 - test_spec_init(&test, ifobj_tx, ifobj_rx, i, &tests[j]); + test_init(&test, ifobj_tx, ifobj_rx, i, &tests[j]); run_pkt_test(&test); usleep(USLEEP_MAX); =20 diff --git a/tools/testing/selftests/bpf/xskxceiver.h b/tools/testing/selft= ests/bpf/xskxceiver.h index 4df3a5d329acf80bc8f83ea4a0f776196f2dd0ab..3ca518df23adaf44fe460f627b4= 5cb018e9c4b4a 100644 --- a/tools/testing/selftests/bpf/xskxceiver.h +++ b/tools/testing/selftests/bpf/xskxceiver.h @@ -22,169 +22,13 @@ #define PF_XDP AF_XDP #endif =20 -#ifndef SO_BUSY_POLL_BUDGET -#define SO_BUSY_POLL_BUDGET 70 -#endif - -#ifndef SO_PREFER_BUSY_POLL -#define SO_PREFER_BUSY_POLL 69 -#endif - -#define TEST_PASS 0 -#define TEST_FAILURE -1 -#define TEST_CONTINUE 1 -#define TEST_SKIP 2 -#define MAX_INTERFACES 2 -#define MAX_INTERFACE_NAME_CHARS 16 -#define MAX_TEST_NAME_SIZE 48 #define MAX_TEARDOWN_ITER 10 -#define PKT_HDR_SIZE (sizeof(struct ethhdr) + 2) /* Just to align the data= in the packet */ -#define MIN_PKT_SIZE 64 -#define MAX_ETH_PKT_SIZE 1518 #define MAX_ETH_JUMBO_SIZE 9000 -#define USLEEP_MAX 10000 #define SOCK_RECONF_CTR 10 -#define DEFAULT_BATCH_SIZE 64 -#define POLL_TMOUT 1000 -#define THREAD_TMOUT 3 -#define DEFAULT_PKT_CNT (4 * 1024) -#define DEFAULT_UMEM_BUFFERS (DEFAULT_PKT_CNT / 4) #define RX_FULL_RXQSIZE 32 #define UMEM_HEADROOM_TEST_SIZE 128 #define XSK_UMEM__INVALID_FRAME_SIZE (MAX_ETH_JUMBO_SIZE + 1) -#define XSK_UMEM__LARGE_FRAME_SIZE (3 * 1024) -#define XSK_UMEM__MAX_FRAME_SIZE (4 * 1024) -#define XSK_DESC__INVALID_OPTION (0xffff) -#define HUGEPAGE_SIZE (2 * 1024 * 1024) -#define PKT_DUMP_NB_TO_PRINT 16 #define RUN_ALL_TESTS UINT_MAX #define NUM_MAC_ADDRESSES 4 =20 -#define print_verbose(x...) do { if (opt_verbose) ksft_print_msg(x); } whi= le (0) - -enum test_mode { - TEST_MODE_SKB, - TEST_MODE_DRV, - TEST_MODE_ZC, - TEST_MODE_ALL -}; - -struct xsk_umem_info { - struct xsk_ring_prod fq; - struct xsk_ring_cons cq; - struct xsk_umem *umem; - u64 next_buffer; - u32 num_frames; - u32 frame_headroom; - void *buffer; - u32 frame_size; - u32 base_addr; - u32 fill_size; - u32 comp_size; - bool unaligned_mode; -}; - -struct xsk_socket_info { - struct xsk_ring_cons rx; - struct xsk_ring_prod tx; - struct xsk_umem_info *umem; - struct xsk_socket *xsk; - struct pkt_stream *pkt_stream; - u32 outstanding_tx; - u32 rxqsize; - u32 batch_size; - u8 dst_mac[ETH_ALEN]; - u8 src_mac[ETH_ALEN]; - bool check_consumer; -}; - -struct pkt { - int offset; - u32 len; - u32 pkt_nb; - bool valid; - u16 options; -}; - -struct pkt_stream { - u32 nb_pkts; - u32 current_pkt_nb; - struct pkt *pkts; - u32 max_pkt_len; - u32 nb_rx_pkts; - u32 nb_valid_entries; - bool verbatim; -}; - -struct set_hw_ring { - u32 default_tx; - u32 default_rx; -}; - -struct ifobject; -struct test_spec; -typedef int (*validation_func_t)(struct ifobject *ifobj); -typedef void *(*thread_func_t)(void *arg); -typedef int (*test_func_t)(struct test_spec *test); - -struct ifobject { - char ifname[MAX_INTERFACE_NAME_CHARS]; - struct xsk_socket_info *xsk; - struct xsk_socket_info *xsk_arr; - struct xsk_umem_info *umem; - thread_func_t func_ptr; - validation_func_t validation_func; - struct xsk_xdp_progs *xdp_progs; - struct bpf_map *xskmap; - struct bpf_program *xdp_prog; - struct ethtool_ringparam ring; - struct set_hw_ring set_ring; - enum test_mode mode; - int ifindex; - int mtu; - u32 bind_flags; - u32 xdp_zc_max_segs; - bool tx_on; - bool rx_on; - bool use_poll; - bool busy_poll; - bool use_fill_ring; - bool release_rx; - bool shared_umem; - bool use_metadata; - bool unaligned_supp; - bool multi_buff_supp; - bool multi_buff_zc_supp; - bool hw_ring_size_supp; -}; - -struct test_spec { - struct ifobject *ifobj_tx; - struct ifobject *ifobj_rx; - struct pkt_stream *tx_pkt_stream_default; - struct pkt_stream *rx_pkt_stream_default; - struct bpf_program *xdp_prog_rx; - struct bpf_program *xdp_prog_tx; - struct bpf_map *xskmap_rx; - struct bpf_map *xskmap_tx; - test_func_t test_func; - int mtu; - u16 total_steps; - u16 current_step; - u16 nb_sockets; - bool fail; - bool set_ring; - bool adjust_tail; - bool adjust_tail_support; - enum test_mode mode; - char name[MAX_TEST_NAME_SIZE]; -}; - -pthread_barrier_t barr; -pthread_mutex_t pacing_mutex =3D PTHREAD_MUTEX_INITIALIZER; - -int pkts_in_flight; - -static const u8 g_mac[ETH_ALEN] =3D {0x55, 0x44, 0x33, 0x22, 0x11, 0x00}; - #endif /* XSKXCEIVER_H_ */ --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B95230AAC2; Wed, 24 Sep 2025 14:50:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.84.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725409; cv=none; b=qXDnb4s9MQZXYq15/X4RNn/ixmQaJyG91MGedZlgCC2HKgVMmh7/KFz6p3kIbbraIfKM67+NE8ADsFXOQ1JbTKBmFzCmjAOkhLJW1JrdA+Nt0OX38tj3ToH9rwYWCCz01kKsoApbeN3cMCZ8bxuh25xbVDqh8BhHMfdHigIcqR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725409; c=relaxed/simple; bh=2nZ+xS75+GYRbPbEr1s2AotVB+FzRxeAkPwF05aNpcg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UKMVhgqjST4i1ltD93xEsr37OuUkQWPl4aFvynOYhOTtqk3ZN0oCFwW2brNdkw2VlHhNSQegCX+6eeBSk0daY23JjWdFacVbpTP7SvF6TajSOKwjNtrHegrXhewAGamY5/yJe3s+/aJ+uyvNCSIGAPEy36IBRpykAK7zlERZnts= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=qqhpxicJ; arc=none smtp.client-ip=185.246.84.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="qqhpxicJ" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-02.galae.net (Postfix) with ESMTPS id B00E01A0F84; Wed, 24 Sep 2025 14:50:05 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 84DA560634; Wed, 24 Sep 2025 14:50:05 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 3FA22102F18D8; Wed, 24 Sep 2025 16:50:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725404; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=3RHLVmk5+u09dFacYEQB1nNTSpeaQW6IgPoQKUHTSJA=; b=qqhpxicJ3nT6tP8vOjipWOKU9C6vNsKRfUJfZf7w5wdRiQYVkFR+8ap2g2hJsp6hq1zPvy +Hq8VVkEXVgGue85JcdaLgo7dLMy4s6r/Pq+DO5xR/KGvhfozEBZjwL27OgZeXrRYiTGW4 TThSZAepZ0w4o5uNCqh7aVC4kPYHSq9UoSPt5zWVmiLIOqZ2x5TGlgYM92vBP3YL2BM9uX TktR5cjDV9ABu0LfV4gv23UGgIEevWOLXjrB8vMFQsO5PqGByYxg84sGxyaAzpdMbuBlL3 4TZH5zQG7W7ZJsQl7+hEjjAoyB9dI82A7LkTBsIu4hGVMyrQ7VPCQfp6oje06w== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:37 +0200 Subject: [PATCH bpf-next v4 02/15] selftests/bpf: test_xsk: Initialize bitmap before use Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-2-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 bitmap is used before being initialized. Initialize it to zero before using it. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 02250f29f9946d1ca701e30a900617056d91587b..679491b6b9dd80ccb2b92729141= fb8715b874c6d 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -1069,6 +1069,8 @@ static int receive_pkts(struct test_spec *test) u32 sock_num =3D 0; int res, ret; =20 + bitmap_zero(bitmap, test->nb_sockets); + ret =3D gettimeofday(&tv_now, NULL); if (ret) exit_with_error(errno); @@ -1259,6 +1261,8 @@ static int send_pkts(struct test_spec *test, struct i= fobject *ifobject) DECLARE_BITMAP(bitmap, test->nb_sockets); u32 i, ret; =20 + bitmap_zero(bitmap, test->nb_sockets); + while (!(all_packets_sent(test, bitmap))) { for (i =3D 0; i < test->nb_sockets; i++) { struct pkt_stream *pkt_stream; --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D1C230B534 for ; Wed, 24 Sep 2025 14:50:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.84.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725415; cv=none; b=ANVQdDmACBAuGrW0SF2kzz2iewWz7lIHtfr9k/VVBFy3XWVY/iLnVz38M95Lt5f+gXmE80tThpCON9udx/iGdP0IXJJKxdDaQNFMvWJ072dl9NrkLYyuC743RctMptyy7chx+E4J+L7wsZWdSrd/esRC75Sc68AYbpjYBHJtLrw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725415; c=relaxed/simple; bh=QntO1gbQ2O/QD7BJzXedroNY+5AyUr6I8y1Tc2DnWpc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TSuAbapA3O4OQNSrcZgAxI8DM0kh8m3JrfdaPTbpipn9LSSr7mmBzDrrMAdYBqahpb4Cx8lSOptzuvZ4JhCR+o0EGg7q2Pm8rHgz7d7vh1Npl4IQBiDsEJw6T69y2e7sX5FCPjx2N1uIRjfZHAUN/yp8krTemLJ+s8xtWRXvLsE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=F6D5gP4Z; arc=none smtp.client-ip=185.246.84.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="F6D5gP4Z" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-02.galae.net (Postfix) with ESMTPS id 723FA1A0F85; Wed, 24 Sep 2025 14:50:08 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 41D2160634; Wed, 24 Sep 2025 14:50:08 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id B6D02102F1916; Wed, 24 Sep 2025 16:50:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725407; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=r25cJKSJ2aGWddtaZ7/M8QgHMa9Wnxi3zbsK79tDlrA=; b=F6D5gP4Z8R00x6PcLAz9BBR7I2C6QEfXHApj9pUygMf9srO1z/DIzUZURM58nF/CBuY9LK jaroQXixgh9qyNAteeaGKG1gPr4tNPVsQloLZYH9pcxf7igGyQDRJ6uYIdMKQN2bRLlxEp d7bldqdtVWG9hZKkWVuDlyyKZxXKex6OJnVy9HsLEJOQfZrpcu9n8v/YVPJ433KcL6/rwD gQSjsMxnhSUBK/PTQzKMVDtx2Wu9TwiGpBAtF0X9cW+snHdxoutwoeNibXrpC6k6x9FdYZ mWHt2dB9/7xLAfkUJiiVtDOOlzvlW31jBtwoSC4uk3LZfnDH+STthZ9UQhdKhg== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:38 +0200 Subject: [PATCH bpf-next v4 03/15] selftests/bpf: test_xsk: Fix __testapp_validate_traffic()'s return value Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-3-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 __testapp_validate_traffic is supposed to return an integer value that tells if the test passed (0), failed (-1) or was skiped (2). It actually returns a boolean in the end. This doesn't harm when the test is successful but can lead to misinterpretation in case of failure as 1 will be returned instead of -1. Return TEST_FAILURE (-1) in case of failure, TEST_PASS (0) otherwise. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 679491b6b9dd80ccb2b92729141fb8715b874c6d..8d7c38eb32ca3537cb019f120c3= 350ebd9f8c6bc 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -1725,7 +1725,10 @@ static int __testapp_validate_traffic(struct test_sp= ec *test, struct ifobject *i testapp_clean_xsk_umem(ifobj2); } =20 - return !!test->fail; + if (test->fail) + return TEST_FAILURE; + + return TEST_PASS; } =20 static int testapp_validate_traffic(struct test_spec *test) --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86EAD30C113; Wed, 24 Sep 2025 14:50:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.84.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725415; cv=none; b=jQ/N2GN97R4/06OGHDSC3R1hAXgo1xZGSWyeSsSHrQNIAeLvkXwAG1TNu5KFwdF7IWJRVuXtuKMyINAP/olid7PxcHMz9DyvWGiim7Kc68HWZDVgYn45Y1czkdxwJ1Yw6r2s3eGNIXDVEjz9jEhIwSXCweFgoHFOT3J3MF5vwvI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725415; c=relaxed/simple; bh=TZieB9lYTRDug/RT2WE0zKfhPR3/EVFWH3rMKlquW68=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mF1bPJ/F5gGRp5nPDnvNbtcw2Z33DYEiU9l6SkaEKmufozYrTRRlh+gum7WfsAUdv+//rn4HkMtyEDjsKXUNlbN5pPd10qrAYB5JroJPQylux6MrItvGMFYdt8UT1Vm+LLFP/lUwPE+Zante03iB38g0E4kS2Ho/GukaNctijtA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=N5TsXfjp; arc=none smtp.client-ip=185.246.84.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="N5TsXfjp" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-02.galae.net (Postfix) with ESMTPS id 05EA11A0F81; Wed, 24 Sep 2025 14:50:11 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id CA66960634; Wed, 24 Sep 2025 14:50:10 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 60B1F102F1973; Wed, 24 Sep 2025 16:50:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725409; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=jkYcjm583dXSVSeZq1srRFe8I29KOIeE29N37oNBMvk=; b=N5TsXfjpr94309e0xIvefosKxSx+FDUVRJiG0GUIaSOflG9c9CqXR81vBYy0UK2O1VLRVo +zt2yudqC43Cuz2tSo0w+bgPrbLcnSNYfVS2fm/5ZD6RXLgBPPJhyVB1UHzDx50Z5cc4hC Uw/cd+Wwvu+Xd1qLSZlh0b2DGnx79A/R+VY+qMuLN834WjcaRh8rykRHs8Ybdv+SfVP5Ts RvvwXO4Jay/NVP8XlJHeZODdy1/J1a2m59Ksb1eC7QP3FCcFh33T4i0CM6HOnSMY1cwbA7 7RIRcF5L5tPuco8e801tKasorc2p9CwUxXRzbb7SiS3wbg88wsQoNqvVmByUwg== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:39 +0200 Subject: [PATCH bpf-next v4 04/15] selftests/bpf: test_xsk: fix memory leak in testapp_stats_rx_dropped() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-4-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 testapp_stats_rx_dropped() generates pkt_stream twice. The last generated is released by pkt_stream_restore_default() at the end of the test but we lose the pointer of the first pkt_stream. Release the 'middle' pkt_stream when it's getting replaced to prevent memory leaks. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 8d7c38eb32ca3537cb019f120c3350ebd9f8c6bc..eb18288ea1e4aa1c9337d16333b= 7174ecaed0999 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -536,6 +536,13 @@ static void pkt_stream_receive_half(struct test_spec *= test) struct pkt_stream *pkt_stream =3D test->ifobj_tx->xsk->pkt_stream; u32 i; =20 + if (test->ifobj_rx->xsk->pkt_stream !=3D test->rx_pkt_stream_default) + /* Packet stream has already been replaced so we have to release this on= e. + * The newly created one will be freed by the restore_default() at the + * end of the test + */ + pkt_stream_delete(test->ifobj_rx->xsk->pkt_stream); + test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(pkt_stream->nb_pk= ts, pkt_stream->pkts[0].len); pkt_stream =3D test->ifobj_rx->xsk->pkt_stream; --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-03.galae.net (smtpout-03.galae.net [185.246.85.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8097730DD34; Wed, 24 Sep 2025 14:50:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.85.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725419; cv=none; b=bQh5xxml4va2BX73BvPKN7RjEb3GZWXlADi+pxn3WqpqK3cwJoc5rzrKcjWoEzCUL5z7QnJGvAgWpq5mWeEukeWoYHc5HoKSMYSKZrMBri3jDsTtzscF7b7V36BNcE/UlnLOBIhfQhQ1pPPWZFA4ad52NiQobpm+PdQsxeX920w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725419; c=relaxed/simple; bh=p5/Ue/EDWrCLygK9OUD9Vn8dsndfDHqeLNBvRGglxCk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cwQEeTtAVD7vugf6PFlVyIW59isuSo/y5wM3ImJrnWmLsnuv2rjN5iVuTVmala6rtE2Oq2RRWehWzk5hzYKOnfx/gDNDn4MQ/zEN3pMpVFBmivhtxT1ScvKmpNROxbf4Sp1pV5M1aPTOOMuyFsjQjgdUZ8HaLTSEGdCA9dS9GGk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=XNg8TocZ; arc=none smtp.client-ip=185.246.85.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="XNg8TocZ" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-03.galae.net (Postfix) with ESMTPS id C4D004E40D58; Wed, 24 Sep 2025 14:50:13 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 9A75860634; Wed, 24 Sep 2025 14:50:13 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id EAE84102F18D8; Wed, 24 Sep 2025 16:50:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725412; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=ez1VzhqefwOv/TLGGLfSuvB3WSc3tsfGzKqORDO0iCU=; b=XNg8TocZoYR9DFgkjwpAElOvJVPmjc7W/W1zUUMJ2EVgqJrDlSWIpXHoxPn+WBeGeCrAAW On8bhnSmDzwe28+vCD2iaaBDVV3y81EQ+W/awfc+Nl+Aav8hODfllJWgRIvv9dqJ+i4ecI i92iHcChbaqHyRSvaLl9/zw3M8ymJ08olWkJt9JNOuayiHdsVryDF1txplfVflzar/U/6r xbhKQ2SOX9AJnTScTaNp/yExhJ5ex6R2NiYTIvywrkSs5eggFMudnQ61fn3T0RgKwhwNUo 6agJRT9MY9sLVILNGGbxLUtF8tQHIGPvwK9KcZOY62yrSZ1YM7fihddAu2PViw== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:40 +0200 Subject: [PATCH bpf-next v4 05/15] selftests/bpf: test_xsk: fix memory leak in testapp_xdp_shared_umem() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-5-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 testapp_xdp_shared_umem() generates pkt_stream on each xsk from xsk_arr, where normally xsk_arr[0] gets pkt_streams and xsk_arr[1] have them NULLed. At the end of the test pkt_stream_restore_default() only releases xsk_arr[0] which leads to memory leaks. Release the missing pkt_stream at the end of testapp_xdp_shared_umem() Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index eb18288ea1e4aa1c9337d16333b7174ecaed0999..d7cb2821469c62abd0d532821e8= 36336a2177eb5 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -570,6 +570,22 @@ static void pkt_stream_even_odd_sequence(struct test_s= pec *test) } } =20 +static void release_even_odd_sequence(struct test_spec *test) +{ + struct pkt_stream *later_free_tx =3D test->ifobj_tx->xsk->pkt_stream; + struct pkt_stream *later_free_rx =3D test->ifobj_rx->xsk->pkt_stream; + int i; + + for (i =3D 0; i < test->nb_sockets; i++) { + /* later_free_{rx/tx} will be freed by restore_default() */ + if (test->ifobj_tx->xsk_arr[i].pkt_stream !=3D later_free_tx) + pkt_stream_delete(test->ifobj_tx->xsk_arr[i].pkt_stream); + if (test->ifobj_rx->xsk_arr[i].pkt_stream !=3D later_free_rx) + pkt_stream_delete(test->ifobj_rx->xsk_arr[i].pkt_stream); + } + +} + static u64 pkt_get_addr(struct pkt *pkt, struct xsk_umem_info *umem) { if (!pkt->valid) @@ -2043,6 +2059,7 @@ int testapp_xdp_shared_umem(struct test_spec *test) { struct xsk_xdp_progs *skel_rx =3D test->ifobj_rx->xdp_progs; struct xsk_xdp_progs *skel_tx =3D test->ifobj_tx->xdp_progs; + int ret; =20 test->total_steps =3D 1; test->nb_sockets =3D 2; @@ -2053,7 +2070,11 @@ int testapp_xdp_shared_umem(struct test_spec *test) =20 pkt_stream_even_odd_sequence(test); =20 - return testapp_validate_traffic(test); + ret =3D testapp_validate_traffic(test); + + release_even_odd_sequence(test); + + return ret; } =20 int testapp_poll_txq_tmout(struct test_spec *test) --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-04.galae.net (smtpout-04.galae.net [185.171.202.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A00EF30DEAE; Wed, 24 Sep 2025 14:50:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.171.202.116 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725419; cv=none; b=JryMDKXBfwCH5QGxAILpbZWqQ9jDtAmo0YVkHCwRS/eSXUnOTx2Un6QM/R4YI6bPEqkSF+2O5aMMGUJPqG4uUpT0CtIYMHY1Xc5+VOZiIi3YTcPK/EJV7v6IAFgdJ2fY6eOX6Fvr6RmZ9hd1tBePt+X4PJy7m36FIDEw+Wm28yw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725419; c=relaxed/simple; bh=Aq4WSStk0rooKIc9npdEwOo7UWQ2PGDevEB4+t8QRUw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fjD02stM8lixDXBY/mR9xVztCbXQgXZS+uX7S4qQs68A9WvYjkvo4/LaxXx9dNCqEL6ooSmJp0vD7t3XgdksUswsGwWHhI5yHMVD8Ydjwcx+jEZc46QCSXdIaueLPve+5h5RDEKERXoADR+AnrBQd++7j6pAhbshz4uzyQv4qVY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=DnjMFKCv; arc=none smtp.client-ip=185.171.202.116 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="DnjMFKCv" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-04.galae.net (Postfix) with ESMTPS id 42CF4C011C1; Wed, 24 Sep 2025 14:49:59 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 707F660634; Wed, 24 Sep 2025 14:50:16 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 97231102F1972; Wed, 24 Sep 2025 16:50:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725415; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=NXLxX7aIyZUMbZFpVGJXKN2UBcI9+3K+7D1mGbTAaUI=; b=DnjMFKCvzCwF9iEbVd9EYcC7HrvoJDzXp3s9r8DNoDDS45rKtxaMmvqx/hFdMxAPwz2+zQ /GyY/BWKrmJGWbvrqpfylY0F5zs7k9WZm2Wg4cl5K8vTbXmhIqYE5pbjGefLDRV4Pc55sf 7rteo2530USfoI6Q1n2l/nbOC8K2fe8s588HSdsVJ5PyxSLZ5g/p44yno7o8q89cwHOj2I Wi5AhMpDXpOAb+a9LAdhKY+7+Ej0ti9HdydyAg3f9XhISgrZCjOjrYYsx1aXJGafP02S8b wxv2NosLJAWfs3jK1mxbD4DWT9WV0aiN5Dq8hqekx5pRcytfBvNC7HbPWvAoqQ== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:41 +0200 Subject: [PATCH bpf-next v4 06/15] selftests/bpf: test_xsk: Wrap test clean-up in functions Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-6-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 The clean-up done at the end of a test in __testapp_validate_traffic() isn't wrapped in a function. It isn't convenient if we want to use it somewhere else in the code. Wrap the clean-up in two new functions : the first deletes the sockets, the second releases the umem. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 36 ++++++++++++++++++++++--------= ---- 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index d7cb2821469c62abd0d532821e836336a2177eb5..84b724731e26d0c7e67131ec1bd= 562e223d3d09d 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -1679,6 +1679,27 @@ static void xsk_attach_xdp_progs(struct test_spec *t= est, struct ifobject *ifobj_ xsk_reattach_xdp(ifobj_tx, test->xdp_prog_tx, test->xskmap_tx, test->mod= e); } =20 +static void clean_sockets(struct test_spec *test, struct ifobject *ifobj) +{ + u32 i; + + if (!ifobj || !test) + return; + + for (i =3D 0; i < test->nb_sockets; i++) + xsk_socket__delete(ifobj->xsk_arr[i].xsk); +} + +static void clean_umem(struct test_spec *test, struct ifobject *ifobj1, st= ruct ifobject *ifobj2) +{ + if (!ifobj1) + return; + + testapp_clean_xsk_umem(ifobj1); + if (ifobj2 && !ifobj2->shared_umem) + testapp_clean_xsk_umem(ifobj2); +} + static int __testapp_validate_traffic(struct test_spec *test, struct ifobj= ect *ifobj1, struct ifobject *ifobj2) { @@ -1734,18 +1755,9 @@ static int __testapp_validate_traffic(struct test_sp= ec *test, struct ifobject *i pthread_join(t0, NULL); =20 if (test->total_steps =3D=3D test->current_step || test->fail) { - u32 i; - - if (ifobj2) - for (i =3D 0; i < test->nb_sockets; i++) - xsk_socket__delete(ifobj2->xsk_arr[i].xsk); - - for (i =3D 0; i < test->nb_sockets; i++) - xsk_socket__delete(ifobj1->xsk_arr[i].xsk); - - testapp_clean_xsk_umem(ifobj1); - if (ifobj2 && !ifobj2->shared_umem) - testapp_clean_xsk_umem(ifobj2); + clean_sockets(test, ifobj1); + clean_sockets(test, ifobj2); + clean_umem(test, ifobj1, ifobj2); } =20 if (test->fail) --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-03.galae.net (smtpout-03.galae.net [185.246.85.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5372330FF1E; Wed, 24 Sep 2025 14:50:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.85.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725424; cv=none; b=k89vbRz5/s6iKBY/fFomWDZ+aQTiHaVi1vf49715MpYEgtUN80kKm3nVUsxDMfljkDSjsrzp7USTMLQxPNGKu2GZQUOdyOuag4dja4oiiXuUsIERSz0/EXQ6Hrt7ALbE5gEsf5EQ86s6l5mSozgMNugh2VvmyFrBaqz7kGJ8vRk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725424; c=relaxed/simple; bh=UTdAvwzTJVJsJdRBIZqx8UYFILV84IUDYUjjD/LqFMo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TyfGX7eVo2bxbsEuaVN8M1YKWxYYOtQ5usd0JzZTNARuVGvkxcs/tp93RJAu/lxBQ+kfmKuIyZu9hyy1B98tIxK+B7MNJ/Ju17m0U0Dp/ICchO/w7g72Z1QKibd7nliMy0tWlKOQT9MCjVMhvES6ihhV8OvrNT7w922q2F4tbic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=KMYvXsPe; arc=none smtp.client-ip=185.246.85.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="KMYvXsPe" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-03.galae.net (Postfix) with ESMTPS id C8C074E40D42; Wed, 24 Sep 2025 14:50:20 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 9880F60634; Wed, 24 Sep 2025 14:50:20 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 930A0102F18D8; Wed, 24 Sep 2025 16:50:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725419; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=Q1E9t3HMoWk9uJ51+vn11N7Peundo5w+fDZjVfqj1wI=; b=KMYvXsPexTAT3Zn+96Jc8Fmdd82VplypsaD0xAbHsUNycXNbBELH4z2k6j1pEUy7Eg+y8L e0NQN+cF46+E+bthAY2FbF8KaUaZruBOGb5mJz/LIbdkKVSG+mE1e9PJJl9mX3Dd/LYJul Tbo9InXRwx6XZofqiwL6HdFZOtAl0UeHugEryyMzRvXNLIvRDTcF3oP++D5mKCKzh9UqR0 9f/HOpEavmKykYZTd6OR0Xyi4b+42SY0Fr+nqmj2kQXgILs/Uk1SPZd1oEGFypyVzgUaUQ k4Bw5yOS8Vj58+VCEMWaXXFEP2T9e57Z4EfHsQEzTBUWHzVQm6A56ES49prcjA== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:42 +0200 Subject: [PATCH bpf-next v4 07/15] selftests/bpf: test_xsk: Release resources when swap fails Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-7-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 testapp_validate_traffic() doesn't release the sockets and the umem created by the threads if the test isn't currently in its last step. Thus, if the swap_xsk_resources() fails before the last step, the created resources aren't cleaned up. Clean the sockets and the umem in case of swap_xsk_resources() failure. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 84b724731e26d0c7e67131ec1bd562e223d3d09d..8fe75845d7a6aa5342229fa419f= cbaa411ae9e70 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -1866,8 +1866,13 @@ int testapp_xdp_prog_cleanup(struct test_spec *test) if (testapp_validate_traffic(test)) return TEST_FAILURE; =20 - if (swap_xsk_resources(test)) + if (swap_xsk_resources(test)) { + clean_sockets(test, test->ifobj_rx); + clean_sockets(test, test->ifobj_tx); + clean_umem(test, test->ifobj_rx, test->ifobj_tx); return TEST_FAILURE; + } + return testapp_validate_traffic(test); } =20 --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-04.galae.net (smtpout-04.galae.net [185.171.202.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED5DC311967; Wed, 24 Sep 2025 14:50:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.171.202.116 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725427; cv=none; b=j2Q5pjq4bfEANHGxZnF3PPwKcAv1/+Xkqaz8jwrOIJ6FDFQ6L5MCGjBN3QkJkc80NCDakfIAqyYS1fzKCEqVU9ISjLq97sBndYCKb3W3r125BXgXdq69pAw7DG+KHVLH2ES782au8NVWgxDXY6fk4hPIsHpveBtYMKS92/T6bzY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725427; c=relaxed/simple; bh=XlK93pgR1V4ENMQ1L1wbdIXNtETXitk355cTtwwbOME=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=s/C6/5+GREdEszaVrqunSyo/0FocM9j53J/u6V/lDivIMAx9tql3CMUyV62XpQSRA0OSkFS5RL5TyWlwaxXz9snxws4MXcOsmzcfDVzKpTItvlZC8AMEDgOn6jjL0dGRCrKDjyMGVOHqNntSwvtD7SUlMxkAssLW9HiTvmJcHDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=Qf0/4ygD; arc=none smtp.client-ip=185.171.202.116 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="Qf0/4ygD" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-04.galae.net (Postfix) with ESMTPS id 6EFA1C011C7; Wed, 24 Sep 2025 14:50:06 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 9C63A60634; Wed, 24 Sep 2025 14:50:23 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 82F2B102F1916; Wed, 24 Sep 2025 16:50:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725422; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=RSp3Q4mkJ46b3+5fyiaK2zTo2iqcaDAhtmMFM8OB9Xo=; b=Qf0/4ygDj1X3haEyYtSZ8cuZwp5FV53t0P7PODs3oSf72TsW1b5fpZW7ic9KEdbsZHBJtN 43MoNqL7tN3gII7bXxTYsJYwoEmPa+0Of7ayn4rKtnfqRS9PRS/B4xhrCGyzoRsmckcMfR svRkphJcRFI/Csmcd16wMK5I0hK0lJtjPzxA35fJVjAll+1SNtJ0K5pOq3a0P3naqmLU4z MYWuZxIoQx+cPRUoWvjaEDPwhOCfgr2j2WjzgVVhMzs2bobUzvbw9Ff/oONthi9qzSUB6D dZETB4uSviNJFEqoxKRGAvP3dribGf3EohX1LDQMVG3etd8RwfAfcSKRr2/j8Q== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:43 +0200 Subject: [PATCH bpf-next v4 08/15] selftests/bpf: test_xsk: Add return value to init_iface() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-8-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 init_iface() doesn't have any return value while it can fail. In case of failure it calls exit_on_error() which exits the application immediately. This prevents the following tests from being run and isn't compliant with the CI Add a return value to init_iface() so errors can be handled more smoothly. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 8 +++++--- tools/testing/selftests/bpf/test_xsk.h | 2 +- tools/testing/selftests/bpf/xskxceiver.c | 7 +++++-- 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 8fe75845d7a6aa5342229fa419fcbaa411ae9e70..7db1d974e31511e93b05bf70be9= 91cee4cd444c6 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -2189,7 +2189,7 @@ static bool hugepages_present(void) return true; } =20 -void init_iface(struct ifobject *ifobj, thread_func_t func_ptr) +int init_iface(struct ifobject *ifobj, thread_func_t func_ptr) { LIBBPF_OPTS(bpf_xdp_query_opts, query_opts); int err; @@ -2199,7 +2199,7 @@ void init_iface(struct ifobject *ifobj, thread_func_t= func_ptr) err =3D xsk_load_xdp_programs(ifobj); if (err) { ksft_print_msg("Error loading XDP program\n"); - exit_with_error(err); + return err; } =20 if (hugepages_present()) @@ -2208,7 +2208,7 @@ void init_iface(struct ifobject *ifobj, thread_func_t= func_ptr) err =3D bpf_xdp_query(ifobj->ifindex, XDP_FLAGS_DRV_MODE, &query_opts); if (err) { ksft_print_msg("Error querying XDP capabilities\n"); - exit_with_error(-err); + return err; } if (query_opts.feature_flags & NETDEV_XDP_ACT_RX_SG) ifobj->multi_buff_supp =3D true; @@ -2220,6 +2220,8 @@ void init_iface(struct ifobject *ifobj, thread_func_t= func_ptr) ifobj->xdp_zc_max_segs =3D 0; } } + + return 0; } =20 int testapp_send_receive(struct test_spec *test) diff --git a/tools/testing/selftests/bpf/test_xsk.h b/tools/testing/selftes= ts/bpf/test_xsk.h index fb546cab39fdfbd22dcb352784a7c5ef383f8ac6..f4e192264b140c21cc861192fd0= df991c46afd24 100644 --- a/tools/testing/selftests/bpf/test_xsk.h +++ b/tools/testing/selftests/bpf/test_xsk.h @@ -137,7 +137,7 @@ struct ifobject { }; struct ifobject *ifobject_create(void); void ifobject_delete(struct ifobject *ifobj); -void init_iface(struct ifobject *ifobj, thread_func_t func_ptr); +int init_iface(struct ifobject *ifobj, thread_func_t func_ptr); =20 int xsk_configure_umem(struct ifobject *ifobj, struct xsk_umem_info *umem,= void *buffer, u64 size); int xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info= *umem, diff --git a/tools/testing/selftests/bpf/xskxceiver.c b/tools/testing/selft= ests/bpf/xskxceiver.c index 8e108e3162695d5d50b3e3805672601024e385e2..a874f27b590d8ba615e16c61272= 8b2f515ac8dff 100644 --- a/tools/testing/selftests/bpf/xskxceiver.c +++ b/tools/testing/selftests/bpf/xskxceiver.c @@ -373,8 +373,11 @@ int main(int argc, char **argv) ifobj_tx->set_ring.default_rx =3D ifobj_tx->ring.rx_pending; } =20 - init_iface(ifobj_rx, worker_testapp_validate_rx); - init_iface(ifobj_tx, worker_testapp_validate_tx); + if (init_iface(ifobj_rx, worker_testapp_validate_rx) || + init_iface(ifobj_tx, worker_testapp_validate_tx)) { + ksft_print_msg("Error : can't initialize interfaces\n"); + ksft_exit_xfail(); + } =20 test_init(&test, ifobj_tx, ifobj_rx, 0, &tests[0]); tx_pkt_stream_default =3D pkt_stream_generate(DEFAULT_PKT_CNT, MIN_PKT_SI= ZE); --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-04.galae.net (smtpout-04.galae.net [185.171.202.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C53A3126D8 for ; Wed, 24 Sep 2025 14:50:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.171.202.116 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725429; cv=none; b=jcqzCPyUys9Ntrg0jhPZnSJpbApc4n3HeE1H0/Kn8JvmMy7H+ml1/UhSN+XkEwaY58p6doPNKKjyag4uxkRKLQeOVqVCqgh/w6PopTH5QjV5nh8muOaQorqpfngPMOmvAx1/lhmruVzMnFIKJRRkTiyAVc98TgiPA+zydsQwxec= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725429; c=relaxed/simple; bh=tN+KK502mRn5IU2MfOua2IbsgatQICalJNBq1gn4xBU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OnYHJMPF4DhsOiAjRCc22fNfq0eYTY+Ajo0KWcLVyP//+Pj3fdKYk5JZTj/mgsuZHku+WJ845PlXfmKnVhrn3jw8h0Y8dLVdFJW1TmICZJNx1tIAi3D2aih27Ik9xILdT7ovo7CDbHq2rdqkrBVn4kZaLPDJntXp/rDEpLCHmGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=PjJ2y6oG; arc=none smtp.client-ip=185.171.202.116 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="PjJ2y6oG" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-04.galae.net (Postfix) with ESMTPS id E5620C011C1; Wed, 24 Sep 2025 14:50:08 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 1E7A160634; Wed, 24 Sep 2025 14:50:26 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id C92D8102F1972; Wed, 24 Sep 2025 16:50:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725424; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=WGWlPkl4pGG2y4n8r70mjy/ZV+JI6orK8Oq0psEM/Wg=; b=PjJ2y6oGs6Qy4QsMcuo7ez/7P7hirhWs0vo3ASxlceKQXUjvwEH7SIUwtkRXuV7kuxJxzv +8pQd/UndoztOAApaekuHPImdUIjvtEffqwdd0SVcH9titqy7gX45h21SefuRRPbaDva02 fHTD0LLsgpMDjBTrUNFRpxAe/2lXIT5+OUHhg3OuTuSwMlyHNsn5HFhrDS54xC70aSIg3J XFVBo1ruDketINLFj6DpQ2OiFlf8kcxB7gLcWd1vDejUKfV1QB+1q0TdYlvNC/MPYOZbWB wIhmRuxI1MF3HWjIHi67PefIEH8iJey7cThLsQ4qiirsGVcT4IMR0X1tCcTRRg== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:44 +0200 Subject: [PATCH bpf-next v4 09/15] selftests/bpf: test_xsk: Don't exit immediately when xsk_attach fails Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-9-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 xsk_reattach_xdp calls exit_with_error() on failures. This exits the program immediately. It prevents the following tests from being run and isn't compliant with the CI. Add a return value to the functions handling XDP attachments to handle errors more smoothly. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 7db1d974e31511e93b05bf70be991cee4cd444c6..0adb6c0b948f6216b24d0562bcd= a26097dbb9dbc 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -1643,7 +1643,7 @@ static bool xdp_prog_changed_tx(struct test_spec *tes= t) return ifobj->xdp_prog !=3D test->xdp_prog_tx || ifobj->mode !=3D test->m= ode; } =20 -static void xsk_reattach_xdp(struct ifobject *ifobj, struct bpf_program *x= dp_prog, +static int xsk_reattach_xdp(struct ifobject *ifobj, struct bpf_program *xd= p_prog, struct bpf_map *xskmap, enum test_mode mode) { int err; @@ -1652,31 +1652,40 @@ static void xsk_reattach_xdp(struct ifobject *ifobj= , struct bpf_program *xdp_pro err =3D xsk_attach_xdp_program(xdp_prog, ifobj->ifindex, mode_to_xdp_flag= s(mode)); if (err) { ksft_print_msg("Error attaching XDP program\n"); - exit_with_error(-err); + return err; } =20 if (ifobj->mode !=3D mode && (mode =3D=3D TEST_MODE_DRV || mode =3D=3D TE= ST_MODE_ZC)) if (!xsk_is_in_mode(ifobj->ifindex, XDP_FLAGS_DRV_MODE)) { ksft_print_msg("ERROR: XDP prog not in DRV mode\n"); - exit_with_error(EINVAL); + return -EINVAL; } =20 ifobj->xdp_prog =3D xdp_prog; ifobj->xskmap =3D xskmap; ifobj->mode =3D mode; + + return 0; } =20 -static void xsk_attach_xdp_progs(struct test_spec *test, struct ifobject *= ifobj_rx, +static int xsk_attach_xdp_progs(struct test_spec *test, struct ifobject *i= fobj_rx, struct ifobject *ifobj_tx) { - if (xdp_prog_changed_rx(test)) - xsk_reattach_xdp(ifobj_rx, test->xdp_prog_rx, test->xskmap_rx, test->mod= e); + int err =3D 0; + + if (xdp_prog_changed_rx(test)) { + err =3D xsk_reattach_xdp(ifobj_rx, test->xdp_prog_rx, test->xskmap_rx, t= est->mode); + if (err) + return err; + } =20 if (!ifobj_tx || ifobj_tx->shared_umem) - return; + return 0; =20 if (xdp_prog_changed_tx(test)) - xsk_reattach_xdp(ifobj_tx, test->xdp_prog_tx, test->xskmap_tx, test->mod= e); + err =3D xsk_reattach_xdp(ifobj_tx, test->xdp_prog_tx, test->xskmap_tx, t= est->mode); + + return err; } =20 static void clean_sockets(struct test_spec *test, struct ifobject *ifobj) @@ -1789,7 +1798,8 @@ static int testapp_validate_traffic(struct test_spec = *test) } } =20 - xsk_attach_xdp_progs(test, ifobj_rx, ifobj_tx); + if (xsk_attach_xdp_progs(test, ifobj_rx, ifobj_tx)) + return TEST_FAILURE; return __testapp_validate_traffic(test, ifobj_rx, ifobj_tx); } =20 --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-03.galae.net (smtpout-03.galae.net [185.246.85.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 655B7312827; Wed, 24 Sep 2025 14:50:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.85.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725432; cv=none; b=VwfHjnyZSiGEhsQYXNSmWssslA1auiUWyLQ4l98nmLThD+Ai6oD/6CjgHu6tlCin75CaYOBCWA+bl3qT6P8LCpmTFPHvAwwAGm4MYokjoWpcknfaduYOayI7VJKnnenMjg3bf9tueorVSph6T1pXuIMaKkYCU5+kDqG2q2FR7qc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725432; c=relaxed/simple; bh=v4Hzeqdb7Z4hEux4t+JJgDM2laAqTJTu0/gQKSRMpIU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=iMovjJ3Dz8XeEsIWXSA5fbXvxbI+atZK0xD6WIWKVHpny9+rgl1RLhAxFLIB4WrUoYaeIiWYcoLWpM2rO+r9z3z3NiugWaMdCqhXZU7hwEgJAhiJpw+BLXj6eIJu/7eOfgW1OQRD4iA/t2prfUa9rXu0MF1NW1CQ0Mgb9fUIP8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=jOZzbbPh; arc=none smtp.client-ip=185.246.85.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="jOZzbbPh" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-03.galae.net (Postfix) with ESMTPS id E8BAB4E40D58; Wed, 24 Sep 2025 14:50:28 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id BE9DD60634; Wed, 24 Sep 2025 14:50:28 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 31E5D102F1973; Wed, 24 Sep 2025 16:50:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725427; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=TK9L/dZGJ+Bzm/DsHznC4dwU9CdtNX9rrae1jJI01ds=; b=jOZzbbPhRLBxIB3iLegxUKAU6qJ02CoWyEszBFekwkike7c6Fb1yuONGw+w0l64OreZmh7 ZQ2LFXlIskjwRaYui9An4O/ximJGy2xymlkKpwZl7+nfuf21C3CIQJGH7vLy0L3msLpDug fupHvVFxVFr0pWda8uGM7/K/hnljDNFb6WITWbViybxpjasx9ByIdXkGQUhr6ZT3v8ITbk Fm1odY2EQLw1Pcwq/C7tLm187FyOwBdKJ0B7hSZgoRtYZCKFioHsH+U+A7P7Nu8EHorGKE 7+KAh8oMTmC3womKVIUgaJQ5fZMaoBjDEvXS0e5XU//RtarDAp63McSdeuVSqQ== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:45 +0200 Subject: [PATCH bpf-next v4 10/15] selftests/bpf: test_xsk: Don't exit immediately when gettimeofday fails Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-10-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 exit_with_error() is called when gettimeofday() fails. This exits the program immediately. It prevents the following tests from being run and isn't compliant with the CI. Return TEST_FAILURE instead of calling exit_on_error(). Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 0adb6c0b948f6216b24d0562bcda26097dbb9dbc..2c955e1099439c377cd28f5a9be= 2a17e65d49f78 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -1096,7 +1096,7 @@ static int receive_pkts(struct test_spec *test) =20 ret =3D gettimeofday(&tv_now, NULL); if (ret) - exit_with_error(errno); + return TEST_FAILURE; =20 timeradd(&tv_now, &tv_timeout, &tv_end); =20 @@ -1112,7 +1112,7 @@ static int receive_pkts(struct test_spec *test) =20 ret =3D gettimeofday(&tv_now, NULL); if (ret) - exit_with_error(errno); + return TEST_FAILURE; =20 if (timercmp(&tv_now, &tv_end, >)) { ksft_print_msg("ERROR: [%s] Receive loop timed out\n", __func__); @@ -1255,13 +1255,13 @@ static int wait_for_tx_completion(struct xsk_socket= _info *xsk) =20 ret =3D gettimeofday(&tv_now, NULL); if (ret) - exit_with_error(errno); + return TEST_FAILURE; timeradd(&tv_now, &tv_timeout, &tv_end); =20 while (xsk->outstanding_tx) { ret =3D gettimeofday(&tv_now, NULL); if (ret) - exit_with_error(errno); + return TEST_FAILURE; if (timercmp(&tv_now, &tv_end, >)) { ksft_print_msg("ERROR: [%s] Transmission loop timed out\n", __func__); return TEST_FAILURE; --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1685313292; Wed, 24 Sep 2025 14:50:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.84.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725434; cv=none; b=ebLyIq2PSz3V73fBd8y7psOCF6flm/s6KcT6mWo1B0qRc6zgHUFap1OjOyMCrz1dAgPtNBKD2s8hXBmeeOI5sR0yPWSqfVxYPVLYDI3hGSsS9Lz2KBFTXdRffHU3rG5bqZV0XOqwOdGkZWOnL2QNfdPXiSepeWy1gP1sNdr5DA4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725434; c=relaxed/simple; bh=k1mLiNqPfeMkvfcF3JQJr1r5KJA6A9pctoPLVoSvfIw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pMoErTDGDrNPt8bxvBTNmIx3gOZATWFuXoN4+1MzQ/N+MR9gklc1BQJ1NnmgfRXHxCg5BGfhVpAk3MfA5eKSyWpiHdvw+hmD9DfX3aKTrE5XTjUI/YTm9kseccRyb0Rm3IrjFXMln3w0uqO1IAin/Ux4kiUmJVXf7Ns/xJW9PvQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=ed4JoDyU; arc=none smtp.client-ip=185.246.84.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="ed4JoDyU" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-02.galae.net (Postfix) with ESMTPS id 5D5481A0F7D; Wed, 24 Sep 2025 14:50:31 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 33E3260634; Wed, 24 Sep 2025 14:50:31 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id E8D1C102F1916; Wed, 24 Sep 2025 16:50:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725430; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=DJxs9UOoCXTkOveg+K++uzdEpMkxFH5v0o1qUIViqTg=; b=ed4JoDyUIXFBzL4bMum4VyDPP6lJhU0eDFV8CYWlyzZ5koxobBg56mV7zwZEdQOP8Og/WH 6DS3t9ug0sZJxeTHe6tYtVmPt83pAPb6oWuYaGyWg8voo1W+0uslg1EPSa1UOX4inH9Roe DmtCH0BzF7IBJHbPsHwS/gVNveIigKPZyWk3lw09Bp5R3UDl6gYGQ9R0a0Mj/q7t8sUSNL lClRMlgREkHjZ1/m1lVxycMzRcBVhs5aKtO9au0g+eZOul/ziAZhGJvNelU6cInw23Anna oX7Kel8WtpmHLcsKmo60KWsh1nIwvE3NhWgmxYNCoxMWKYj9wLzz9od2yWThxQ== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:46 +0200 Subject: [PATCH bpf-next v4 11/15] selftests/bpf: test_xsk: Don't exit immediately when workers fail Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-11-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 TX and RX workers can fail in many places. These failures trigger a call to exit_with_error() which exits the program immediately. It prevents the following tests from running and isn't compliant with the CI. Add return value to functions that can fail. Handle failures more smoothly through report_failure(). Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 110 +++++++++++++++++++++++------= ---- 1 file changed, 76 insertions(+), 34 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 2c955e1099439c377cd28f5a9be2a17e65d49f78..2c392d5b9f30145cf7b0ea8a499= 90f1673bef6c9 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -132,24 +132,26 @@ static void umem_reset_alloc(struct xsk_umem_info *um= em) umem->next_buffer =3D 0; } =20 -static void enable_busy_poll(struct xsk_socket_info *xsk) +static int enable_busy_poll(struct xsk_socket_info *xsk) { int sock_opt; =20 sock_opt =3D 1; if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_PREFER_BUSY_POLL, (void *)&sock_opt, sizeof(sock_opt)) < 0) - exit_with_error(errno); + return -errno; =20 sock_opt =3D 20; if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL, (void *)&sock_opt, sizeof(sock_opt)) < 0) - exit_with_error(errno); + return -errno; =20 sock_opt =3D xsk->batch_size; if (setsockopt(xsk_socket__fd(xsk->xsk), SOL_SOCKET, SO_BUSY_POLL_BUDGET, (void *)&sock_opt, sizeof(sock_opt)) < 0) - exit_with_error(errno); + return -errno; + + return 0; } =20 int xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info= *umem, @@ -759,7 +761,7 @@ static bool is_metadata_correct(struct pkt *pkt, void *= buffer, u64 addr) return true; } =20 -static bool is_adjust_tail_supported(struct xsk_xdp_progs *skel_rx) +static int is_adjust_tail_supported(struct xsk_xdp_progs *skel_rx, bool *s= upported) { struct bpf_map *data_map; int adjust_value =3D 0; @@ -769,19 +771,21 @@ static bool is_adjust_tail_supported(struct xsk_xdp_p= rogs *skel_rx) data_map =3D bpf_object__find_map_by_name(skel_rx->obj, "xsk_xdp_.bss"); if (!data_map || !bpf_map__is_internal(data_map)) { ksft_print_msg("Error: could not find bss section of XDP program\n"); - exit_with_error(errno); + return -EINVAL; } =20 ret =3D bpf_map_lookup_elem(bpf_map__fd(data_map), &key, &adjust_value); if (ret) { ksft_print_msg("Error: bpf_map_lookup_elem failed with error %d\n", ret); - exit_with_error(errno); + return ret; } =20 /* Set the 'adjust_value' variable to -EOPNOTSUPP in the XDP program if t= he adjust_tail * helper is not supported. Skip the adjust_tail test case in this scenar= io. */ - return adjust_value !=3D -EOPNOTSUPP; + *supported =3D adjust_value !=3D -EOPNOTSUPP; + + return 0; } =20 static bool is_frag_valid(struct xsk_umem_info *umem, u64 addr, u32 len, u= 32 expected_pkt_nb, @@ -1433,7 +1437,7 @@ static int validate_tx_invalid_descs(struct ifobject = *ifobject) return TEST_PASS; } =20 -static void xsk_configure(struct test_spec *test, struct ifobject *ifobjec= t, +static int xsk_configure(struct test_spec *test, struct ifobject *ifobject, struct xsk_umem_info *umem, bool tx) { int i, ret; @@ -1450,24 +1454,34 @@ static void xsk_configure(struct test_spec *test, s= truct ifobject *ifobject, =20 /* Retry if it fails as xsk_socket__create() is asynchronous */ if (ctr >=3D SOCK_RECONF_CTR) - exit_with_error(-ret); + return ret; usleep(USLEEP_MAX); } - if (ifobject->busy_poll) - enable_busy_poll(&ifobject->xsk_arr[i]); + if (ifobject->busy_poll) { + ret =3D enable_busy_poll(&ifobject->xsk_arr[i]); + if (ret) + return ret; + } } + + return 0; } =20 -static void thread_common_ops_tx(struct test_spec *test, struct ifobject *= ifobject) +static int thread_common_ops_tx(struct test_spec *test, struct ifobject *i= fobject) { - xsk_configure(test, ifobject, test->ifobj_rx->umem, true); + int ret =3D xsk_configure(test, ifobject, test->ifobj_rx->umem, true); + + if (ret) + return ret; ifobject->xsk =3D &ifobject->xsk_arr[0]; ifobject->xskmap =3D test->ifobj_rx->xskmap; memcpy(ifobject->umem, test->ifobj_rx->umem, sizeof(struct xsk_umem_info)= ); ifobject->umem->base_addr =3D 0; + + return 0; } =20 -static void xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_= stream *pkt_stream, +static int xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_s= tream *pkt_stream, bool fill_up) { u32 rx_frame_size =3D umem->frame_size - XDP_PACKET_HEADROOM; @@ -1481,7 +1495,7 @@ static void xsk_populate_fill_ring(struct xsk_umem_in= fo *umem, struct pkt_stream =20 ret =3D xsk_ring_prod__reserve(&umem->fq, buffers_to_fill, &idx); if (ret !=3D buffers_to_fill) - exit_with_error(ENOSPC); + return -ENOSPC; =20 while (filled < buffers_to_fill) { struct pkt *pkt =3D pkt_stream_get_next_rx_pkt(pkt_stream, &nb_pkts); @@ -1509,9 +1523,11 @@ static void xsk_populate_fill_ring(struct xsk_umem_i= nfo *umem, struct pkt_stream =20 pkt_stream_reset(pkt_stream); umem_reset_alloc(umem); + + return 0; } =20 -static void thread_common_ops(struct test_spec *test, struct ifobject *ifo= bject) +static int thread_common_ops(struct test_spec *test, struct ifobject *ifob= ject) { LIBBPF_OPTS(bpf_xdp_query_opts, opts); int mmap_flags; @@ -1531,27 +1547,34 @@ static void thread_common_ops(struct test_spec *tes= t, struct ifobject *ifobject) =20 bufs =3D mmap(NULL, umem_sz, PROT_READ | PROT_WRITE, mmap_flags, -1, 0); if (bufs =3D=3D MAP_FAILED) - exit_with_error(errno); + return -errno; =20 ret =3D xsk_configure_umem(ifobject, ifobject->umem, bufs, umem_sz); if (ret) - exit_with_error(-ret); + return ret; =20 - xsk_configure(test, ifobject, ifobject->umem, false); + ret =3D xsk_configure(test, ifobject, ifobject->umem, false); + if (ret) + return ret; =20 ifobject->xsk =3D &ifobject->xsk_arr[0]; =20 if (!ifobject->rx_on) - return; + return 0; =20 - xsk_populate_fill_ring(ifobject->umem, ifobject->xsk->pkt_stream, ifobjec= t->use_fill_ring); + ret =3D xsk_populate_fill_ring(ifobject->umem, ifobject->xsk->pkt_stream, + ifobject->use_fill_ring); + if (ret) + return ret; =20 for (i =3D 0; i < test->nb_sockets; i++) { ifobject->xsk =3D &ifobject->xsk_arr[i]; ret =3D xsk_update_xskmap(ifobject->xskmap, ifobject->xsk->xsk, i); if (ret) - exit_with_error(errno); + return ret; } + + return 0; } =20 void *worker_testapp_validate_tx(void *arg) @@ -1561,10 +1584,17 @@ void *worker_testapp_validate_tx(void *arg) int err; =20 if (test->current_step =3D=3D 1) { - if (!ifobject->shared_umem) - thread_common_ops(test, ifobject); - else - thread_common_ops_tx(test, ifobject); + if (!ifobject->shared_umem) { + if (thread_common_ops(test, ifobject)) { + test->fail =3D true; + pthread_exit(NULL); + } + } else { + if (thread_common_ops_tx(test, ifobject)) { + test->fail =3D true; + pthread_exit(NULL); + } + } } =20 err =3D send_pkts(test, ifobject); @@ -1584,29 +1614,41 @@ void *worker_testapp_validate_rx(void *arg) int err; =20 if (test->current_step =3D=3D 1) { - thread_common_ops(test, ifobject); + err =3D thread_common_ops(test, ifobject); } else { xsk_clear_xskmap(ifobject->xskmap); err =3D xsk_update_xskmap(ifobject->xskmap, ifobject->xsk->xsk, 0); - if (err) { + if (err) ksft_print_msg("Error: Failed to update xskmap, error %s\n", strerror(-err)); - exit_with_error(-err); - } } =20 pthread_barrier_wait(&barr); =20 + /* We leave only now in case of error to avoid getting stuck in the barri= er */ + if (err) { + test->fail =3D true; + pthread_exit(NULL); + } + err =3D receive_pkts(test); =20 if (!err && ifobject->validation_func) err =3D ifobject->validation_func(ifobject); =20 if (err) { - if (test->adjust_tail && !is_adjust_tail_supported(ifobject->xdp_progs)) - test->adjust_tail_support =3D false; - else + if (!test->adjust_tail) { test->fail =3D true; + } else { + bool supported; + + if (is_adjust_tail_supported(ifobject->xdp_progs, &supported)) + test->fail =3D true; + if (!supported) + test->adjust_tail_support =3D false; + else + test->fail =3D true; + } } =20 pthread_exit(NULL); --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-03.galae.net (smtpout-03.galae.net [185.246.85.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47BB530B53B; Wed, 24 Sep 2025 14:50:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.85.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725437; cv=none; b=p6OqsOHuPq08PfN2bHi/BrthxFvjLhEDlo7qHxd104xEV9eJYnyQlsVZ5Jl1NZcJuhM/Lm78WwFngPQSIBAwzjI5d/zyYM6Lhggu4q4qi2ygZ3LIDuNvAEXlhGdMa94ZapL4VNIh6CIKAAtlSne4au3OxJrrS8pJqkx1Y5Hvzxw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725437; c=relaxed/simple; bh=13mlIcN22MoVBJKPQYpDZafSJMWRWUcz6N9AWEC0wKU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=P9cT+avrbCeVErSX9Ppx1oGXyMu1vmmOsGbAPAnLhobSF+/Op52pku3AkaMRLQIG7G6NeGf7MgPhwYrRuh0mbHPyrVZbSK+z7J5EQudUm3AqZ9AeC9pcETG5KodjT/Jbd0fq94bmgXwYgKZ2FSdOX0MF2qOf/k2FJ3jU4KV+9s8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=VQJPzBgv; arc=none smtp.client-ip=185.246.85.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="VQJPzBgv" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-03.galae.net (Postfix) with ESMTPS id 03ABD4E40D4B; Wed, 24 Sep 2025 14:50:34 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id CD51F60634; Wed, 24 Sep 2025 14:50:33 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 7E0E4102F1975; Wed, 24 Sep 2025 16:50:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725432; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=PtAOV6pl4nR4Ob8glEmR2K44VxFkck1hyOMiXeH+b3c=; b=VQJPzBgvQGaP9GkqafObH4o7S1iYAr/AngpKrnDjjsoo0BIG6rGmM8Wps8LxYBleKC6k6P U7DxWoPxR/feEp97HdF4IB7Mtgr5XyhsiMZtEHcdpUJx3Pq/WD2zo9jHGZkdrAMNbdRVXk tHBa4dSiE1DhhJaa9BygaFtmtfrVj6ff6N1ficIBz2w8BJ+aCRUqCS9+RviBgfQmjTyHr1 /z5DiNAOfgpPOM5N+ReJtK6ozGooei40MtbfNONZcqj8AUIE/lAgxrmOQb9F7z8aeeLBFY N0g5RONxv9TJLT+3yy97bxwd0fUrKk85G4Yp15s+Cn/kjXgGOBpuyFxIRPlEGw== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:47 +0200 Subject: [PATCH bpf-next v4 12/15] selftests/bpf: test_xsk: Don't exit immediately if validate_traffic fails Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-12-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 __testapp_validate_traffic() calls exit_with_error() on failures. This exits the program immediately. It prevents the following tests from running and isn't compliant with the CI. Return TEST_FAILURE instead of calling exit_with_error(). Release the resource of the 1st thread if a failure happens between its creation and the creation of the second thread. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 2c392d5b9f30145cf7b0ea8a49990f1673bef6c9..18a6ce648461de1975aa25997a2= 2c77cecb97a76 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -1772,12 +1772,12 @@ static int __testapp_validate_traffic(struct test_s= pec *test, struct ifobject *i err =3D test_spec_set_mtu(test, test->mtu); if (err) { ksft_print_msg("Error, could not set mtu.\n"); - exit_with_error(err); + return TEST_FAILURE; } =20 if (ifobj2) { if (pthread_barrier_init(&barr, NULL, 2)) - exit_with_error(errno); + return TEST_FAILURE; pkt_stream_reset(ifobj2->xsk->pkt_stream); } =20 @@ -1791,8 +1791,11 @@ static int __testapp_validate_traffic(struct test_sp= ec *test, struct ifobject *i =20 if (ifobj2) { pthread_barrier_wait(&barr); - if (pthread_barrier_destroy(&barr)) - exit_with_error(errno); + if (pthread_barrier_destroy(&barr)) { + clean_sockets(test, ifobj1); + clean_umem(test, ifobj1, NULL); + return TEST_FAILURE; + } =20 /*Spawn TX thread */ pthread_create(&t1, NULL, ifobj2->func_ptr, test); --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 327E63148C6 for ; Wed, 24 Sep 2025 14:50:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.84.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725440; cv=none; b=toYq5JdvbdDF9tY5gKVfpGACEgwiA3sRJjtLMKGQ973pvubbv27gAsSI6B5j/7v86Hse5gW5+9+jo4JwXD4i1xpidsVNgEjE/25oWp9T3TrKLor39rvYt1ArzH7y/9NqsFC8dm/ejff9qCQ5Uji79aBpCCgfZq0vy4Ky27PWNaI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725440; c=relaxed/simple; bh=Sl7qyhFQO6cs/JT4ZzWKFa/Vo1Tj+0vv5OnnWd0PYEs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fXIId54XhyECmBJXlGjoD08K3okTNrrLjsiqvkHzfRI9f3ddkn0NA6LoHml/aVBNyAZMb7iUaw8aEOJmlsIzdDArkWjq8gleJk1updqX+5ZfKR5z7rWxLSs2mrgPwOc56hNhewDoVs2qBLnQR6cDI/nTA3vs8h7eJsyyWWxrVGg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=aJjhDPjT; arc=none smtp.client-ip=185.246.84.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="aJjhDPjT" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-02.galae.net (Postfix) with ESMTPS id B17B01A0F7D; Wed, 24 Sep 2025 14:50:36 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 8607C60634; Wed, 24 Sep 2025 14:50:36 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id EBEF2102F1916; Wed, 24 Sep 2025 16:50:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725435; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=IMwoaA0KIJB8w7w/2HWONK/hMoiPI90VMZ2K/F1AZ4M=; b=aJjhDPjTJwHwvJX5ZU1OILOjM2ArMimoBouviK2jmMiCQcB/SM+N2FMCMEcVyC8kJDIYEe +XiYZ9tDsME8DiXeG9K4dnV+85SSHF1AphkrNfok1H9Um+9wt58yD30qMBVS6fxPOvi/C7 JAvtjQS5uhcnyeO57ufieY4MaWYoUgmTg4xsg1kjc32Kq088BmyZw1YC4k5G0VHqdug0U8 bhFgvbn2Dr+N7f3MDzKKaKTaJLSP3h6F3MTyVCVmA5/3rTm8NPSo7zaSlw7HPDghxzwSVJ FBaW9cO1f5ouNfMi1WFfRcs+fRrv/p6RprOmTZvPMC/ljL9waN/4s187gRmHxw== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:48 +0200 Subject: [PATCH bpf-next v4 13/15] selftests/bpf: test_xsk: Don't exit immediately on allocation failures Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-13-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 If any allocation in the pkt_stream_*() helpers fail, exit_with_error() is called. This terminates the program immediately. It prevents the following tests from running and isn't compliant with the CI. Return NULL in case of allocation failure. Return TEST_FAILURE when something goes wrong in the packet generation. Clean up the resources if a failure happens between two steps of a test. Move exit_with_error()'s definition into xskxceiver.c as it isn't used anywhere else now. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.c | 136 +++++++++++++++++++++++----= ---- tools/testing/selftests/bpf/test_xsk.h | 7 -- tools/testing/selftests/bpf/xskxceiver.c | 9 ++ 3 files changed, 110 insertions(+), 42 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/test_xsk.c index 18a6ce648461de1975aa25997a22c77cecb97a76..453b55463ee8dcd2738d6605752= ad305532211b0 100644 --- a/tools/testing/selftests/bpf/test_xsk.c +++ b/tools/testing/selftests/bpf/test_xsk.c @@ -479,7 +479,7 @@ static struct pkt_stream *__pkt_stream_generate(u32 nb_= pkts, u32 pkt_len, u32 nb =20 pkt_stream =3D __pkt_stream_alloc(nb_pkts); if (!pkt_stream) - exit_with_error(ENOMEM); + return NULL; =20 pkt_stream->nb_pkts =3D nb_pkts; pkt_stream->max_pkt_len =3D pkt_len; @@ -503,37 +503,56 @@ static struct pkt_stream *pkt_stream_clone(struct pkt= _stream *pkt_stream) return pkt_stream_generate(pkt_stream->nb_pkts, pkt_stream->pkts[0].len); } =20 -static void pkt_stream_replace_ifobject(struct ifobject *ifobj, u32 nb_pkt= s, u32 pkt_len) +static int pkt_stream_replace_ifobject(struct ifobject *ifobj, u32 nb_pkts= , u32 pkt_len) { ifobj->xsk->pkt_stream =3D pkt_stream_generate(nb_pkts, pkt_len); + + if (!ifobj->xsk->pkt_stream) + return -ENOMEM; + + return 0; } =20 -static void pkt_stream_replace(struct test_spec *test, u32 nb_pkts, u32 pk= t_len) +static int pkt_stream_replace(struct test_spec *test, u32 nb_pkts, u32 pkt= _len) { - pkt_stream_replace_ifobject(test->ifobj_tx, nb_pkts, pkt_len); - pkt_stream_replace_ifobject(test->ifobj_rx, nb_pkts, pkt_len); + int ret; + + ret =3D pkt_stream_replace_ifobject(test->ifobj_tx, nb_pkts, pkt_len); + if (ret) + return ret; + + return pkt_stream_replace_ifobject(test->ifobj_rx, nb_pkts, pkt_len); } =20 -static void __pkt_stream_replace_half(struct ifobject *ifobj, u32 pkt_len, +static int __pkt_stream_replace_half(struct ifobject *ifobj, u32 pkt_len, int offset) { struct pkt_stream *pkt_stream; u32 i; =20 pkt_stream =3D pkt_stream_clone(ifobj->xsk->pkt_stream); + if (!pkt_stream) + return -ENOMEM; + for (i =3D 1; i < ifobj->xsk->pkt_stream->nb_pkts; i +=3D 2) pkt_stream_pkt_set(pkt_stream, &pkt_stream->pkts[i], offset, pkt_len); =20 ifobj->xsk->pkt_stream =3D pkt_stream; + + return 0; } =20 -static void pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, i= nt offset) +static int pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, in= t offset) { - __pkt_stream_replace_half(test->ifobj_tx, pkt_len, offset); - __pkt_stream_replace_half(test->ifobj_rx, pkt_len, offset); + int ret =3D __pkt_stream_replace_half(test->ifobj_tx, pkt_len, offset); + + if (ret) + return ret; + + return __pkt_stream_replace_half(test->ifobj_rx, pkt_len, offset); } =20 -static void pkt_stream_receive_half(struct test_spec *test) +static int pkt_stream_receive_half(struct test_spec *test) { struct pkt_stream *pkt_stream =3D test->ifobj_tx->xsk->pkt_stream; u32 i; @@ -547,14 +566,19 @@ static void pkt_stream_receive_half(struct test_spec = *test) =20 test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(pkt_stream->nb_pk= ts, pkt_stream->pkts[0].len); + if (!test->ifobj_rx->xsk->pkt_stream) + return -ENOMEM; + pkt_stream =3D test->ifobj_rx->xsk->pkt_stream; for (i =3D 1; i < pkt_stream->nb_pkts; i +=3D 2) pkt_stream->pkts[i].valid =3D false; =20 pkt_stream->nb_valid_entries /=3D 2; + + return 0; } =20 -static void pkt_stream_even_odd_sequence(struct test_spec *test) +static int pkt_stream_even_odd_sequence(struct test_spec *test) { struct pkt_stream *pkt_stream; u32 i; @@ -563,13 +587,19 @@ static void pkt_stream_even_odd_sequence(struct test_= spec *test) pkt_stream =3D test->ifobj_tx->xsk_arr[i].pkt_stream; pkt_stream =3D __pkt_stream_generate(pkt_stream->nb_pkts / 2, pkt_stream->pkts[0].len, i, 2); + if (!pkt_stream) + return -ENOMEM; test->ifobj_tx->xsk_arr[i].pkt_stream =3D pkt_stream; =20 pkt_stream =3D test->ifobj_rx->xsk_arr[i].pkt_stream; pkt_stream =3D __pkt_stream_generate(pkt_stream->nb_pkts / 2, pkt_stream->pkts[0].len, i, 2); + if (!pkt_stream) + return -ENOMEM; test->ifobj_rx->xsk_arr[i].pkt_stream =3D pkt_stream; } + + return 0; } =20 static void release_even_odd_sequence(struct test_spec *test) @@ -628,7 +658,7 @@ static struct pkt_stream *__pkt_stream_generate_custom(= struct ifobject *ifobj, s =20 pkt_stream =3D __pkt_stream_alloc(nb_frames); if (!pkt_stream) - exit_with_error(ENOMEM); + return NULL; =20 for (i =3D 0; i < nb_frames; i++) { struct pkt *pkt =3D &pkt_stream->pkts[pkt_nb]; @@ -671,15 +701,21 @@ static struct pkt_stream *__pkt_stream_generate_custo= m(struct ifobject *ifobj, s return pkt_stream; } =20 -static void pkt_stream_generate_custom(struct test_spec *test, struct pkt = *pkts, u32 nb_pkts) +static int pkt_stream_generate_custom(struct test_spec *test, struct pkt *= pkts, u32 nb_pkts) { struct pkt_stream *pkt_stream; =20 pkt_stream =3D __pkt_stream_generate_custom(test->ifobj_tx, pkts, nb_pkts= , true); + if (!pkt_stream) + return -ENOMEM; test->ifobj_tx->xsk->pkt_stream =3D pkt_stream; =20 pkt_stream =3D __pkt_stream_generate_custom(test->ifobj_rx, pkts, nb_pkts= , false); + if (!pkt_stream) + return -ENOMEM; test->ifobj_rx->xsk->pkt_stream =3D pkt_stream; + + return 0; } =20 static void pkt_print_data(u32 *data, u32 cnt) @@ -1944,24 +1980,28 @@ int testapp_stats_rx_dropped(struct test_spec *test) return TEST_SKIP; } =20 - pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0); + if (pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0)) + return TEST_FAILURE; test->ifobj_rx->umem->frame_headroom =3D test->ifobj_rx->umem->frame_size= - XDP_PACKET_HEADROOM - MIN_PKT_SIZE * 3; - pkt_stream_receive_half(test); + if (pkt_stream_receive_half(test)) + return TEST_FAILURE; test->ifobj_rx->validation_func =3D validate_rx_dropped; return testapp_validate_traffic(test); } =20 int testapp_stats_tx_invalid_descs(struct test_spec *test) { - pkt_stream_replace_half(test, XSK_UMEM__INVALID_FRAME_SIZE, 0); + if (pkt_stream_replace_half(test, XSK_UMEM__INVALID_FRAME_SIZE, 0)) + return TEST_FAILURE; test->ifobj_tx->validation_func =3D validate_tx_invalid_descs; return testapp_validate_traffic(test); } =20 int testapp_stats_rx_full(struct test_spec *test) { - pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS / 2,= MIN_PKT_SIZE); + if (pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS = / 2, MIN_PKT_SIZE)) + return TEST_FAILURE; test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(DEFAULT_UMEM_BUFF= ERS, MIN_PKT_SIZE); =20 test->ifobj_rx->xsk->rxqsize =3D DEFAULT_UMEM_BUFFERS; @@ -1972,7 +2012,8 @@ int testapp_stats_rx_full(struct test_spec *test) =20 int testapp_stats_fill_empty(struct test_spec *test) { - pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS / 2,= MIN_PKT_SIZE); + if (pkt_stream_replace(test, DEFAULT_UMEM_BUFFERS + DEFAULT_UMEM_BUFFERS = / 2, MIN_PKT_SIZE)) + return TEST_FAILURE; test->ifobj_rx->xsk->pkt_stream =3D pkt_stream_generate(DEFAULT_UMEM_BUFF= ERS, MIN_PKT_SIZE); =20 test->ifobj_rx->use_fill_ring =3D false; @@ -1985,7 +2026,8 @@ int testapp_send_receive_unaligned(struct test_spec *= test) test->ifobj_tx->umem->unaligned_mode =3D true; test->ifobj_rx->umem->unaligned_mode =3D true; /* Let half of the packets straddle a 4K buffer boundary */ - pkt_stream_replace_half(test, MIN_PKT_SIZE, -MIN_PKT_SIZE / 2); + if (pkt_stream_replace_half(test, MIN_PKT_SIZE, -MIN_PKT_SIZE / 2)) + return TEST_FAILURE; =20 return testapp_validate_traffic(test); } @@ -1995,7 +2037,8 @@ int testapp_send_receive_unaligned_mb(struct test_spe= c *test) test->mtu =3D MAX_ETH_JUMBO_SIZE; test->ifobj_tx->umem->unaligned_mode =3D true; test->ifobj_rx->umem->unaligned_mode =3D true; - pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE); + if (pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE)) + return TEST_FAILURE; return testapp_validate_traffic(test); } =20 @@ -2003,14 +2046,16 @@ int testapp_single_pkt(struct test_spec *test) { struct pkt pkts[] =3D {{0, MIN_PKT_SIZE, 0, true}}; =20 - pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); + if (pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts))) + return TEST_FAILURE; return testapp_validate_traffic(test); } =20 int testapp_send_receive_mb(struct test_spec *test) { test->mtu =3D MAX_ETH_JUMBO_SIZE; - pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE); + if (pkt_stream_replace(test, DEFAULT_PKT_CNT, MAX_ETH_JUMBO_SIZE)) + return TEST_FAILURE; =20 return testapp_validate_traffic(test); } @@ -2051,7 +2096,8 @@ int testapp_invalid_desc_mb(struct test_spec *test) } =20 test->mtu =3D MAX_ETH_JUMBO_SIZE; - pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); + if (pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts))) + return TEST_FAILURE; return testapp_validate_traffic(test); } =20 @@ -2096,7 +2142,8 @@ int testapp_invalid_desc(struct test_spec *test) pkts[6].offset +=3D umem_size; } =20 - pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts)); + if (pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts))) + return TEST_FAILURE; return testapp_validate_traffic(test); } =20 @@ -2108,7 +2155,8 @@ int testapp_xdp_drop(struct test_spec *test) test_spec_set_xdp_prog(test, skel_rx->progs.xsk_xdp_drop, skel_tx->progs.= xsk_xdp_drop, skel_rx->maps.xsk, skel_tx->maps.xsk); =20 - pkt_stream_receive_half(test); + if (pkt_stream_receive_half(test)) + return TEST_FAILURE; return testapp_validate_traffic(test); } =20 @@ -2140,7 +2188,8 @@ int testapp_xdp_shared_umem(struct test_spec *test) skel_tx->progs.xsk_xdp_shared_umem, skel_rx->maps.xsk, skel_tx->maps.xsk); =20 - pkt_stream_even_odd_sequence(test); + if (pkt_stream_even_odd_sequence(test)) + return TEST_FAILURE; =20 ret =3D testapp_validate_traffic(test); =20 @@ -2154,7 +2203,8 @@ int testapp_poll_txq_tmout(struct test_spec *test) test->ifobj_tx->use_poll =3D true; /* create invalid frame by set umem frame_size and pkt length equal to 20= 48 */ test->ifobj_tx->umem->frame_size =3D 2048; - pkt_stream_replace(test, 2 * DEFAULT_PKT_CNT, 2048); + if (pkt_stream_replace(test, 2 * DEFAULT_PKT_CNT, 2048)) + return TEST_FAILURE; return testapp_validate_traffic_single_thread(test, test->ifobj_tx); } =20 @@ -2168,7 +2218,7 @@ int testapp_too_many_frags(struct test_spec *test) { struct pkt *pkts; u32 max_frags, i; - int ret; + int ret =3D TEST_FAILURE; =20 if (test->mode =3D=3D TEST_MODE_ZC) { max_frags =3D test->ifobj_tx->xdp_zc_max_segs; @@ -2212,9 +2262,12 @@ int testapp_too_many_frags(struct test_spec *test) pkts[2 * max_frags + 1].len =3D MIN_PKT_SIZE; pkts[2 * max_frags + 1].valid =3D true; =20 - pkt_stream_generate_custom(test, pkts, 2 * max_frags + 2); - ret =3D testapp_validate_traffic(test); + if (pkt_stream_generate_custom(test, pkts, 2 * max_frags + 2)) { + free(pkts); + return TEST_FAILURE; + } =20 + ret =3D testapp_validate_traffic(test); free(pkts); return ret; } @@ -2288,7 +2341,8 @@ int testapp_send_receive_2k_frame(struct test_spec *t= est) { test->ifobj_tx->umem->frame_size =3D 2048; test->ifobj_rx->umem->frame_size =3D 2048; - pkt_stream_replace(test, DEFAULT_PKT_CNT, MIN_PKT_SIZE); + if (pkt_stream_replace(test, DEFAULT_PKT_CNT, MIN_PKT_SIZE)) + return TEST_FAILURE; return testapp_validate_traffic(test); } =20 @@ -2410,7 +2464,13 @@ int testapp_hw_sw_max_ring_size(struct test_spec *te= st) */ test->ifobj_tx->xsk->batch_size =3D test->ifobj_tx->ring.tx_max_pending -= 8; test->ifobj_rx->xsk->batch_size =3D test->ifobj_tx->ring.tx_max_pending -= 8; - pkt_stream_replace(test, max_descs, MIN_PKT_SIZE); + if (pkt_stream_replace(test, max_descs, MIN_PKT_SIZE)) { + clean_sockets(test, test->ifobj_tx); + clean_sockets(test, test->ifobj_rx); + clean_umem(test, test->ifobj_rx, test->ifobj_tx); + return TEST_FAILURE; + } + return testapp_validate_traffic(test); } =20 @@ -2436,8 +2496,13 @@ static int testapp_adjust_tail(struct test_spec *tes= t, u32 value, u32 pkt_len) test->adjust_tail =3D true; test->total_steps =3D 1; =20 - pkt_stream_replace_ifobject(test->ifobj_tx, DEFAULT_BATCH_SIZE, pkt_len); - pkt_stream_replace_ifobject(test->ifobj_rx, DEFAULT_BATCH_SIZE, pkt_len += value); + ret =3D pkt_stream_replace_ifobject(test->ifobj_tx, DEFAULT_BATCH_SIZE, p= kt_len); + if (ret) + return TEST_FAILURE; + + ret =3D pkt_stream_replace_ifobject(test->ifobj_rx, DEFAULT_BATCH_SIZE, p= kt_len + value); + if (ret) + return TEST_FAILURE; =20 ret =3D testapp_xdp_adjust_tail(test, value); if (ret) @@ -2489,7 +2554,8 @@ int testapp_tx_queue_consumer(struct test_spec *test) } =20 nr_packets =3D MAX_TX_BUDGET_DEFAULT + 1; - pkt_stream_replace(test, nr_packets, MIN_PKT_SIZE); + if (pkt_stream_replace(test, nr_packets, MIN_PKT_SIZE)) + return TEST_FAILURE; test->ifobj_tx->xsk->batch_size =3D nr_packets; test->ifobj_tx->xsk->check_consumer =3D true; =20 diff --git a/tools/testing/selftests/bpf/test_xsk.h b/tools/testing/selftes= ts/bpf/test_xsk.h index f4e192264b140c21cc861192fd0df991c46afd24..b068b25ea5da728fad1e17b894d= 6a1b1c9794f74 100644 --- a/tools/testing/selftests/bpf/test_xsk.h +++ b/tools/testing/selftests/bpf/test_xsk.h @@ -34,13 +34,6 @@ extern bool opt_verbose; #define print_verbose(x...) do { if (opt_verbose) ksft_print_msg(x); } whi= le (0) =20 -static void __exit_with_error(int error, const char *file, const char *fun= c, int line) -{ - ksft_test_result_fail("[%s:%s:%i]: ERROR: %d/\"%s\"\n", file, func, line,= error, - strerror(error)); - ksft_exit_xfail(); -} -#define exit_with_error(error) __exit_with_error(error, __FILE__, __func__= , __LINE__) =20 static inline u32 ceil_u32(u32 a, u32 b) { diff --git a/tools/testing/selftests/bpf/xskxceiver.c b/tools/testing/selft= ests/bpf/xskxceiver.c index a874f27b590d8ba615e16c612728b2f515ac8dff..a16d3ed3629a995e2bcdd735743= 7451f059d213e 100644 --- a/tools/testing/selftests/bpf/xskxceiver.c +++ b/tools/testing/selftests/bpf/xskxceiver.c @@ -107,6 +107,15 @@ static u32 opt_run_test =3D RUN_ALL_TESTS; =20 void test__fail(void) { /* for network_helpers.c */ } =20 +static void __exit_with_error(int error, const char *file, const char *fun= c, int line) +{ + ksft_test_result_fail("[%s:%s:%i]: ERROR: %d/\"%s\"\n", file, func, line, + error, strerror(error)); + ksft_exit_xfail(); +} + +#define exit_with_error(error) __exit_with_error(error, __FILE__, __func__= , __LINE__) + static bool ifobj_zc_avail(struct ifobject *ifobject) { size_t umem_sz =3D DEFAULT_UMEM_BUFFERS * XSK_UMEM__DEFAULT_FRAME_SIZE; --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F643314A94; Wed, 24 Sep 2025 14:50:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.84.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725442; cv=none; b=a43DdSLQUYWUBVXPSeuals9RSDk9CWShmPfq52oSa4PIdbsVkPwRcl8wSNB7omda2JZeir2er3GCnc/nCLcBMpJNBVNg+RLYDQuQKHT0XL5M+WI41MLTkbvIURxBfJOGwI/mJOWegTOECCIi5Dm1uhYmmRESS2VKGdp46UedblU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725442; c=relaxed/simple; bh=Hs4+AUs3krBlVcGNstavnFhp/svJKbtUXTQMgJjRYgw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TuiIitP70hSdHHSI6ox1JaF2Cbuak2hvjOfdsEEAQ8lajQyU1snF0QFsRigMfI+/oek6xTizuEkuSn9gY5sKennBlyn0qEwSpczryuHH8necfcGwl9jVQiguO9UPTOxWJLdGlN2zsLBVmyYFoaUjuDBuraE/VsbX1FfYLVmb6O8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=ulkTM4A2; arc=none smtp.client-ip=185.246.84.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="ulkTM4A2" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-02.galae.net (Postfix) with ESMTPS id 3F9611A0F81; Wed, 24 Sep 2025 14:50:39 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 15BB460634; Wed, 24 Sep 2025 14:50:39 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id C0956102F1973; Wed, 24 Sep 2025 16:50:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725437; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=3/WEqKstae0b12RLPmjaq1abjYRG9ZTZTzDs8xwjMu8=; b=ulkTM4A22y4wfF3v9/v/dLmwxx2fEe/+ELWr9kvM34izrh3EsRTSoe82z408qdNpTc2T6I 0jO4n3WceByong0f4iCuFnEc40appQrixa4AsbjkROH/lMORK+Wh+pT75CHNwaX/GRpj2L NcXV3vU4/7XO4+jmzhnMZKzDyXUEVU+D0RCYSMN3OSBQRgUQNafUfMtYF3w3i/g41iPDU2 GFKPQfZhsCBj0HlhIQI2MwYlTNpil5/3zOwx+f5G22v1avFRSoml58R6ab19lBdc0FOkae cEyBKVcAZp+jDRc96BaH4BFUr0TheWCWuj2R3bStxhXkDQKGvLJZ5gQsygafHA== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:49 +0200 Subject: [PATCH bpf-next v4 14/15] selftests/bpf: test_xsk: Isolate flaky tests Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-14-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 Some tests are flaky and fail from time to time on virtual interfaces. Adding them to the CI would trigger lots of 'false' errors. Remove the flaky tests from the nominal tests table so they won't be run by the CI in upcoming patch. Create a flaky_tests table to hold them. Use this flaky table in xskxceiver.c to keep all the tests available from the test_xsk.sh script. Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/test_xsk.h | 8 ++++++-- tools/testing/selftests/bpf/xskxceiver.c | 15 +++++++++++---- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/bpf/test_xsk.h b/tools/testing/selftes= ts/bpf/test_xsk.h index b068b25ea5da728fad1e17b894d6a1b1c9794f74..b86694d0eef53521e2f245670bb= c8176991466bd 100644 --- a/tools/testing/selftests/bpf/test_xsk.h +++ b/tools/testing/selftests/bpf/test_xsk.h @@ -272,7 +272,6 @@ static const struct test_spec tests[] =3D { {.name =3D "XDP_SHARED_UMEM", .test_func =3D testapp_xdp_shared_umem}, {.name =3D "XDP_METADATA_COPY", .test_func =3D testapp_xdp_metadata}, {.name =3D "XDP_METADATA_COPY_MULTI_BUFF", .test_func =3D testapp_xdp_met= adata_mb}, - {.name =3D "SEND_RECEIVE_9K_PACKETS", .test_func =3D testapp_send_receive= _mb}, {.name =3D "SEND_RECEIVE_UNALIGNED_9K_PACKETS", .test_func =3D testapp_send_receive_unaligned_mb}, {.name =3D "ALIGNED_INV_DESC_MULTI_BUFF", .test_func =3D testapp_aligned_= inv_desc_mb}, @@ -282,9 +281,14 @@ static const struct test_spec tests[] =3D { {.name =3D "HW_SW_MAX_RING_SIZE", .test_func =3D testapp_hw_sw_max_ring_s= ize}, {.name =3D "XDP_ADJUST_TAIL_SHRINK", .test_func =3D testapp_adjust_tail_s= hrink}, {.name =3D "XDP_ADJUST_TAIL_SHRINK_MULTI_BUFF", .test_func =3D testapp_ad= just_tail_shrink_mb}, - {.name =3D "XDP_ADJUST_TAIL_GROW", .test_func =3D testapp_adjust_tail_gro= w}, {.name =3D "XDP_ADJUST_TAIL_GROW_MULTI_BUFF", .test_func =3D testapp_adju= st_tail_grow_mb}, {.name =3D "TX_QUEUE_CONSUMER", .test_func =3D testapp_tx_queue_consumer}, }; =20 +static const struct test_spec flaky_tests[] =3D { + {.name =3D "XDP_ADJUST_TAIL_GROW", .test_func =3D testapp_adjust_tail_gro= w}, + {.name =3D "SEND_RECEIVE_9K_PACKETS", .test_func =3D testapp_send_receive= _mb}, +}; + + #endif /* TEST_XSK_H_ */ diff --git a/tools/testing/selftests/bpf/xskxceiver.c b/tools/testing/selft= ests/bpf/xskxceiver.c index a16d3ed3629a995e2bcdd7357437451f059d213e..8707f4a0fac64e1ebb6a4241edf= 8e874a1eb67c3 100644 --- a/tools/testing/selftests/bpf/xskxceiver.c +++ b/tools/testing/selftests/bpf/xskxceiver.c @@ -326,10 +326,13 @@ static void print_tests(void) printf("Tests:\n"); for (i =3D 0; i < ARRAY_SIZE(tests); i++) printf("%u: %s\n", i, tests[i].name); + for (i =3D ARRAY_SIZE(tests); i < ARRAY_SIZE(tests) + ARRAY_SIZE(flaky_te= sts); i++) + printf("%u: %s\n", i, flaky_tests[i - ARRAY_SIZE(tests)].name); } =20 int main(int argc, char **argv) { + const size_t total_tests =3D ARRAY_SIZE(tests) + ARRAY_SIZE(flaky_tests); struct pkt_stream *rx_pkt_stream_default; struct pkt_stream *tx_pkt_stream_default; struct ifobject *ifobj_tx, *ifobj_rx; @@ -357,7 +360,7 @@ int main(int argc, char **argv) print_tests(); ksft_exit_xpass(); } - if (opt_run_test !=3D RUN_ALL_TESTS && opt_run_test >=3D ARRAY_SIZE(tests= )) { + if (opt_run_test !=3D RUN_ALL_TESTS && opt_run_test >=3D total_tests) { ksft_print_msg("Error: test %u does not exist.\n", opt_run_test); ksft_exit_xfail(); } @@ -397,7 +400,7 @@ int main(int argc, char **argv) test.rx_pkt_stream_default =3D rx_pkt_stream_default; =20 if (opt_run_test =3D=3D RUN_ALL_TESTS) - nb_tests =3D ARRAY_SIZE(tests); + nb_tests =3D total_tests; else nb_tests =3D 1; if (opt_mode =3D=3D TEST_MODE_ALL) { @@ -419,11 +422,15 @@ int main(int argc, char **argv) if (opt_mode !=3D TEST_MODE_ALL && i !=3D opt_mode) continue; =20 - for (j =3D 0; j < ARRAY_SIZE(tests); j++) { + for (j =3D 0; j < total_tests; j++) { if (opt_run_test !=3D RUN_ALL_TESTS && j !=3D opt_run_test) continue; =20 - test_init(&test, ifobj_tx, ifobj_rx, i, &tests[j]); + if (j < ARRAY_SIZE(tests)) + test_init(&test, ifobj_tx, ifobj_rx, i, &tests[j]); + else + test_init(&test, ifobj_tx, ifobj_rx, i, + &flaky_tests[j - ARRAY_SIZE(tests)]); run_pkt_test(&test); usleep(USLEEP_MAX); =20 --=20 2.51.0 From nobody Thu Oct 2 01:01:40 2025 Received: from smtpout-02.galae.net (smtpout-02.galae.net [185.246.84.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26F97314B90; Wed, 24 Sep 2025 14:50:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.246.84.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725445; cv=none; b=G6BtwV2tOcbhiguROefxuLGVmZGHcRnprIEC3e9ym90lbIblhibL/hDWkouPdIPBey2+sFz3+cK7pKU1q3YS7cMJsD6k3Ltq17wo2BkSaVA+X2IwZYbBEqcdWXOMj2biEj3jYDr5Zm3a9Edd5igbJD81WQ2fUuv8emHJUEsujVM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758725445; c=relaxed/simple; bh=7SKzH8PxYzpAySCoXAtnf3AzABu9MFjk+10aWHUVLi0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ErEvoTnDybm4M2ZrGAlWb0Tm8dTq80rfSwUcSJ4GvDiuwd4qRtCb5e5VhVxQV+QKbMakVKZOqjcJugtGFLGkPxKhDpDAcVVuS1me0PUTqwiiEiNB8yLiYs8UuNQByzvawE8wP0CJcbKLpIv2uOUwfpBm4fLSSxgBO8hA+c3+yug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=xgAxSahX; arc=none smtp.client-ip=185.246.84.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="xgAxSahX" Received: from smtpout-01.galae.net (smtpout-01.galae.net [212.83.139.233]) by smtpout-02.galae.net (Postfix) with ESMTPS id B4C301A0F82; Wed, 24 Sep 2025 14:50:41 +0000 (UTC) Received: from mail.galae.net (mail.galae.net [212.83.136.155]) by smtpout-01.galae.net (Postfix) with ESMTPS id 83FEB60634; Wed, 24 Sep 2025 14:50:41 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 58B7D102F1916; Wed, 24 Sep 2025 16:50:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=dkim; t=1758725440; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=oPEzeTnQ7Or1TuaZXMgQBye/vToi6RorrJA8q5MfX8w=; b=xgAxSahXYK8tzd2470KLTRxFZnW5OqM8zs3ibyWTQ1PBZP/AoO1iH+YGzVEmG92eQtDAQo KSqwOrTR8Et0nyBBM060kEDf9HceqlOMsFvsJOtMonFhjhp0qDmNyLYThfjKNq/dQH37oL NiUnDniBpLBijK4IEV9J0D1r9ETo7OfumAn8AOL4h+N9fJCq6wPLurDPGDP3s44biNrZBx uw6ZmEG/0LfC3XdNFCJHaucZEpmuoCV4LBWDCAbWZhRQoVOr/ufiiPNtUGY1UZO7L/NkBp wpq44H3WnvJs72ICzYkrpQ7eCWKRVzJ5qUlgx6uApZgw4Y0Qz/rlv8mziiIidA== From: "Bastien Curutchet (eBPF Foundation)" Date: Wed, 24 Sep 2025 16:49:50 +0200 Subject: [PATCH bpf-next v4 15/15] selftests/bpf: test_xsk: Integrate test_xsk.c to test_progs framework Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250924-xsk-v4-15-20e57537b876@bootlin.com> References: <20250924-xsk-v4-0-20e57537b876@bootlin.com> In-Reply-To: <20250924-xsk-v4-0-20e57537b876@bootlin.com> To: =?utf-8?q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Mykola Lysenko , Shuah Khan , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer Cc: Thomas Petazzoni , Alexis Lothore , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Bastien Curutchet (eBPF Foundation)" X-Mailer: b4 0.14.2 X-Last-TLS-Session-Version: TLSv1.3 test_xsk.c isn't part of the test_progs framework. Integrate the tests defined by test_xsk.c into the test_progs framework through a new file : prog_tests/xsk.c. ZeroCopy mode isn't tested in it as veth peers don't support it. Move test_xsk{.c/.h} to prog_tests/. Add the find_bit library to test_progs sources in the Makefile as it is is used by test_xsk.c Signed-off-by: Bastien Curutchet (eBPF Foundation) Reviewed-by: Maciej Fijalkowski --- tools/testing/selftests/bpf/Makefile | 13 +- .../selftests/bpf/{ =3D> prog_tests}/test_xsk.c | 0 .../selftests/bpf/{ =3D> prog_tests}/test_xsk.h | 0 tools/testing/selftests/bpf/prog_tests/xsk.c | 146 +++++++++++++++++= ++++ tools/testing/selftests/bpf/xskxceiver.c | 2 +- 5 files changed, 158 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests= /bpf/Makefile index ecd6f6fb540d968473227c770c6617f56257c7d8..ff2de16eafdade22c97c6a632bc= 200fb67e83b2f 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -543,6 +543,8 @@ TRUNNER_TEST_OBJS :=3D $$(patsubst %.c,$$(TRUNNER_OUTPU= T)/%.test.o, \ $$(notdir $$(wildcard $(TRUNNER_TESTS_DIR)/*.c))) TRUNNER_EXTRA_OBJS :=3D $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.o, \ $$(filter %.c,$(TRUNNER_EXTRA_SOURCES))) +TRUNNER_LIB_OBJS :=3D $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.o, \ + $$(filter %.c,$(TRUNNER_LIB_SOURCES))) TRUNNER_EXTRA_HDRS :=3D $$(filter %.h,$(TRUNNER_EXTRA_SOURCES)) TRUNNER_TESTS_HDR :=3D $(TRUNNER_TESTS_DIR)/tests.h TRUNNER_BPF_SRCS :=3D $$(notdir $$(wildcard $(TRUNNER_BPF_PROGS_DIR)/*.c)) @@ -686,6 +688,10 @@ $(TRUNNER_EXTRA_OBJS): $(TRUNNER_OUTPUT)/%.o: \ $$(call msg,EXT-OBJ,$(TRUNNER_BINARY),$$@) $(Q)$$(CC) $$(CFLAGS) -c $$< $$(LDLIBS) -o $$@ =20 +$(TRUNNER_LIB_OBJS): $(TRUNNER_OUTPUT)/%.o:$(TOOLSDIR)/lib/%.c + $$(call msg,LIB-OBJ,$(TRUNNER_BINARY),$$@) + $(Q)$$(CC) $$(CFLAGS) -c $$< $$(LDLIBS) -o $$@ + # non-flavored in-srctree builds receive special treatment, in particular,= we # do not need to copy extra resources (see e.g. test_btf_dump_case()) $(TRUNNER_BINARY)-extras: $(TRUNNER_EXTRA_FILES) | $(TRUNNER_OUTPUT) @@ -699,6 +705,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): | $(TRUNNER_BPF_OBJS) =20 $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(TRUNNER_EXTRA_OBJS) $$(BPFOBJ) \ + $(TRUNNER_LIB_OBJS) \ $(RESOLVE_BTFIDS) \ $(TRUNNER_BPFTOOL) \ $(OUTPUT)/veristat \ @@ -745,6 +752,7 @@ TRUNNER_EXTRA_SOURCES :=3D test_progs.c \ $(VERIFY_SIG_HDR) \ flow_dissector_load.h \ ip_check_defrag_frags.h +TRUNNER_LIB_SOURCES :=3D find_bit.c TRUNNER_EXTRA_FILES :=3D $(OUTPUT)/urandom_read \ $(OUTPUT)/liburandom_read.so \ $(OUTPUT)/xdp_synproxy \ @@ -782,6 +790,7 @@ endif TRUNNER_TESTS_DIR :=3D map_tests TRUNNER_BPF_PROGS_DIR :=3D progs TRUNNER_EXTRA_SOURCES :=3D test_maps.c +TRUNNER_LIB_SOURCES :=3D TRUNNER_EXTRA_FILES :=3D TRUNNER_BPF_BUILD_RULE :=3D $$(error no BPF objects should be built) TRUNNER_BPF_CFLAGS :=3D @@ -803,8 +812,8 @@ $(OUTPUT)/test_verifier: test_verifier.c verifier/tests= .h $(BPFOBJ) | $(OUTPUT) $(Q)$(CC) $(CFLAGS) $(filter %.a %.o %.c,$^) $(LDLIBS) -o $@ =20 # Include find_bit.c to compile xskxceiver. -EXTRA_SRC :=3D $(TOOLSDIR)/lib/find_bit.c -$(OUTPUT)/xskxceiver: $(EXTRA_SRC) test_xsk.c test_xsk.h xskxceiver.c xskx= ceiver.h $(OUTPUT)/network_helpers.o $(OUTPUT)/xsk.o $(OUTPUT)/xsk_xdp_prog= s.skel.h $(BPFOBJ) | $(OUTPUT) +EXTRA_SRC :=3D $(TOOLSDIR)/lib/find_bit.c prog_tests/test_xsk.c prog_tests= /test_xsk.h +$(OUTPUT)/xskxceiver: $(EXTRA_SRC) xskxceiver.c xskxceiver.h $(OUTPUT)/net= work_helpers.o $(OUTPUT)/xsk.o $(OUTPUT)/xsk_xdp_progs.skel.h $(BPFOBJ) | $= (OUTPUT) $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(filter %.a %.o %.c,$^) $(LDLIBS) -o $@ =20 diff --git a/tools/testing/selftests/bpf/test_xsk.c b/tools/testing/selftes= ts/bpf/prog_tests/test_xsk.c similarity index 100% rename from tools/testing/selftests/bpf/test_xsk.c rename to tools/testing/selftests/bpf/prog_tests/test_xsk.c diff --git a/tools/testing/selftests/bpf/test_xsk.h b/tools/testing/selftes= ts/bpf/prog_tests/test_xsk.h similarity index 100% rename from tools/testing/selftests/bpf/test_xsk.h rename to tools/testing/selftests/bpf/prog_tests/test_xsk.h diff --git a/tools/testing/selftests/bpf/prog_tests/xsk.c b/tools/testing/s= elftests/bpf/prog_tests/xsk.c new file mode 100644 index 0000000000000000000000000000000000000000..7ce5ddd7d3fc848df27534f00a6= a9f82fbc797c5 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/xsk.c @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +#include "network_helpers.h" +#include "test_progs.h" +#include "test_xsk.h" +#include "xsk_xdp_progs.skel.h" + +#define VETH_RX "veth0" +#define VETH_TX "veth1" +#define MTU 1500 + +int setup_veth(bool busy_poll) +{ + SYS(fail, + "ip link add %s numtxqueues 4 numrxqueues 4 type veth peer name %s numtxq= ueues 4 numrxqueues 4", + VETH_RX, VETH_TX); + SYS(fail, "sysctl -wq net.ipv6.conf.%s.disable_ipv6=3D1", VETH_RX); + SYS(fail, "sysctl -wq net.ipv6.conf.%s.disable_ipv6=3D1", VETH_TX); + + if (busy_poll) { + SYS(fail, "echo 2 > /sys/class/net/%s/napi_defer_hard_irqs", VETH_RX); + SYS(fail, "echo 200000 > /sys/class/net/%s/gro_flush_timeout", VETH_RX); + SYS(fail, "echo 2 > /sys/class/net/%s/napi_defer_hard_irqs", VETH_TX); + SYS(fail, "echo 200000 > /sys/class/net/%s/gro_flush_timeout", VETH_TX); + } + + SYS(fail, "ip link set %s mtu %d", VETH_RX, MTU); + SYS(fail, "ip link set %s mtu %d", VETH_TX, MTU); + SYS(fail, "ip link set %s up", VETH_RX); + SYS(fail, "ip link set %s up", VETH_TX); + + return 0; + +fail: + return -1; +} + +void delete_veth(void) +{ + SYS_NOFAIL("ip link del %s", VETH_RX); + SYS_NOFAIL("ip link del %s", VETH_TX); +} + +int configure_ifobj(struct ifobject *tx, struct ifobject *rx) +{ + rx->ifindex =3D if_nametoindex(VETH_RX); + if (!ASSERT_OK_FD(rx->ifindex, "get RX ifindex")) + return -1; + + tx->ifindex =3D if_nametoindex(VETH_TX); + if (!ASSERT_OK_FD(tx->ifindex, "get TX ifindex")) + return -1; + + tx->shared_umem =3D false; + rx->shared_umem =3D false; + + + return 0; +} + +static void test_xsk(const struct test_spec *test_to_run, enum test_mode m= ode) +{ + struct ifobject *ifobj_tx, *ifobj_rx; + struct test_spec test; + int ret; + + ifobj_tx =3D ifobject_create(); + if (!ASSERT_OK_PTR(ifobj_tx, "create ifobj_tx")) + return; + + ifobj_rx =3D ifobject_create(); + if (!ASSERT_OK_PTR(ifobj_rx, "create ifobj_rx")) + goto delete_tx; + + if (!ASSERT_OK(setup_veth(false), "setup veth")) + goto delete_rx; + + if (!ASSERT_OK(configure_ifobj(ifobj_tx, ifobj_rx), "conigure ifobj")) + goto delete_veth; + + ret =3D get_hw_ring_size(ifobj_tx->ifname, &ifobj_tx->ring); + if (!ret) { + ifobj_tx->hw_ring_size_supp =3D true; + ifobj_tx->set_ring.default_tx =3D ifobj_tx->ring.tx_pending; + ifobj_tx->set_ring.default_rx =3D ifobj_tx->ring.rx_pending; + } + + if (!ASSERT_OK(init_iface(ifobj_rx, worker_testapp_validate_rx), "init RX= ")) + goto delete_veth; + if (!ASSERT_OK(init_iface(ifobj_tx, worker_testapp_validate_tx), "init TX= ")) + goto delete_veth; + + test_init(&test, ifobj_tx, ifobj_rx, 0, &tests[0]); + + test.tx_pkt_stream_default =3D pkt_stream_generate(DEFAULT_PKT_CNT, MIN_P= KT_SIZE); + if (!ASSERT_OK_PTR(test.tx_pkt_stream_default, "TX pkt generation")) + goto delete_veth; + test.rx_pkt_stream_default =3D pkt_stream_generate(DEFAULT_PKT_CNT, MIN_P= KT_SIZE); + if (!ASSERT_OK_PTR(test.rx_pkt_stream_default, "RX pkt generation")) + goto delete_veth; + + + test_init(&test, ifobj_tx, ifobj_rx, mode, test_to_run); + ret =3D test.test_func(&test); + if (ret !=3D TEST_SKIP) + ASSERT_OK(ret, "Run test"); + pkt_stream_restore_default(&test); + + if (ifobj_tx->hw_ring_size_supp) + hw_ring_size_reset(ifobj_tx); + + pkt_stream_delete(test.tx_pkt_stream_default); + pkt_stream_delete(test.rx_pkt_stream_default); + xsk_xdp_progs__destroy(ifobj_tx->xdp_progs); + xsk_xdp_progs__destroy(ifobj_rx->xdp_progs); + +delete_veth: + delete_veth(); +delete_rx: + ifobject_delete(ifobj_rx); +delete_tx: + ifobject_delete(ifobj_tx); +} + +void test_ns_xsk_skb(void) +{ + int i; + + for (i =3D 0; i < ARRAY_SIZE(tests); i++) { + if (test__start_subtest(tests[i].name)) + test_xsk(&tests[i], TEST_MODE_SKB); + } +} + +void test_ns_xsk_drv(void) +{ + int i; + + for (i =3D 0; i < ARRAY_SIZE(tests); i++) { + if (test__start_subtest(tests[i].name)) + test_xsk(&tests[i], TEST_MODE_DRV); + } +} + diff --git a/tools/testing/selftests/bpf/xskxceiver.c b/tools/testing/selft= ests/bpf/xskxceiver.c index 8707f4a0fac64e1ebb6a4241edf8e874a1eb67c3..a54904783c757d282e3b99194aa= ed5f74d510763 100644 --- a/tools/testing/selftests/bpf/xskxceiver.c +++ b/tools/testing/selftests/bpf/xskxceiver.c @@ -90,7 +90,7 @@ #include #include =20 -#include "test_xsk.h" +#include "prog_tests/test_xsk.h" #include "xsk_xdp_progs.skel.h" #include "xsk.h" #include "xskxceiver.h" --=20 2.51.0