From nobody Sat Sep 26 01:05:00 2026 Received: from out-utut-a193.jellyfish.systems (out-utut-a193.jellyfish.systems [198.177.127.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1237435A93C for ; Sun, 6 Sep 2026 17:08:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.177.127.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788714540; cv=none; b=JVeoNB66KjuhPiBAMjBZIYojIqwJTURkRzilNxWRFQQ2HXsFvD5SPsQSQF7DO+V94CUuc+wOLVcMTM71LAPcQzXf3fMDEzI5HkZlpK0h9NOe32DuU+logKbZF1DYe1IuAraFaQiHMCOA5iosW68V4yh4xP0YX9lcRsqce+5xY4I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788714540; c=relaxed/simple; bh=dJy8K7QpTEAQGyMtp7rRVTUQldhttr9jIZVdtHql6D8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nBuqp91K0RNYSkgnVS0jyfGoA00nDr4TkjQPwE/hQDg9nyq3RO0u8yRPy8wAYrRk8p02VOxyh6N3/eECnO462VL0hFYrzW+VHBtMITEDMxiyr2BmcV/bgVicBZiDl5jSbdYY6E2R6BEsyWoaoNWVJYBxJZvQWQRIrZyupojlKTY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=tychen.cc; spf=pass smtp.mailfrom=tychen.cc; dkim=pass (2048-bit key) header.d=tychen.cc header.i=@tychen.cc header.b=d3QRuOHM; arc=none smtp.client-ip=198.177.127.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=tychen.cc Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=tychen.cc Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=tychen.cc header.i=@tychen.cc header.b="d3QRuOHM" Received: from fedora (unknown [69.5.53.41]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.spacemail.com (Postfix) with ESMTPSA id 4hdGr41rpLz8sXJ; Sun, 06 Sep 2026 17:08:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tychen.cc; s=spacemail; t=1788714523; bh=pEaJ0IworLq94kzj0A5eN/+Ym81HOT70PvZyiKxaOOI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d3QRuOHM0NXOvHlwAaAO95wMHLjxSgm0qPB5e2lxiCi7mN3VQl2Xk6rCWz6q2+967 MKqR/1jQ7jo3nmkL+v2wvoko9roMg3eytXJBqHRl9+qVar4c/nAfSIl4/hf4GmJK5M XmQ7ET70TTMjVJ+HuwEdKdt+4xMBER7Sr8rixLga1LKk3NGHJefE2b6zerGRZZHUg9 zCkH03HTRUHocKo5hBLGhtNFAWRqk8Og4jLT3Il6DGc04wvn9cwzAhBWPWCcnErL30 ho2zFG1urzQz5LnfDujGOdjdWpuC8Sxp8iBwbC6SKOiFfsXcRj73tQP43Hg51YW2BS nO505y0Q1AW5Q== From: Tianyi Chen To: bpf@vger.kernel.org Cc: Tianyi Chen , Quentin Monnet , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Eduard Zingerman , Kumar Kartikeya Dwivedi , Martin KaFai Lau , Song Liu , Yonghong Song , Jiri Olsa , Emil Tsalapatis , Ihor Solodrai , Shuah Khan , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [PATCH bpf-next 1/2] bpftool: Use batch lookups for bounded hash map dumps Date: Mon, 7 Sep 2026 01:08:18 +0800 Message-ID: <20260906170819.1212661-2-hi@tychen.cc> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260906170819.1212661-1-hi@tychen.cc> References: <20260906170819.1212661-1-hi@tychen.cc> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Envelope-From: hi@tychen.cc Content-Type: text/plain; charset="utf-8" Use BPF_MAP_LOOKUP_BATCH when dumping hash maps to reduce the number of BPF syscalls. Share element formatting with individual lookups to preserve plain, JSON and BTF output. Start with up to 256 entries and grow on ENOSPC without advancing the input cursor. Restrict the optimization to maps whose maximum key and value storage fits in 4 MiB, so even a worst-case bucket can fit without restarting a partially printed dump. Preserve aligned element buffers for formatting keys and values with odd sizes. Fall back to individual lookups only if the initial batch operation is unsupported. Process the final partial batch on ENOENT, but never use count or output buffers after other errors. Report errors after batch traversal starts without restarting and duplicating output. Assisted-by: Codex:GPT-6 Signed-off-by: Tianyi Chen --- tools/bpf/bpftool/map.c | 113 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 105 insertions(+), 8 deletions(-) diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index 684a8fb7241..1eccdc0d1e9 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -740,15 +740,10 @@ static int do_show(int argc, char **argv) return errno =3D=3D ENOENT ? 0 : -1; } =20 -static int dump_map_elem(int fd, void *key, void *value, - struct bpf_map_info *map_info, struct btf *btf, - json_writer_t *btf_wtr) +static void print_map_elem(void *key, void *value, + struct bpf_map_info *map_info, struct btf *btf, + json_writer_t *btf_wtr) { - if (bpf_map_lookup_elem(fd, key, value)) { - print_entry_error(map_info, key, errno); - return -1; - } - if (json_output) { print_entry_json(map_info, key, value, btf); } else if (btf) { @@ -762,10 +757,108 @@ static int dump_map_elem(int fd, void *key, void *va= lue, } else { print_entry_plain(map_info, key, value); } +} + +static int dump_map_elem(int fd, void *key, void *value, + struct bpf_map_info *map_info, struct btf *btf, + json_writer_t *btf_wtr) +{ + if (bpf_map_lookup_elem(fd, key, value)) { + print_entry_error(map_info, key, errno); + return -1; + } =20 + print_map_elem(key, value, map_info, btf, btf_wtr); return 0; } =20 +#define MAP_DUMP_BATCH_SIZE 256U +#define MAP_DUMP_BATCH_MAX_BYTES (4 * 1024 * 1024) + +/* Return 1 to use individual lookups, but only before batch traversal sta= rts. */ +static int dump_map_batch(int fd, void *key, void *value, + struct bpf_map_info *info, struct btf *btf, + json_writer_t *wtr, unsigned int *num_elems) +{ + __u32 capacity, count, batch =3D 0, next_batch =3D 0, i; + void *keys =3D NULL, *values =3D NULL, *buf; + bool first =3D true, can_fallback =3D true; + int err; + + /* Hash lookup batches must accommodate a whole bucket. Restrict the + * optimization to maps whose worst-case bucket fits the memory budget, + * so a later ENOSPC never forces a restart after printing some entries. + * Division also bounds the allocation multiplications on 32-bit hosts. + */ + if (info->type !=3D BPF_MAP_TYPE_HASH || !info->max_entries || + (__u64)info->key_size + info->value_size > + MAP_DUMP_BATCH_MAX_BYTES / info->max_entries) + return 1; + + capacity =3D min(info->max_entries, MAP_DUMP_BATCH_SIZE); +resize: + buf =3D realloc(keys, (size_t)capacity * info->key_size); + if (!buf) { + err =3D ENOMEM; + goto error; + } + keys =3D buf; + buf =3D realloc(values, (size_t)capacity * info->value_size); + if (!buf) { + err =3D ENOMEM; + goto error; + } + values =3D buf; + + while (true) { + count =3D capacity; + err =3D bpf_map_lookup_batch(fd, first ? NULL : &batch, + &next_batch, keys, values, &count, NULL); + err =3D err ? errno : 0; + /* Older kernels reject the command before updating count. Do not + * inspect the buffers on these errors, or fall back after progress. + */ + if (can_fallback && (err =3D=3D EINVAL || err =3D=3D EOPNOTSUPP || + err =3D=3D 524 /* ENOTSUPP */)) { + err =3D 1; + goto out; + } + can_fallback =3D false; + if (err =3D=3D ENOSPC) { + if (capacity =3D=3D info->max_entries) + goto error; + capacity +=3D min(capacity, info->max_entries - capacity); + /* Preserve the input cursor: the oversized bucket was not read. */ + goto resize; + } + /* In particular, EFAULT can leave count and the buffers invalid. */ + if (err && err !=3D ENOENT) + goto error; + for (i =3D 0; i < count; i++) { + /* Keep the alignment provided by individual lookups, including + * for BTF types whose map key/value size is not aligned. + */ + memcpy(key, keys + (size_t)i * info->key_size, info->key_size); + memcpy(value, values + (size_t)i * info->value_size, info->value_size); + print_map_elem(key, value, info, btf, wtr); + (*num_elems)++; + } + if (err =3D=3D ENOENT) { + err =3D 0; + goto out; + } + first =3D false; + batch =3D next_batch; + } +error: + p_err("can't lookup map batch: %s", strerror(err)); + err =3D -1; +out: + free(keys); + free(values); + return err; +} + static int maps_have_btf(int *fds, int nb_fds) { struct bpf_map_info info =3D {}; @@ -869,6 +962,9 @@ map_dump(int fd, struct bpf_map_info *info, json_writer= _t *wtr, p_info("Warning: cannot read values from %s map with value_size !=3D 8", map_type_str); } + err =3D dump_map_batch(fd, key, value, info, btf, wtr, &num_elems); + if (err !=3D 1) + goto end_dump; while (true) { err =3D bpf_map_get_next_key(fd, prev_key, key); if (err) { @@ -881,6 +977,7 @@ map_dump(int fd, struct bpf_map_info *info, json_writer= _t *wtr, prev_key =3D key; } =20 +end_dump: if (wtr) { jsonw_end_array(wtr); /* elements */ if (show_header) --=20 2.55.0 From nobody Sat Sep 26 01:05:00 2026 Received: from out-utut-a220.jellyfish.systems (out-utut-a220.jellyfish.systems [198.177.127.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65B8036167F for ; Sun, 6 Sep 2026 17:08:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.177.127.220 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788714539; cv=none; b=Bz8pZ6S6FO3+9V9PtDjXN1bqefpYvT2TwciYE5iy5M+ADHxvsC2OPzmv2nBO7/Bm+4ADqcrbjPx06YIxL5EGOLubjec43/aNRGzkIQCkT4PRGmXWQHtDzRhURB1ghOAeqCdo3hKPtVu/n6yaI0ruoJOyIs3peD9YvmBbWRKHhog= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788714539; c=relaxed/simple; bh=4ps7r/6jj3ma8R0P6CV2XFY5qF3oZ+AM5Lmlft6RKv4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cIGH1PDd31b9VnXLzr+Qr+I5EBHd44iiTu0EwFvDD8SzK3efsgva6XX3gvZMxQikCrl4vmLaQ3eNNf49PhqkgbGqitGBl/XpAqpRkO9vqGzkU/gv1ZjcN7l/xTVJGOriw5mgNxb46xvsRFtz6xzXgrKm7h1Y6EsXMefXBO55ghk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=tychen.cc; spf=pass smtp.mailfrom=tychen.cc; dkim=pass (2048-bit key) header.d=tychen.cc header.i=@tychen.cc header.b=HpK5/WeN; arc=none smtp.client-ip=198.177.127.220 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=tychen.cc Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=tychen.cc Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=tychen.cc header.i=@tychen.cc header.b="HpK5/WeN" Received: from fedora (unknown [69.5.53.41]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.spacemail.com (Postfix) with ESMTPSA id 4hdGrC5Fx6z8sXJ; Sun, 06 Sep 2026 17:08:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tychen.cc; s=spacemail; t=1788714530; bh=13sUXRu+h1QQZIBRc6w1+PeJXdnRzfbybdudh1z1K1A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HpK5/WeNsrVqTWsHFKvc2ybhQhuNOUGjDRDZPxC+iS3zBL6h/iOov2VgOxkNNytrq sNlqLmTdIzF54l9tTqMIKkpoxTTm1Mo6UQiLE7YoD2Sr0VbrPOjz8uyCCD8GU+71ti XdXHkTX/4URgycXDrKY/MAcld0RHLQfYqn8BrvyuZp5E3GvwmRFN9m+fSnxNWYHq1O 6jHfRpDFOSxJbSkVP0rwbZSgnh3H1Kbq6fZX47ZZVPln1pr/a1QMZhxB9l19tvOeHR DfwWyVvCg/57ckohuPQRpvKKRJneueIRWci1QEJXAkNKKDRJCQgZmwXQDL5/UO5CMv Ga5QlpJEXOAAQ== From: Tianyi Chen To: bpf@vger.kernel.org Cc: Tianyi Chen , Quentin Monnet , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Eduard Zingerman , Kumar Kartikeya Dwivedi , Martin KaFai Lau , Song Liu , Yonghong Song , Jiri Olsa , Emil Tsalapatis , Ihor Solodrai , Shuah Khan , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents Date: Mon, 7 Sep 2026 01:08:19 +0800 Message-ID: <20260906170819.1212661-3-hi@tychen.cc> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260906170819.1212661-1-hi@tychen.cc> References: <20260906170819.1212661-1-hi@tychen.cc> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Envelope-From: hi@tychen.cc Content-Type: text/plain; charset="utf-8" Exercise hash map dumps around the initial batch size and across multiple batches, including empty and single-entry maps. Compare each complete unordered key/value set with the input data in plain, JSON and pretty JSON output. Cover one-byte keys, three-byte values and BTF-formatted maps to catch cursor sizing, buffer alignment and formatting regressions. Assisted-by: Codex:GPT-6 Signed-off-by: Tianyi Chen --- .../bpf/prog_tests/bpftool_map_batch.c | 186 ++++++++++++++++++ 1 file changed, 186 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_map_batc= h.c diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/t= ools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c new file mode 100644 index 00000000000..139d13a49dc --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include + +#define MAX_ENTRIES 1025 +#define RECORD_SIZE 256 +#define OUTPUT_SIZE (MAX_ENTRIES * RECORD_SIZE + 1024) + +struct dump_case { + const char *name; + unsigned int count; + unsigned int key_size; + unsigned int value_size; + bool btf; +}; + +static void hex_bytes(char *out, const void *data, unsigned int size, bool= json) +{ + const unsigned char *bytes =3D data; + unsigned int i; + + if (json) + *out++ =3D '['; + for (i =3D 0; i < size; i++) { + if (json && i) + *out++ =3D ','; + out +=3D sprintf(out, json ? "\"0x%02x\"" : "%02x", bytes[i]); + } + if (json) + *out++ =3D ']'; + *out =3D '\0'; +} + +static void expected_record(char *record, const struct dump_case *test, + unsigned int index, bool json) +{ + __u32 key =3D index, value =3D index * 37 + 11; + unsigned char short_key =3D index; + char key_hex[64], value_hex[64], formatted[96]; + + hex_bytes(key_hex, test->key_size =3D=3D 1 ? (void *)&short_key : &key, + test->key_size, json); + hex_bytes(value_hex, &value, test->value_size, json); + snprintf(formatted, sizeof(formatted), "{\"key\":%u,\"value\":%u}", + key, value); + if (json && test->btf) + snprintf(record, RECORD_SIZE, + "{\"key\":%s,\"value\":%s,\"formatted\":%s}", + key_hex, value_hex, formatted); + else if (json) + snprintf(record, RECORD_SIZE, "{\"key\":%s,\"value\":%s}", + key_hex, value_hex); + else if (test->btf) + snprintf(record, RECORD_SIZE, "%s", formatted); + else + snprintf(record, RECORD_SIZE, "key:%svalue:%s", key_hex, value_hex); +} + +static void check_dump(const struct dump_case *test, __u32 id, bool json, = bool pretty) +{ + bool array =3D json || test->btf; + char command[MAX_BPFTOOL_CMD_LEN], expected[RECORD_SIZE], footer[64]; + bool seen[MAX_ENTRIES] =3D {}; + char *output, *src, *dst, *cursor; + unsigned int i, n; + int err; + + output =3D calloc(1, OUTPUT_SIZE); + if (!ASSERT_OK_PTR(output, "alloc_output")) + return; + snprintf(command, sizeof(command), "%smap dump id %u", + pretty ? "-p " : json ? "-j " : "", id); + err =3D get_bpftool_command_output(command, output, OUTPUT_SIZE); + if (!ASSERT_OK(err, "map_dump")) + goto out; + /* Ignore presentation whitespace, but compare complete records and all + * punctuation. Expected contents come only from the input data, never + * from another map walk or bpftool invocation. + */ + for (src =3D output, dst =3D output; *src; src++) + if (!isspace((unsigned char)*src)) + *dst++ =3D *src; + *dst =3D '\0'; + cursor =3D output; + if (array) { + if (!ASSERT_EQ(*cursor, '[', "array_start")) + goto out; + cursor++; + } + for (n =3D 0; n < test->count; n++) { + if (array && n) { + if (!ASSERT_EQ(*cursor, ',', "record_separator")) + goto out; + cursor++; + } + for (i =3D 0; i < test->count; i++) { + if (seen[i]) + continue; + expected_record(expected, test, i, json); + if (!strncmp(cursor, expected, strlen(expected))) + break; + } + if (!ASSERT_LT(i, test->count, "unique_expected_record")) + goto out; + seen[i] =3D true; + cursor +=3D strlen(expected); + } + if (array) { + ASSERT_STREQ(cursor, "]", "array_end_and_count"); + } else { + snprintf(footer, sizeof(footer), "Found%uelement%s", test->count, + test->count =3D=3D 1 ? "" : "s"); + ASSERT_STREQ(cursor, footer, "plain_count"); + } +out: + free(output); +} + +static void run_dump_case(const struct dump_case *test) +{ + LIBBPF_OPTS(bpf_map_create_opts, opts); + struct bpf_map_info info =3D {}; + __u32 info_len =3D sizeof(info); + struct btf *btf =3D NULL; + unsigned int i; + int fd =3D -1; + + if (test->btf) { + btf =3D btf__new_empty(); + if (!ASSERT_OK_PTR(btf, "btf_new")) + return; + if (!ASSERT_EQ(btf__add_int(btf, "unsigned int", 4, 0), 1, + "btf_int") || + !ASSERT_OK(btf__load_into_kernel(btf), "btf_load")) + goto out; + opts.btf_fd =3D btf__fd(btf); + opts.btf_key_type_id =3D 1; + opts.btf_value_type_id =3D 1; + } + fd =3D bpf_map_create(BPF_MAP_TYPE_HASH, "dump_batch", test->key_size, + test->value_size, test->count ?: 1, &opts); + if (!ASSERT_OK_FD(fd, "map_create")) + goto out; + for (i =3D 0; i < test->count; i++) { + __u32 key =3D i, value =3D i * 37 + 11; + unsigned char short_key =3D i; + void *key_ptr =3D test->key_size =3D=3D 1 ? (void *)&short_key : &key; + + if (!ASSERT_OK(bpf_map_update_elem(fd, key_ptr, &value, BPF_ANY), + "map_update")) + goto out; + } + if (!ASSERT_OK(bpf_map_get_info_by_fd(fd, &info, &info_len), "map_info")) + goto out; + check_dump(test, info.id, false, false); + check_dump(test, info.id, true, false); + check_dump(test, info.id, true, true); +out: + if (fd >=3D 0) + close(fd); + btf__free(btf); +} + +void test_bpftool_map_batch(void) +{ + static const struct dump_case cases[] =3D { + { "empty", 0, 4, 4 }, + { "single", 1, 4, 4 }, + { "below_batch", 255, 4, 4 }, + { "exact_batch", 256, 4, 4 }, + { "above_batch", 257, 4, 4 }, + { "multiple_batches", 1025, 4, 4 }, + { "one_byte_key", 256, 1, 4 }, + { "odd_value_size", 257, 4, 3 }, + { "btf_empty", 0, 4, 4, true }, + { "btf_single", 1, 4, 4, true }, + { "btf_multiple_batches", 1025, 4, 4, true }, + }; + unsigned int i; + + for (i =3D 0; i < ARRAY_SIZE(cases); i++) + if (test__start_subtest(cases[i].name)) + run_dump_case(&cases[i]); +} --=20 2.55.0