From nobody Fri Dec 19 09:29:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39D84C4167B for ; Thu, 7 Dec 2023 05:05:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230304AbjLGFEw (ORCPT ); Thu, 7 Dec 2023 00:04:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230264AbjLGFEr (ORCPT ); Thu, 7 Dec 2023 00:04:47 -0500 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96DEDD44 for ; Wed, 6 Dec 2023 21:04:53 -0800 (PST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5d3a1e5f8d6so3973157b3.3 for ; Wed, 06 Dec 2023 21:04:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701925493; x=1702530293; darn=vger.kernel.org; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :from:to:cc:subject:date:message-id:reply-to; bh=HLfF4gdeUWNr8FF8ACObQvFWnydI6Js6QIkQSQodKEI=; b=16JKr/OAlmIySJ2uZLXZC4p70yZiecAN8EhnkomFKLRuI8/eJ+HXfBmc9iI1Cq67rv 1T+ncKFZBzub0uuI67C/Rrf8+HL+kC1+1EqjdnqUxmJ3pi9L9/Rk2RPxBbsbizJ52Fkg 11JhkT1Z12L1LsPQQielsZdIbFLpz+B5rV8wxCJ7BEomwakbQlHHrx+ZFVgY/akc2lNG dafpmPc5KgqTa9V9dx9FVoqVmKL+WvcmeyWJhwIfhmjUc07tO+YB+PbDQrLwND2A+HNR oYA0PxhpM3VYtNN19I+CY83SNFgJ6ep0KgUE3vrs2pFRS3r2YQyxym5EvW8sJNcj1I6J WxCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701925493; x=1702530293; h=to:from:subject:references:mime-version:message-id:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HLfF4gdeUWNr8FF8ACObQvFWnydI6Js6QIkQSQodKEI=; b=iEhRYRed2mT5NVF2wk98XY4NYHbb1aEdx088aizpD8EvgIMRDJgkAJlhF2xfLB/Uvf hXYUaqT2xGEJBLKzb41prVzORAFu9fNMTNnWEl7rTOLKiJjQHaUpzXKbeQYcW36U4cVE 0KFiRQidsVRJqiGgG2nDpjr4joX+gNr3im7Jet09Rk3o//xNAKbDHz8zYeMTSuk3V2h8 OqQ/LJPwTteFXKpUIYeWrjVeo/d0nTrJTkcvckycbFMRbexySW+JGndSwF1EqmegLWbw DNIVFHDnFjR6NAAEomClZ3FzZ5SLAjgDBb/UXNJFozjREwSgb6Jj/zvBuJp4gIAPCy8n HMDQ== X-Gm-Message-State: AOJu0YwNce6hrN376jGGUQUkvO/EjI7Jq+IWqbDn+BbtSKX+D1yyYcvj Yh/dlHSM7oB0x6diEVLjH5igrTGRPypa X-Google-Smtp-Source: AGHT+IFJ8VLzkhmv5lIHBwHao+8hbxbTzxh0PLIX/Y/8B0uG5Cn/P5rXMoDTUFtOLFwHal2ivb+PmHh5k+sj X-Received: from irogers.svl.corp.google.com ([2620:15c:2a3:200:35bf:293e:7696:34e9]) (user=irogers job=sendgmr) by 2002:a81:ad14:0:b0:5d5:6a4c:b0b3 with SMTP id l20-20020a81ad14000000b005d56a4cb0b3mr26965ywh.8.1701925492804; Wed, 06 Dec 2023 21:04:52 -0800 (PST) Date: Wed, 6 Dec 2023 21:04:29 -0800 In-Reply-To: <20231207050433.1426834-1-irogers@google.com> Message-Id: <20231207050433.1426834-3-irogers@google.com> Mime-Version: 1.0 References: <20231207050433.1426834-1-irogers@google.com> X-Mailer: git-send-email 2.43.0.rc2.451.g8631bc7472-goog Subject: [PATCH v1 2/6] tools lib api: Add io_dir an allocation free readdir alternative From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , "Masami Hiramatsu (Google)" , Thomas Richter , Quentin Monnet , Fangrui Song , Kan Liang , Yanteng Si , K Prateek Nayak , Artem Savkov , James Clark , Ravi Bangoria , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" glibc's opendir allocates a minimum of 32kb, when called recursively for a directory tree the memory consumption can add up - nearly 300kb during perf start-up when processing modules. Add a stack allocated variant of readdir sized a little more than 1kb. As getdents64 may be missing from libc, add support using syscall. Signed-off-by: Ian Rogers --- tools/lib/api/Makefile | 2 +- tools/lib/api/io_dir.h | 84 ++++++++++++++++++++++++++++++++++++++ tools/perf/Makefile.config | 4 ++ 3 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 tools/lib/api/io_dir.h diff --git a/tools/lib/api/Makefile b/tools/lib/api/Makefile index 044860ac1ed1..186aa407de8c 100644 --- a/tools/lib/api/Makefile +++ b/tools/lib/api/Makefile @@ -99,7 +99,7 @@ install_lib: $(LIBFILE) $(call do_install_mkdir,$(libdir_SQ)); \ cp -fpR $(LIBFILE) $(DESTDIR)$(libdir_SQ) =20 -HDRS :=3D cpu.h debug.h io.h +HDRS :=3D cpu.h debug.h io.h io_dir.h FD_HDRS :=3D fd/array.h FS_HDRS :=3D fs/fs.h fs/tracing_path.h INSTALL_HDRS_PFX :=3D $(DESTDIR)$(prefix)/include/api diff --git a/tools/lib/api/io_dir.h b/tools/lib/api/io_dir.h new file mode 100644 index 000000000000..9b702497e05c --- /dev/null +++ b/tools/lib/api/io_dir.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +/* + * Lightweight directory reading library. + */ +#ifndef __API_IO_DIR__ +#define __API_IO_DIR__ + +#include +#include +#include +#include +#include + +#ifndef HAVE_GETDENTS64 +#include + +static inline ssize_t getdents64(int fd, void *dirp, size_t count) +{ + return syscall(SYS_getdents64, fd, dirp, count); +} +#endif + +struct io_dirent64 { + ino64_t d_ino; /* 64-bit inode number */ + off64_t d_off; /* 64-bit offset to next structure */ + unsigned short d_reclen; /* Size of this dirent */ + unsigned char d_type; /* File type */ + char d_name[NAME_MAX + 1]; /* Filename (null-terminated) */ +}; + +struct io_dir { + int dirfd; + ssize_t available_bytes; + struct io_dirent64 *next; + struct io_dirent64 buff[4]; +}; + +static inline void io_dir__init(struct io_dir *iod, int dirfd) +{ + iod->dirfd =3D dirfd; + iod->available_bytes =3D 0; +} + +static inline void io_dir__rewinddir(struct io_dir *iod) +{ + lseek(iod->dirfd, 0, SEEK_SET); + iod->available_bytes =3D 0; +} + +static inline struct io_dirent64 *io_dir__readdir(struct io_dir *iod) +{ + struct io_dirent64 *entry; + + if (iod->available_bytes <=3D 0) { + ssize_t rc =3D getdents64(iod->dirfd, iod->buff, sizeof(iod->buff)); + + if (rc <=3D 0) + return NULL; + iod->available_bytes =3D rc; + iod->next =3D iod->buff; + } + entry =3D iod->next; + iod->next =3D (struct io_dirent64 *)((char *)entry + entry->d_reclen); + iod->available_bytes -=3D entry->d_reclen; + return entry; +} + +static inline bool io_dir__is_dir(const struct io_dir *iod, struct io_dire= nt64 *dent) +{ + if (dent->d_type =3D=3D DT_UNKNOWN) { + struct stat st; + + if (fstatat(iod->dirfd, dent->d_name, &st, /*flags=3D*/0)) + return false; + + if (S_ISDIR(st.st_mode)) { + dent->d_type =3D DT_DIR; + return true; + } + } + return dent->d_type =3D=3D DT_DIR; +} + +#endif diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config index aa55850fbc21..1cef1ab4ddb7 100644 --- a/tools/perf/Makefile.config +++ b/tools/perf/Makefile.config @@ -357,6 +357,10 @@ ifeq ($(feature-stackprotector-all), 1) CORE_CFLAGS +=3D -fstack-protector-all endif =20 +ifeq ($(feature-getdents64), 1) + CFLAGS +=3D -DHAVE_GETDENTS64 +endif + ifeq ($(DEBUG),0) ifeq ($(feature-fortify-source), 1) CORE_CFLAGS +=3D -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3D2 --=20 2.43.0.rc2.451.g8631bc7472-goog