From nobody Thu Sep 24 16:07:10 2026 Received: from mta0.migadu.com (out-19.mta0.migadu.com [91.218.175.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95DD652F294 for ; Tue, 22 Sep 2026 10:19:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790072359; cv=none; b=jNRLT0npuODyt8JPRI2Y+nz2fzJNoTwrwinLRT6hY11iofWQ68ghISR3E8codPfrjyFAWK6oflWsYtlEITjioFdRoCWI7vW+G4iQtkSvftmJEbMyWhcek4d2C6LUhJrsLX5nxdLHT4Xt89gyIUYzX9k1FshniJlV4qG+MB0Rc68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790072359; c=relaxed/simple; bh=1UxtsbzBrTGi5DkzONXT8zvBmsTaML+Fc5YEC45PMdQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aD03xo8bnGPGK6E/9awFTCcVNv+7DGj41XkxaaS0nkg0ESlPZCAPZP6QhjikMIhn9Yj4dDBkkKLcwgNdl4zEJQ50A71PTr6gDNUi9ohb1m3hjRQb95rqzPrvJEw2BJ1cN+IaYpxKuPIvIH/9WTV6DT1UFPhqxmbcjIgG7i5y+34= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=iVIwc91B; arc=none smtp.client-ip=91.218.175.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="iVIwc91B" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=1UxtsbzBrTGi5DkzONXT8zvBmsTaML+Fc5YEC45PMdQ=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790072354; v=1; x=1790677154; b=iVIwc91B+5PgcaN7U2oDoakSW6z6mDykBrqgUM67SFlpPRk8ENzdGVvNivPPeBD6PNxxTTw/ aMRHWDrBtoAHeBSerDdzhddR6eo8oG68EMCUVfAX1NsHxXiOga3aMgZj/sVswWJ/hebTo8Qnrpc sKEL4RUiiV9yUc88uc5Vo5Dk= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id 80c0348aef9e1eb5; Tue, 22 Sep 2026 10:19:14 +0000 X-Mizu-Trace-ID: 80c0348aef9e1eb5 X-Migadu-Flow: FLOW_OUT From: Jiayuan Chen To: bpf@vger.kernel.org Cc: Jiayuan Chen , Emil Tsalapatis , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Eduard Zingerman , Kumar Kartikeya Dwivedi , Martin KaFai Lau , Song Liu , Yonghong Song , Jiri Olsa , Ihor Solodrai , Shuah Khan , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [PATCH bpf-next v9 1/3] bpf: arena: allocate the fault-in page outside the lock Date: Tue, 22 Sep 2026 18:17:33 +0800 Message-ID: <20260922101831.192102-2-jiayuan.chen@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260922101831.192102-1-jiayuan.chen@linux.dev> References: <20260922101831.192102-1-jiayuan.chen@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" arena_vm_fault() allocated the page while holding arena->spinlock, so it could only use the non-blocking allocator, which never reclaims. Once the memcg is at memory.max that allocation just fails, the fault turns into VM_FAULT_SIGSEGV, and the process gets a SIGSEGV on a perfectly valid arena address. Hitting memory.max is routine (e.g. page cache from reading a big file), so this kills innocent processes over memory that reclaim could have freed. Rework the fault handler: - Preallocate the page before taking the lock, like do_anonymous_page() does, so it can sleep and reclaim, instead of turning a routine memory.max into a fake segfault. The allocation uses __GFP_RETRY_MAYFAIL so it never invokes the OOM killer: the page is charged to the map's memcg, which need not be the faulting task's, so an OOM there could kill unrelated tasks in the map's cgroup while a foreign faulter could never be its victim. map->numa_node is always NUMA_NO_NODE for an arena (BPF_F_NUMA_NODE is rejected), so the page comes from the local node, as before. - On allocation failure fall through to the locked recheck rather than failing right away: a page a concurrent allocator installed meanwhile is used, otherwise the non-blocking fallback fails and we return VM_FAULT_SIGBUS. Not VM_FAULT_OOM: nothing ran the OOM killer, and the fault path would just retry it forever. - A lockless probe skips that preallocation when a page is already mapped (e.g. allocated by the bpf program), so the common case wastes no allocation. The rare race where such a page is freed before we take the lock falls back to the non-blocking allocator under the lock. - Return VM_FAULT_SIGBUS for the other non-recoverable errors (allocation, range-tree and page-table failures) instead of VM_FAULT_SIGSEGV; the lock failure already did. Only BPF_F_SEGV_ON_FAULT, and a scratch-page hole under that flag, is a real user addressing error and keeps VM_FAULT_SIGSEGV. - Tidy up the error labels. Reviewed-by: Emil Tsalapatis Signed-off-by: Jiayuan Chen --- To sashiko: - The non-blocking fallback under the lock still zeroes: alloc_pages_nolock() forces __GFP_ZERO internally and only accepts __GFP_ACCOUNT. - __GFP_ZERO matches the existing __bpf_alloc_page(), and no arena-capable arch has D-cache aliasing, so there is no dcache concern. --- kernel/bpf/arena.c | 83 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 63 insertions(+), 20 deletions(-) diff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c index 7b6847200b431..c6369ea5e2082 100644 --- a/kernel/bpf/arena.c +++ b/kernel/bpf/arena.c @@ -481,7 +481,8 @@ static vm_fault_t arena_vm_fault(struct vm_fault *vmf) struct bpf_map *map =3D vmf->vma->vm_file->private_data; struct bpf_arena *arena =3D container_of(map, struct bpf_arena, map); struct mem_cgroup *new_memcg, *old_memcg; - struct page *page; + struct page *page, *new_page =3D NULL; + vm_fault_t fault_ret; long kbase, kaddr; unsigned long flags; int ret; @@ -489,59 +490,101 @@ static vm_fault_t arena_vm_fault(struct vm_fault *vm= f) kbase =3D bpf_arena_get_kern_vm_start(arena); kaddr =3D kbase + (u32)(vmf->address); =20 - if (raw_res_spin_lock_irqsave(&arena->spinlock, flags)) + page =3D vmalloc_to_page((void *)kaddr); + if (!page && !(arena->map.map_flags & BPF_F_SEGV_ON_FAULT)) { + /* + * Preallocate outside the lock so the allocation can reclaim; + * __GFP_RETRY_MAYFAIL keeps the OOM killer out of it. + */ + bpf_map_memcg_enter(&arena->map, &old_memcg, &new_memcg); + new_page =3D alloc_pages_node(map->numa_node, + GFP_KERNEL | __GFP_ZERO | + __GFP_ACCOUNT | __GFP_NOWARN | + __GFP_RETRY_MAYFAIL, 0); + bpf_map_memcg_exit(old_memcg, new_memcg); + } + + if (raw_res_spin_lock_irqsave(&arena->spinlock, flags)) { /* * A failed lock means a possible deadlock was detected. Don't * return VM_FAULT_RETRY: this handler never took mmap_lock, but * the fault path would re-take it on retry and deadlock. Fail. */ + if (new_page) + free_pages_nolock(new_page, 0); return VM_FAULT_SIGBUS; + } =20 page =3D vmalloc_to_page((void *)kaddr); if (page) { - if (page =3D=3D arena->scratch_page) + if (page =3D=3D arena->scratch_page) { /* BPF triggered scratch here; don't lazy-alloc over it */ - goto out_sigsegv; + fault_ret =3D (arena->map.map_flags & BPF_F_SEGV_ON_FAULT) + ? VM_FAULT_SIGSEGV : VM_FAULT_SIGBUS; + goto out_err_locked; + } /* already have a page vmap-ed */ goto out; } =20 + if (arena->map.map_flags & BPF_F_SEGV_ON_FAULT) { + /* + * User space requested to segfault when page is not allocated + * by bpf prog + */ + fault_ret =3D VM_FAULT_SIGSEGV; + goto out_err_locked; + } + bpf_map_memcg_enter(&arena->map, &old_memcg, &new_memcg); =20 - if (arena->map.map_flags & BPF_F_SEGV_ON_FAULT) - /* User space requested to segfault when page is not allocated by bpf pr= og */ - goto out_sigsegv_memcg; + if (!new_page) { + /* + * The probed page was freed meanwhile or preallocation failed; + * try the non-blocking allocator, we cannot sleep here. + */ + ret =3D bpf_map_alloc_pages(map, map->numa_node, 1, &new_page); + if (ret) { + fault_ret =3D VM_FAULT_SIGBUS; + goto out_err_locked_memcg; + } + } =20 ret =3D range_tree_clear(&arena->rt, vmf->pgoff, 1); - if (ret) - goto out_sigsegv_memcg; - - struct apply_range_data data =3D { .arena =3D arena, .pages =3D &page, .i= =3D 0 }; - /* Account into memcg of the process that created bpf_arena */ - ret =3D bpf_map_alloc_pages(map, NUMA_NO_NODE, 1, &page); if (ret) { - range_tree_set(&arena->rt, vmf->pgoff, 1); - goto out_sigsegv_memcg; + fault_ret =3D VM_FAULT_SIGBUS; + goto out_err_locked_memcg; } + struct apply_range_data data =3D { + .arena =3D arena, .pages =3D &new_page, .i =3D 0 + }; =20 ret =3D apply_to_page_range(&init_mm, kaddr, PAGE_SIZE, apply_range_set_c= b, &data); if (ret) { range_tree_set(&arena->rt, vmf->pgoff, 1); - free_pages_nolock(page, 0); - goto out_sigsegv_memcg; + fault_ret =3D VM_FAULT_SIGBUS; + goto out_err_locked_memcg; } flush_vmap_cache(kaddr, PAGE_SIZE); bpf_map_memcg_exit(old_memcg, new_memcg); + /* new_page was consumed */ + page =3D new_page; + new_page =3D NULL; out: page_ref_add(page, 1); raw_res_spin_unlock_irqrestore(&arena->spinlock, flags); + if (new_page) + free_pages_nolock(new_page, 0); vmf->page =3D page; return 0; -out_sigsegv_memcg: + +out_err_locked_memcg: bpf_map_memcg_exit(old_memcg, new_memcg); -out_sigsegv: +out_err_locked: raw_res_spin_unlock_irqrestore(&arena->spinlock, flags); - return VM_FAULT_SIGSEGV; + if (new_page) + free_pages_nolock(new_page, 0); + return fault_ret; } =20 static const struct vm_operations_struct arena_vm_ops =3D { --=20 2.43.0 From nobody Thu Sep 24 16:07:10 2026 Received: from mta1.migadu.com (out-100.mta1.migadu.com [95.215.58.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84BD35328D1 for ; Tue, 22 Sep 2026 10:19:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.100 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790072379; cv=none; b=UzyBQX1CchplSEW2OY+nuq0hqP1eFbxCOwiWdbedUtleAzzxxtkrQThMBTnOG3p5sJq8tm32Fh6NCY93Tch3A7w+RMmeVugF/jRAMNzf4QJF9gnCNkQno/sd5l8Ip7dUYkSMhkrHpASiUgTFki2yDiT2YQB3etc+ryY53M/3GAM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790072379; c=relaxed/simple; bh=4ZMgtbSq1fLEo9A153G8EUdhTbvG1BhTa+Q06muy2Bc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kSbxXqv6b59MRyxIp6XFmEn6s1Z5TV0ZFt8ME8mpEL4QmyMO9p6iVI4CdlIbFlma1cwaaaXS1MoZC8Og7BTGWIEFYIj9unPOKnUePK4sImLHcWC1PkAaYpeF8Qw7lPjof0pRCGcE4z0ZMAmBr6mcAl8iO8UX2F2iONBY6hhgCKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=wzxTifWS; arc=none smtp.client-ip=95.215.58.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="wzxTifWS" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=4ZMgtbSq1fLEo9A153G8EUdhTbvG1BhTa+Q06muy2Bc=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790072375; v=1; x=1790677175; b=wzxTifWSgi+8YpXa30hSTn5d7kCLH/U2UrLUH4LgP7OwhZzYLd2Y4d9s0lNNBpUov0qErY0q a3u1qe6Exlib+BMMTQEU7BKcQbcsTbSQiqAxQouuK9kNnUArE3JlfwrEVJLHIM1KfJeLDqQ9ewB BtsS6z0Hi1o4L4gmEDaXE4Do= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id 228bac855f9d92f3; Tue, 22 Sep 2026 10:19:34 +0000 X-Mizu-Trace-ID: 228bac855f9d92f3 X-Migadu-Flow: FLOW_OUT From: Jiayuan Chen To: bpf@vger.kernel.org Cc: Jiayuan Chen , Emil Tsalapatis , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Eduard Zingerman , Kumar Kartikeya Dwivedi , Martin KaFai Lau , Song Liu , Yonghong Song , Jiri Olsa , Ihor Solodrai , Shuah Khan , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [PATCH bpf-next v9 2/3] selftests/bpf: Add read_cgroup_file() to cgroup_helpers Date: Tue, 22 Sep 2026 18:17:34 +0800 Message-ID: <20260922101831.192102-3-jiayuan.chen@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260922101831.192102-1-jiayuan.chen@linux.dev> References: <20260922101831.192102-1-jiayuan.chen@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" cgroup_helpers has write_cgroup_file()/write_cgroup_file_parent() but no read counterpart. Add read_cgroup_file() and read_cgroup_file_parent() so a forked child can read a cgroup file (e.g. memory.current) from the work dir owned by the parent that set the environment up, without hand-building the /mnt/... path. Reviewed-by: Emil Tsalapatis Signed-off-by: Jiayuan Chen --- tools/testing/selftests/bpf/cgroup_helpers.c | 67 ++++++++++++++++++++ tools/testing/selftests/bpf/cgroup_helpers.h | 4 ++ 2 files changed, 71 insertions(+) diff --git a/tools/testing/selftests/bpf/cgroup_helpers.c b/tools/testing/s= elftests/bpf/cgroup_helpers.c index 45cd0b479fe35..4183ff6150c28 100644 --- a/tools/testing/selftests/bpf/cgroup_helpers.c +++ b/tools/testing/selftests/bpf/cgroup_helpers.c @@ -188,6 +188,73 @@ int write_cgroup_file_parent(const char *relative_path= , const char *file, return __write_cgroup_file(cgroup_path, file, buf); } =20 +static int __read_cgroup_file(const char *cgroup_path, const char *file, + char *buf, size_t len) +{ + char file_path[PATH_MAX + 1]; + ssize_t got; + int fd; + + snprintf(file_path, sizeof(file_path), "%s/%s", cgroup_path, file); + fd =3D open(file_path, O_RDONLY); + if (fd < 0) { + log_err("Opening %s", file_path); + return 1; + } + + got =3D read(fd, buf, len - 1); + if (got < 0) { + log_err("Reading %s", file_path); + close(fd); + return 1; + } + buf[got] =3D '\0'; + close(fd); + return 0; +} + +/** + * read_cgroup_file() - Read from a cgroup file + * @relative_path: The cgroup path, relative to the workdir + * @file: The name of the file in cgroupfs to read from + * @buf: Buffer to read into, NUL-terminated on success + * @len: Size of @buf + * + * Read from a file in the given cgroup's directory. + * + * If successful, 0 is returned. + */ +int read_cgroup_file(const char *relative_path, const char *file, + char *buf, size_t len) +{ + char cgroup_path[PATH_MAX - 24]; + + format_cgroup_path(cgroup_path, relative_path); + return __read_cgroup_file(cgroup_path, file, buf, len); +} + +/** + * read_cgroup_file_parent() - Read from a cgroup file in the parent proce= ss + * workdir + * @relative_path: The cgroup path, relative to the parent process workdir + * @file: The name of the file in cgroupfs to read from + * @buf: Buffer to read into, NUL-terminated on success + * @len: Size of @buf + * + * Read from a file in the given cgroup's directory under the parent proce= ss + * workdir. + * + * If successful, 0 is returned. + */ +int read_cgroup_file_parent(const char *relative_path, const char *file, + char *buf, size_t len) +{ + char cgroup_path[PATH_MAX - 24]; + + format_parent_cgroup_path(cgroup_path, relative_path); + return __read_cgroup_file(cgroup_path, file, buf, len); +} + /** * setup_cgroup_environment() - Setup the cgroup environment * diff --git a/tools/testing/selftests/bpf/cgroup_helpers.h b/tools/testing/s= elftests/bpf/cgroup_helpers.h index 3857304be8741..d42d2e13044e5 100644 --- a/tools/testing/selftests/bpf/cgroup_helpers.h +++ b/tools/testing/selftests/bpf/cgroup_helpers.h @@ -15,6 +15,10 @@ int write_cgroup_file(const char *relative_path, const c= har *file, const char *buf); int write_cgroup_file_parent(const char *relative_path, const char *file, const char *buf); +int read_cgroup_file(const char *relative_path, const char *file, + char *buf, size_t len); +int read_cgroup_file_parent(const char *relative_path, const char *file, + char *buf, size_t len); int cgroup_setup_and_join(const char *relative_path); int get_root_cgroup(void); int create_and_get_cgroup(const char *relative_path); --=20 2.43.0 From nobody Thu Sep 24 16:07:10 2026 Received: from mta1.migadu.com (out-114.mta1.migadu.com [95.215.58.114]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 658785328DA for ; Tue, 22 Sep 2026 10:19:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.114 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790072400; cv=none; b=SoWAdq+Jo6c4G/b1CnKXxTY0wpRzho5OHcxRLC+ykKbCbkG5spiD4PKqfCJma+WFKYBixrOX70chebRfbTpd5FRRASMwaEzusCDWYc7NDkIZAx2a9f5d+qWRR/PyRbezRhv10c/sXyq8hZj2223Z0itrb9w/QxlFzODZNaN5RhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790072400; c=relaxed/simple; bh=COPF3YEMiQ3Yab0KaIhKYX8PYpi8LggU23AZSVaYTms=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gmpu646Rwf/o0plb8KXyGy5tHlu47JcL9fQ/wNtK5wGoNhmRyjL0DeuSpat4hRYCep0Dh82MnKdBNViY6Wv7kDVCEKb55nUEHI3hvjSjDEPyqi91EWbV5eAEVddSo6Oo1D1foFL1qW3XNsTmPoBSBMw5ovAETYLB5Rvbh60NpXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=U8IWTWZT; arc=none smtp.client-ip=95.215.58.114 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="U8IWTWZT" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=COPF3YEMiQ3Yab0KaIhKYX8PYpi8LggU23AZSVaYTms=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790072395; v=1; x=1790677195; b=U8IWTWZT5W1t0qmxAVvSAqw/Ag++8VW77d9MWjxQG5oloPHJBOuzvMTrtVGgQsg8ymqD6pqf gSLKwI5a8TkvSgLvLN+Z7p+0v5ZGauReiXwBhgZyJjP2i9+4XwCga4boRRmPu9VuSdZCSwdSC/5 UrvlRkF/347r+UrlU7RmGNiM= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id 6e927b65a3285982; Tue, 22 Sep 2026 10:19:55 +0000 X-Mizu-Trace-ID: 6e927b65a3285982 X-Migadu-Flow: FLOW_OUT From: Jiayuan Chen To: bpf@vger.kernel.org Cc: Jiayuan Chen , Emil Tsalapatis , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Eduard Zingerman , Kumar Kartikeya Dwivedi , Martin KaFai Lau , Song Liu , Yonghong Song , Jiri Olsa , Ihor Solodrai , Shuah Khan , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [PATCH bpf-next v9 3/3] selftests/bpf: Add a test for arena fault-in under memory.max Date: Tue, 22 Sep 2026 18:17:35 +0800 Message-ID: <20260922101831.192102-4-jiayuan.chen@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260922101831.192102-1-jiayuan.chen@linux.dev> References: <20260922101831.192102-1-jiayuan.chen@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A child joins a memcg, fills it with 128M of clean page cache by reading a sparse temp file (the way the cgroup selftests do), caps memory.max 8M above its usage and then faults 64M of arena in, which only fits by reclaiming that cache. With the fix the arena fault-in reclaims, every fault succeeds and the child exits 0. Without it the allocation cannot reclaim, fails once the headroom is used up, and the child dies with SIGSEGV on a valid arena address, so the test fails. # test_progs -v -t arena_memcg serial_test_arena_memcg:PASS:child faulted the arena in #8 arena_memcg:OK # without the fix child killed by signal 11 serial_test_arena_memcg:FAIL:child faulted the arena in The page cache must be reclaimable, so the temp file has to live on a disk-backed filesystem, not tmpfs - the same assumption the cgroup selftests make. Reviewed-by: Emil Tsalapatis Signed-off-by: Jiayuan Chen --- To sashiko: - The non-arm64 1<<44 map_extra is copied from the existing arena tests. - waitpid() without an EINTR retry is copied from the existing tests. - fork() then work without exec follows the existing tests (the SIGEV_THREAD watchdog is test_progs-wide). - Not skipping on EOPNOTSUPP (unsupported arena JIT) follows most of the existing arena tests; only arena_direct_value/arena_spin_lock skip. - cgroup.memory=3Dnobpf would charge the arena pages to the root memcg, so the limit would not bind and the test would pass either way; CI does not configure it. - The test SKIPs when the working directory cannot host the reserve: it detects tmpfs (shmem pages are not reclaimable without swap) and a filesystem without O_TMPFILE support. The cgroup selftests make the same assumption about the working directory. --- .../selftests/bpf/prog_tests/arena_memcg.c | 196 ++++++++++++++++++ .../testing/selftests/bpf/progs/arena_memcg.c | 23 ++ 2 files changed, 219 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_memcg.c create mode 100644 tools/testing/selftests/bpf/progs/arena_memcg.c diff --git a/tools/testing/selftests/bpf/prog_tests/arena_memcg.c b/tools/t= esting/selftests/bpf/prog_tests/arena_memcg.c new file mode 100644 index 0000000000000..c76a7eb2f01d9 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/arena_memcg.c @@ -0,0 +1,196 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#ifndef PAGE_SIZE /* on some archs it comes in sys/user.h */ +#define PAGE_SIZE getpagesize() +#endif + +#include "cgroup_helpers.h" +#include "arena_memcg.skel.h" + +#define CG_PATH "/arena_memcg" + +/* Reclaimable page cache the child builds up before it gets capped. */ +#define RECLAIMABLE (128 * 1024 * 1024) +/* Headroom left under memory.max, far less than the arena we fault in. */ +#define HEADROOM (8 * 1024 * 1024) +/* Arena to fault in; it only fits by reclaiming the page cache. */ +#define ARENA_FAULT (64 * 1024 * 1024) +/* Child exit code for "this environment cannot host the test". */ +#define CHILD_UNSUPPORTED 9 + +static void dump_memcg(void) +{ + char buf[512]; + + if (!read_cgroup_file(CG_PATH, "memory.max", buf, sizeof(buf))) + fprintf(stderr, "memory.max: %s", buf); + if (!read_cgroup_file(CG_PATH, "memory.peak", buf, sizeof(buf))) + fprintf(stderr, "memory.peak: %s", buf); + if (!read_cgroup_file(CG_PATH, "memory.events", buf, sizeof(buf))) + fprintf(stderr, "memory.events:\n%s", buf); +} + +/* + * Fill the page cache with @size bytes of clean, reclaimable pages by + * reading a sparse temp file, the way the cgroup selftests do. Returns the + * fd, which must stay open: closing it drops the cache. Returns -EOPNOTSU= PP + * if the working directory cannot back such a file, -1 on error. + */ +static int alloc_pagecache(size_t size) +{ + struct statfs stfs; + char buf[4096]; + size_t off; + int fd; + + fd =3D open(".", O_TMPFILE | O_RDWR | O_EXCL, 0600); + if (fd < 0) + return errno =3D=3D EOPNOTSUPP ? -EOPNOTSUPP : -1; + /* tmpfs hands out shmem pages, which are not reclaimable without swap */ + if (fstatfs(fd, &stfs) || stfs.f_type =3D=3D TMPFS_MAGIC) { + close(fd); + return -EOPNOTSUPP; + } + if (ftruncate(fd, size)) + goto err; + for (off =3D 0; off < size; off +=3D sizeof(buf)) + if (read(fd, buf, sizeof(buf)) < 0) + goto err; + return fd; +err: + close(fd); + return -1; +} + +void serial_test_arena_memcg(void) +{ + int cgroup_fd =3D -1, status, err; + const long ps =3D PAGE_SIZE; + char buf[64]; + pid_t pid; + + err =3D setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto out; + + cgroup_fd =3D create_and_get_cgroup(CG_PATH); + if (!ASSERT_OK_FD(cgroup_fd, "create_and_get_cgroup")) + goto out; + + /* No memory controller -> nothing to test. */ + if (read_cgroup_file(CG_PATH, "memory.current", buf, sizeof(buf))) { + fprintf(stderr, "%s:SKIP:no memory controller or other env error\n", + __func__); + test__skip(); + goto out; + } + + pid =3D fork(); + if (!ASSERT_GE(pid, 0, "fork")) + goto out; + if (pid =3D=3D 0) { + struct arena_memcg *cskel; + __u32 i, npages; + char *base; + size_t sz; + long cur; + int fd; + + /* + * Everything runs in the child: the arena vma is VM_DONTCOPY so + * it does not survive fork(), and only the child should be under + * the limit. The work dir belongs to the parent, so use the + * _parent() helpers; errors come back as an exit code, ASSERT_* + * does not reach the parent from here. + */ + + /* Step 1: join the memcg, so what follows is charged to it. */ + snprintf(buf, sizeof(buf), "%d", getpid()); + if (write_cgroup_file_parent(CG_PATH, "cgroup.procs", buf)) + _exit(2); + + /* + * Step 2: load the arena. A map is charged to whoever creates + * it, hence joining first. + */ + cskel =3D arena_memcg__open_and_load(); + if (!cskel) + _exit(3); + base =3D bpf_map__initial_value(cskel->maps.arena, &sz); + if (!base) + _exit(4); + npages =3D ARENA_FAULT / ps; + if (npages > bpf_map__max_entries(cskel->maps.arena)) + _exit(5); + + /* Step 3: make RECLAIMABLE bytes of clean page cache. */ + fd =3D alloc_pagecache(RECLAIMABLE); + if (fd =3D=3D -EOPNOTSUPP) + _exit(CHILD_UNSUPPORTED); + if (fd < 0) + _exit(6); + + /* + * Step 4: set memory.max to what we use now plus HEADROOM. The + * page cache is already inside the limit, so only HEADROOM is + * left. + */ + if (read_cgroup_file_parent(CG_PATH, "memory.current", buf, sizeof(buf))) + _exit(7); + cur =3D strtol(buf, NULL, 10); + snprintf(buf, sizeof(buf), "%ld", cur + HEADROOM); + if (write_cgroup_file_parent(CG_PATH, "memory.max", buf)) + _exit(8); + + /* + * Step 5: fault ARENA_FAULT of arena in, much more than + * HEADROOM. Once it hits memory.max every further page has to + * come from reclaiming the page cache. With the fix the + * fault-in reclaims and all of it succeeds; without it the + * allocation cannot reclaim and we die on a valid address. + */ + for (i =3D 0; i < npages; i++) + base[(size_t)i * ps] =3D 1; + _exit(0); /* fd deliberately kept open until here */ + } + + if (!ASSERT_EQ(waitpid(pid, &status, 0), pid, "waitpid")) + goto out; + + /* The working directory cannot hold a reclaimable page cache. */ + if (WIFEXITED(status) && WEXITSTATUS(status) =3D=3D CHILD_UNSUPPORTED) { + fprintf(stderr, "%s:SKIP:no disk-backed O_TMPFILE in cwd\n", __func__); + test__skip(); + goto out; + } + + /* A non-zero exit means the child failed to set up; the code says where.= */ + if (WIFEXITED(status) && WEXITSTATUS(status)) { + ASSERT_OK(WEXITSTATUS(status), "child setup"); + goto out; + } + + /* + * With the fix the arena fault-in reclaims the page cache and every + * fault succeeds, so the child exits 0. Without it the allocation + * cannot reclaim, fails once the headroom is used up, and the child + * dies with SIGSEGV on a valid arena address. + */ + if (!ASSERT_TRUE(WIFEXITED(status) && !WEXITSTATUS(status), + "child faulted the arena in")) { + if (WIFSIGNALED(status)) + fprintf(stderr, "child killed by signal %d\n", WTERMSIG(status)); + dump_memcg(); + } +out: + if (cgroup_fd >=3D 0) + close(cgroup_fd); + cleanup_cgroup_environment(); +} diff --git a/tools/testing/selftests/bpf/progs/arena_memcg.c b/tools/testin= g/selftests/bpf/progs/arena_memcg.c new file mode 100644 index 0000000000000..aff73757e7941 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/arena_memcg.c @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include "bpf_arena_common.h" + +struct { + __uint(type, BPF_MAP_TYPE_ARENA); + __uint(map_flags, BPF_F_MMAPABLE); + /* + * Number of pages. Must cover ARENA_FAULT on the smallest page size + * (64M/4K =3D 16384) yet stay under the 4G arena limit on 64K pages + * (50000*64K =3D 3.2G). + */ + __uint(max_entries, 50000); +#ifdef __TARGET_ARCH_arm64 + __ulong(map_extra, 0x1ull << 32); /* start of mmap() region */ +#else + __ulong(map_extra, 0x1ull << 44); /* start of mmap() region */ +#endif +} arena SEC(".maps"); + +char _license[] SEC("license") =3D "GPL"; --=20 2.43.0