From nobody Mon Dec 1 21:30:04 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9BA52C027C for ; Fri, 28 Nov 2025 03:12:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299524; cv=none; b=eTu/OBxnbknuuBGqEyJ3t16WdXG+Kv5G8AegEnDgOr91qXWKDdoUOrrG1zaKFaGgqeY9zidIGg0rkLJ2SdoICdXgSrysloEdm2GRZihR0Rav/YykGFjuKmHOWIyN1DulWwnIk6/o3AueSZfcLc3IRACmn3ziAE6qNYDlGDvru70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299524; c=relaxed/simple; bh=peKOX3tDaJjI+d9s6OHERNkephqMZYbfewyZNQKm5d4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s0zcxHxOCWKFlfd1PIowWaOaPMBtFpaQISrkSxPj1IvFGRL3LVYNOeBKUTszToQgVeoVTguhHdZgnCK4wJBO6/NHbpw20BNow39fXHnJsvirqvJXxW39Ao0XvVyPf1kqOncATuJJJulCu5mahRs6kjVv1OhRtzgl7SKOyeTC0Jc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=m8qqQfH3; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="m8qqQfH3" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-7ade456b6abso1096923b3a.3 for ; Thu, 27 Nov 2025 19:12:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764299522; x=1764904322; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UURN4qing9DSF3ysybnLESXjHzZIzZMUCK3v5bSZiLw=; b=m8qqQfH35mZrDH1hrNSqoJ8lkxBvQJ7xTc3nXT3fmGQp2ngCgqCZeKHhmvlgPItNum abIR32tZmrHQBVvbjHh1tEKXLHAJ8Fs1edIx9/HeJKlNsS7CQ7wsRLoMn9JF3CQGCCRx N8n+KYRPZgCEZJSp+a7Yo7Pk4zZ0m+Vvvkc8nzFTYLjyIUPW5Es9aJzMedUX7UJa+1wK EIj23cMvmZRylbbEju+H+Rr1evKQnydp9uEViDRgmu/1CnS5zB61p9i+O2VPqOAiKhpJ cMPo0GvL677P34xHbCLw5FyKPbybGCvbbrLa+jRSBKoVxYYWEW+CJv8jx2/2K3eVFiv6 I6oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764299522; x=1764904322; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UURN4qing9DSF3ysybnLESXjHzZIzZMUCK3v5bSZiLw=; b=NsbsvL2upc6xgKgXWziyU5cHbUqAvSGr+iRSCbfb8G7T4pM91fCAwpXNqAso+dnqRD 1ims6Ngy9QdW/gBll5t0ETc6/T2OChSgZUwCrEJpEuavdEHrpwxog+ZF54LbtzbIndi8 g+DNWGL7c35xy+3LmBSuPZRyH5xJ43bSC7FdIKnoWZm9wKi+aplV4bCsiG01anYqDh+/ Q9LnFcTjhWQ4APh2vuq1x0/Nf7jXDTO08rMjeZiYGYGNvDjhvZ6FlfRWu+yxKpfvU+bK VZ/9hvQaavhp5uX4dBbxU/oMHbMw2AOv3Pm+0x1jWoN2TW3O66mPfo5gk8sHqqiB7VDI YDIQ== X-Forwarded-Encrypted: i=1; AJvYcCVL4+F/Pth0Dryq/Yj5JS79zzrvpDcJv0P0r0eK5h6miWMiGWJGXX2qtCF2mQbFEgJl/V6NpiWKQkVoSoU=@vger.kernel.org X-Gm-Message-State: AOJu0YylttdJu+40fMFQ3+tVybIqLd2SV1AtD5e+781gk/6Mhw4xWfh6 YIpYPpCsJBPq3iWKF9WW0OvIWFYU5Ul3jlmQ+AsUOl0r3RyB2A/xCl+r X-Gm-Gg: ASbGnctqPMl4lX8oDf+RZt+SfrwBIVL8DhX11RB/v6iSP2MGU5hqZaPo8caRMZ8XLkS ZCAx3BAwmDtHDGzkN877ydpjMooM7gGZCqWeDno7FIr6FgTxq4n32p2YTrg8sINGqflUfkwkZVB KCajWRZ0kyNSvjmnSAH8RZi9QTEA91077HVgaf+WAO0TnJd6dRjDFPsc16jR+K5T5yPCP/y2Ma/ EGqGenWrDfVAuOMbTAD58oEybe8C1QJ0FfVsa8lLxxAEkM8osZ9FC0B6HX0bRL5WtijVNatw9gt ITgX8snSejAKDPdxK6VCQxLX1qvhoZFPu17j8xYsjxoHOAxt/MgOdlsvpPk52TZuKYhN0n5kOum +qISvpG8OKxkjOVVeT8a8bWUoqXFlQwrLxDRFjAgJocJ1u/BfWi7cZWfGRUoCX6vSCmbL7yc5Gb POev+bz0ioskpajCbYO5t9vKIerpqPqPM039gL8Q== X-Google-Smtp-Source: AGHT+IH01szTUvjUWPgozI/rwQ/A0QsBtn0PFJolC4W+66Z3l+siWZ7TigqC/fsl6tVQiWrY7nIbtw== X-Received: by 2002:a05:6a00:8d6:b0:7ab:e844:1e76 with SMTP id d2e1a72fcca58-7c58c2b20b5mr23898778b3a.5.1764299522057; Thu, 27 Nov 2025 19:12:02 -0800 (PST) Received: from zhr-ThinkStation-K.mioffice.cn ([2408:8607:1b00:8:3638:7863:b439:f5a7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7d15f080cf1sm3245651b3a.46.2025.11.27.19.11.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Nov 2025 19:12:01 -0800 (PST) From: Hongru Zhang X-Google-Original-From: Hongru Zhang To: akpm@linux-foundation.org, vbabka@suse.cz, david@kernel.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, Hongru Zhang Subject: [PATCH 1/3] mm/page_alloc: add per-migratetype counts to buddy allocator Date: Fri, 28 Nov 2025 11:11:42 +0800 Message-ID: <37a6c6c35e029c3429f236d5895c898140d991eb.1764297987.git.zhanghongru@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hongru Zhang On mobile devices, some user-space memory management components check memory pressure and fragmentation status periodically or via PSI, and take actions such as killing processes or performing memory compaction based on this information. Under high load scenarios, reading /proc/pagetypeinfo causes memory management components or memory allocation/free paths to be blocked for extended periods waiting for the zone lock, leading to the following issues: 1. Long interrupt-disabled spinlocks - occasionally exceeding 10ms on Qcom 8750 platforms, reducing system real-time performance 2. Memory management components being blocked for extended periods, preventing rapid acquisition of memory fragmentation information for critical memory management decisions and actions 3. Increased latency in memory allocation and free paths due to prolonged zone lock contention This patch adds per-migratetype counts to the buddy allocator in preparation for optimizing /proc/pagetypeinfo access. The optimized implementation: - Make per-migratetype count updates protected by zone lock on the write side while /proc/pagetypeinfo reads are lock-free, which reduces interrupt-disabled spinlock duration and improves system real-time performance (addressing issue #1) - Reduce blocking time for memory management components when reading /proc/pagetypeinfo, enabling more rapid acquisition of memory fragmentation information (addressing issue #2) - Minimize the critical section held during /proc/pagetypeinfo reads to reduce zone lock contention on memory allocation and free paths (addressing issue #3) The main overhead is a slight increase in latency on the memory allocation and free paths due to additional per-migratetype counting, with theoretically minimal impact on overall performance. Signed-off-by: Hongru Zhang --- include/linux/mmzone.h | 1 + mm/mm_init.c | 1 + mm/page_alloc.c | 7 ++++++- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 7fb7331c5725..6eeefe6a3727 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -138,6 +138,7 @@ extern int page_group_by_mobility_disabled; struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; + unsigned long mt_nr_free[MIGRATE_TYPES]; }; =20 struct pglist_data; diff --git a/mm/mm_init.c b/mm/mm_init.c index 7712d887b696..dca2be8cc3b1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1439,6 +1439,7 @@ static void __meminit zone_init_free_lists(struct zon= e *zone) for_each_migratetype_order(order, t) { INIT_LIST_HEAD(&zone->free_area[order].free_list[t]); zone->free_area[order].nr_free =3D 0; + zone->free_area[order].mt_nr_free[t] =3D 0; } =20 #ifdef CONFIG_UNACCEPTED_MEMORY diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ed82ee55e66a..9431073e7255 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -818,6 +818,7 @@ static inline void __add_to_free_list(struct page *page= , struct zone *zone, else list_add(&page->buddy_list, &area->free_list[migratetype]); area->nr_free++; + area->mt_nr_free[migratetype]++; =20 if (order >=3D pageblock_order && !is_migrate_isolate(migratetype)) __mod_zone_page_state(zone, NR_FREE_PAGES_BLOCKS, nr_pages); @@ -840,6 +841,8 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, get_pageblock_migratetype(page), old_mt, nr_pages); =20 list_move_tail(&page->buddy_list, &area->free_list[new_mt]); + area->mt_nr_free[old_mt]--; + area->mt_nr_free[new_mt]++; =20 account_freepages(zone, -nr_pages, old_mt); account_freepages(zone, nr_pages, new_mt); @@ -855,6 +858,7 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, static inline void __del_page_from_free_list(struct page *page, struct zon= e *zone, unsigned int order, int migratetype) { + struct free_area *area =3D &zone->free_area[order]; int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, @@ -868,7 +872,8 @@ static inline void __del_page_from_free_list(struct pag= e *page, struct zone *zon list_del(&page->buddy_list); __ClearPageBuddy(page); set_page_private(page, 0); - zone->free_area[order].nr_free--; + area->nr_free--; + area->mt_nr_free[migratetype]--; =20 if (order >=3D pageblock_order && !is_migrate_isolate(migratetype)) __mod_zone_page_state(zone, NR_FREE_PAGES_BLOCKS, -nr_pages); --=20 2.43.0 From nobody Mon Dec 1 21:30:04 2025 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A5AF19CD03 for ; Fri, 28 Nov 2025 03:12:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299558; cv=none; b=iet1gSIiNUvDMqmOxg69L4Ld/gYgsPnFT2Lw/qm5vibijvVGhz+MgYkFVcGSU5/dpqRt81/AanW0Y/+GqMpNQE/FXXhVzTutdJzdERL3G4nCB0cFHFEbc11vGxGFwEHi2WvfiOy65KVRw+b1AUu4vafhY4tuCd/GrgNbUpUwVDY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299558; c=relaxed/simple; bh=iTMALNhwMLLu1PAzFEZ5VgMbxyiNk6FcqhCJoqmCvis=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RZ7oxGpLWtT+QeGV5Lx0vYQsHBFpGlqJNr4nBVWrcVtC2Mp7W/QYVGD+4Rdbv46DWrw7EqGooRGOabIxEwlrF9GfphbvqXyhWq3zd8/3ZLvsSoFdeiByvVfZpl1/kQtTHHh0QY6e3A8L4OeLc2Hnmm3vn/I1v3Z8eWZZ4fwSBrk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KwuwtRDI; arc=none smtp.client-ip=209.85.214.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KwuwtRDI" Received: by mail-pl1-f194.google.com with SMTP id d9443c01a7336-2958db8ae4fso14036495ad.2 for ; Thu, 27 Nov 2025 19:12:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764299556; x=1764904356; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S9VXBJ60a07RGP4hbMBT/KJ4u6vYKF5t5iuO5lexYnI=; b=KwuwtRDIiCvBytkLhH8mUJCTrTs0BhuA3VF1MqzIQ+/Dv3phhSZ1eT34K8KkybN65Z J0OMz31paCK5Ni9dobpMOKpyr4ArnM4zIG42+su8jEeU30A47561cHp/h98OaJuujNPB QgYcaJOP78jLmDa8pK/Ujk0qFJARRCXn1rpcMPw2bfJ61yy4Gdd/HjmGsKACpV3RiFf8 cF9I8t6zQYkn3rpzUuNmRGpO2SxuvizH8/yCCuy9acYnKBMoRLMD4C1+eNjSRs239YzC 3gnBMceO48GKeWSl+Ugd7Lr6ac4PAXe4fFK9dzZRheOUF4XQlyF4Mbm3jKEq4O9foxF8 GzAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764299556; x=1764904356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=S9VXBJ60a07RGP4hbMBT/KJ4u6vYKF5t5iuO5lexYnI=; b=gKOdgS75sDJe7RAU2oPfUhab0qs3Q75Ci9fgTlWAoQc5nKbbPVz33MBb76x7v5pFJr gj7xkhtkU9jOkN/pZTUGkRHHkrtcdi0Ya+PvrMEN7NtgUZ/Mt5yBUAfxt1DEePA65Uq5 1X66tA4s5Le2+NgdzqL/n+i6UTKNp948lR9O5qP9bWc3sVTf2vJqq5S7pbbdOepc0zLk DB88fj5bM2U5UbY8uMpQqgqz9EeevuZ6O2oznw+g0uK7fuLu/1PuGzA49J9BaVyRF19F S0doCqtSiJnCxim80ullFNinR4NVE0eIeytaQB8jetJAQXYM02zGHC3pb5GZIva9L2Vq NoHQ== X-Forwarded-Encrypted: i=1; AJvYcCW78tMFFA+Pn+w7BOpDDTKl4RlsatuxzyqTKzwWvJCgniwhrI4hUCFrsBhuInNGSffVy4eQF2EVInRzFUk=@vger.kernel.org X-Gm-Message-State: AOJu0Yz84KFa1pj1okiYeexkh6DAOWpcjCa3cBT665yUDZHA/S+Mw3VR Ra5Lg2BHtud/stow6XBr7Qkz1WhxZg3oeAL5aR1+o7iRgw8zJaPcS/iW X-Gm-Gg: ASbGncse0VYasKcJUbVrdAkgFCIpZKU+O5+w2ZX02f4mwY88F06jpErdOJJoWdDZWy3 ImxhUP0XdTzKO0VVtypnjhmVkjuiA/Kgieh5E3S2GlbzaZ4nsZ3U9lTYlAI09gucMK3aa3TZuT6 0+dv0134ganbIVYsUK3SWo6Sffi4F4hr//yiuKyezJtRPnySwJ6XnkfZwutJYFF2trekOABw6d6 UKMGPwQUa7Rzc8vjgAaVji21gkA4r+hlKf3A9Y5xdgmMnmaoMANFjlaEtWtBAhBd/16GbT62B94 5OTD5deljP8TNrGE5ghzLptGjchRDYRRLaL7sB9Mu5yDTeI45hIzQzmxRbmDvY9uTbZO3lAsSIv 6aoXOxjUMs0+b+0OonzRSi6SkYicwgqByL/NMWNwoI/Rp3A36c12DqjuyH+tDwlT0puw+s0GlhI ADWbM5lY3ccyH3lVOTKgUKfuOGamWPOxzwJ+Io4NvRPp/li/k4 X-Google-Smtp-Source: AGHT+IE1lwR9Ofq4Qqg5eKpVX0cSDeD8qkn4ka7DL91z29m60LfJEWybbHRIS4gz9OtjqCITSYWmuQ== X-Received: by 2002:a17:902:ce01:b0:295:557e:746a with SMTP id d9443c01a7336-29b6c3e375emr257843275ad.13.1764299555542; Thu, 27 Nov 2025 19:12:35 -0800 (PST) Received: from zhr-ThinkStation-K.mioffice.cn ([2408:8607:1b00:8:3638:7863:b439:f5a7]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29bceb275a3sm29674945ad.66.2025.11.27.19.12.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Nov 2025 19:12:35 -0800 (PST) From: Hongru Zhang X-Google-Original-From: Hongru Zhang To: akpm@linux-foundation.org, vbabka@suse.cz, david@kernel.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, Hongru Zhang Subject: [PATCH 2/3] mm/vmstat: get fragmentation statistics from per-migragetype count Date: Fri, 28 Nov 2025 11:12:10 +0800 Message-ID: <780ded63aca04eb513d6e7e2c8862e6972b84b45.1764297987.git.zhanghongru@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hongru Zhang This patch optimizes /proc/pagetypeinfo access by utilizing the per-migratetype free page block counts already maintained by the buddy allocator, instead of iterating through free lists under zone lock. Accuracy. Both implementations have accuracy limitations. The previous implementation required acquiring and releasing the zone lock for counting each order and migratetype, making it potentially inaccurate. Under high memory pressure, accuracy would further degrade due to zone lock contention or fragmentation. The new implementation collects data within a short time window, which helps maintain relatively small errors, and is unaffected by memory pressure. Furthermore, user-space memory management components inherently experience decision latency - by the time they process the collected data and execute actions, the memory state has already changed. This means that even perfectly accurate data at collection time becomes stale by decision time. Considering these factors, the accuracy trade-off introduced by the new implementation should be acceptable for practical use cases, offering a balance between performance and accuracy requirements. Performance benefits: System setup: - 12th Gen Intel(R) Core(TM) i7-12700 - 1 NUMA node, 16G memory in total - Turbo disabled - cpufreq governor set to performance 1. Average latency over 10,000 /proc/pagetypeinfo accesses +-----------------------+----------+------------+ | | no-patch | with-patch | +-----------------------+----------+------------+ | Just after boot | 700.9 us | 268.6 us | +-----------------------+----------+------------+ | After building kernel | 28.7 ms | 269.8 us | +-----------------------+----------+------------+ 2. Page alloc/free latency with concurrent /proc/pagetypeinfo access Test setup: - Using config-pagealloc-micro - Monitor set to proc-pagetypeinfo, update frequency set to 10ms - PAGEALLOC_ORDER_MIN=3D4, PAGEALLOC_ORDER_MAX=3D4 Without patch test results: vanilla vanilla no-monitor monitor Min alloc-odr4-1 8539.00 ( 0.00%) 8762.00 ( -2.61%) Min alloc-odr4-2 6501.00 ( 0.00%) 6683.00 ( -2.80%) Min alloc-odr4-4 5537.00 ( 0.00%) 5873.00 ( -6.07%) Min alloc-odr4-8 5030.00 ( 0.00%) 5361.00 ( -6.58%) Min alloc-odr4-16 4782.00 ( 0.00%) 5162.00 ( -7.95%) Min alloc-odr4-32 5838.00 ( 0.00%) 6499.00 ( -11.32%) Min alloc-odr4-64 6565.00 ( 0.00%) 7413.00 ( -12.92%) Min alloc-odr4-128 6896.00 ( 0.00%) 7898.00 ( -14.53%) Min alloc-odr4-256 7303.00 ( 0.00%) 8163.00 ( -11.78%) Min alloc-odr4-512 10179.00 ( 0.00%) 11985.00 ( -17.74%) Min alloc-odr4-1024 11000.00 ( 0.00%) 12165.00 ( -10.59%) Min free-odr4-1 820.00 ( 0.00%) 1230.00 ( -50.00%) Min free-odr4-2 511.00 ( 0.00%) 952.00 ( -86.30%) Min free-odr4-4 347.00 ( 0.00%) 434.00 ( -25.07%) Min free-odr4-8 286.00 ( 0.00%) 399.00 ( -39.51%) Min free-odr4-16 250.00 ( 0.00%) 405.00 ( -62.00%) Min free-odr4-32 294.00 ( 0.00%) 405.00 ( -37.76%) Min free-odr4-64 333.00 ( 0.00%) 363.00 ( -9.01%) Min free-odr4-128 340.00 ( 0.00%) 412.00 ( -21.18%) Min free-odr4-256 339.00 ( 0.00%) 329.00 ( 2.95%) Min free-odr4-512 361.00 ( 0.00%) 409.00 ( -13.30%) Min free-odr4-1024 300.00 ( 0.00%) 361.00 ( -20.33%) Stddev alloc-odr4-1 7.29 ( 0.00%) 90.78 (-1146.00%) Stddev alloc-odr4-2 3.87 ( 0.00%) 51.30 (-1225.75%) Stddev alloc-odr4-4 3.20 ( 0.00%) 50.90 (-1491.24%) Stddev alloc-odr4-8 4.67 ( 0.00%) 52.23 (-1019.35%) Stddev alloc-odr4-16 5.72 ( 0.00%) 27.53 (-381.04%) Stddev alloc-odr4-32 6.25 ( 0.00%) 641.23 (-10154.46%) Stddev alloc-odr4-64 2.06 ( 0.00%) 386.99 (-18714.22%) Stddev alloc-odr4-128 14.36 ( 0.00%) 52.39 (-264.77%) Stddev alloc-odr4-256 32.42 ( 0.00%) 326.19 (-906.05%) Stddev alloc-odr4-512 65.58 ( 0.00%) 184.49 (-181.31%) Stddev alloc-odr4-1024 8.88 ( 0.00%) 153.01 (-1622.67%) Stddev free-odr4-1 2.29 ( 0.00%) 152.27 (-6549.85%) Stddev free-odr4-2 10.99 ( 0.00%) 73.10 (-564.89%) Stddev free-odr4-4 1.99 ( 0.00%) 28.40 (-1324.45%) Stddev free-odr4-8 2.51 ( 0.00%) 52.93 (-2007.64%) Stddev free-odr4-16 2.85 ( 0.00%) 26.04 (-814.88%) Stddev free-odr4-32 4.04 ( 0.00%) 27.05 (-569.79%) Stddev free-odr4-64 2.10 ( 0.00%) 48.07 (-2185.66%) Stddev free-odr4-128 2.63 ( 0.00%) 26.23 (-897.86%) Stddev free-odr4-256 6.29 ( 0.00%) 37.04 (-488.71%) Stddev free-odr4-512 2.56 ( 0.00%) 10.65 (-315.28%) Stddev free-odr4-1024 0.95 ( 0.00%) 6.46 (-582.22%) Max alloc-odr4-1 8564.00 ( 0.00%) 9099.00 ( -6.25%) Max alloc-odr4-2 6511.00 ( 0.00%) 6844.00 ( -5.11%) Max alloc-odr4-4 5549.00 ( 0.00%) 6038.00 ( -8.81%) Max alloc-odr4-8 5045.00 ( 0.00%) 5551.00 ( -10.03%) Max alloc-odr4-16 4800.00 ( 0.00%) 5257.00 ( -9.52%) Max alloc-odr4-32 5861.00 ( 0.00%) 8115.00 ( -38.46%) Max alloc-odr4-64 6571.00 ( 0.00%) 8292.00 ( -26.19%) Max alloc-odr4-128 6930.00 ( 0.00%) 8081.00 ( -16.61%) Max alloc-odr4-256 7372.00 ( 0.00%) 9150.00 ( -24.12%) Max alloc-odr4-512 10333.00 ( 0.00%) 12636.00 ( -22.29%) Max alloc-odr4-1024 11035.00 ( 0.00%) 12590.00 ( -14.09%) Max free-odr4-1 828.00 ( 0.00%) 1724.00 (-108.21%) Max free-odr4-2 543.00 ( 0.00%) 1192.00 (-119.52%) Max free-odr4-4 354.00 ( 0.00%) 519.00 ( -46.61%) Max free-odr4-8 293.00 ( 0.00%) 617.00 (-110.58%) Max free-odr4-16 260.00 ( 0.00%) 483.00 ( -85.77%) Max free-odr4-32 308.00 ( 0.00%) 488.00 ( -58.44%) Max free-odr4-64 341.00 ( 0.00%) 505.00 ( -48.09%) Max free-odr4-128 346.00 ( 0.00%) 497.00 ( -43.64%) Max free-odr4-256 353.00 ( 0.00%) 463.00 ( -31.16%) Max free-odr4-512 367.00 ( 0.00%) 442.00 ( -20.44%) Max free-odr4-1024 303.00 ( 0.00%) 381.00 ( -25.74%) With patch test results: patched patched no-monitor monitor Min alloc-odr4-1 8488.00 ( 0.00%) 8514.00 ( -0.31%) Min alloc-odr4-2 6551.00 ( 0.00%) 6527.00 ( 0.37%) Min alloc-odr4-4 5536.00 ( 0.00%) 5591.00 ( -0.99%) Min alloc-odr4-8 5008.00 ( 0.00%) 5098.00 ( -1.80%) Min alloc-odr4-16 4760.00 ( 0.00%) 4857.00 ( -2.04%) Min alloc-odr4-32 5827.00 ( 0.00%) 5919.00 ( -1.58%) Min alloc-odr4-64 6561.00 ( 0.00%) 6680.00 ( -1.81%) Min alloc-odr4-128 6898.00 ( 0.00%) 7014.00 ( -1.68%) Min alloc-odr4-256 7311.00 ( 0.00%) 7464.00 ( -2.09%) Min alloc-odr4-512 10181.00 ( 0.00%) 10286.00 ( -1.03%) Min alloc-odr4-1024 11205.00 ( 0.00%) 11725.00 ( -4.64%) Min free-odr4-1 789.00 ( 0.00%) 867.00 ( -9.89%) Min free-odr4-2 490.00 ( 0.00%) 526.00 ( -7.35%) Min free-odr4-4 350.00 ( 0.00%) 360.00 ( -2.86%) Min free-odr4-8 272.00 ( 0.00%) 287.00 ( -5.51%) Min free-odr4-16 247.00 ( 0.00%) 254.00 ( -2.83%) Min free-odr4-32 298.00 ( 0.00%) 304.00 ( -2.01%) Min free-odr4-64 334.00 ( 0.00%) 325.00 ( 2.69%) Min free-odr4-128 334.00 ( 0.00%) 329.00 ( 1.50%) Min free-odr4-256 336.00 ( 0.00%) 336.00 ( 0.00%) Min free-odr4-512 360.00 ( 0.00%) 342.00 ( 5.00%) Min free-odr4-1024 327.00 ( 0.00%) 355.00 ( -8.56%) Stddev alloc-odr4-1 5.19 ( 0.00%) 45.38 (-775.09%) Stddev alloc-odr4-2 6.99 ( 0.00%) 37.63 (-437.98%) Stddev alloc-odr4-4 3.91 ( 0.00%) 17.85 (-356.28%) Stddev alloc-odr4-8 5.15 ( 0.00%) 9.34 ( -81.47%) Stddev alloc-odr4-16 3.83 ( 0.00%) 5.34 ( -39.34%) Stddev alloc-odr4-32 1.96 ( 0.00%) 10.28 (-425.09%) Stddev alloc-odr4-64 1.32 ( 0.00%) 333.30 (-25141.39%) Stddev alloc-odr4-128 2.06 ( 0.00%) 7.37 (-258.28%) Stddev alloc-odr4-256 15.56 ( 0.00%) 113.48 (-629.25%) Stddev alloc-odr4-512 61.25 ( 0.00%) 165.09 (-169.53%) Stddev alloc-odr4-1024 18.89 ( 0.00%) 2.93 ( 84.51%) Stddev free-odr4-1 4.45 ( 0.00%) 40.12 (-800.98%) Stddev free-odr4-2 1.50 ( 0.00%) 29.30 (-1850.31%) Stddev free-odr4-4 1.27 ( 0.00%) 19.49 (-1439.40%) Stddev free-odr4-8 0.97 ( 0.00%) 8.93 (-823.07%) Stddev free-odr4-16 8.38 ( 0.00%) 4.51 ( 46.21%) Stddev free-odr4-32 3.18 ( 0.00%) 6.59 (-107.42%) Stddev free-odr4-64 2.40 ( 0.00%) 3.09 ( -28.50%) Stddev free-odr4-128 1.55 ( 0.00%) 2.53 ( -62.92%) Stddev free-odr4-256 0.41 ( 0.00%) 2.80 (-585.57%) Stddev free-odr4-512 1.60 ( 0.00%) 4.84 (-202.08%) Stddev free-odr4-1024 0.66 ( 0.00%) 1.19 ( -80.68%) Max alloc-odr4-1 8505.00 ( 0.00%) 8676.00 ( -2.01%) Max alloc-odr4-2 6572.00 ( 0.00%) 6651.00 ( -1.20%) Max alloc-odr4-4 5552.00 ( 0.00%) 5646.00 ( -1.69%) Max alloc-odr4-8 5024.00 ( 0.00%) 5131.00 ( -2.13%) Max alloc-odr4-16 4774.00 ( 0.00%) 4875.00 ( -2.12%) Max alloc-odr4-32 5834.00 ( 0.00%) 5950.00 ( -1.99%) Max alloc-odr4-64 6565.00 ( 0.00%) 7434.00 ( -13.24%) Max alloc-odr4-128 6907.00 ( 0.00%) 7034.00 ( -1.84%) Max alloc-odr4-256 7347.00 ( 0.00%) 7843.00 ( -6.75%) Max alloc-odr4-512 10315.00 ( 0.00%) 10866.00 ( -5.34%) Max alloc-odr4-1024 11278.00 ( 0.00%) 11733.00 ( -4.03%) Max free-odr4-1 803.00 ( 0.00%) 1009.00 ( -25.65%) Max free-odr4-2 495.00 ( 0.00%) 607.00 ( -22.63%) Max free-odr4-4 354.00 ( 0.00%) 417.00 ( -17.80%) Max free-odr4-8 275.00 ( 0.00%) 313.00 ( -13.82%) Max free-odr4-16 273.00 ( 0.00%) 272.00 ( 0.37%) Max free-odr4-32 309.00 ( 0.00%) 324.00 ( -4.85%) Max free-odr4-64 340.00 ( 0.00%) 335.00 ( 1.47%) Max free-odr4-128 340.00 ( 0.00%) 338.00 ( 0.59%) Max free-odr4-256 338.00 ( 0.00%) 346.00 ( -2.37%) Max free-odr4-512 364.00 ( 0.00%) 359.00 ( 1.37%) Max free-odr4-1024 329.00 ( 0.00%) 359.00 ( -9.12%) Signed-off-by: Hongru Zhang --- mm/page_alloc.c | 10 ++++++---- mm/vmstat.c | 30 +++++++----------------------- 2 files changed, 13 insertions(+), 27 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 9431073e7255..a90f2bf735f6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -818,7 +818,8 @@ static inline void __add_to_free_list(struct page *page= , struct zone *zone, else list_add(&page->buddy_list, &area->free_list[migratetype]); area->nr_free++; - area->mt_nr_free[migratetype]++; + WRITE_ONCE(area->mt_nr_free[migratetype], + area->mt_nr_free[migratetype] + 1); =20 if (order >=3D pageblock_order && !is_migrate_isolate(migratetype)) __mod_zone_page_state(zone, NR_FREE_PAGES_BLOCKS, nr_pages); @@ -841,8 +842,8 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, get_pageblock_migratetype(page), old_mt, nr_pages); =20 list_move_tail(&page->buddy_list, &area->free_list[new_mt]); - area->mt_nr_free[old_mt]--; - area->mt_nr_free[new_mt]++; + WRITE_ONCE(area->mt_nr_free[old_mt], area->mt_nr_free[old_mt] - 1); + WRITE_ONCE(area->mt_nr_free[new_mt], area->mt_nr_free[new_mt] + 1); =20 account_freepages(zone, -nr_pages, old_mt); account_freepages(zone, nr_pages, new_mt); @@ -873,7 +874,8 @@ static inline void __del_page_from_free_list(struct pag= e *page, struct zone *zon __ClearPageBuddy(page); set_page_private(page, 0); area->nr_free--; - area->mt_nr_free[migratetype]--; + WRITE_ONCE(area->mt_nr_free[migratetype], + area->mt_nr_free[migratetype] - 1); =20 if (order >=3D pageblock_order && !is_migrate_isolate(migratetype)) __mod_zone_page_state(zone, NR_FREE_PAGES_BLOCKS, -nr_pages); diff --git a/mm/vmstat.c b/mm/vmstat.c index bb09c032eecf..9334bbbe1e16 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1590,32 +1590,16 @@ static void pagetypeinfo_showfree_print(struct seq_= file *m, zone->name, migratetype_names[mtype]); for (order =3D 0; order < NR_PAGE_ORDERS; ++order) { - unsigned long freecount =3D 0; - struct free_area *area; - struct list_head *curr; + unsigned long freecount; bool overflow =3D false; =20 - area =3D &(zone->free_area[order]); - - list_for_each(curr, &area->free_list[mtype]) { - /* - * Cap the free_list iteration because it might - * be really large and we are under a spinlock - * so a long time spent here could trigger a - * hard lockup detector. Anyway this is a - * debugging tool so knowing there is a handful - * of pages of this order should be more than - * sufficient. - */ - if (++freecount >=3D 100000) { - overflow =3D true; - break; - } + /* Keep the same output format for user-space tools compatibility */ + freecount =3D READ_ONCE(zone->free_area[order].mt_nr_free[mtype]); + if (freecount >=3D 100000) { + overflow =3D true; + freecount =3D 100000; } seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount); - spin_unlock_irq(&zone->lock); - cond_resched(); - spin_lock_irq(&zone->lock); } seq_putc(m, '\n'); } @@ -1633,7 +1617,7 @@ static void pagetypeinfo_showfree(struct seq_file *m,= void *arg) seq_printf(m, "%6d ", order); seq_putc(m, '\n'); =20 - walk_zones_in_node(m, pgdat, true, false, pagetypeinfo_showfree_print); + walk_zones_in_node(m, pgdat, true, true, pagetypeinfo_showfree_print); } =20 static void pagetypeinfo_showblockcount_print(struct seq_file *m, --=20 2.43.0 From nobody Mon Dec 1 21:30:04 2025 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 041B32BE622 for ; Fri, 28 Nov 2025 03:13:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299584; cv=none; b=eLlOxv8njAJrUc4HM9heu3lHPNn8dk36QTtTfJMwmhB16UIEfBx7hcMoDBXCbPlJWDVABAd+0xXre0vnzOw6H+Nco3booHUM9jXIM3D1juiMaNEqjrGp7NzbESRI1d9Loec8+dKqRCxElEud19bot8E4+eMs2r+vE1RZWcR4Dik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299584; c=relaxed/simple; bh=2LHvaznvK7+TwxC/vfZ6UNoyw4VExsHG9XDRUDCd6kY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gQbPyL4v+8PDHtlL/pd7+eqKoAAKMksjOn2KpXlg0Isk5Gd0BgrKzvQ2fZVRFHZ+FSveDIByd7sSgxk/Gvt2LPs7Wr4C1NtOI7nDR4RmkWZvuTGM1xAly15lh581f3JbRECR6GDsYEL2wPOgm3/GYo50+RAa8Oerpw09oDiez1c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GZFaS722; arc=none smtp.client-ip=209.85.214.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GZFaS722" Received: by mail-pl1-f194.google.com with SMTP id d9443c01a7336-298456bb53aso16855025ad.0 for ; Thu, 27 Nov 2025 19:13:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764299582; x=1764904382; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kGL2VMPMFa8R+g6JKXrGDSEsUVYbVq9e7DRPdheTEI4=; b=GZFaS722ZbxVuDOTfYa0vUQsALO6Yqk5T4Q6CJkcfyMk5E0bws27Fp4CR0mMRYElZs Ijc8w7KmG2gf+qmCi4MhGyhZxcYM8GKnLxZ/FZ7lJ+BUk+wf+aZ9Ay1c+JdEB+On9uAD OMhFDhcMxQ86BONmfFpDQc7646GsMdIAmsOwD8J+dSHkAa+y/fyffFzL6xu1TW6FRrM2 +hW3/OD1zWphbIAMNgnu1NV5oRl8TlzxQATJV+tjxHEUm3W2jDrLqypGeXHI56K92UVW TeAIciyteccWyvO0I+H7wpFdOpvY7SaF0EewNQvfKriqSaBnavjpGH+tTlb7GKVhwD9z XiYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764299582; x=1764904382; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=kGL2VMPMFa8R+g6JKXrGDSEsUVYbVq9e7DRPdheTEI4=; b=AFCl9Irlz86gfi38zYXowanc1SwS10fKT+mPFRQHFlP/vT+ArnRHxIU0Yu2fRr47pj +IJjZzczSS7Crp805FMizlTN+sr87YYQqhD/n/opI4VuOvYXxIlzYr7s1A3CdR2LLW5B kuyNm9qH5Y1+lyDInxuXeAbFp821MaYFy87Wg6LMlvenxLUyZz83PnmCxCM67adOxzN7 v46w1J5q4o2G/59qiFiHIHd79laKNStHnz+kA5q6KYGX6mv0q+5/9VKiXTEILEzX8uu8 cVJ70AoK8lKWh8xMPZa1B+gQDaBGT3Z3Htx5dGqxUBcXupSp+vok8gn6veB5J5tng45/ gPrg== X-Forwarded-Encrypted: i=1; AJvYcCXXTUQdAfld1Yh7km/408r/HG/mMvWMJBt/rs3xvV1elbjvSWS9OpUYPXIJKkpbu8/NQHqcyOHyMJu5zCQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzmW673kL8GTNlZK7/vUZ0or37+NXEJnAbSgVtHVoekZQTYcy0E hGUNIFL4ZpcuUH5ACbuzGs/iant2NEGdLRULydRWjKLlCp50JO/Go1F0 X-Gm-Gg: ASbGncsgC4Hc04hBfeOddtw1+jU1nL6pMnucsOdDtMJEE0pneTV+QTqAjeNihJok1GK GQihTRlI3oHlbIVvp+p2Rivdq4B0smOalmV0KZSg8Dh5cHITQ7pqdGX/rV+GGbqm21zwkDAauQg E3PZd5/k+BzIMwt5bOB/M2SvcZGbEBwK9ap+CqGvBlaciBUMeNhDJTi8q9v8RDoiDiHcuFN8XAv x62uHmTeCnh0IzF5pjR6MM3vtftv6UDxK7OSGWogTdQ6FCTzreXSGiS1sWEzk5Skw2OJTgeRI1k 0466UGnhCVdo6EVtqxbCTFdH27QuXKousi2/anMefAyJwN666jeSnlKpwxriuiWlXGozdoah9l9 4zXyWxhuRmbqYLRDiaOrIo1aUKZ7EZLXh50D5EE3hRz+LMEqETqn/k2xbZzXaAXtK3s0JvFuQWO cXpxrU7zhXzgQ6OxdBcMFTqKIn3n/TJ+x7W2INNw== X-Google-Smtp-Source: AGHT+IFO3Qtu6YGB56UoXogoMpEAtR4mV4R5vs3RIIh4n0plscvgAihdnr/7wFahZAc/A1wHqxqldQ== X-Received: by 2002:a17:902:e545:b0:295:8c80:fb94 with SMTP id d9443c01a7336-29b6c6f1516mr261806555ad.59.1764299582259; Thu, 27 Nov 2025 19:13:02 -0800 (PST) Received: from zhr-ThinkStation-K.mioffice.cn ([2408:8607:1b00:8:3638:7863:b439:f5a7]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29bceb29fbbsm29576855ad.49.2025.11.27.19.12.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Nov 2025 19:13:01 -0800 (PST) From: Hongru Zhang X-Google-Original-From: Hongru Zhang To: akpm@linux-foundation.org, vbabka@suse.cz, david@kernel.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, Hongru Zhang Subject: [PATCH 3/3] mm: optimize free_area_empty() check using per-migratetype counts Date: Fri, 28 Nov 2025 11:12:42 +0800 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hongru Zhang Use per-migratetype counts instead of list_empty() helps reduce a few cpu instructions. Signed-off-by: Hongru Zhang --- mm/internal.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/internal.h b/mm/internal.h index 1561fc2ff5b8..7759f8fdf445 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -954,7 +954,7 @@ int find_suitable_fallback(struct free_area *area, unsi= gned int order, =20 static inline bool free_area_empty(struct free_area *area, int migratetype) { - return list_empty(&area->free_list[migratetype]); + return !READ_ONCE(area->mt_nr_free[migratetype]); } =20 /* mm/util.c */ --=20 2.43.0