From nobody Mon Dec 1 22:03:50 2025 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9BA52C027C for ; Fri, 28 Nov 2025 03:12:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299524; cv=none; b=eTu/OBxnbknuuBGqEyJ3t16WdXG+Kv5G8AegEnDgOr91qXWKDdoUOrrG1zaKFaGgqeY9zidIGg0rkLJ2SdoICdXgSrysloEdm2GRZihR0Rav/YykGFjuKmHOWIyN1DulWwnIk6/o3AueSZfcLc3IRACmn3ziAE6qNYDlGDvru70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764299524; c=relaxed/simple; bh=peKOX3tDaJjI+d9s6OHERNkephqMZYbfewyZNQKm5d4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s0zcxHxOCWKFlfd1PIowWaOaPMBtFpaQISrkSxPj1IvFGRL3LVYNOeBKUTszToQgVeoVTguhHdZgnCK4wJBO6/NHbpw20BNow39fXHnJsvirqvJXxW39Ao0XvVyPf1kqOncATuJJJulCu5mahRs6kjVv1OhRtzgl7SKOyeTC0Jc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=m8qqQfH3; arc=none smtp.client-ip=209.85.210.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="m8qqQfH3" Received: by mail-pf1-f194.google.com with SMTP id d2e1a72fcca58-7ade456b6abso1096923b3a.3 for ; Thu, 27 Nov 2025 19:12:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764299522; x=1764904322; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UURN4qing9DSF3ysybnLESXjHzZIzZMUCK3v5bSZiLw=; b=m8qqQfH35mZrDH1hrNSqoJ8lkxBvQJ7xTc3nXT3fmGQp2ngCgqCZeKHhmvlgPItNum abIR32tZmrHQBVvbjHh1tEKXLHAJ8Fs1edIx9/HeJKlNsS7CQ7wsRLoMn9JF3CQGCCRx N8n+KYRPZgCEZJSp+a7Yo7Pk4zZ0m+Vvvkc8nzFTYLjyIUPW5Es9aJzMedUX7UJa+1wK EIj23cMvmZRylbbEju+H+Rr1evKQnydp9uEViDRgmu/1CnS5zB61p9i+O2VPqOAiKhpJ cMPo0GvL677P34xHbCLw5FyKPbybGCvbbrLa+jRSBKoVxYYWEW+CJv8jx2/2K3eVFiv6 I6oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764299522; x=1764904322; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UURN4qing9DSF3ysybnLESXjHzZIzZMUCK3v5bSZiLw=; b=NsbsvL2upc6xgKgXWziyU5cHbUqAvSGr+iRSCbfb8G7T4pM91fCAwpXNqAso+dnqRD 1ims6Ngy9QdW/gBll5t0ETc6/T2OChSgZUwCrEJpEuavdEHrpwxog+ZF54LbtzbIndi8 g+DNWGL7c35xy+3LmBSuPZRyH5xJ43bSC7FdIKnoWZm9wKi+aplV4bCsiG01anYqDh+/ Q9LnFcTjhWQ4APh2vuq1x0/Nf7jXDTO08rMjeZiYGYGNvDjhvZ6FlfRWu+yxKpfvU+bK VZ/9hvQaavhp5uX4dBbxU/oMHbMw2AOv3Pm+0x1jWoN2TW3O66mPfo5gk8sHqqiB7VDI YDIQ== X-Forwarded-Encrypted: i=1; AJvYcCVL4+F/Pth0Dryq/Yj5JS79zzrvpDcJv0P0r0eK5h6miWMiGWJGXX2qtCF2mQbFEgJl/V6NpiWKQkVoSoU=@vger.kernel.org X-Gm-Message-State: AOJu0YylttdJu+40fMFQ3+tVybIqLd2SV1AtD5e+781gk/6Mhw4xWfh6 YIpYPpCsJBPq3iWKF9WW0OvIWFYU5Ul3jlmQ+AsUOl0r3RyB2A/xCl+r X-Gm-Gg: ASbGnctqPMl4lX8oDf+RZt+SfrwBIVL8DhX11RB/v6iSP2MGU5hqZaPo8caRMZ8XLkS ZCAx3BAwmDtHDGzkN877ydpjMooM7gGZCqWeDno7FIr6FgTxq4n32p2YTrg8sINGqflUfkwkZVB KCajWRZ0kyNSvjmnSAH8RZi9QTEA91077HVgaf+WAO0TnJd6dRjDFPsc16jR+K5T5yPCP/y2Ma/ EGqGenWrDfVAuOMbTAD58oEybe8C1QJ0FfVsa8lLxxAEkM8osZ9FC0B6HX0bRL5WtijVNatw9gt ITgX8snSejAKDPdxK6VCQxLX1qvhoZFPu17j8xYsjxoHOAxt/MgOdlsvpPk52TZuKYhN0n5kOum +qISvpG8OKxkjOVVeT8a8bWUoqXFlQwrLxDRFjAgJocJ1u/BfWi7cZWfGRUoCX6vSCmbL7yc5Gb POev+bz0ioskpajCbYO5t9vKIerpqPqPM039gL8Q== X-Google-Smtp-Source: AGHT+IH01szTUvjUWPgozI/rwQ/A0QsBtn0PFJolC4W+66Z3l+siWZ7TigqC/fsl6tVQiWrY7nIbtw== X-Received: by 2002:a05:6a00:8d6:b0:7ab:e844:1e76 with SMTP id d2e1a72fcca58-7c58c2b20b5mr23898778b3a.5.1764299522057; Thu, 27 Nov 2025 19:12:02 -0800 (PST) Received: from zhr-ThinkStation-K.mioffice.cn ([2408:8607:1b00:8:3638:7863:b439:f5a7]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7d15f080cf1sm3245651b3a.46.2025.11.27.19.11.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Nov 2025 19:12:01 -0800 (PST) From: Hongru Zhang X-Google-Original-From: Hongru Zhang To: akpm@linux-foundation.org, vbabka@suse.cz, david@kernel.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, Hongru Zhang Subject: [PATCH 1/3] mm/page_alloc: add per-migratetype counts to buddy allocator Date: Fri, 28 Nov 2025 11:11:42 +0800 Message-ID: <37a6c6c35e029c3429f236d5895c898140d991eb.1764297987.git.zhanghongru@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hongru Zhang On mobile devices, some user-space memory management components check memory pressure and fragmentation status periodically or via PSI, and take actions such as killing processes or performing memory compaction based on this information. Under high load scenarios, reading /proc/pagetypeinfo causes memory management components or memory allocation/free paths to be blocked for extended periods waiting for the zone lock, leading to the following issues: 1. Long interrupt-disabled spinlocks - occasionally exceeding 10ms on Qcom 8750 platforms, reducing system real-time performance 2. Memory management components being blocked for extended periods, preventing rapid acquisition of memory fragmentation information for critical memory management decisions and actions 3. Increased latency in memory allocation and free paths due to prolonged zone lock contention This patch adds per-migratetype counts to the buddy allocator in preparation for optimizing /proc/pagetypeinfo access. The optimized implementation: - Make per-migratetype count updates protected by zone lock on the write side while /proc/pagetypeinfo reads are lock-free, which reduces interrupt-disabled spinlock duration and improves system real-time performance (addressing issue #1) - Reduce blocking time for memory management components when reading /proc/pagetypeinfo, enabling more rapid acquisition of memory fragmentation information (addressing issue #2) - Minimize the critical section held during /proc/pagetypeinfo reads to reduce zone lock contention on memory allocation and free paths (addressing issue #3) The main overhead is a slight increase in latency on the memory allocation and free paths due to additional per-migratetype counting, with theoretically minimal impact on overall performance. Signed-off-by: Hongru Zhang --- include/linux/mmzone.h | 1 + mm/mm_init.c | 1 + mm/page_alloc.c | 7 ++++++- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 7fb7331c5725..6eeefe6a3727 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -138,6 +138,7 @@ extern int page_group_by_mobility_disabled; struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; + unsigned long mt_nr_free[MIGRATE_TYPES]; }; =20 struct pglist_data; diff --git a/mm/mm_init.c b/mm/mm_init.c index 7712d887b696..dca2be8cc3b1 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1439,6 +1439,7 @@ static void __meminit zone_init_free_lists(struct zon= e *zone) for_each_migratetype_order(order, t) { INIT_LIST_HEAD(&zone->free_area[order].free_list[t]); zone->free_area[order].nr_free =3D 0; + zone->free_area[order].mt_nr_free[t] =3D 0; } =20 #ifdef CONFIG_UNACCEPTED_MEMORY diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ed82ee55e66a..9431073e7255 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -818,6 +818,7 @@ static inline void __add_to_free_list(struct page *page= , struct zone *zone, else list_add(&page->buddy_list, &area->free_list[migratetype]); area->nr_free++; + area->mt_nr_free[migratetype]++; =20 if (order >=3D pageblock_order && !is_migrate_isolate(migratetype)) __mod_zone_page_state(zone, NR_FREE_PAGES_BLOCKS, nr_pages); @@ -840,6 +841,8 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, get_pageblock_migratetype(page), old_mt, nr_pages); =20 list_move_tail(&page->buddy_list, &area->free_list[new_mt]); + area->mt_nr_free[old_mt]--; + area->mt_nr_free[new_mt]++; =20 account_freepages(zone, -nr_pages, old_mt); account_freepages(zone, nr_pages, new_mt); @@ -855,6 +858,7 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, static inline void __del_page_from_free_list(struct page *page, struct zon= e *zone, unsigned int order, int migratetype) { + struct free_area *area =3D &zone->free_area[order]; int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, @@ -868,7 +872,8 @@ static inline void __del_page_from_free_list(struct pag= e *page, struct zone *zon list_del(&page->buddy_list); __ClearPageBuddy(page); set_page_private(page, 0); - zone->free_area[order].nr_free--; + area->nr_free--; + area->mt_nr_free[migratetype]--; =20 if (order >=3D pageblock_order && !is_migrate_isolate(migratetype)) __mod_zone_page_state(zone, NR_FREE_PAGES_BLOCKS, -nr_pages); --=20 2.43.0