From nobody Tue Dec 16 16:24:03 2025 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CCE654FBE for ; Fri, 12 Apr 2024 11:49:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922559; cv=none; b=khRy1WfFWoo9zJy/syQToFDGt1NJr2tyyiE5yY7yhHzlcIeGtVtYUM/1Kinba61gYEdwPB6xwXBNnfuvKAaP+q21bjNwKRTP9VLRY19SDMblUTEMf5/uds0JNXQOMNr4/8zA8ZIA846L9kH4PkXWnOgz8V8cVbgVzNtePE4QpXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922559; c=relaxed/simple; bh=/NCU8L1nyBke5rCK/5TqCMZiK2pSQoaU6KW5LiSJJ8o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IX5nBfewmGQ43cuMc13WdWaTAbUzDq1e76xyncdb8ZvmOw9Y7Gugi06bTnb+bNH7Y5MbjGtZQKMu6+TnUf5KL5MKeIVHGKdY/hFX9QrtcVOoF0XUlcrZeSQtoHjz8wDqi6GYRWD/uJkgRowQRPFsyNYMTsfCfn+4l9XliiVfQ/4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZWDWZRAW; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZWDWZRAW" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-6ecf8ebff50so567684b3a.1 for ; Fri, 12 Apr 2024 04:49:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712922558; x=1713527358; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EBqcW3yGOLnf8fCkLXd+q0+NdDQ6JnFQ/vzKSqccdZY=; b=ZWDWZRAWuC8vPzK65YDH7lrK0I4uudtfAYXgOfXVW5VbLHmnROIM8vcNoj3fU6wtDr u7chhslYFJDjpG+tsqTt062s82WlLvgFDH8iWn6ZoSIBdNbDa////v4hfPPCI8V67Kvd e9DqDw9MGR8f43rd3okH4tefQClbjmhHi4DzJAKwODndH6nngquLXLuDVQmOjR/TuiCn +5qjuqQuymhaQtSiuZz+rQO2T70qldF6fhEgw88GGiDAbxWa1yO2QlxAVpe13NjRwJl5 rLDgV4/KYPXpL6FrWc8l5Rg1xUnGlnSnYJHB6KMTw6LI77IAoiiXVwmle4fyKYfhTE8M VGRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712922558; x=1713527358; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EBqcW3yGOLnf8fCkLXd+q0+NdDQ6JnFQ/vzKSqccdZY=; b=fuvrOI3FUs4Qr5iwdUqfR0FomfziSvaPFZYb/9yExgiFVBWmUxsnnhz+TIOtUHWZDx X/W2lljd0lFDeigFBLF1UYecCznOswnBq2eI+bW0quc23MzXU+HxLXz3rwCnRKAYC45o zxTiDhLjmmG9E1mYQxfY3ER/BkBWdXAUXR8zx31gMPaI2kUu31L6rroluu3ZQP9+awte ukjnVhXAJ6tnHV79EGFdA7dlLiquIQ5+I81ezlznFjpkYhXM2gK3yhWsJ6Urk/qTg04A ILrQtSdjPl57VT4i50SCnZURTOLZMmIcwG/fmMaHUtmQczSlci3PYhl5lQ+x5ugDujA9 Grlg== X-Forwarded-Encrypted: i=1; AJvYcCXIcio6leFQ72ZYg9C3EjM1e9w65z/PSMnAeMiTAUWLCgwLYJHhcqtGHnr7qzfE4/STiy65nCCKhSEj44y8whfAW9ZI7wTFXqr2vb6w X-Gm-Message-State: AOJu0Yx59o+JVuoR4JgjjRfQj74lkYxm4WtcKZ88XJ2GEa+Exa+HPLUK Ecw9cSeAku7u0IeWyEvwlkecVqstQFghD2mZhZETJN0i8RmIvCGQOT54GO8d X-Google-Smtp-Source: AGHT+IGaTHWz4rwtGHc+sgkHopCT3nCaKwwOngZTn+zC3VlvQfYHhe+0iaWXVFi2u44WZCge+8Zqcg== X-Received: by 2002:a05:6a21:3406:b0:1a8:58a2:ea41 with SMTP id yn6-20020a056a21340600b001a858a2ea41mr3325699pzb.9.1712922557669; Fri, 12 Apr 2024 04:49:17 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id q1-20020a634301000000b005e43cce33f8sm2541048pga.88.2024.04.12.04.49.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 04:49:17 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v6 1/4] mm: add per-order mTHP anon_fault_alloc and anon_fault_fallback counters Date: Fri, 12 Apr 2024 23:48:55 +1200 Message-Id: <20240412114858.407208-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412114858.407208-1-21cnbao@gmail.com> References: <20240412114858.407208-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Barry Song Profiling a system blindly with mTHP has become challenging due to the lack of visibility into its operations. Presenting the success rate of mTHP allocations appears to be pressing need. Recently, I've been experiencing significant difficulty debugging performance improvements and regressions without these figures. It's crucial for us to understand the true effectiveness of mTHP in real-world scenarios, especially in systems with fragmented memory. This patch establishes the framework for per-order mTHP counters. It begins by introducing the anon_fault_alloc and anon_fault_fallback counters. Additionally, to maintain consistency with thp_fault_fallback_charge in /proc/vmstat, this patch also tracks anon_fault_fallback_charge when mem_cgroup_charge fails for mTHP. Incorporating additional counters should now be straightforward as well. Signed-off-by: Barry Song Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao Acked-by: David Hildenbrand Reviewed-by: Ryan Roberts --- include/linux/huge_mm.h | 21 +++++++++++++++++ mm/huge_memory.c | 52 +++++++++++++++++++++++++++++++++++++++++ mm/memory.c | 5 ++++ 3 files changed, 78 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index e896ca4760f6..d4fdb2641070 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -264,6 +264,27 @@ unsigned long thp_vma_allowable_orders(struct vm_area_= struct *vma, enforce_sysfs, orders); } =20 +enum mthp_stat_item { + MTHP_STAT_ANON_FAULT_ALLOC, + MTHP_STAT_ANON_FAULT_FALLBACK, + MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + __MTHP_STAT_COUNT +}; + +struct mthp_stat { + unsigned long stats[ilog2(MAX_PTRS_PER_PTE) + 1][__MTHP_STAT_COUNT]; +}; + +DECLARE_PER_CPU(struct mthp_stat, mthp_stats); + +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ + if (order <=3D 0 || order > PMD_ORDER) + return; + + this_cpu_inc(mthp_stats.stats[order][item]); +} + #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ (1<stats[order][item]; + } + + return sum; +} + +#define DEFINE_MTHP_STAT_ATTR(_name, _index) \ +static ssize_t _name##_show(struct kobject *kobj, \ + struct kobj_attribute *attr, char *buf) \ +{ \ + int order =3D to_thpsize(kobj)->order; \ + \ + return sysfs_emit(buf, "%lu\n", sum_mthp_stat(order, _index)); \ +} \ +static struct kobj_attribute _name##_attr =3D __ATTR_RO(_name) + +DEFINE_MTHP_STAT_ATTR(anon_fault_alloc, MTHP_STAT_ANON_FAULT_ALLOC); +DEFINE_MTHP_STAT_ATTR(anon_fault_fallback, MTHP_STAT_ANON_FAULT_FALLBACK); +DEFINE_MTHP_STAT_ATTR(anon_fault_fallback_charge, MTHP_STAT_ANON_FAULT_FAL= LBACK_CHARGE); + +static struct attribute *stats_attrs[] =3D { + &anon_fault_alloc_attr.attr, + &anon_fault_fallback_attr.attr, + &anon_fault_fallback_charge_attr.attr, + NULL, +}; + +static struct attribute_group stats_attr_group =3D { + .name =3D "stats", + .attrs =3D stats_attrs, +}; + static struct thpsize *thpsize_create(int order, struct kobject *parent) { unsigned long size =3D (PAGE_SIZE << order) / SZ_1K; @@ -549,6 +591,12 @@ static struct thpsize *thpsize_create(int order, struc= t kobject *parent) return ERR_PTR(ret); } =20 + ret =3D sysfs_create_group(&thpsize->kobj, &stats_attr_group); + if (ret) { + kobject_put(&thpsize->kobj); + return ERR_PTR(ret); + } + thpsize->order =3D order; return thpsize; } @@ -880,6 +928,8 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct v= m_fault *vmf, folio_put(folio); count_vm_event(THP_FAULT_FALLBACK); count_vm_event(THP_FAULT_FALLBACK_CHARGE); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_FALLBACK); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); return VM_FAULT_FALLBACK; } folio_throttle_swaprate(folio, gfp); @@ -929,6 +979,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct v= m_fault *vmf, mm_inc_nr_ptes(vma->vm_mm); spin_unlock(vmf->ptl); count_vm_event(THP_FAULT_ALLOC); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC); count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC); } =20 @@ -1050,6 +1101,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault= *vmf) folio =3D vma_alloc_folio(gfp, HPAGE_PMD_ORDER, vma, haddr, true); if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_FALLBACK); return VM_FAULT_FALLBACK; } return __do_huge_pmd_anonymous_page(vmf, &folio->page, gfp); diff --git a/mm/memory.c b/mm/memory.c index 649a547fe8e3..f31da2de19c6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4368,6 +4368,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) folio =3D vma_alloc_folio(gfp, order, vma, addr, true); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { + count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); folio_put(folio); goto next; } @@ -4376,6 +4377,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) return folio; } next: + count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK); order =3D next_order(&orders, order); } =20 @@ -4485,6 +4487,9 @@ static vm_fault_t do_anonymous_page(struct vm_fault *= vmf) =20 folio_ref_add(folio, nr_pages - 1); add_mm_counter(vma->vm_mm, MM_ANONPAGES, nr_pages); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_FAULT_ALLOC); +#endif folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); setpte: --=20 2.34.1 From nobody Tue Dec 16 16:24:03 2025 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 437D654FBE for ; Fri, 12 Apr 2024 11:49:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922565; cv=none; b=ryxWyZqPjw6PUoy4BPt9auuM/xcBmg9x1fGKEuktrAWehRjcwkXR6eEB2GCkytN31p14pRRCiB2Am5PfT4xGth33zX4rPAejoSOgeYlmVKnEpafmxjNk8Xwbj8Nh01KfsCKu7GMEmbzNbVCfYGxzKqhNFOEbVMXVA8b/9mlDgCc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922565; c=relaxed/simple; bh=S6XPYVNWwHAZsyPPLbqtfAowLOKQYeJfl0zecQgrprE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KvvIbjxEXvH/W2M6B7gI/Y9tBThmv/SpNCgccR7JI27kdojWFKpMgB7ngjRqhuwMvSRRdYs/R86TclrOUOJmVqNdT1Laq3HD2DaxoTwRC+kCSqopEDZtcdxshVrsE8Btb7ZSeOCXh3GhVEzbuougTE+ELma6jcx1DvU6OSL1CFM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hyhRuWbx; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hyhRuWbx" Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6ee12766586so553303b3a.0 for ; Fri, 12 Apr 2024 04:49:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712922563; x=1713527363; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rsGNE4XVP3pLkOAKr41E6LqkIK87UvM+YC6wxpcMFv8=; b=hyhRuWbxHsUXDlOA1ozuuqJILYLSXaotKAR+MNCTamLBwr6AMNg6xoDuW+C3A322EN 2N47NNwUNNca/r7uU/Yjt88IukpyDMi0YT60VqpgdwlB9/K+lbLeZRk+smGFRgFMIHyx LGb8FCUnpy4b1are5GMJzb5PS9oX0vFVDY1PODpxUJisZsw8zHmo+WUlGRxpBw+r0AlC uuJiuAXil9K1tcC6nzjRKEd5m7bkDgjfUA76YhxHTor86ieb4ZnikuPrGsax2YNi/oJ/ 8CeYeTz9g8rwAZl182JP2uRaUP5gsoJy8dAAryc6uSrxquXjueBE24Op515K0w4+RWDc +iBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712922563; x=1713527363; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rsGNE4XVP3pLkOAKr41E6LqkIK87UvM+YC6wxpcMFv8=; b=Ov9ATtS3RnacgNJm9eRDnkr0bSbaJrIUi4Hd6XCuzFLYeIJ7VaccmpLPq6GSlorfGs nEQ5X78YcXpH5Djjaw8NRbU0l1B6dK84eFLpixUgvdtrVZOt/Q1FbCiGfqBBz0K3vB7e y7J3h9zGRuVoZyDoeUEbllq1p9kMjgbziFTo19rU+Rl3IAXDp4Acou5JCTtGUqHssfQP 9H5O6tfjjbvjvc00NN9uky09lDe90hO8TfVSngvTiSE2qzFz7TppvJNC605s+CLYkHy0 pqD0zGSoH6+A74h+KMEMCACRwXwm7g8w5aCD5l9uV8BYAt1Hr7Orq8OKwz4H1KkWRDvG oEvw== X-Forwarded-Encrypted: i=1; AJvYcCUPvTV3u/Q7uvFWIXf1LGgiO8xpZYVZ3Dt+4d0uGYcZ4+wPnL508FSW2ums1FIFVmc77tzwsM7qZ9g0Z3CS9Vhea0eDNfG+/jNQMDnF X-Gm-Message-State: AOJu0Yy21LGYaCLp64PQTyZZWUHNvJXwIxNLFx8hs9bb4v8+9B2bHYAe eYsLYq8KyuKlc2BAI+9fgPI9/kDhH11fkXeP8N7LgP/YhxtdE2lnTw0tlW7W X-Google-Smtp-Source: AGHT+IGT5HsRUdIKeuS97P1LcQaXZ4m6uHGIP6/+AXNUFnrOjD3L9oLUEEO5s/R9jaTkOrY3eJcOCw== X-Received: by 2002:a05:6a00:10c7:b0:6ed:825b:30c0 with SMTP id d7-20020a056a0010c700b006ed825b30c0mr7228196pfu.15.1712922563594; Fri, 12 Apr 2024 04:49:23 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id q1-20020a634301000000b005e43cce33f8sm2541048pga.88.2024.04.12.04.49.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 04:49:23 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v6 2/4] mm: add per-order mTHP anon_swpout and anon_swpout_fallback counters Date: Fri, 12 Apr 2024 23:48:56 +1200 Message-Id: <20240412114858.407208-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412114858.407208-1-21cnbao@gmail.com> References: <20240412114858.407208-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Barry Song This helps to display the fragmentation situation of the swapfile, knowing the proportion of how much we haven't split large folios. So far, we only support non-split swapout for anon memory, with the possibility of expanding to shmem in the future. So, we add the "anon" prefix to the counter names. Signed-off-by: Barry Song Reviewed-by: Ryan Roberts Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao Acked-by: David Hildenbrand --- include/linux/huge_mm.h | 2 ++ mm/huge_memory.c | 4 ++++ mm/page_io.c | 1 + mm/vmscan.c | 3 +++ 4 files changed, 10 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index d4fdb2641070..7cd07b83a3d0 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -268,6 +268,8 @@ enum mthp_stat_item { MTHP_STAT_ANON_FAULT_ALLOC, MTHP_STAT_ANON_FAULT_FALLBACK, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + MTHP_STAT_ANON_SWPOUT, + MTHP_STAT_ANON_SWPOUT_FALLBACK, __MTHP_STAT_COUNT }; =20 diff --git a/mm/huge_memory.c b/mm/huge_memory.c index dfc38cc83a04..58f2c4745d80 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -555,11 +555,15 @@ static struct kobj_attribute _name##_attr =3D __ATTR_= RO(_name) DEFINE_MTHP_STAT_ATTR(anon_fault_alloc, MTHP_STAT_ANON_FAULT_ALLOC); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback, MTHP_STAT_ANON_FAULT_FALLBACK); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback_charge, MTHP_STAT_ANON_FAULT_FAL= LBACK_CHARGE); +DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT); +DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK= ); =20 static struct attribute *stats_attrs[] =3D { &anon_fault_alloc_attr.attr, &anon_fault_fallback_attr.attr, &anon_fault_fallback_charge_attr.attr, + &anon_swpout_attr.attr, + &anon_swpout_fallback_attr.attr, NULL, }; =20 diff --git a/mm/page_io.c b/mm/page_io.c index a9a7c236aecc..46c603dddf04 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -217,6 +217,7 @@ static inline void count_swpout_vm_event(struct folio *= folio) count_memcg_folio_events(folio, THP_SWPOUT, 1); count_vm_event(THP_SWPOUT); } + count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_SWPOUT); #endif count_vm_events(PSWPOUT, folio_nr_pages(folio)); } diff --git a/mm/vmscan.c b/mm/vmscan.c index bca2d9981c95..49bd94423961 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1231,6 +1231,8 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, goto activate_locked; } if (!add_to_swap(folio)) { + int __maybe_unused order =3D folio_order(folio); + if (!folio_test_large(folio)) goto activate_locked_split; /* Fallback to swap normal pages */ @@ -1242,6 +1244,7 @@ static unsigned int shrink_folio_list(struct list_hea= d *folio_list, THP_SWPOUT_FALLBACK, 1); count_vm_event(THP_SWPOUT_FALLBACK); } + count_mthp_stat(order, MTHP_STAT_ANON_SWPOUT_FALLBACK); #endif if (!add_to_swap(folio)) goto activate_locked_split; --=20 2.34.1 From nobody Tue Dec 16 16:24:03 2025 Received: from mail-oo1-f47.google.com (mail-oo1-f47.google.com [209.85.161.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF68E3FE55 for ; Fri, 12 Apr 2024 11:49:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922572; cv=none; b=ca8P0UTIHbPvLVVj24jIYyUJoMscKPGV4tpZc08o7yr9yo/p420R3M1/L2QWmVOzsHlVE1CvTUrRnbyiDXm5rDGr/8xtT0PnIAtyP1KSxIBhyrhTUftMGsReIpjsmhmLxrUBe4bYYQ8iaEcZ7BPazAOIxp76dWOXvEUlTNuxC6Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922572; c=relaxed/simple; bh=ttBpAQxgjjw4lzD2Bvn2GcELVPDXaZwZiKdPqI4o07g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=K2IInc0Hp/hW6zGdfDg/RrW6eL6Qn2CORz0tFai2RRYV3hFC5y6PCsj0xzOVEaY864rlKz7WBqolAQrIdUADNCGA86R4jLu1RAjdV3aWoEhRJoCoLQdJMNC/ciVQULXJafPXCQaWXGTumHtKFdOCNgs0Wv+D4JCTx3dH1XI/FXI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=X4U1NLp8; arc=none smtp.client-ip=209.85.161.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="X4U1NLp8" Received: by mail-oo1-f47.google.com with SMTP id 006d021491bc7-5aa25e99414so554437eaf.3 for ; Fri, 12 Apr 2024 04:49:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712922570; x=1713527370; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dh6hvogc8WrNbjngKn7rl7SAnmLGjB7yuHqyvbwFdGE=; b=X4U1NLp8LUNjVtL0MckGrHi3kJSnIACcGO1vvohfYvbEW8TS2XVrZlr3amggTlPh8W 5G5J82IpD+vKw48/9jk4agYdocaXW1EoUZbh27dMBojW+V1ExRGgiMm2/2vBesmjkWlN XxzZ88+n2OWykamW42eX3TWF41j6tim6j3FIX2aj5C9iHkDaZOjVBkyFA3G/NSt+FKvp q8RdTRXChkGNUFW9dzGLDGWpZ4XgyRYSFHj57v56urIxGp1Ui1gs8MdCrbHxJ+UNwZHC J+pmX4pYv/lW2/k1/LWe7oIjmcPMJ7UdY9XUNEsDBwJZMLyLDphj45Ex/LDAE1EpRewX 24YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712922570; x=1713527370; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dh6hvogc8WrNbjngKn7rl7SAnmLGjB7yuHqyvbwFdGE=; b=wpJgwH+iTX6tslJsni5hz4FxzPdFGYHYDGQQihaGgTDRtf2F0J/nxrbWloo/qVF3c6 y1Jbr8ufAaoAeDgOmLidk8qps/e1utVLNltiyNQvLTvpWECuzqT/o1w2cxmlLMyOjldT 8Z8fzduXYv6jQcxZB0CBlRyWCaduaGVe4cejI4O/94owHlLSnFmo0luqpML9yirEKOwv 8yPIRAgqXyBEgC5heY31gDzKVn9NmsICGMUhxTADGrk/QHsT94A1+D7iZQxBnko6Tkmh nToX3ucnMnzixt58ec5wej3eQoxQ8VDdRjtHgqEXpAcShnz0pvkc9MLuuLQ9ZwQeOLN+ pOyA== X-Forwarded-Encrypted: i=1; AJvYcCWssDb5u/rVcHT5viOmHnlEDI6kslcdbxDRiHZEpjWUDhTOhq5TsDHdVGyVcOHPwGUYhrvRUfb+dLWeoYRKe5+hc1k/ZLh2ehmSjALE X-Gm-Message-State: AOJu0YyMIf0YP4PZqCRZTcJiOeGj8TbI9gPEu5CpH3lpheqBryky45xw GMOetqlCIHs/2mI6EqB7ERytey0reZBn4XRu5PFElLWxb5PhB4qmOEScjGkO X-Google-Smtp-Source: AGHT+IHSbMtHAQIh+PfqRG0gW0Y1lSnuI10KtMX9wyKQO0P5Kw6DL1x3fzgwy290LKu7YQyO4BqMtg== X-Received: by 2002:a05:6359:4608:b0:186:2a96:a5b9 with SMTP id my8-20020a056359460800b001862a96a5b9mr2626632rwb.6.1712922569322; Fri, 12 Apr 2024 04:49:29 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id q1-20020a634301000000b005e43cce33f8sm2541048pga.88.2024.04.12.04.49.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 04:49:28 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v6 3/4] mm: add docs for per-order mTHP counters and transhuge_page ABI Date: Fri, 12 Apr 2024 23:48:57 +1200 Message-Id: <20240412114858.407208-4-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412114858.407208-1-21cnbao@gmail.com> References: <20240412114858.407208-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Barry Song This patch includes documentation for mTHP counters and an ABI file for sys-kernel-mm-transparent-hugepage, which appears to have been missing for some time. Signed-off-by: Barry Song Reviewed-by: Ryan Roberts Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao Cc: Jonathan Corbet Reviewed-by: David Hildenbrand --- .../sys-kernel-mm-transparent-hugepage | 17 +++++++++++ Documentation/admin-guide/mm/transhuge.rst | 28 +++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 Documentation/ABI/testing/sys-kernel-mm-transparent-hug= epage diff --git a/Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage b= /Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage new file mode 100644 index 000000000000..33163eba5342 --- /dev/null +++ b/Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage @@ -0,0 +1,17 @@ +What: /sys/kernel/mm/transparent_hugepage/ +Date: April 2024 +Contact: Linux memory management mailing list +Description: + /sys/kernel/mm/transparent_hugepage/ contains a number of files and + subdirectories, + - defrag + - enabled + - hpage_pmd_size + - khugepaged + - shmem_enabled + - use_zero_page + - subdirectories of the form hugepages-kB, where + is the page size of the hugepages supported by the kernel/CPU + combination. + + See Documentation/admin-guide/mm/transhuge.rst for details. diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/adm= in-guide/mm/transhuge.rst index 04eb45a2f940..e0fe17affeb3 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -447,6 +447,34 @@ thp_swpout_fallback Usually because failed to allocate some continuous swap space for the huge page. =20 +In /sys/kernel/mm/transparent_hugepage/hugepages-kB/stats, There are +also individual counters for each huge page size, which can be utilized to +monitor the system's effectiveness in providing huge pages for usage. Each +counter has its own corresponding file. + +anon_fault_alloc + is incremented every time a huge page is successfully + allocated and charged to handle a page fault. + +anon_fault_fallback + is incremented if a page fault fails to allocate or charge + a huge page and instead falls back to using huge pages with + lower orders or small pages. + +anon_fault_fallback_charge + is incremented if a page fault fails to charge a huge page and + instead falls back to using huge pages with lower orders or + small pages even though the allocation was successful. + +anon_swpout + is incremented every time a huge page is swapped out in one + piece without splitting. + +anon_swpout_fallback + is incremented if a huge page has to be split before swapout. + Usually because failed to allocate some continuous swap space + for the huge page. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help --=20 2.34.1 From nobody Tue Dec 16 16:24:03 2025 Received: from mail-oo1-f41.google.com (mail-oo1-f41.google.com [209.85.161.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAE553FE55 for ; Fri, 12 Apr 2024 11:49:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922577; cv=none; b=rfG/BFGWxGfcAu+RK1GVH2+oa+YPgeq0WcpwpKt5/J0XqBDH7EmgixYsJg6lfMhkAc7xLyt2Xr37Nupr9OjypHKzECbmCO7+E0z/QDjH+XZ00Sw4GpNiaCwQhh+gdzL9NFa577Rb1rdi+r+2CPLVg0VHz0RJdH5Hq8Hrw6LsSK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712922577; c=relaxed/simple; bh=+fRyfCSzpNT7RQu6k6IdskSk6S1/8EuA2BI6cr2KdHY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WTn5NDfGHluOnYYNpjYqSSokiWfB8bSp4Da/pUbewUhZor0rXOBU3RPHyv95AMPrN0cYLE09+cAceKSPmIwmb3wtA2zMfCmTCKlzQdYQQigTz2LNbFIEzqsbD9T/JFGrHobGPRDnkoM3MaNND7HvyZFuVlForiwOqG01eDO2Hmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gVsV8e3K; arc=none smtp.client-ip=209.85.161.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gVsV8e3K" Received: by mail-oo1-f41.google.com with SMTP id 006d021491bc7-5aa17c69cf7so570514eaf.2 for ; Fri, 12 Apr 2024 04:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712922575; x=1713527375; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JkmaL5dnbNToitX+dr2yruhYVAPZI2DFDH1TSVGRV7w=; b=gVsV8e3KdSMmsdTgBQ6ljgP/+/l1Qreppdw9qiYrpJDKZeFomparVunLxmk7JdrgkU P20QR7aIvlz9jGdJcwl3W8EeZWz2rk/PDO/yS7ONGFEy2jUGWp5xAYFo8nNa5ikypkuH VYVv6fgcq+PxOPCGbUJfidOXLFrT9nn/htDwnEe8KzDxkgs+78hxDu13Kbbaq/v9OA8e TEq+dlmalHQppv45uDT9KF0ePI3VLioWhK2VjsQkUtEGcU1rNhSXBzI3SAVFc6G4fFnp /pmZX4Rs+/+blphOVJKqWNAedVrz4t0QugFrTQ96UU7FF1eXFFfJfUFQaRbRULPCShrx a2XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712922575; x=1713527375; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JkmaL5dnbNToitX+dr2yruhYVAPZI2DFDH1TSVGRV7w=; b=o+ZMekDwajH7VJu33jHKxRjN5+OsBjCmoQfdu5dBGq1WsQsCQfZ/XxfcdnwJyRczXV he7SIKrumhWj8rYovXLRT1A2YyJOQWKo7r8331SOVOK8Pgu6P+TTl6VUyZKLSBtHf7+D ibGsjnma0CovkFvR2sAspoiSjVCI8/a5cj/XkMJEHvwEw2aO63EcsxMriZ+ygs54JUET urgenF5biKcxxy/pLaZls4YV2Yzs9CTvxIWlAuZ+OrIPYZk9Y0vg0yHgxC4H/a5Pk1ha zcKUQ/29sqpy3Z3R+CPHCimtHIt6xjuX78BlEnGWOORx+hWYJfJfSOFbQj01QFPyGW1X jWuA== X-Forwarded-Encrypted: i=1; AJvYcCXXTHGga6OYuhgiACERSdNqRQdxbbtzUhz/TgAOXGr3jVfUYmG64K05ouWUdspSWi4SL9j4wWBP9GsHrZUdq0ERILOfUFuVYVF5hmwW X-Gm-Message-State: AOJu0YyTgTdkoVJCLxiaKyZ7N3ybhpclayjCXBYN2az+JgMP38Q3ObQU GMY7yRTCq6AUEbs6U+K06uSkPDKk7ppIzko5RU99vKDsSe66F9Oo X-Google-Smtp-Source: AGHT+IELpsWoBmGsNvBBR/x1yJEqJQU0ioXXjh7/o28t3fv1i7+x6sXnF0k1SDCwzsvXK5vX3cvAmg== X-Received: by 2002:a05:6871:b27:b0:22e:a204:9bb5 with SMTP id fq39-20020a0568710b2700b0022ea2049bb5mr2485122oab.53.1712922575076; Fri, 12 Apr 2024 04:49:35 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id q1-20020a634301000000b005e43cce33f8sm2541048pga.88.2024.04.12.04.49.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 04:49:34 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v6 4/4] mm: correct the docs for thp_fault_alloc and thp_fault_fallback Date: Fri, 12 Apr 2024 23:48:58 +1200 Message-Id: <20240412114858.407208-5-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412114858.407208-1-21cnbao@gmail.com> References: <20240412114858.407208-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Barry Song The documentation does not align with the code. In __do_huge_pmd_anonymous_page(), THP_FAULT_FALLBACK is incremented when mem_cgroup_charge() fails, despite the allocation succeeding, whereas THP_FAULT_ALLOC is only incremented after a successful charge. Signed-off-by: Barry Song Reviewed-by: Ryan Roberts Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao Cc: Jonathan Corbet Reviewed-by: David Hildenbrand --- Documentation/admin-guide/mm/transhuge.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/adm= in-guide/mm/transhuge.rst index e0fe17affeb3..f82300b9193f 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -369,7 +369,7 @@ monitor how successfully the system is providing huge p= ages for use. =20 thp_fault_alloc is incremented every time a huge page is successfully - allocated to handle a page fault. + allocated and charged to handle a page fault. =20 thp_collapse_alloc is incremented by khugepaged when it has found @@ -377,7 +377,7 @@ thp_collapse_alloc successfully allocated a new huge page to store the data. =20 thp_fault_fallback - is incremented if a page fault fails to allocate + is incremented if a page fault fails to allocate or charge a huge page and instead falls back to using small pages. =20 thp_fault_fallback_charge --=20 2.34.1