From nobody Sun Feb 8 17:41:52 2026 Received: from mail-qt1-f174.google.com (mail-qt1-f174.google.com [209.85.160.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1524E2904 for ; Wed, 20 Mar 2024 02:42:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902546; cv=none; b=OTBFeBnR2yB2CKM/Io9/ZpDwz2w0C2K3/MtPdyCuFAj2UIm17x011rDlMTycV6zD7Q/3IuodYQfvQbQzCTEKnJc5SFEGMW1hKbu3/1+ptikGMfYU4n3YoOOVyMl4oYIHKkYQK/dsD9M8ZCcvn6mK7wB0gDsCzsxoP8t5VFDRJGo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902546; c=relaxed/simple; bh=bfxKiZ2Bpuz4dSJFBwllmbdyHFW8D4nedhUWJ4eIxpQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=h6KVruLs+gvIa6T09CrEWSiNIPsKgrA05QBN5L936OpDudjv4KtZM4bTiNrwaYbOq1sbEziN1u/gXZ5SYYRUEZZx1U6ZEjCm/OIfFZjDeoAQwZYMEeKvofEtu+Ztg8e2s0FMT8pNrrwTBRoO6bWJ8GW7l3FYaba3kiI9NB9kvVg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu; spf=pass smtp.mailfrom=andrew.cmu.edu; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b=LjTcULmh; arc=none smtp.client-ip=209.85.160.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=andrew.cmu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b="LjTcULmh" Received: by mail-qt1-f174.google.com with SMTP id d75a77b69052e-430c4d0408eso19373171cf.3 for ; Tue, 19 Mar 2024 19:42:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902541; x=1711507341; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=iqOS+o0oT1l/HNw34tsU/JsvphKmYASr0yvHH9hC1Ss=; b=LjTcULmhjpKAL5THoapSzirOjowk90jWVYzNP011ANH3Dwm9/G6h9g8Tp03ULfyYH8 TyC33xyfqSHfzPOnQBiK1y9653WESNrmXQqZsR0GO8PSEJBwhLUjlRy16PoFYx4bSj+P tHdIUMwf4lShVopKCwGthTT3QkYHCN3oPRjrVw7YDU8kaYDVpDn6Ad0kes93p15N+bO+ XRZEZWchH2ZsWo+e950cjobgKSggUVMjP4S2sSeBLbDPqFlB12cwTtW2lLjx3wwGWmpn uuVjAwEDC9K7dCvErZ0aYu8BmAKJ+lLdfjbXvzQ1beMSNkd/q15dJxeoBcql+kBH/zI4 Ft6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902541; x=1711507341; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=iqOS+o0oT1l/HNw34tsU/JsvphKmYASr0yvHH9hC1Ss=; b=uzove2Mn79yvI7Sda6aq2jqz4HNVVGXoK6YMFZgzhwvtjqeSd2PVJ2Axt4gPImGdhO 1+hfQxNPeLPjFd/HtfhBLnfjWz49I8UtVBlrMwyV1v40X9Oc4JGdWQ8spatOW1i8Yznw Yi7Fl8qjG484cKRXssDRdA9W4DfNnai/vjyzJEinxeoQIyj5Ijw5jzE5Z8xaSLjpNeUu 6pYxHTKkXcGPs2ckSeCuCgGk5yr+oldwZOD1pMQSox1sOG3AphXgX6rXomnAXIsrB4r5 3rb+WDsY/e+oERcUG6LfTnoC8Al7MNj5lz3X/yQczNJWDB7qnI5F8Iu9SaUQ6eE4rZDU 0TTQ== X-Forwarded-Encrypted: i=1; AJvYcCULhMKabHNB8Cleu+eMo0mxQQx9uHZTl7k/SsA12RDe+TncbB4zfraVvCx9/l7nH0k3zHy7oqzCHvgU4P+okPpd6I8MZcgzxMER9XOn X-Gm-Message-State: AOJu0YwtXqbUz7oZN8PqQWhOcQf6ftKMZXMvAEGYOXvyw0tXGcBSFqJp psd4hYP7ft3q9A38HghChqyH//gb8W8I1++Rkq4SeuIdgQ6FShAgGnNVNO3jvA== X-Google-Smtp-Source: AGHT+IFk+lAGf3V5CLYbVG5wQXqw/ssb09CFDjYho2uIfDPdBFEW3or4buauRl9+pxp4i58SExaXDQ== X-Received: by 2002:a05:622a:1896:b0:430:d8c9:b523 with SMTP id v22-20020a05622a189600b00430d8c9b523mr5159062qtc.8.1710902540913; Tue, 19 Mar 2024 19:42:20 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id hb4-20020a05622a2b4400b0042f0008b8b7sm4428856qtb.35.2024.03.19.19.42.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:20 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 1/7] sysfs interface for the boundary of movable zone Date: Wed, 20 Mar 2024 02:42:12 +0000 Message-Id: <20240320024218.203491-2-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaiyang Zhao Exports the pfn and memory block id for boundary Signed-off-by: Kaiyang Zhao --- drivers/base/memory.c | 2 +- drivers/base/node.c | 32 ++++++++++++++++++++++++++++++++ include/linux/memory.h | 1 + 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index b456ac213610..281b229d7019 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -55,7 +55,7 @@ static inline unsigned long memory_block_id(unsigned long= section_nr) return section_nr / sections_per_block; } =20 -static inline unsigned long pfn_to_block_id(unsigned long pfn) +unsigned long pfn_to_block_id(unsigned long pfn) { return memory_block_id(pfn_to_section_nr(pfn)); } diff --git a/drivers/base/node.c b/drivers/base/node.c index b46db17124f3..f29ce07565ba 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -486,6 +486,37 @@ static ssize_t node_read_meminfo(struct device *dev, #undef K static DEVICE_ATTR(meminfo, 0444, node_read_meminfo, NULL); =20 +static ssize_t node_read_movable_zone(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int len =3D 0; + struct zone *unmovable_zone; + unsigned long movable_start_pfn, unmovable_end_pfn; + unsigned long movable_start_block_id, unmovable_end_block_id; + + movable_start_pfn =3D NODE_DATA(dev->id)->node_zones[ZONE_MOVABLE].zone_s= tart_pfn; + movable_start_block_id =3D pfn_to_block_id(movable_start_pfn); + + if (populated_zone(&(NODE_DATA(dev->id)->node_zones[ZONE_NORMAL]))) + unmovable_zone =3D &(NODE_DATA(dev->id)->node_zones[ZONE_NORMAL]); + else + unmovable_zone =3D &(NODE_DATA(dev->id)->node_zones[ZONE_DMA32]); + + unmovable_end_pfn =3D zone_end_pfn(unmovable_zone); + unmovable_end_block_id =3D pfn_to_block_id(unmovable_end_pfn); + + len =3D sysfs_emit_at(buf, len, + "movable_zone_start_pfn %lu\n" + "movable_zone_start_block_id %lu\n" + "unmovable_zone_end_pfn %lu\n" + "unmovable_zone_end_block_id %lu\n", + movable_start_pfn, movable_start_block_id, + unmovable_end_pfn, unmovable_end_block_id); + + return len; +} +static DEVICE_ATTR(movable_zone, 0444, node_read_movable_zone, NULL); + static ssize_t node_read_numastat(struct device *dev, struct device_attribute *attr, char *buf) { @@ -565,6 +596,7 @@ static DEVICE_ATTR(distance, 0444, node_read_distance, = NULL); =20 static struct attribute *node_dev_attrs[] =3D { &dev_attr_meminfo.attr, + &dev_attr_movable_zone.attr, &dev_attr_numastat.attr, &dev_attr_distance.attr, &dev_attr_vmstat.attr, diff --git a/include/linux/memory.h b/include/linux/memory.h index 31343566c221..17a92a5c1ae5 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -92,6 +92,7 @@ struct memory_block { int arch_get_memory_phys_device(unsigned long start_pfn); unsigned long memory_block_size_bytes(void); int set_memory_block_size_order(unsigned int order); +unsigned long pfn_to_block_id(unsigned long pfn); =20 /* These states are exposed to userspace as text strings in sysfs */ #define MEM_ONLINE (1<<0) /* exposed to userspace */ --=20 2.40.1 From nobody Sun Feb 8 17:41:52 2026 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC087F9E8 for ; Wed, 20 Mar 2024 02:42:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902546; cv=none; b=gA3TxpETtY1ADT27wJlXuPGXkW9S2okqH5HAsa7rLfpWVNWg45LuY7MCsrgkNwQmqR6Akw7EJZW1M/boOwbO/jgSxx4m4kOtxX22p21qEgBsbm5RvnKH3UlHj8o9z5TUhRh1relZGl/ld5xLqf+8w/v9nNcYpUuuav/aS/OOjIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902546; c=relaxed/simple; bh=qrzIBtExBXCUXHgWYojzqnQkh/sdDvI8P9QCLSv5BYE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qpiw2pidyB70QOX58EK6QNB/EpXbBJQP0H+aAAQJFRd8w169A7Qm38g6DcgP9erL2JMoyjfA9uBoguNQXqCuJlLQ3i8rF6O29DVdXpbQ7meB49EGA25Vy9yuL9Dde2IEIOxWD6fKbj8Mhz3YlTOpHyUJrqyP3obAGyqKkxR9jy4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu; spf=pass smtp.mailfrom=andrew.cmu.edu; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b=R5q+Vv4l; arc=none smtp.client-ip=209.85.222.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=andrew.cmu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b="R5q+Vv4l" Received: by mail-qk1-f175.google.com with SMTP id af79cd13be357-789ddcd57f4so376556785a.2 for ; Tue, 19 Mar 2024 19:42:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902541; x=1711507341; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=3nx6/aDk3VJUuN8aHF02HR1BsWsxIjtwUpot61xzcjI=; b=R5q+Vv4lYkMVZG43PwoUBAvHC4CcqWkyhX+gSGfpGf5sF7Jpba8MTEWto87m6xo8bJ v2CNiRT5MJ1u1ZU/CoG6tihqWa/wE82KP8SpqgjumeTKh+L6lLzap380INLdeY5IS1O0 LHuQmWldNc3p2N7Lt4SOeNtvE6seGRWLpaEHeiev7A/AxTHRhizxqAAUdE2dSAQ5WR5J qvcL7kSeiClHicaLjgSqjTTLo4CkKjhDdTDG80nYbP5UMm+2WYd7iK82paSzY7PNmNQq RIXXhrQ2FlFwmZSc3PiOw1TwYNw6GjpRUSlHATybkDNFv1c56i9+MvB7Dhx0owCLck3x +7aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902541; x=1711507341; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=3nx6/aDk3VJUuN8aHF02HR1BsWsxIjtwUpot61xzcjI=; b=gUWn0pH27chjdx4AqPgF9cWfTlfoI65xo3fsAaQ9LRMaxKuwG662aeanyaDfW1Q1ZK 3nC7AARWK4in9dJHR0EagmtrGztjAgNVHMydImbKI7DpQtisq10lA+IoV2S2kvPKvHcU Mvu0AjcRf/OkeukaB9C1DkUFlLRm+MGkzPmpTGqH1faSI4TTIcHn5/QPjyLll1OHBFhD aUYAqTEt38wgrHLGbPQtTaVvhW+BpFQyrgg75x7mgLkKvuDN5zGG8Z2KHPjJOjSqg2TI YIoFfLgSgc7cd6hdVrjzBGC/IWwgpY0hkaD69CN2UnPBMhSUpqO2OqB0XbeaE5tMzmni EPJw== X-Forwarded-Encrypted: i=1; AJvYcCXUaXQBNUYrqLqVilsOnvglVot3LnDigJc1BIRLASPrlxfAoZ7Og9BliQ7dFIy0f+Guk+8UIcQ+Juo0cGT+vB/bzFSyN/zpgMES+QtC X-Gm-Message-State: AOJu0YzIhIOU4UnwHAtVs5+zx/+d1hafu03XinTIfeUMKiuah7nh73GA z1kduNKPeZytNqN54qPfr8g8vr6SFz+8ubtwBSlGRBWptxbtPyUNPoSO2o1cuA== X-Google-Smtp-Source: AGHT+IE8ey/Y7RHs4C66DCwZWoviXuK1UpbUhJUPjIEBgTvetqJ0Lw+B0vwcdezjuc587wrWuN/E9w== X-Received: by 2002:a05:620a:4484:b0:788:30a5:6aaf with SMTP id x4-20020a05620a448400b0078830a56aafmr22805144qkp.52.1710902541638; Tue, 19 Mar 2024 19:42:21 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id vr17-20020a05620a55b100b00789ea5b08bcsm4074525qkn.23.2024.03.19.19.42.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:21 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 2/7] Disallows high-order movable allocations in other zones if ZONE_MOVABLE is populated Date: Wed, 20 Mar 2024 02:42:13 +0000 Message-Id: <20240320024218.203491-3-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaiyang Zhao Use ZONE_MOVABLE exclusively for non-0 order allocations Signed-off-by: Kaiyang Zhao --- mm/page_alloc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 47421bedc12b..9ad9357e340a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3403,6 +3403,16 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int = order, int alloc_flags, struct page *page; unsigned long mark; =20 + /* + * Disallows high-order movable allocations in other zones if + * ZONE_MOVABLE is populated on this node. + */ + if (ac->highest_zoneidx >=3D ZONE_MOVABLE && + order > 0 && + zone_idx(zone) !=3D ZONE_MOVABLE && + populated_zone(&(zone->zone_pgdat->node_zones[ZONE_MOVABLE]))) + continue; + if (cpusets_enabled() && (alloc_flags & ALLOC_CPUSET) && !__cpuset_zone_allowed(zone, gfp_mask)) --=20 2.40.1 From nobody Sun Feb 8 17:41:52 2026 Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A644BF9F5 for ; Wed, 20 Mar 2024 02:42:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902547; cv=none; b=XHOqP0huvLTKcAzgUuyjFod0i51YO71RUq4fSgo1JrMQzmuHQ3b/nvLlTXoJtx8vn3g1yMbGcGLcHjKZ1yNdQPXOS0ztJMwQl4j39bVM9zTTmvaYyydaM4Dmn51v6dknP1B5xrWcjMvU98/atwP2xcT0PeNNvX2xHQ5J/V2qDGo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902547; c=relaxed/simple; bh=l52JrmiqoXyGIuFFDgBlXrC/zh8TWuGP8g41ayonsYU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qPHhUzG4y9+v7R0aXPuQkXSC1Jg3yr9Z6wq3uYK48klPWbIh19YLPGOR7Cu6UUN2qFYu7jBzaTaN2w6RlaCTTgm9WRdRtubEtXeWdVSWmcsNUyVjItz+GGfha08N9DGbV+53zNYpWlhZaWL4GeEAzwti+uarvEcOPfTINXg0oP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu; spf=pass smtp.mailfrom=andrew.cmu.edu; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b=b26ysJaW; arc=none smtp.client-ip=209.85.210.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=andrew.cmu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b="b26ysJaW" Received: by mail-ot1-f51.google.com with SMTP id 46e09a7af769-6e6969855c8so1602626a34.0 for ; Tue, 19 Mar 2024 19:42:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902542; x=1711507342; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=UljsMw4YXxvG2im1vY4a6XAU2ZaTJQdUEua3lUBpIog=; b=b26ysJaWaMFF1aefAi/9HdyoC665IxctzqD06rPgFth8PFOfayGvSVkUNd3BEyXVLv 8j+dRLDyKWXzkRO1SZuWmr6xDhoEp+8p5eNB4qyVYbLdYIFRk/DiOgGOCRq8xCBMqkTD 7mXnKZJOWYN48M8/ME8or95lUrtslfKAgVr1oHmFmiHwBLAY9reVwsUje1eTDs0PVTx3 TuV1CVy0kg3fXinS24JYVCg1LE+4xl2AE/29XCIP7iQQJNQ8pozlZNnhTm7DXC2TSxGo xbVOrxsnTBQ0SBuGLrG4tsQ55E+ktcbgpYvzOgxkYoXIGOt+lP+97+TjSjFbrRjOEcqD nsrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902542; x=1711507342; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=UljsMw4YXxvG2im1vY4a6XAU2ZaTJQdUEua3lUBpIog=; b=c3h9iN0GweR7/L5Ke9/CJP2sDFiK6yGRlxsZbzTaIntoFl1f0q9wwOcnDCI+Pyl0pD vdajrmox/DZMAcF8YKs3Tut69+SG3Hx7AeJLY0CME9jajLlS0nzyDFmA1/kNAD6DMuPk yXVP2rZOJ22kiXmZMZ2RC/YEEHlq30/Q6eIRy9hfgUOLUzmqVlHIWPb5pIYsWsk/61Tv EjIlPp0Vvt2nBhlZoH8BAtOb2uWFdxrKx6MX0UI7fEvUwi2XbxTF6hWa2lf167geDygO sPSewBHewTH7mPK0Deozq0WF8TWpSYH/uvmtbSm9RZfsGpDEeHekTo48jxcU0NlF9x8Q 4rgQ== X-Forwarded-Encrypted: i=1; AJvYcCWgbdrlMntLKFwmp+Q/GVCoMtibJHDSHJ7BxUZ0rnnLc8DTmqpWGIBENF0xCrBoOakIlbxrR1Op51iWGnLO+fDUYSmLPukiBCYXKaho X-Gm-Message-State: AOJu0Yyxe0k63aKwlQ89sHnrtQ/lGJWBIYWwyijHMRwnde6K9Ii708/n CtU8wHrVKFtfeBXKGFKV3qvpQiHoyAvE+7huEl3A8qMK+ftSbdLL4yY44roCag== X-Google-Smtp-Source: AGHT+IESptNQKDLD8MOe2hQxpKnyZd4H1raJLsnnYtjWGvQGBbhnaHWmiZ6IFtzpyT4LI3UuB+xfsg== X-Received: by 2002:a9d:74c5:0:b0:6e4:f961:e444 with SMTP id a5-20020a9d74c5000000b006e4f961e444mr5504940otl.4.1710902542503; Tue, 19 Mar 2024 19:42:22 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id y27-20020a05620a09db00b00789f0d9e6dcsm3560797qky.93.2024.03.19.19.42.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:22 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 3/7] compaction accepts a destination zone Date: Wed, 20 Mar 2024 02:42:14 +0000 Message-Id: <20240320024218.203491-4-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaiyang Zhao Distinguishes the source and destination zones in compaction Signed-off-by: Kaiyang Zhao --- include/linux/compaction.h | 4 +- mm/compaction.c | 106 +++++++++++++++++++++++-------------- mm/internal.h | 1 + mm/vmscan.c | 4 +- 4 files changed, 70 insertions(+), 45 deletions(-) diff --git a/include/linux/compaction.h b/include/linux/compaction.h index a6e512cfb670..11f5a1a83abb 100644 --- a/include/linux/compaction.h +++ b/include/linux/compaction.h @@ -90,7 +90,7 @@ extern enum compact_result try_to_compact_pages(gfp_t gfp= _mask, struct page **page); extern void reset_isolation_suitable(pg_data_t *pgdat); extern enum compact_result compaction_suitable(struct zone *zone, int orde= r, - unsigned int alloc_flags, int highest_zoneidx); + unsigned int alloc_flags, int highest_zoneidx, struct zone *dst_zone); =20 extern void compaction_defer_reset(struct zone *zone, int order, bool alloc_success); @@ -180,7 +180,7 @@ static inline void reset_isolation_suitable(pg_data_t *= pgdat) } =20 static inline enum compact_result compaction_suitable(struct zone *zone, i= nt order, - int alloc_flags, int highest_zoneidx) + int alloc_flags, int highest_zoneidx, struct zone *dst_zone) { return COMPACT_SKIPPED; } diff --git a/mm/compaction.c b/mm/compaction.c index c8bcdea15f5f..03b5c4debc17 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -435,7 +435,7 @@ static void update_cached_migrate(struct compact_contro= l *cc, unsigned long pfn) static void update_pageblock_skip(struct compact_control *cc, struct page *page, unsigned long pfn) { - struct zone *zone =3D cc->zone; + struct zone *dst_zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; =20 if (cc->no_set_skip_hint) return; @@ -446,8 +446,8 @@ static void update_pageblock_skip(struct compact_contro= l *cc, set_pageblock_skip(page); =20 /* Update where async and sync compaction should restart */ - if (pfn < zone->compact_cached_free_pfn) - zone->compact_cached_free_pfn =3D pfn; + if (pfn < dst_zone->compact_cached_free_pfn) + dst_zone->compact_cached_free_pfn =3D pfn; } #else static inline bool isolation_suitable(struct compact_control *cc, @@ -550,6 +550,7 @@ static unsigned long isolate_freepages_block(struct com= pact_control *cc, bool locked =3D false; unsigned long blockpfn =3D *start_pfn; unsigned int order; + struct zone *dst_zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; =20 /* Strict mode is for isolation, speed is secondary */ if (strict) @@ -568,7 +569,7 @@ static unsigned long isolate_freepages_block(struct com= pact_control *cc, * pending. */ if (!(blockpfn % COMPACT_CLUSTER_MAX) - && compact_unlock_should_abort(&cc->zone->lock, flags, + && compact_unlock_should_abort(&dst_zone->lock, flags, &locked, cc)) break; =20 @@ -596,7 +597,7 @@ static unsigned long isolate_freepages_block(struct com= pact_control *cc, =20 /* If we already hold the lock, we can skip some rechecking. */ if (!locked) { - locked =3D compact_lock_irqsave(&cc->zone->lock, + locked =3D compact_lock_irqsave(&dst_zone->lock, &flags, cc); =20 /* Recheck this is a buddy page under lock */ @@ -634,7 +635,7 @@ static unsigned long isolate_freepages_block(struct com= pact_control *cc, } =20 if (locked) - spin_unlock_irqrestore(&cc->zone->lock, flags); + spin_unlock_irqrestore(&dst_zone->lock, flags); =20 /* * There is a tiny chance that we have read bogus compound_order(), @@ -683,11 +684,12 @@ isolate_freepages_range(struct compact_control *cc, { unsigned long isolated, pfn, block_start_pfn, block_end_pfn; LIST_HEAD(freelist); + struct zone *dst_zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; =20 pfn =3D start_pfn; block_start_pfn =3D pageblock_start_pfn(pfn); - if (block_start_pfn < cc->zone->zone_start_pfn) - block_start_pfn =3D cc->zone->zone_start_pfn; + if (block_start_pfn < dst_zone->zone_start_pfn) + block_start_pfn =3D dst_zone->zone_start_pfn; block_end_pfn =3D pageblock_end_pfn(pfn); =20 for (; pfn < end_pfn; pfn +=3D isolated, @@ -710,7 +712,7 @@ isolate_freepages_range(struct compact_control *cc, } =20 if (!pageblock_pfn_to_page(block_start_pfn, - block_end_pfn, cc->zone)) + block_end_pfn, dst_zone)) break; =20 isolated =3D isolate_freepages_block(cc, &isolate_start_pfn, @@ -1359,6 +1361,7 @@ fast_isolate_around(struct compact_control *cc, unsig= ned long pfn) { unsigned long start_pfn, end_pfn; struct page *page; + struct zone *dst_zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; =20 /* Do not search around if there are enough pages already */ if (cc->nr_freepages >=3D cc->nr_migratepages) @@ -1369,10 +1372,10 @@ fast_isolate_around(struct compact_control *cc, uns= igned long pfn) return; =20 /* Pageblock boundaries */ - start_pfn =3D max(pageblock_start_pfn(pfn), cc->zone->zone_start_pfn); - end_pfn =3D min(pageblock_end_pfn(pfn), zone_end_pfn(cc->zone)); + start_pfn =3D max(pageblock_start_pfn(pfn), dst_zone->zone_start_pfn); + end_pfn =3D min(pageblock_end_pfn(pfn), zone_end_pfn(dst_zone)); =20 - page =3D pageblock_pfn_to_page(start_pfn, end_pfn, cc->zone); + page =3D pageblock_pfn_to_page(start_pfn, end_pfn, dst_zone); if (!page) return; =20 @@ -1414,6 +1417,7 @@ fast_isolate_freepages(struct compact_control *cc) struct page *page =3D NULL; bool scan_start =3D false; int order; + struct zone *dst_zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; =20 /* Full compaction passes in a negative order */ if (cc->order <=3D 0) @@ -1423,7 +1427,7 @@ fast_isolate_freepages(struct compact_control *cc) * If starting the scan, use a deeper search and use the highest * PFN found if a suitable one is not found. */ - if (cc->free_pfn >=3D cc->zone->compact_init_free_pfn) { + if (cc->free_pfn >=3D dst_zone->compact_init_free_pfn) { limit =3D pageblock_nr_pages >> 1; scan_start =3D true; } @@ -1448,7 +1452,7 @@ fast_isolate_freepages(struct compact_control *cc) for (order =3D cc->search_order; !page && order >=3D 0; order =3D next_search_order(cc, order)) { - struct free_area *area =3D &cc->zone->free_area[order]; + struct free_area *area =3D &dst_zone->free_area[order]; struct list_head *freelist; struct page *freepage; unsigned long flags; @@ -1458,7 +1462,7 @@ fast_isolate_freepages(struct compact_control *cc) if (!area->nr_free) continue; =20 - spin_lock_irqsave(&cc->zone->lock, flags); + spin_lock_irqsave(&dst_zone->lock, flags); freelist =3D &area->free_list[MIGRATE_MOVABLE]; list_for_each_entry_reverse(freepage, freelist, lru) { unsigned long pfn; @@ -1469,7 +1473,7 @@ fast_isolate_freepages(struct compact_control *cc) =20 if (pfn >=3D highest) highest =3D max(pageblock_start_pfn(pfn), - cc->zone->zone_start_pfn); + dst_zone->zone_start_pfn); =20 if (pfn >=3D low_pfn) { cc->fast_search_fail =3D 0; @@ -1516,7 +1520,7 @@ fast_isolate_freepages(struct compact_control *cc) } } =20 - spin_unlock_irqrestore(&cc->zone->lock, flags); + spin_unlock_irqrestore(&dst_zone->lock, flags); =20 /* * Smaller scan on next order so the total scan is related @@ -1541,17 +1545,17 @@ fast_isolate_freepages(struct compact_control *cc) if (cc->direct_compaction && pfn_valid(min_pfn)) { page =3D pageblock_pfn_to_page(min_pfn, min(pageblock_end_pfn(min_pfn), - zone_end_pfn(cc->zone)), - cc->zone); + zone_end_pfn(dst_zone)), + dst_zone); cc->free_pfn =3D min_pfn; } } } } =20 - if (highest && highest >=3D cc->zone->compact_cached_free_pfn) { + if (highest && highest >=3D dst_zone->compact_cached_free_pfn) { highest -=3D pageblock_nr_pages; - cc->zone->compact_cached_free_pfn =3D highest; + dst_zone->compact_cached_free_pfn =3D highest; } =20 cc->total_free_scanned +=3D nr_scanned; @@ -1569,7 +1573,7 @@ fast_isolate_freepages(struct compact_control *cc) */ static void isolate_freepages(struct compact_control *cc) { - struct zone *zone =3D cc->zone; + struct zone *zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; struct page *page; unsigned long block_start_pfn; /* start of current pageblock */ unsigned long isolate_start_pfn; /* exact pfn we start at */ @@ -2089,11 +2093,19 @@ static enum compact_result __compact_finished(struc= t compact_control *cc) unsigned int order; const int migratetype =3D cc->migratetype; int ret; + struct zone *dst_zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; =20 - /* Compaction run completes if the migrate and free scanner meet */ - if (compact_scanners_met(cc)) { + /* + * Compaction run completes if the migrate and free scanner meet + * or when either the src or dst zone has been completely scanned + */ + if (compact_scanners_met(cc) || + cc->migrate_pfn >=3D zone_end_pfn(cc->zone) || + cc->free_pfn < dst_zone->zone_start_pfn) { /* Let the next compaction start anew. */ reset_cached_positions(cc->zone); + if (cc->dst_zone) + reset_cached_positions(cc->dst_zone); =20 /* * Mark that the PG_migrate_skip information should be cleared @@ -2196,10 +2208,13 @@ static enum compact_result compact_finished(struct = compact_control *cc) static enum compact_result __compaction_suitable(struct zone *zone, int or= der, unsigned int alloc_flags, int highest_zoneidx, - unsigned long wmark_target) + unsigned long wmark_target, struct zone *dst_zone) { unsigned long watermark; =20 + if (!dst_zone) + dst_zone =3D zone; + if (is_via_compact_memory(order)) return COMPACT_CONTINUE; =20 @@ -2227,9 +2242,9 @@ static enum compact_result __compaction_suitable(stru= ct zone *zone, int order, * suitable migration targets */ watermark =3D (order > PAGE_ALLOC_COSTLY_ORDER) ? - low_wmark_pages(zone) : min_wmark_pages(zone); + low_wmark_pages(dst_zone) : min_wmark_pages(dst_zone); watermark +=3D compact_gap(order); - if (!__zone_watermark_ok(zone, 0, watermark, highest_zoneidx, + if (!__zone_watermark_ok(dst_zone, 0, watermark, highest_zoneidx, ALLOC_CMA, wmark_target)) return COMPACT_SKIPPED; =20 @@ -2245,13 +2260,16 @@ static enum compact_result __compaction_suitable(st= ruct zone *zone, int order, */ enum compact_result compaction_suitable(struct zone *zone, int order, unsigned int alloc_flags, - int highest_zoneidx) + int highest_zoneidx, struct zone *dst_zone) { enum compact_result ret; int fragindex; =20 + if (!dst_zone) + dst_zone =3D zone; + ret =3D __compaction_suitable(zone, order, alloc_flags, highest_zoneidx, - zone_page_state(zone, NR_FREE_PAGES)); + zone_page_state(dst_zone, NR_FREE_PAGES), dst_zone); /* * fragmentation index determines if allocation failures are due to * low memory or external fragmentation @@ -2305,7 +2323,7 @@ bool compaction_zonelist_suitable(struct alloc_contex= t *ac, int order, available =3D zone_reclaimable_pages(zone) / order; available +=3D zone_page_state_snapshot(zone, NR_FREE_PAGES); compact_result =3D __compaction_suitable(zone, order, alloc_flags, - ac->highest_zoneidx, available); + ac->highest_zoneidx, available, NULL); if (compact_result =3D=3D COMPACT_CONTINUE) return true; } @@ -2317,8 +2335,9 @@ static enum compact_result compact_zone(struct compact_control *cc, struct capture_control *capc) { enum compact_result ret; + struct zone *dst_zone =3D cc->dst_zone ? cc->dst_zone : cc->zone; unsigned long start_pfn =3D cc->zone->zone_start_pfn; - unsigned long end_pfn =3D zone_end_pfn(cc->zone); + unsigned long end_pfn =3D zone_end_pfn(dst_zone); unsigned long last_migrated_pfn; const bool sync =3D cc->mode !=3D MIGRATE_ASYNC; bool update_cached; @@ -2337,7 +2356,7 @@ compact_zone(struct compact_control *cc, struct captu= re_control *capc) =20 cc->migratetype =3D gfp_migratetype(cc->gfp_mask); ret =3D compaction_suitable(cc->zone, cc->order, cc->alloc_flags, - cc->highest_zoneidx); + cc->highest_zoneidx, dst_zone); /* Compaction is likely to fail */ if (ret =3D=3D COMPACT_SUCCESS || ret =3D=3D COMPACT_SKIPPED) return ret; @@ -2346,14 +2365,19 @@ compact_zone(struct compact_control *cc, struct cap= ture_control *capc) * Clear pageblock skip if there were failures recently and compaction * is about to be retried after being deferred. */ - if (compaction_restarting(cc->zone, cc->order)) + if (compaction_restarting(cc->zone, cc->order)) { __reset_isolation_suitable(cc->zone); + if (dst_zone !=3D cc->zone) + __reset_isolation_suitable(dst_zone); + } =20 /* * Setup to move all movable pages to the end of the zone. Used cached * information on where the scanners should start (unless we explicitly * want to compact the whole zone), but check that it is initialised * by ensuring the values are within zone boundaries. + * + * If a destination zone is provided, use it for free pages. */ cc->fast_start_pfn =3D 0; if (cc->whole_zone) { @@ -2361,12 +2385,12 @@ compact_zone(struct compact_control *cc, struct cap= ture_control *capc) cc->free_pfn =3D pageblock_start_pfn(end_pfn - 1); } else { cc->migrate_pfn =3D cc->zone->compact_cached_migrate_pfn[sync]; - cc->free_pfn =3D cc->zone->compact_cached_free_pfn; - if (cc->free_pfn < start_pfn || cc->free_pfn >=3D end_pfn) { + cc->free_pfn =3D dst_zone->compact_cached_free_pfn; + if (cc->free_pfn < dst_zone->zone_start_pfn || cc->free_pfn >=3D end_pfn= ) { cc->free_pfn =3D pageblock_start_pfn(end_pfn - 1); - cc->zone->compact_cached_free_pfn =3D cc->free_pfn; + dst_zone->compact_cached_free_pfn =3D cc->free_pfn; } - if (cc->migrate_pfn < start_pfn || cc->migrate_pfn >=3D end_pfn) { + if (cc->migrate_pfn < start_pfn || cc->migrate_pfn >=3D zone_end_pfn(cc-= >zone)) { cc->migrate_pfn =3D start_pfn; cc->zone->compact_cached_migrate_pfn[0] =3D cc->migrate_pfn; cc->zone->compact_cached_migrate_pfn[1] =3D cc->migrate_pfn; @@ -2522,8 +2546,8 @@ compact_zone(struct compact_control *cc, struct captu= re_control *capc) * Only go back, not forward. The cached pfn might have been * already reset to zone end in compact_finished() */ - if (free_pfn > cc->zone->compact_cached_free_pfn) - cc->zone->compact_cached_free_pfn =3D free_pfn; + if (free_pfn > dst_zone->compact_cached_free_pfn) + dst_zone->compact_cached_free_pfn =3D free_pfn; } =20 count_compact_events(COMPACTMIGRATE_SCANNED, cc->total_migrate_scanned); @@ -2834,7 +2858,7 @@ static bool kcompactd_node_suitable(pg_data_t *pgdat) continue; =20 if (compaction_suitable(zone, pgdat->kcompactd_max_order, 0, - highest_zoneidx) =3D=3D COMPACT_CONTINUE) + highest_zoneidx, NULL) =3D=3D COMPACT_CONTINUE) return true; } =20 @@ -2871,7 +2895,7 @@ static void kcompactd_do_work(pg_data_t *pgdat) if (compaction_deferred(zone, cc.order)) continue; =20 - if (compaction_suitable(zone, cc.order, 0, zoneid) !=3D + if (compaction_suitable(zone, cc.order, 0, zoneid, NULL) !=3D COMPACT_CONTINUE) continue; =20 diff --git a/mm/internal.h b/mm/internal.h index 68410c6d97ac..349223cc0359 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -465,6 +465,7 @@ struct compact_control { unsigned long migrate_pfn; unsigned long fast_start_pfn; /* a pfn to start linear scan from */ struct zone *zone; + struct zone *dst_zone; /* use another zone as the destination */ unsigned long total_migrate_scanned; unsigned long total_free_scanned; unsigned short fast_search_fail;/* failures to use free list searches */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 5bf98d0a22c9..aa21da983804 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -6383,7 +6383,7 @@ static inline bool should_continue_reclaim(struct pgl= ist_data *pgdat, if (!managed_zone(zone)) continue; =20 - switch (compaction_suitable(zone, sc->order, 0, sc->reclaim_idx)) { + switch (compaction_suitable(zone, sc->order, 0, sc->reclaim_idx, NULL)) { case COMPACT_SUCCESS: case COMPACT_CONTINUE: return false; @@ -6580,7 +6580,7 @@ static inline bool compaction_ready(struct zone *zone= , struct scan_control *sc) unsigned long watermark; enum compact_result suitable; =20 - suitable =3D compaction_suitable(zone, sc->order, 0, sc->reclaim_idx); + suitable =3D compaction_suitable(zone, sc->order, 0, sc->reclaim_idx, NUL= L); if (suitable =3D=3D COMPACT_SUCCESS) /* Allocation should succeed already. Don't reclaim. */ return true; --=20 2.40.1 From nobody Sun Feb 8 17:41:52 2026 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AB06FC17 for ; Wed, 20 Mar 2024 02:42:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902547; cv=none; b=MpxwWBLiyCxngnIMkwL3vaVgnUva9mWxmFBptjf+xHA4yxtkrHGIqlAZ3xy1su3jFFi96nI6Xfk1bgxjtzGBL1xvnd1J5TeYvX+UkCBb4/c1NAxEQtTXJshrc772MaeSs+Xray+xZ+od96MgW59EBBsIubfJ14Tp0Y5Jd9iD1C4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902547; c=relaxed/simple; bh=YzPNp/bNBe/NK++T3MgP34oObjeH/a7wt40c91gratA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KWtZfDmeD0f706R709d+91J0e1Mrhw1lNmepbEJ8chSH25oM6pRHkWWbxkti1fcsKOYXCp2DyaR/PrNyWbw1G51HzFroJkbBwvU33I0jYavsBc9dCMaWGkIzFKLOmNtAI0e21QXvUPdAVRrss4rW2UonvwmXCZyCngV9YMF1Xjs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu; spf=pass smtp.mailfrom=andrew.cmu.edu; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b=NyWllr7l; arc=none smtp.client-ip=209.85.222.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=andrew.cmu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b="NyWllr7l" Received: by mail-qk1-f173.google.com with SMTP id af79cd13be357-78a13117a3dso41383385a.1 for ; Tue, 19 Mar 2024 19:42:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902543; x=1711507343; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Gwth4DHc3n7Jk8RbQant6q4qmoLsduwQhc9A8tF9LNs=; b=NyWllr7lZI070fBxf7aqVYRREUG4OKRS/DVLH10s2ofC4Tz1HOulNy+cfe/LNG5QS5 IQktS84gTIf2sIDOM/+QgSURQPit27g9OAp+WzqnxCXpLtvkfj2L5md9/nw6ujGVavNf BG+0HpffJq+Xp01WIBgcFxnFfAcuQpYeW2vjl9/G9zErl2If6d+NCMtWyzxzfedTgtz0 Nh+TkzE5VSJkYo6sX3F8DMcdjJZbQZbftUHjmLAL4qCl3iFiL+18mL293wyoT9iNIWcD ZBtI0XVepGgzq4hf6bYNsjDXqfA+OMiayTKX9ibAmvnicFtSLYzqk3d1hUikAkW+DM/Q E0hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902543; x=1711507343; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Gwth4DHc3n7Jk8RbQant6q4qmoLsduwQhc9A8tF9LNs=; b=g9Ri5or2ONo4j8BWVGl/qnoUR+YmU0Gy2xZ+gEHmmosJLdZU3BJGbQLm/yx5L8zfPX mjL1TLxSEXPVlbb77QPnBKX2HDrfB/SG1IFWkdXnnxCNMjZ6Ic5mQDtQehqm7MY5DO91 pWxoHihp57pnYW9G1lud7Rud6xQJuFUmMcWc7qePwqh9pUueT3NUfJu1LnrKqKQpMHEZ Zd7f6qLern921LykvdNySBzGkvkY2F4X+MAAMDuikno8eFIa8c9KlebkNykurO8z6k6Q q6peLvok8iwuzq9XOpHu8nuWkPKm+RJ+Ou6b8Ks+Jqo6pXSgQ4B6KKbc+IhXsjvl/mxh mTUA== X-Forwarded-Encrypted: i=1; AJvYcCU8PTjHR8hKNh/CMSVshKJBBF6bQn0e2lSkWdYfyBGUp/I8k+sH9THqA7QS4R5hGDW+INq7kwI+1GLBiw0OI3i3MMaY0ndB4w4JRw9c X-Gm-Message-State: AOJu0YzVzxTa107xQId012HjM47YplwKuInxXFuzpqbGquCQh4v0+7DQ /zOM+CwI8Lrz8vIUCz2CTBssWC2jSX+7uJ1ZTLeW+62ypTCNj62JQBZocaGy+JsDcPNxL4M24+4 woQ== X-Google-Smtp-Source: AGHT+IGHswrTM4FTiBPD0YXAQUMLyxW4PRx4gQjvpjDC8ST6/12GayLSbc0ihrs1ql7V1uIy1W3KoA== X-Received: by 2002:a05:620a:4691:b0:78a:120f:ee44 with SMTP id bq17-20020a05620a469100b0078a120fee44mr2253073qkb.28.1710902543396; Tue, 19 Mar 2024 19:42:23 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id d7-20020a05620a136700b0078863e0f829sm5213075qkl.12.2024.03.19.19.42.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:23 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 4/7] vmstat counter for pages migrated across zones Date: Wed, 20 Mar 2024 02:42:15 +0000 Message-Id: <20240320024218.203491-5-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaiyang Zhao Add a counter for the number of pages migrated across zones in vmstat Signed-off-by: Kaiyang Zhao --- include/linux/vm_event_item.h | 1 + mm/compaction.c | 2 ++ mm/vmstat.c | 1 + 3 files changed, 4 insertions(+) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index 8abfa1240040..be88819085b6 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -79,6 +79,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, COMPACTSTALL, COMPACTFAIL, COMPACTSUCCESS, KCOMPACTD_WAKE, KCOMPACTD_MIGRATE_SCANNED, KCOMPACTD_FREE_SCANNED, + COMPACT_CROSS_ZONE_MIGRATED, #endif #ifdef CONFIG_HUGETLB_PAGE HTLB_BUDDY_PGALLOC, HTLB_BUDDY_PGALLOC_FAIL, diff --git a/mm/compaction.c b/mm/compaction.c index 03b5c4debc17..dea10ad8ec64 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -2552,6 +2552,8 @@ compact_zone(struct compact_control *cc, struct captu= re_control *capc) =20 count_compact_events(COMPACTMIGRATE_SCANNED, cc->total_migrate_scanned); count_compact_events(COMPACTFREE_SCANNED, cc->total_free_scanned); + if (dst_zone !=3D cc->zone) + count_compact_events(COMPACT_CROSS_ZONE_MIGRATED, nr_succeeded); =20 trace_mm_compaction_end(cc, start_pfn, end_pfn, sync, ret); =20 diff --git a/mm/vmstat.c b/mm/vmstat.c index c28046371b45..98af82e65ad9 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1324,6 +1324,7 @@ const char * const vmstat_text[] =3D { "compact_daemon_wake", "compact_daemon_migrate_scanned", "compact_daemon_free_scanned", + "compact_cross_zone_migrated", #endif =20 #ifdef CONFIG_HUGETLB_PAGE --=20 2.40.1 From nobody Sun Feb 8 17:41:52 2026 Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57ABA101C6 for ; Wed, 20 Mar 2024 02:42:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902548; cv=none; b=u9Bi6eaLLY87fM7mR14vUFOS1dX5MMm8eB1FIumNxU/THtdmKMZiFZiz0ejD6ZfYUiRaBbslOP6WOGHiMNpNdmfsnBMlIFjBS5oaoLpoCnC43z4jLQKeD0bvsOKYjyIHtb5/qMiR1/e7cBlREiPB3mdKXAtLv5RKjdNQklclFMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902548; c=relaxed/simple; bh=W7Eye2Q63n7l9RaClb+x2PjywYvN68crJrLesJ3QC3g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=L96r5QY5dydSLfRumZbeyq+GqqgszkxQt+dVoOqiO27/9o2OYYnH8JzyjlfPSG0+37ey28P91fc22Gu3E5sPOYJuVC0DZzKDnm3RHjnkxeXor59URe+AjKzyQGuu1X9nkQwYrLtBlqPfe7RUOtrWoisjqtCLhZ/cAxxe4Rg57/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu; spf=pass smtp.mailfrom=andrew.cmu.edu; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b=iKNDCQVA; arc=none smtp.client-ip=209.85.219.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=andrew.cmu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b="iKNDCQVA" Received: by mail-qv1-f41.google.com with SMTP id 6a1803df08f44-690b24973beso4583096d6.0 for ; Tue, 19 Mar 2024 19:42:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902544; x=1711507344; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Yh8JatHXgZIgb1eyxT1DhEdr2aiRbmcRrkpZMUB4hHw=; b=iKNDCQVAyB1+2Ye6Ok08YuQqqrwWagYHyynu6U5cYTtME+gYNa6qzbjctbI6p4knP0 kDO8fsBkCHlX4xEHUuC1CGokI/VCgHfK37SSPkTrT//2GIkDR7GqBip3rr5cqQXzA7Nz Qa8Wmk7XVeEGmBOk1Lht1g6m2t7Bx6nuEqPDFpIm1ad+j7MwzI8ow1GLV0LjRWNgN6BC J48/unZRZj4LQKAlErgRoit7MztuTOU3ZZAK2JxnQcuDtZVc4o3YyxBsA6JxAR8jEO6T EQWskq2Kfa3Q+6fLdyOH4H9jCeU3xrvBr7p1tjECJC4/8ulI6HxIwlxSfa9LXtU3FCt6 nJAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902544; x=1711507344; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Yh8JatHXgZIgb1eyxT1DhEdr2aiRbmcRrkpZMUB4hHw=; b=YJnSwrPvyUxhKJiG/Y/wzuy2+oN9UzA89xpHVdC6ugyNhhS/XrIv79FyfM6dF3ruBV I4WsTDspkhMcAZf+xa7Mb1mB+OgZGr4y0UOwc3/RlzyTq18w5DFC6MJ+rcKEZo7r/Dow 0WC60xyaSfHXfp0qdL1j0Cenz48KVcS7cMsSzbIvCBrYfvhxXb6tGFKHA4BAYXctT9JU rxhSBOmLnmwE4vB+EU1WOJvmFGpfc1vcAVkZJDhT1ZdazZWuxePIzmW+Fndgf9uLbvHz TWRxisPrbag/wgHIKNsOJRY5iZzca68h4yG8mhTPwuDEPdjvZqbL2RGxn8dnb9NnBB4a pnMg== X-Forwarded-Encrypted: i=1; AJvYcCWfC+VWcU2a1lxZ/yRdP05gjcaIruKWteY/d1vbG5cjV9w85ZjxGQopKTWaExgOoLZlsUBFiiqi9G2ldWkLwPNX6Hp0c/PQ8KqTrCkB X-Gm-Message-State: AOJu0Yzi9bNM5bxMvox7H48XOW+gTAjwv/bMWlNUqveHm9prcGVr0dG2 8FiAoATqbP9QMUShec1pu0Ri6RNfFYDGtUEspRzf40KlS/lEGD6kwiINhFFPNw== X-Google-Smtp-Source: AGHT+IGR8vrTWY1WaI5oE4IRdj4GRFNX5p3VLQAJ1dsWkq6eebPZOQskjAK08wq1ThkjZxRjS5rZPw== X-Received: by 2002:ad4:430b:0:b0:691:4d1f:6a65 with SMTP id c11-20020ad4430b000000b006914d1f6a65mr2449565qvs.27.1710902544204; Tue, 19 Mar 2024 19:42:24 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id k12-20020ad45bec000000b006961199b96asm3707324qvc.13.2024.03.19.19.42.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:23 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 5/7] proactively move pages out of unmovable zones in kcompactd Date: Wed, 20 Mar 2024 02:42:16 +0000 Message-Id: <20240320024218.203491-6-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaiyang Zhao Proactively move pages out of unmovable zones in kcompactd Debug only: zone start and end pfn printed in vmstat Added counters for cross zone compaction start and scan Signed-off-by: Kaiyang Zhao --- include/linux/vm_event_item.h | 3 + mm/compaction.c | 101 +++++++++++++++++++++++++++++++--- mm/vmstat.c | 11 +++- 3 files changed, 104 insertions(+), 11 deletions(-) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index be88819085b6..c9183117c8f7 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -80,6 +80,9 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, KCOMPACTD_WAKE, KCOMPACTD_MIGRATE_SCANNED, KCOMPACTD_FREE_SCANNED, COMPACT_CROSS_ZONE_MIGRATED, + KCOMPACTD_CROSS_ZONE_START, + COMPACT_CROSS_ZONE_MIGRATE_SCANNED, + COMPACT_CROSS_ZONE_FREE_SCANNED, #endif #ifdef CONFIG_HUGETLB_PAGE HTLB_BUDDY_PGALLOC, HTLB_BUDDY_PGALLOC_FAIL, diff --git a/mm/compaction.c b/mm/compaction.c index dea10ad8ec64..94ce1282f17b 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1436,7 +1436,10 @@ fast_isolate_freepages(struct compact_control *cc) * Preferred point is in the top quarter of the scan space but take * a pfn from the top half if the search is problematic. */ - distance =3D (cc->free_pfn - cc->migrate_pfn); + if (cc->zone !=3D dst_zone) + distance =3D (cc->free_pfn - dst_zone->zone_start_pfn) >> 1; + else + distance =3D (cc->free_pfn - cc->migrate_pfn); low_pfn =3D pageblock_start_pfn(cc->free_pfn - (distance >> 2)); min_pfn =3D pageblock_start_pfn(cc->free_pfn - (distance >> 1)); =20 @@ -1602,7 +1605,10 @@ static void isolate_freepages(struct compact_control= *cc) block_start_pfn =3D pageblock_start_pfn(isolate_start_pfn); block_end_pfn =3D min(block_start_pfn + pageblock_nr_pages, zone_end_pfn(zone)); - low_pfn =3D pageblock_end_pfn(cc->migrate_pfn); + if (cc->dst_zone && cc->zone !=3D cc->dst_zone) + low_pfn =3D pageblock_end_pfn(cc->dst_zone->zone_start_pfn); + else + low_pfn =3D pageblock_end_pfn(cc->migrate_pfn); stride =3D cc->mode =3D=3D MIGRATE_ASYNC ? COMPACT_CLUSTER_MAX : 1; =20 /* @@ -1822,7 +1828,11 @@ static unsigned long fast_find_migrateblock(struct c= ompact_control *cc) * within the first eighth to reduce the chances that a migration * target later becomes a source. */ - distance =3D (cc->free_pfn - cc->migrate_pfn) >> 1; + if (cc->dst_zone && cc->zone !=3D cc->dst_zone) + distance =3D (zone_end_pfn(cc->zone) - cc->migrate_pfn) >> 1; + else + distance =3D (cc->free_pfn - cc->migrate_pfn) >> 1; + if (cc->migrate_pfn !=3D cc->zone->zone_start_pfn) distance >>=3D 2; high_pfn =3D pageblock_start_pfn(cc->migrate_pfn + distance); @@ -1897,7 +1907,7 @@ static isolate_migrate_t isolate_migratepages(struct = compact_control *cc) { unsigned long block_start_pfn; unsigned long block_end_pfn; - unsigned long low_pfn; + unsigned long low_pfn, high_pfn; struct page *page; const isolate_mode_t isolate_mode =3D (sysctl_compact_unevictable_allowed ? ISOLATE_UNEVICTABLE : 0) | @@ -1924,11 +1934,16 @@ static isolate_migrate_t isolate_migratepages(struc= t compact_control *cc) /* Only scan within a pageblock boundary */ block_end_pfn =3D pageblock_end_pfn(low_pfn); =20 + if (cc->dst_zone && cc->zone !=3D cc->dst_zone) + high_pfn =3D zone_end_pfn(cc->zone); + else + high_pfn =3D cc->free_pfn; + /* * Iterate over whole pageblocks until we find the first suitable. * Do not cross the free scanner. */ - for (; block_end_pfn <=3D cc->free_pfn; + for (; block_end_pfn <=3D high_pfn; fast_find_block =3D false, cc->migrate_pfn =3D low_pfn =3D block_end_pfn, block_start_pfn =3D block_end_pfn, @@ -1954,6 +1969,7 @@ static isolate_migrate_t isolate_migratepages(struct = compact_control *cc) * before making it "skip" so other compaction instances do * not scan the same block. */ + if (pageblock_aligned(low_pfn) && !fast_find_block && !isolation_suitable(cc, page)) continue; @@ -1976,6 +1992,10 @@ static isolate_migrate_t isolate_migratepages(struct= compact_control *cc) isolate_mode)) return ISOLATE_ABORT; =20 + /* free_pfn may have changed. update high_pfn. */ + if (!cc->dst_zone || cc->zone =3D=3D cc->dst_zone) + high_pfn =3D cc->free_pfn; + /* * Either we isolated something and proceed with migration. Or * we failed and compact_zone should decide if we should @@ -2141,7 +2161,9 @@ static enum compact_result __compact_finished(struct = compact_control *cc) goto out; } =20 - if (is_via_compact_memory(cc->order)) + /* Don't check if a suitable page is free if doing cross zone compaction.= */ + if (is_via_compact_memory(cc->order) || + (cc->dst_zone && cc->dst_zone !=3D cc->zone)) return COMPACT_CONTINUE; =20 /* @@ -2224,7 +2246,8 @@ static enum compact_result __compaction_suitable(stru= ct zone *zone, int order, * should be no need for compaction at all. */ if (zone_watermark_ok(zone, order, watermark, highest_zoneidx, - alloc_flags)) + alloc_flags) && + dst_zone =3D=3D zone) return COMPACT_SUCCESS; =20 /* @@ -2270,6 +2293,11 @@ enum compact_result compaction_suitable(struct zone = *zone, int order, =20 ret =3D __compaction_suitable(zone, order, alloc_flags, highest_zoneidx, zone_page_state(dst_zone, NR_FREE_PAGES), dst_zone); + + /* Allow migrating movable pages to ZONE_MOVABLE regardless of frag index= */ + if (ret =3D=3D COMPACT_CONTINUE && dst_zone !=3D zone) + return ret; + /* * fragmentation index determines if allocation failures are due to * low memory or external fragmentation @@ -2841,6 +2869,14 @@ void compaction_unregister_node(struct node *node) } #endif /* CONFIG_SYSFS && CONFIG_NUMA */ =20 +static inline bool should_compact_unmovable_zones(pg_data_t *pgdat) +{ + if (populated_zone(&pgdat->node_zones[ZONE_MOVABLE])) + return true; + else + return false; +} + static inline bool kcompactd_work_requested(pg_data_t *pgdat) { return pgdat->kcompactd_max_order > 0 || kthread_should_stop() || @@ -2942,6 +2978,48 @@ static void kcompactd_do_work(pg_data_t *pgdat) pgdat->kcompactd_highest_zoneidx =3D pgdat->nr_zones - 1; } =20 +static void kcompactd_clean_unmovable_zones(pg_data_t *pgdat) +{ + int zoneid; + struct zone *zone; + struct compact_control cc =3D { + .order =3D 0, + .search_order =3D 0, + .highest_zoneidx =3D ZONE_MOVABLE, + .mode =3D MIGRATE_SYNC, + .ignore_skip_hint =3D true, + .gfp_mask =3D GFP_KERNEL, + .dst_zone =3D &pgdat->node_zones[ZONE_MOVABLE], + .whole_zone =3D true + }; + count_compact_event(KCOMPACTD_CROSS_ZONE_START); + + for (zoneid =3D 0; zoneid < ZONE_MOVABLE; zoneid++) { + int status; + + zone =3D &pgdat->node_zones[zoneid]; + if (!populated_zone(zone)) + continue; + + if (compaction_suitable(zone, cc.order, 0, zoneid, cc.dst_zone) !=3D + COMPACT_CONTINUE) + continue; + + if (kthread_should_stop()) + return; + + /* Not participating in compaction defer. */ + + cc.zone =3D zone; + status =3D compact_zone(&cc, NULL); + + count_compact_events(COMPACT_CROSS_ZONE_MIGRATE_SCANNED, + cc.total_migrate_scanned); + count_compact_events(COMPACT_CROSS_ZONE_FREE_SCANNED, + cc.total_free_scanned); + } +} + void wakeup_kcompactd(pg_data_t *pgdat, int order, int highest_zoneidx) { if (!order) @@ -2994,9 +3072,10 @@ static int kcompactd(void *p) =20 /* * Avoid the unnecessary wakeup for proactive compaction - * when it is disabled. + * and cleanup of unmovable zones + * when they are disabled. */ - if (!sysctl_compaction_proactiveness) + if (!sysctl_compaction_proactiveness && !should_compact_unmovable_zones(= pgdat)) timeout =3D MAX_SCHEDULE_TIMEOUT; trace_mm_compaction_kcompactd_sleep(pgdat->node_id); if (wait_event_freezable_timeout(pgdat->kcompactd_wait, @@ -3017,6 +3096,10 @@ static int kcompactd(void *p) continue; } =20 + /* Migrates movable pages out of unmovable zones if ZONE_MOVABLE exists = */ + if (should_compact_unmovable_zones(pgdat)) + kcompactd_clean_unmovable_zones(pgdat); + /* * Start the proactive work with default timeout. Based * on the fragmentation score, this timeout is updated. diff --git a/mm/vmstat.c b/mm/vmstat.c index 98af82e65ad9..444740605f2f 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1325,6 +1325,9 @@ const char * const vmstat_text[] =3D { "compact_daemon_migrate_scanned", "compact_daemon_free_scanned", "compact_cross_zone_migrated", + "compact_cross_zone_start", + "compact_cross_zone_migrate_scanned", + "compact_cross_zone_free_scanned", #endif =20 #ifdef CONFIG_HUGETLB_PAGE @@ -1692,7 +1695,9 @@ static void zoneinfo_show_print(struct seq_file *m, p= g_data_t *pgdat, "\n spanned %lu" "\n present %lu" "\n managed %lu" - "\n cma %lu", + "\n cma %lu" + "\n start %lu" + "\n end %lu", zone_page_state(zone, NR_FREE_PAGES), zone->watermark_boost, min_wmark_pages(zone), @@ -1701,7 +1706,9 @@ static void zoneinfo_show_print(struct seq_file *m, p= g_data_t *pgdat, zone->spanned_pages, zone->present_pages, zone_managed_pages(zone), - zone_cma_pages(zone)); + zone_cma_pages(zone), + zone->zone_start_pfn, + zone_end_pfn(zone)); =20 seq_printf(m, "\n protection: (%ld", --=20 2.40.1 From nobody Sun Feb 8 17:41:52 2026 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9DBB107A6 for ; Wed, 20 Mar 2024 02:42:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902548; cv=none; b=e/v7KO+hyB3Vj6ahbAgZs6xKr8HDfD8QMPcff1wXtx32CUNlTp83MCF/jTTqnkIb2FSFfeLq6ZYVF9fOhpPtOQTlamhzYUpx1GbbAdwQjiitV3FTUytLl+ooIFudTMRoSuXd3quWuhRP79yIEtiOAsNQCvoxPoFn0Jo0PDgKVYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902548; c=relaxed/simple; bh=xfUq48QO6OVoxR0oxw0nGWYHlbBWHnOlmAnISATgV2g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jlhTiKcGMtjh+5Ww5cOZuwTgYxZ6KPnP+ynvqxfu1A/Ms55ivBbWSdgRDEJKxSn3otq9TPwh6GFJhtRtxbYiba9X1es0YcTP+kX20Ebb3f78PO4mSrc/ate8BtsskhixuM685auYsTE79EpxpdXrXNktjZV2/rLvVGh2U+KbMLw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu; spf=pass smtp.mailfrom=andrew.cmu.edu; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b=ebfEqXGg; arc=none smtp.client-ip=209.85.160.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=andrew.cmu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b="ebfEqXGg" Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-430a25ed4e7so38551311cf.0 for ; Tue, 19 Mar 2024 19:42:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902545; x=1711507345; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=WzVosZB8NV09/ZiVxYfUQEEiRP/cgORaiI2cKH0KzxM=; b=ebfEqXGg6zInbzB7p1gP3M53nHGWaA5XWy7Fae8D7C2svwmpxRiNO5UYt7NnqZAgyG D2x3EJwSi7B+FE7L8ugadDarYDrtoN99P12ISP5ubQzMJIc5YWTDaB9D5TcQoMdY0L+U GP2SD7/xnUJNUitgdQ3uzXdQmlKKiBkAxXH2SrB1erchbMkzqeSxhk6TDlEkk/1ixFtA ByzYJG13YznlEKkZqDPddbZynW6Nfc+zV84irNNHVwYfbIt4J2LWku85lnO2dzY4lrXF ryh++Pu6PtqrqI+2+hTmDp8GgOc8LrhnxySiUUiiMzU3mr0gubmht78e3zly0F9yfTHO NOSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902545; x=1711507345; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=WzVosZB8NV09/ZiVxYfUQEEiRP/cgORaiI2cKH0KzxM=; b=M+anUt8jLvl7XmO1V364Eez9n5VtcHwM1skF+TPp9t0eh8lXQD8zE3rH4m5fs78jGB z2sAZTb6MM9OJW7ecjbB/8Kmdwfju6ZRrRbBZ+RcYV3TbmmqIygVSGVgTb5Y7OvYwTTZ gBloZeWiQGqdry7eZse2nVbkf9PRyzjEX2lm3fbZc4zML8RroYiTgSqqIbMJkOzIoAAA r7a3WUGth80mz2ERn8B31kcam9xGfrKNn3vMCf/mW/xYNmsNCUpf3DMHRxpzrbJwseuq 9iATjcIzJK/Gz5DRsQS9EvzWu++WKn5EICjAviqpkk3VBPxPeQ+KjlNbU/Qsf63LIQNH IFvg== X-Forwarded-Encrypted: i=1; AJvYcCUiO3DRPFtjp/CMK0h0ZLwGHWa74ZT+AE2Zr3cttUE1ujguTd1o2T1VJ3Xv02RskECkyRq5Hd+3fFX1houuYEqyB+4ZGAtJnO87HENH X-Gm-Message-State: AOJu0YyVIoVEfpV91Pweqogyg65tFYeQ5+UWwvgtHMwg3sdnZK8Fi66Q 6JHaxBLdITkZAyN3VWr7Dwmy/4VSDHtwJgDbotbCy2wiAIb1j63OzEDYFcNocA== X-Google-Smtp-Source: AGHT+IGEfbBr6cutAmN+mlO7n/aShmIQU3GTmhE5CM9Lh4mCEbZf0tbj0IjwPIaD6NVNuXa3bBmlRg== X-Received: by 2002:a05:622a:110a:b0:431:baa:ab0c with SMTP id e10-20020a05622a110a00b004310baaab0cmr135047qty.50.1710902544986; Tue, 19 Mar 2024 19:42:24 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id fg14-20020a05622a580e00b00430bddc75a5sm4582852qtb.23.2024.03.19.19.42.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:24 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 6/7] pass gfp mask of the allocation that waked kswapd to track number of pages scanned on behalf of each alloc type Date: Wed, 20 Mar 2024 02:42:17 +0000 Message-Id: <20240320024218.203491-7-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaiyang Zhao In preparation for exporting the number of pages scanned for each alloc type Signed-off-by: Kaiyang Zhao --- include/linux/mmzone.h | 1 + mm/vmscan.c | 13 +++++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index a4889c9d4055..abc9f1623c82 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1288,6 +1288,7 @@ typedef struct pglist_data { struct task_struct *kswapd; /* Protected by kswapd_lock */ int kswapd_order; enum zone_type kswapd_highest_zoneidx; + gfp_t kswapd_gfp; =20 int kswapd_failures; /* Number of 'reclaimed =3D=3D 0' runs */ =20 diff --git a/mm/vmscan.c b/mm/vmscan.c index aa21da983804..ed0f47e2e810 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -7330,7 +7330,7 @@ clear_reclaim_active(pg_data_t *pgdat, int highest_zo= neidx) * or lower is eligible for reclaim until at least one usable zone is * balanced. */ -static int balance_pgdat(pg_data_t *pgdat, int order, int highest_zoneidx) +static int balance_pgdat(pg_data_t *pgdat, int order, int highest_zoneidx,= gfp_t gfp_mask) { int i; unsigned long nr_soft_reclaimed; @@ -7345,6 +7345,8 @@ static int balance_pgdat(pg_data_t *pgdat, int order,= int highest_zoneidx) .order =3D order, .may_unmap =3D 1, }; + if (is_migrate_movable(gfp_migratetype(gfp_mask))) + sc.gfp_mask |=3D __GFP_MOVABLE; =20 set_task_reclaim_state(current, &sc.reclaim_state); psi_memstall_enter(&pflags); @@ -7659,6 +7661,7 @@ static int kswapd(void *p) pg_data_t *pgdat =3D (pg_data_t *)p; struct task_struct *tsk =3D current; const struct cpumask *cpumask =3D cpumask_of_node(pgdat->node_id); + gfp_t gfp_mask; =20 if (!cpumask_empty(cpumask)) set_cpus_allowed_ptr(tsk, cpumask); @@ -7680,6 +7683,7 @@ static int kswapd(void *p) =20 WRITE_ONCE(pgdat->kswapd_order, 0); WRITE_ONCE(pgdat->kswapd_highest_zoneidx, MAX_NR_ZONES); + WRITE_ONCE(pgdat->kswapd_gfp, 0); atomic_set(&pgdat->nr_writeback_throttled, 0); for ( ; ; ) { bool ret; @@ -7687,6 +7691,7 @@ static int kswapd(void *p) alloc_order =3D reclaim_order =3D READ_ONCE(pgdat->kswapd_order); highest_zoneidx =3D kswapd_highest_zoneidx(pgdat, highest_zoneidx); + gfp_mask =3D READ_ONCE(pgdat->kswapd_gfp); =20 kswapd_try_sleep: kswapd_try_to_sleep(pgdat, alloc_order, reclaim_order, @@ -7696,8 +7701,10 @@ static int kswapd(void *p) alloc_order =3D READ_ONCE(pgdat->kswapd_order); highest_zoneidx =3D kswapd_highest_zoneidx(pgdat, highest_zoneidx); + gfp_mask =3D READ_ONCE(pgdat->kswapd_gfp); WRITE_ONCE(pgdat->kswapd_order, 0); WRITE_ONCE(pgdat->kswapd_highest_zoneidx, MAX_NR_ZONES); + WRITE_ONCE(pgdat->kswapd_gfp, 0); =20 ret =3D try_to_freeze(); if (kthread_should_stop()) @@ -7721,7 +7728,7 @@ static int kswapd(void *p) trace_mm_vmscan_kswapd_wake(pgdat->node_id, highest_zoneidx, alloc_order); reclaim_order =3D balance_pgdat(pgdat, alloc_order, - highest_zoneidx); + highest_zoneidx, gfp_mask); if (reclaim_order < alloc_order) goto kswapd_try_sleep; } @@ -7759,6 +7766,8 @@ void wakeup_kswapd(struct zone *zone, gfp_t gfp_flags= , int order, if (READ_ONCE(pgdat->kswapd_order) < order) WRITE_ONCE(pgdat->kswapd_order, order); =20 + WRITE_ONCE(pgdat->kswapd_gfp, gfp_flags); + if (!waitqueue_active(&pgdat->kswapd_wait)) return; =20 --=20 2.40.1 From nobody Sun Feb 8 17:41:52 2026 Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2E6D11717 for ; Wed, 20 Mar 2024 02:42:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902549; cv=none; b=eb0s0mQaJHopy6HPwJfwzIZHOk/MH/LiBE6SzG55s0Hsh9fNaBIOIJsdihom28c1+tljwAnRzuMlAlutPDw7wk6hPanmgVxC3MevtTrhYJ5DzeeNt1Bgh05MugCBeVaZkWgkY1AjxraIfKU01Nc4PFQd/+v/g65XkcXaE2CwomE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710902549; c=relaxed/simple; bh=WFhBYAwLNBbB8uV/HwzsFmNf8vujXFelMMa+4V8Uzr0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lw6cKxn1tG0F/g1JVyNxCnneNM2RMM20LnmR0fRJfUDCrxhG7BVsrm1SYnogna1UyySOenIBBy97L7d2FHKP59KUqOd50Efl8MrHKslRoq9s1sDxtz6yNnl4HtNSY8JA3GZHU/+yum7Vxpy0XxhIAHpHWgFgOelTtdgrjrAi/Qo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu; spf=pass smtp.mailfrom=andrew.cmu.edu; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b=KLh5EMNq; arc=none smtp.client-ip=209.85.219.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cs.cmu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=andrew.cmu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cs.cmu.edu header.i=@cs.cmu.edu header.b="KLh5EMNq" Received: by mail-qv1-f53.google.com with SMTP id 6a1803df08f44-69629b4ae2bso15004146d6.3 for ; Tue, 19 Mar 2024 19:42:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902545; x=1711507345; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=DIGL+dvsKsYG7Tx7310j/b/qOHwOOB9JaG0dOT6QmVU=; b=KLh5EMNqAhttxh2TlBWT9qn8jbLvEytYduqt77T2iiUTTG+otRGEo1CpM75ZU0kQMS 1mXlSCJi/zxjmUY7GRVuiJ/ZQTYR7B0WC1ZZe/ysWNKILTsgmy6juKN6v5Sb3mnZ7dcX 2pQrFKhoX6LYjSZsB/DCzuU7HpOFtpYpB5BXHQ39cK9ffs9utgLOGti+XonRnEIaVI0A WS2QygmrLMB0BHQe/LMOE/O5E5dpLOQWGk2THiZQWH8i9bTqnDS6QwvZziRFoQB3BggB BGHgLZjjNFKf7BWgl70qKiIh/U3tSvglH5WyFc1bCJiRnoU/5CRejTXqUQ6XwnmokMkP Hhfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902545; x=1711507345; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=DIGL+dvsKsYG7Tx7310j/b/qOHwOOB9JaG0dOT6QmVU=; b=VNiQvb9HJYiNQvnEEXyR8w2ULlopHA9wg0ekqGFTDDON5dDLQpLvd0Ltg3qwEV3V+P aLVgSVHM6G61a8iW5yyeyP21h+4L2l17bmhYD8uTHx1W/NViqn9AyMX4s07+whqy5Ad9 /HQD9vW+5qDYHPtJJygBw22n4cteiH5441fSdylcJddUkA6NX+SHARKWFcZ2D0PPG4r9 7/ms4BUwEZrq0FxPEFBbmhPcXViU2qHRjpi5ohQ/UDuwv1vXOlymGNduerhkPW0Mqzkp oBPJDUhtCP12ta7ck31df2RfKyQKolLvf/Z+OhQBDhCTAMBzeyXQv59pLbRzxqt3yOIt KUgQ== X-Forwarded-Encrypted: i=1; AJvYcCWxxcSeZjMAvQqEsYfEXFoZMo6nbRU8zQ+JWu7x5CESRLUtotSBV60BbsdmL8ug51MiinLJXFcckZ6VWcZesaSRmKz1AMcEMyKUolPr X-Gm-Message-State: AOJu0Yx4rkvlsWyo7K1T3MylkJZszpkh1HWRtwqHttjb1Hm3hJHNcox9 H8DRHJMkFA9vRj42Rp8tX8NGzw9FFtU+cv2mpbPiyxkz7pHLqWbfXvNpi5n6Pp7X+39kD/DcrG1 CGw== X-Google-Smtp-Source: AGHT+IF1l0N2egM3aSC98LzaXNROU1+O+6Y4xqmw+dsS00Fx32Wnx7e/F2IaRaJEP8IIHWZ1DYSFqw== X-Received: by 2002:a0c:dc14:0:b0:696:22ae:eb67 with SMTP id s20-20020a0cdc14000000b0069622aeeb67mr4027527qvk.33.1710902545749; Tue, 19 Mar 2024 19:42:25 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id g15-20020a0562140acf00b0069150ebcc30sm7204402qvi.76.2024.03.19.19.42.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:25 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 7/7] exports the number of pages scanned on behalf of movable/unmovable allocations Date: Wed, 20 Mar 2024 02:42:18 +0000 Message-Id: <20240320024218.203491-8-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kaiyang Zhao exports the number of pages scanned on behalf of movable/unmovable allocations in vmstat Signed-off-by: Kaiyang Zhao --- include/linux/vm_event_item.h | 2 ++ mm/vmscan.c | 11 +++++++++++ mm/vmstat.c | 2 ++ 3 files changed, 15 insertions(+) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index c9183117c8f7..dcfff56c6d29 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -50,6 +50,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, PGSCAN_DIRECT_THROTTLE, PGSCAN_ANON, PGSCAN_FILE, + PGSCAN_MOVABLE, /* number of pages scanned on behalf of a movable all= ocation */ + PGSCAN_UNMOVABLE, PGSTEAL_ANON, PGSTEAL_FILE, #ifdef CONFIG_NUMA diff --git a/mm/vmscan.c b/mm/vmscan.c index ed0f47e2e810..4eadf0254918 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -904,6 +904,12 @@ static unsigned long do_shrink_slab(struct shrink_cont= rol *shrinkctl, cond_resched(); } =20 + /* Arbitrarily consider 16 pages scanned */ + if (is_migrate_movable(gfp_migratetype(shrinkctl->gfp_mask))) + count_vm_events(PGSCAN_MOVABLE, 16); + else + count_vm_events(PGSCAN_UNMOVABLE, 16); + /* * The deferred work is increased by any new work (delta) that wasn't * done, decreased by old deferred work that was done now. @@ -2580,6 +2586,11 @@ static unsigned long shrink_inactive_list(unsigned l= ong nr_to_scan, __count_memcg_events(lruvec_memcg(lruvec), item, nr_scanned); __count_vm_events(PGSCAN_ANON + file, nr_scanned); =20 + if (is_migrate_movable(gfp_migratetype(sc->gfp_mask))) + __count_vm_events(PGSCAN_MOVABLE, nr_scanned); + else + __count_vm_events(PGSCAN_UNMOVABLE, nr_scanned); + spin_unlock_irq(&lruvec->lru_lock); =20 if (nr_taken =3D=3D 0) diff --git a/mm/vmstat.c b/mm/vmstat.c index 444740605f2f..56062d53a36c 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1281,6 +1281,8 @@ const char * const vmstat_text[] =3D { "pgscan_direct_throttle", "pgscan_anon", "pgscan_file", + "pgscan_by_movable", + "pgscan_by_unmovable", "pgsteal_anon", "pgsteal_file", =20 --=20 2.40.1