From nobody Fri Sep 25 13:20:05 2026 Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D02223BD1D for ; Sat, 12 Sep 2026 03:04:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789182273; cv=none; b=OqUnaMC20jwyaW7xpqBbBaO/+/ShCUguouc4RbozpFE8pl8Z1S45Ec/2NUY/MRS/4pS+CMKwOajbGB8bkeKmH47419xj9EgL/beV9ht4jYr63ZymCaEMphPLtV0YfoeVfnLWpdmH1Lbkh0giSKsO4+FtoXDL7YUJiJgH1g/+leo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789182273; c=relaxed/simple; bh=UumXD4YpIiaTE+6DzaH3TiHCesszgH7gFWbWLue+XkQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RoFFRc6iTwWI7CLR/NKe5hvP6zo4loqyQEC+ra75xXX1LCNrABqyQJuGvB2tYC26cYw0FhgUh22Imf61TXyOvCCSEO3ktVtAZmDk7jBtNGzT7JbBCcZ0wbNtp0K7IWDkKbI24Y/sNsjOya1OUlI4iBe2YwsQZOzqUjYzbHCpMGg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=RAFLJaSC; arc=none smtp.client-ip=209.85.219.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="RAFLJaSC" Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-9103228ef53so16838146d6.0 for ; Fri, 11 Sep 2026 20:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1789182270; x=1789787070; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=47nBXTe0p7S8h9Q2JUen+5KKbEFQG9U2loUXl5hvd/I=; b=RAFLJaSCV9/WY60wNvP1Y29xIGkys4wLLQQ6uDBB3vX6EHLUfK+D7CQtLxbSry2n8G 1QbCw13Hy3gL2cnR3Ol5DTyySJGuSOdiMj5axUA54BCBU61d0Cp5lNUgKift+3orNUyb Bn3bAUuneHWRI3CL9QeeitszkEB8xFPaA584lh26yZ6aF6RVake7nWjzDNkC4M8cgAfB 9BaKy4svCnw/aSyRiQp3yt46gIh8lnbPsBYP0mjOK+XZ6vMJLYRbnOby4alDJxBkZIfY HtucsrBGk12FkbBnrlp0hjOBbOFc7LmN5D28LIuuisJ3W1wtYJTIs8jJZ35OQDrTziaA SebA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789182270; x=1789787070; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=47nBXTe0p7S8h9Q2JUen+5KKbEFQG9U2loUXl5hvd/I=; b=g1ZoZ/tdJ4i31EXvmO4Lyo+MpgDvZDIS5gmdmVHPBDAAKhURnSuiBcBKdjXaMtRO0u mezfQ26m9YMFJSscgtB0zqupD3eXXCHmJXRFjLNepmBlw0z45WINtkf2iFhHuiQygeB5 kgeY42mPw+mUUZ8j+QWCpWOHYaLcM2P0bhTAeTRrlY+D50moYs2RxKqOHMAUsc191vVV P9QF58saoaVX+ILCClfjXpyCzksfck1qzSEqwG6itQca3y4oadVzgHbqh6lCUvaosx1f 4kyw3FG8eyI6nAzx9E7hOm7FNmBRPGhOafyVktUgWCNFUtUg82wdlXph+gUEuxsCNxjT PrSA== X-Gm-Message-State: AFuF++liIe9OqiOufdHeHWVOW+W6lr+fXQyEQLXjZmjzJIaMz/FKVUVS cOEXunIZNf00z3a+uoDyA4v0rqGNnDTz9onBFdOi1H36PDVmyYkMZoMeAto0NvSVRkA= X-Gm-Gg: AYBFou0pCmE3mkJcBzTUWmuzu7dpARd0+Ph4EQzFtonRlMR2IkAKeXFdBIPbDVeeuJ0 tnoe5xuQrVobQnCugNYxH1rfNS6BocnON5CWxRiMoNaildWpUXStseIDeam4nc3ztDzv37Xtn3u B0l8iheEoXLEmDiMXollSZi6xqqI09ip6mxJPScFYyqgLT+uNUtE16z2SL7NAalUotnmhGdjFW5 ZP2JtlSTefyi4fsvw0nWV6LymoKaSnASQcRL99+tTnXPToP6kdye484vLpG38ml6MTn5jwTpyRQ yyJT2gz29MBio89XEWPIg9pjvLZjI8fjm6w/hGeX6oED9OIYGEUJwc9vYu3yJ7fNE0jwVn6ZVmo HvU4dunc4ECqHoclU4R8aFLD5lF4tXWMXiw/q5D672xTr8Ir4rz03j7fTKT4ifqFEV4g3xjJy1s HRbKpRFps0uagY7I6l1nudhqs7rFJQPZgfvtZezEEU2Tbs6mDjeALPfvdEd8ZO8Odl79R/TDsx4 XXpcLUSzLsFi7nR+lRZyWMjQ7XQD++Xk6iSqPacLIn3RccHJtLzvngXk8vP X-Received: by 2002:a05:6214:212e:b0:910:44be:42b8 with SMTP id 6a1803df08f44-9121207d62cmr104319336d6.6.1789182270357; Fri, 11 Sep 2026 20:04:30 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-9120f202856sm36085296d6.2.2026.09.11.20.04.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2026 20:04:30 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org, liam@infradead.org, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org, ziy@nvidia.com, Balbir Singh Subject: [PATCH v2 1/2] mm: refactor find_next_best_node to find_next_best_node_in Date: Fri, 11 Sep 2026 23:04:23 -0400 Message-ID: <20260912030424.2889731-2-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260912030424.2889731-1-gourry@gourry.net> References: <20260912030424.2889731-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" find_next_best_node() picks the next-closest node for a fallback list from the full N_MEMORY set. Refactor it into find_next_best_node_in(), which takes an explicit candidates nodemask. This enables building fallback lists with non-N_MEMORY candidates. No functional change: every caller still selects from N_MEMORY. Reviewed-by: Vlastimil Babka (SUSE) Acked-by: Balbir Singh Signed-off-by: Gregory Price Reviewed-by: Zenghui Yu (Huawei) Reviewed-by: Zi Yan --- mm/internal.h | 6 ++++-- mm/memory-tiers.c | 7 ++++--- mm/page_alloc.c | 13 ++++++++----- 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index da14c56fb24e..1519dd21a900 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1133,7 +1133,8 @@ extern int node_reclaim_mode; =20 extern unsigned long node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned int order); -extern int find_next_best_node(int node, nodemask_t *used_node_mask); +int find_next_best_node_in(int node, nodemask_t *used_node_mask, + const nodemask_t *candidates); #else #define node_reclaim_mode 0 =20 @@ -1142,7 +1143,8 @@ static inline unsigned long node_reclaim(struct pglis= t_data *pgdat, { return 0; } -static inline int find_next_best_node(int node, nodemask_t *used_node_mask) +static inline int find_next_best_node_in(int node, nodemask_t *used_node_m= ask, + const nodemask_t *candidates) { return NUMA_NO_NODE; } diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 54851d8a195b..25e121851b58 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -370,7 +370,7 @@ int next_demotion_node(int node, const nodemask_t *allo= wed_mask) * closest demotion target. */ nodes_complement(mask, *allowed_mask); - return find_next_best_node(node, &mask); + return find_next_best_node_in(node, &mask, &node_states[N_MEMORY]); } =20 static void disable_all_demotion_targets(void) @@ -450,7 +450,7 @@ static void establish_demotion_targets(void) memtier =3D list_next_entry(memtier, list); tier_nodes =3D get_memtier_nodemask(memtier); /* - * find_next_best_node, use 'used' nodemask as a skip list. + * find_next_best_node_in, use 'used' nodemask as a skip list. * Add all memory nodes except the selected memory tier * nodelist to skip list so that we find the best node from the * memtier nodelist. @@ -463,7 +463,8 @@ static void establish_demotion_targets(void) * in the preferred mask when allocating pages during demotion. */ do { - target =3D find_next_best_node(node, &tier_nodes); + target =3D find_next_best_node_in(node, &tier_nodes, + &node_states[N_MEMORY]); if (target =3D=3D NUMA_NO_NODE) break; =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1cfbd0582a10..7efce139d562 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5793,9 +5793,10 @@ static int numa_zonelist_order_handler(const struct = ctl_table *table, int write, static int node_load[MAX_NUMNODES]; =20 /** - * find_next_best_node - find the next node that should appear in a given = node's fallback list + * find_next_best_node_in - find the next node that should appear in a giv= en node's fallback list * @node: node whose fallback list we're appending * @used_node_mask: nodemask_t of already used nodes + * @candidates: nodemask_t of nodes eligible for selection * * We use a number of factors to determine which is the next node that sho= uld * appear on a given node's fallback list. The node should not have appea= red @@ -5807,7 +5808,8 @@ static int node_load[MAX_NUMNODES]; * * Return: node id of the found node or %NUMA_NO_NODE if no node is found. */ -int find_next_best_node(int node, nodemask_t *used_node_mask) +int find_next_best_node_in(int node, nodemask_t *used_node_mask, + const nodemask_t *candidates) { int n, val; int min_val =3D INT_MAX; @@ -5817,12 +5819,12 @@ int find_next_best_node(int node, nodemask_t *used_= node_mask) * Use the local node if we haven't already, but for memoryless local * node, we should skip it and fall back to other nodes. */ - if (!node_isset(node, *used_node_mask) && node_state(node, N_MEMORY)) { + if (!node_isset(node, *used_node_mask) && node_isset(node, *candidates)) { node_set(node, *used_node_mask); return node; } =20 - for_each_node_state(n, N_MEMORY) { + for_each_node_mask(n, *candidates) { =20 /* Don't want a node to appear more than once */ if (node_isset(n, *used_node_mask)) @@ -5907,7 +5909,8 @@ static void build_zonelists(pg_data_t *pgdat) prev_node =3D local_node; =20 memset(node_order, 0, sizeof(node_order)); - while ((node =3D find_next_best_node(local_node, &used_mask)) >=3D 0) { + while ((node =3D find_next_best_node_in(local_node, &used_mask, + &node_states[N_MEMORY])) >=3D 0) { /* * We don't want to pressure a particular node. * So adding penalty to the first node in same --=20 2.55.0 From nobody Fri Sep 25 13:20:05 2026 Received: from mail-qv2-f12.google.com (mail-qv2-f12.google.com [74.125.230.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4129B385D83 for ; Sat, 12 Sep 2026 03:04:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.230.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789182274; cv=none; b=Wtquc+q8VfvBvyInUl5+YdVW2R4hXNoR3tWjtwFt8z2CitRfFL+INX8uc3pi7TWyEhEGtvIRKGO9y+i4xqV4yOWBuk+LZRe+uwhatjjXHEbByQ8uPABUDkFgoF2QkoqNTY+7gVl4eKdIVTEF+DAuXLLVbuG0IjYKGSMnvW07wgA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789182274; c=relaxed/simple; bh=gjx27FDXG8SgLeoaq0Wo8ZxlupU6Oj73FPP3NgswvwY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=izAa9D4FOrfacbUEoXucUj2xgAV0EUz+Uv+KttUVnykjkOoYz/SrA+VtaQEF89Lkl5HyTH05JyCX+z4oxUTFqqbHyuaige2vVuLupQknMunmka+qMx5fQDtCDaUEUYAfPRMR26ZxZa7X9kNyQlPB73F7ms3rHitC8P8PbDnD4rc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=QxIy6itq; arc=none smtp.client-ip=74.125.230.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="QxIy6itq" Received: by mail-qv2-f12.google.com with SMTP id 6a1803df08f44-910399d8b09so1892796d6.2 for ; Fri, 11 Sep 2026 20:04:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1789182272; x=1789787072; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=pxIU5xWvvJNvSs/zUIGzI+Vs/buRWaBwnwkaFkIAvLk=; b=QxIy6itqPp2B0r7noOJStE00bUHh+yG0BflmgzX2juoZFQQ2viCFV9H9oMD9S4X7WP k9hb+RJ/vgpkS8IvTpCtbybIghNMKtsJfDKeFP1unG1ZroGPme5lZfxm8x1YvJ/TmbkK nHx5NfGxTl4g5Yegg7vruDCz4umZDTJAHcWJ1E6qvbxK/FCrnB1ZIMZs333GmhWHBs3P kz7L9JPJxHhlxq/pU4R7JnaCZ3JlSgy6J9NHl2C8K+KcxJaYnb4o/64x0UNLajVUy0Dc V5Rfp0UPSi5+wyRPxxMTvfenH5AVu/hZdWNuYZYiyNieA0TLWIWV1BPhGp6kk+5jtQJ/ IYiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789182272; x=1789787072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=pxIU5xWvvJNvSs/zUIGzI+Vs/buRWaBwnwkaFkIAvLk=; b=PSQh2hpvLe+NtKvNgzHB4O1y2ygUFWHxSZ2V83J4L3GPbtCBPSLZBXGwxhcft2IEtH fQHonhbrxGvFW3fc1S6AzjZbjRkKqUrjjDcKiEpCF/qBWaDvRKE9SzkbFlf0qJByR3lD ZHWKHQiSb4JE37VyzkijoFsm6JNtLDL9kLt02q1KCWW7NYkUz/JeVyrZREyfmDq0pAOO Z5vt+mPpZWFKfGWsoQ49tAPj5wAhqvuzJYzlMiwCL4oGACCXlj6gYiWuPEMA78pEu7pT NKDF2Jc1qsQJL6au/leukI5MsiZ7Lsfl67WJLW8EObzZ+ttJ1M+4b4XyoCQ2yevcKWSF UF4A== X-Gm-Message-State: AFuF++miFhKjoP2vtBg2Q3LSVNeqCjCwOk/QN0KPFg3PSt9HxcUsKvR/ Kt3zF2qjVqu13g4IxwHWpIvYf9FiyOtkKwtlRxQnQub4sV/qiAU0s/SP5Eg0O7bvEC4= X-Gm-Gg: AYBFou0DLRmyD91VF8GC70zOPAX2KswF0RpWCwHrEhsQnBxhWcWkmoy2ndx09/IHBhj HT9I1EVo/rsv33odPNhBvjCcPt4o7jloeaHOeNBVePz+ABtccXyyOiPHcwG3ABBQKsMoL0PBDmN x+zQ4uMPqCvlzGxT7kCcNs5MD9IzFfUfw/4bW3rqh0b971/MmtOZlZJ133PgVoAJQHodOLzdV/L 9l4e578p/jZskQJyiAH8GqLcRA84zEIi0YcAXsRyQsdbLQxjGrDSXk8bwZac03P5XCDJfOt8LuY ukAhedOF90Kv9XGmWOgZjyR9bMLqRCuGnK5oy3cfjNxA6/4VvPemlcAx2+7pfHw3Q6KqTjCAteB tWYR6pQD+gkrVgChI5Ql0zd0s+q0PluUthImEv6EXpZtJgRgOVwh5wyfT2iigWI+iVudEHBJosg wl6DQGGbJqihHKrAEM7UucHo7JSyfmiV6ZsbYPNrFBbuuR1SfA1M7lr7rxFEUeP2U+Qi68cuTgt EbIEeR2khVKUERQwg1v5B2YBExsPdhvTZpbbMsZbP86aE4WurkSGCcKamdZ7FQOp6IuTyw= X-Received: by 2002:a05:6214:2685:b0:912:bed:c74 with SMTP id 6a1803df08f44-9121da7ecd7mr19424046d6.30.1789182271965; Fri, 11 Sep 2026 20:04:31 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-9120f202856sm36085296d6.2.2026.09.11.20.04.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2026 20:04:31 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org, liam@infradead.org, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org, ziy@nvidia.com Subject: [PATCH v2 2/2] mm/page_alloc: refactor build_node_zonelist() out of build_zonelists() Date: Fri, 11 Sep 2026 23:04:24 -0400 Message-ID: <20260912030424.2889731-3-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260912030424.2889731-1-gourry@gourry.net> References: <20260912030424.2889731-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Extract per-node fallback-list construction into build_node_zonelist(). Build each selected node directly into the destination zonelist so no intermediate node_order array or node count is needed. Print the fallback order as each node is added. This lets us build new zonelists from candidate nodemasks instead of just the default N_MEMORY node state list. No functional change: build_zonelists() builds and prints the same FALLBACK list over N_MEMORY with node_load updates as before. Signed-off-by: Gregory Price Reviewed-by: Vlastimil Babka (SUSE) Reviewed-by: Zenghui Yu (Huawei) Reviewed-by: Zi Yan --- mm/page_alloc.c | 63 ++++++++++++++++++------------------------------- 1 file changed, 23 insertions(+), 40 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7efce139d562..d1888d5630e0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5857,31 +5857,6 @@ int find_next_best_node_in(int node, nodemask_t *use= d_node_mask, } =20 =20 -/* - * Build zonelists ordered by node and zones within node. - * This results in maximum locality--normal zone overflows into local - * DMA zone, if any--but risks exhausting DMA zone. - */ -static void build_zonelists_in_node_order(pg_data_t *pgdat, int *node_orde= r, - unsigned nr_nodes) -{ - struct zoneref *zonerefs; - int i; - - zonerefs =3D pgdat->node_zonelists[ZONELIST_FALLBACK]._zonerefs; - - for (i =3D 0; i < nr_nodes; i++) { - int nr_zones; - - pg_data_t *node =3D NODE_DATA(node_order[i]); - - nr_zones =3D build_zonerefs_node(node, zonerefs); - zonerefs +=3D nr_zones; - } - zonerefs->zone =3D NULL; - zonerefs->zone_idx =3D 0; -} - /* * Build __GFP_THISNODE zonelists */ @@ -5897,20 +5872,24 @@ static void build_thisnode_zonelists(pg_data_t *pgd= at) zonerefs->zone_idx =3D 0; } =20 -static void build_zonelists(pg_data_t *pgdat) +/* + * Build one zonelist ordered by node and zones within node. This results = in + * maximum locality--normal zone overflows into local DMA zone, if any--but + * risks exhausting DMA zone. + */ +static void build_node_zonelist(pg_data_t *pgdat, const nodemask_t *candid= ates, + int zlidx) { - static int node_order[MAX_NUMNODES]; - int node, nr_nodes =3D 0; + struct zoneref *zonerefs =3D pgdat->node_zonelists[zlidx]._zonerefs; nodemask_t used_mask =3D NODE_MASK_NONE; - int local_node, prev_node; + int local_node =3D pgdat->node_id; + int prev_node =3D local_node; + int node; =20 - /* NUMA-aware ordering of nodes */ - local_node =3D pgdat->node_id; - prev_node =3D local_node; + pr_info("Fallback order for Node %d: ", local_node); =20 - memset(node_order, 0, sizeof(node_order)); while ((node =3D find_next_best_node_in(local_node, &used_mask, - &node_states[N_MEMORY])) >=3D 0) { + candidates)) >=3D 0) { /* * We don't want to pressure a particular node. * So adding penalty to the first node in same @@ -5920,18 +5899,22 @@ static void build_zonelists(pg_data_t *pgdat) node_distance(local_node, prev_node)) node_load[node] +=3D 1; =20 - node_order[nr_nodes++] =3D node; + zonerefs +=3D build_zonerefs_node(NODE_DATA(node), zonerefs); + pr_cont("%d ", node); prev_node =3D node; } =20 - build_zonelists_in_node_order(pgdat, node_order, nr_nodes); - build_thisnode_zonelists(pgdat); - pr_info("Fallback order for Node %d: ", local_node); - for (node =3D 0; node < nr_nodes; node++) - pr_cont("%d ", node_order[node]); + zonerefs->zone =3D NULL; + zonerefs->zone_idx =3D 0; pr_cont("\n"); } =20 +static void build_zonelists(pg_data_t *pgdat) +{ + build_node_zonelist(pgdat, &node_states[N_MEMORY], ZONELIST_FALLBACK); + build_thisnode_zonelists(pgdat); +} + #ifdef CONFIG_HAVE_MEMORYLESS_NODES /* * Return node id of node used for "local" allocations. --=20 2.55.0