From nobody Thu Sep 24 14:26:46 2026 Received: from mail-qk2-f42.google.com (mail-qk2-f42.google.com [74.125.230.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3485369D56 for ; Wed, 23 Sep 2026 02:29:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.230.234 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790130550; cv=none; b=Qp2hy2oah53Arz+h9xRMm7Ym5JYZvFKEx0KbGMCoAaboulp7jtmQwqNs7Dq487ATxqb0hq+Oh9o/ZfoqfTKMAx+VTP1g7vk8BlIhStSA4NNTx0aW1K6V8iFu7k1Iw9JqDLZ0N7cuMbOxzA3fN7xp2uAcW64c0eQNKcjOmnushqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790130550; c=relaxed/simple; bh=/AeIosBLYXEmJJMjp1lQbjdRZXC4yVGPyR4w8hqfOfw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cgni0wIvRQDVIMcODoABZ4yQg7L4l6JuXEWchzi/XCOFJwyqhDB5MdKzEuFPE6y73uNZVF1xlXKT53YT0xESECHZ7jmc3KJEd8Yu98VCwRIF7QttuHoJgDVgKUsBhuVqQrrTBIdiZb2OGkPzINpikcVliXxGSL/1wp48tl3EQnk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=VJpWXgUD; arc=none smtp.client-ip=74.125.230.234 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="VJpWXgUD" Received: by mail-qk2-f42.google.com with SMTP id af79cd13be357-93bef17c191so39547485a.2 for ; Tue, 22 Sep 2026 19:29:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790130548; x=1790735348; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=2DTbYesErX7uqQ2UiAVBtVPVMTumuecPrjo9lCd5uig=; b=VJpWXgUD9cl9fmRHcovDtejzVsZ//OoHovKRlTL97Vx17B4KjTnBS11OO2ZCVWMInJ SGJ0JYpjJoj1VUUdswgdM/XZpNhmCdQzA1EQgZfjqHXoE557nFp4cEo5vSz9uBjJtB/4 3I3dvD/som2xRnaDV9VMu+4iEeiNknfKmgYE6qiON4QMOQCD4RY0xnThDgi1cOYcjCPI 7WzBhVPJDTFtjexdE6wkCC9+vSOtdj2A6SyLNP5J2L4lwYwfHeul2jbS4+f0JcfEO5yk Yis3JNifxO8SGjwBOv2k3/bW5JmGy4RvFUIkaIJwFzoaEi0hCps+ySSq5mxo7qxnKZoK bFgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790130548; x=1790735348; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=2DTbYesErX7uqQ2UiAVBtVPVMTumuecPrjo9lCd5uig=; b=25rr7xFzN5Q4uDlrddqTe5cl5QSSeNVl/YfyQ7XFbcuad8QDvcmkL3b1si7AXYs1ws 4ebn8yznDBQmFADN/H9VhzAklP0C5ibCe6nSr75txn3JgzzSrTPBZTKS4X5yVM+25f+f ibWHx6RF201gW6bEVgDMaSNJQvXeZfIWFeynnZXrEXJmoT/XkthDS0KJwfpPjDLeDc3l brATCEssC+qrw2KmmHuIIIyIAoXVhs/B52xZyEps99qXZUPBsalFMRQxa8ZUc+n1Js74 Bt3R6UcRqXdmUWIFjRDW2HBfR6+8gEkNgNFy2Cm7eFkT08DtneVwEyCXgB1Z8UWsHHER geqA== X-Gm-Message-State: AFuF++lEvSS9lkZHNwL3sVHda7gvPWfua/u/wBN/qBAL4Ef3fScQ6Has oh2xEKGyf+hX4toEX+eDyPYpyb8/4CWGdCJRea4E1osIekE2nOqsja5AVYpRlnv6VD8= X-Gm-Gg: AYBFou3J67ITXnNJUzAVWXx8OhEQrbUHuvdSI+2euS+VnoKU7cZh8e0wRk+gOOf2eEl tjfLHQtA6OjAF+aCr4lcGxKzrcwUTwoSD/oaQC7Z3y5eA4pB66d6XqQieUxj1m59TIMCMmMP5m7 f3iLalqQcIEgr8tuvYMn61wT2z6t8ACBg/fXYfPMYHvzjRg/XK0HYHhXmj4zOCxiL0RQlvXqkw4 Y6rb9U74Hn9gJIVItzfwEH1fbz2ORDIsFHZhsrmht9ySSfqLQH955fTl3D7i81qVnzZmKZ1cdR1 NhMHmW8AZhdfgKdWT9kkIHX1PrKrnTjCz8ApbA80aVFG56ITMrjbA3C+shSV8lhM9bs1JpSjb9E UTA1//wgBBASEDGmwt6JPdjTJ7/W81LSil91ZZUjmao93nroHu96VuoM1IuTGS8W4+5zYPVJZoO 17vDacgm4BtJIAtYsmO1fPgGa4P+yNHWurFAEKOYiAPttRU8Ayq0RocrshoKDJmwZqEe83Hlj15 s5iJp0/nPjqsfMZyiczc7+IjH/xqicQS/6QANkwxXikBsffAnLcl7bO1RLv X-Received: by 2002:a05:620a:3902:b0:93a:2221:5460 with SMTP id af79cd13be357-93c251af18dmr218974285a.40.1790130547657; Tue, 22 Sep 2026 19:29:07 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2482302fsm122626785a.12.2026.09.22.19.29.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 19:29:07 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org, liam@infradead.org, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, brendan.jackman@linux.dev, hannes@cmpxchg.org, Balbir Singh , "Zenghui Yu (Huawei)" Subject: [PATCH v3 1/2] mm: refactor find_next_best_node to find_next_best_node_in Date: Tue, 22 Sep 2026 22:29:01 -0400 Message-ID: <20260923022902.2433614-2-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923022902.2433614-1-gourry@gourry.net> References: <20260923022902.2433614-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" find_next_best_node() picks the next-closest node for a fallback list from the full N_MEMORY set. Refactor it into find_next_best_node_in(), which takes an explicit candidates nodemask. This enables building fallback lists with non-N_MEMORY candidates. No functional change: every caller still selects from N_MEMORY. Reviewed-by: Vlastimil Babka (SUSE) Acked-by: Balbir Singh Reviewed-by: Zi Yan Reviewed-by: Zenghui Yu (Huawei) Signed-off-by: Gregory Price --- mm/internal.h | 6 ++++-- mm/memory-tiers.c | 7 ++++--- mm/page_alloc.c | 13 ++++++++----- 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index cbdec2193fad..0434dfcfc36f 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1133,7 +1133,8 @@ extern int node_reclaim_mode; =20 extern unsigned long node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned int order); -extern int find_next_best_node(int node, nodemask_t *used_node_mask); +int find_next_best_node_in(int node, nodemask_t *used_node_mask, + const nodemask_t *candidates); #else #define node_reclaim_mode 0 =20 @@ -1142,7 +1143,8 @@ static inline unsigned long node_reclaim(struct pglis= t_data *pgdat, { return 0; } -static inline int find_next_best_node(int node, nodemask_t *used_node_mask) +static inline int find_next_best_node_in(int node, nodemask_t *used_node_m= ask, + const nodemask_t *candidates) { return NUMA_NO_NODE; } diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 54851d8a195b..25e121851b58 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -370,7 +370,7 @@ int next_demotion_node(int node, const nodemask_t *allo= wed_mask) * closest demotion target. */ nodes_complement(mask, *allowed_mask); - return find_next_best_node(node, &mask); + return find_next_best_node_in(node, &mask, &node_states[N_MEMORY]); } =20 static void disable_all_demotion_targets(void) @@ -450,7 +450,7 @@ static void establish_demotion_targets(void) memtier =3D list_next_entry(memtier, list); tier_nodes =3D get_memtier_nodemask(memtier); /* - * find_next_best_node, use 'used' nodemask as a skip list. + * find_next_best_node_in, use 'used' nodemask as a skip list. * Add all memory nodes except the selected memory tier * nodelist to skip list so that we find the best node from the * memtier nodelist. @@ -463,7 +463,8 @@ static void establish_demotion_targets(void) * in the preferred mask when allocating pages during demotion. */ do { - target =3D find_next_best_node(node, &tier_nodes); + target =3D find_next_best_node_in(node, &tier_nodes, + &node_states[N_MEMORY]); if (target =3D=3D NUMA_NO_NODE) break; =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 090419c70173..7f706e3d7e50 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5802,9 +5802,10 @@ static int numa_zonelist_order_handler(const struct = ctl_table *table, int write, static int node_load[MAX_NUMNODES]; =20 /** - * find_next_best_node - find the next node that should appear in a given = node's fallback list + * find_next_best_node_in - find the next node that should appear in a giv= en node's fallback list * @node: node whose fallback list we're appending * @used_node_mask: nodemask_t of already used nodes + * @candidates: nodemask_t of nodes eligible for selection * * We use a number of factors to determine which is the next node that sho= uld * appear on a given node's fallback list. The node should not have appea= red @@ -5816,7 +5817,8 @@ static int node_load[MAX_NUMNODES]; * * Return: node id of the found node or %NUMA_NO_NODE if no node is found. */ -int find_next_best_node(int node, nodemask_t *used_node_mask) +int find_next_best_node_in(int node, nodemask_t *used_node_mask, + const nodemask_t *candidates) { int n, val; int min_val =3D INT_MAX; @@ -5826,12 +5828,12 @@ int find_next_best_node(int node, nodemask_t *used_= node_mask) * Use the local node if we haven't already, but for memoryless local * node, we should skip it and fall back to other nodes. */ - if (!node_isset(node, *used_node_mask) && node_state(node, N_MEMORY)) { + if (!node_isset(node, *used_node_mask) && node_isset(node, *candidates)) { node_set(node, *used_node_mask); return node; } =20 - for_each_node_state(n, N_MEMORY) { + for_each_node_mask(n, *candidates) { =20 /* Don't want a node to appear more than once */ if (node_isset(n, *used_node_mask)) @@ -5916,7 +5918,8 @@ static void build_zonelists(pg_data_t *pgdat) prev_node =3D local_node; =20 memset(node_order, 0, sizeof(node_order)); - while ((node =3D find_next_best_node(local_node, &used_mask)) >=3D 0) { + while ((node =3D find_next_best_node_in(local_node, &used_mask, + &node_states[N_MEMORY])) >=3D 0) { /* * We don't want to pressure a particular node. * So adding penalty to the first node in same --=20 2.55.0 From nobody Thu Sep 24 14:26:46 2026 Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F19F037F339 for ; Wed, 23 Sep 2026 02:29:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790130552; cv=none; b=A5xsa3RZqajwZhOKSYdt9dWkLl0QQWxbFUYYdViesOo2zbz0SnbMvXJ3sNAU/1NinJZVpf2yDUu7L7GaZwxpfXpIiLTh5fz1G8n2MXRFntpp3C3wBT4IR9c3IartOhsemDo5Y3YDN5xumdTV7CQfK0+uaYCQwt3xlmmlTMqcYmg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790130552; c=relaxed/simple; bh=5BejYXRDjaXsEFSHm/4GOXt9VbHt5qODsu2loC2kU1k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XUpeo7d1PXWyIBckWSttX/PipI++v7kOHw0lmSRCAvNMY1pSgDWjEcGev20WcS3MoQMDU5+Xc+w7/+ftvYNFlwADZ49XuGUP3Bi/idHc5XVGg9b+J3B46Xe1o2LFOu3yzmz+QV5m2gwPzcWrTs3Ffz4PqA8u/E6TfiRJ8gUpiCc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=Kxa7iJHw; arc=none smtp.client-ip=209.85.222.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="Kxa7iJHw" Received: by mail-qk1-f177.google.com with SMTP id af79cd13be357-93a0fb2f9efso25673185a.1 for ; Tue, 22 Sep 2026 19:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790130550; x=1790735350; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=yWWHCDlvz4uiBBqUhZH0RsVaOlDUvVDcH0PiwNo8xZc=; b=Kxa7iJHwBqgaYl0DKV0tzCnmeV9uA+nDixUuL9QCmibs20WZQnoXp9DDY3DJvDoh0c HmsKGF4NRkOBYuBIOsUNDzMuYRHkSmr18jeLVxmk/XcQpoenwTt+v5u7WKKIDC9jRPON tEDCWzoAB7zFAWTRmzWJsZVcRoJDapQz8DWOReWQvsKbMPNfiykl7b5SgF9RA9VACaoi YZii+3kWlz7ciak5kc8cyGOYo3BgNKyz37zDZIzjpLPXNDxcqMZdsQkmG2fyk4JUi9UT 3/fgpvu9oKnokx54TYPetD0IAwuNU4nPZUse4sISfjYDev0+waZ+Fh6UoUv9wzaALBwp cpFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790130550; x=1790735350; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=yWWHCDlvz4uiBBqUhZH0RsVaOlDUvVDcH0PiwNo8xZc=; b=MAp+ZyGGNlBTfGoLjLJcaQqQCoxz7H0g5fDfag7vXpXnj+cQlOExj1TqwAid6HqIz7 PFz/t3EFVL1b3CQPjPR2hBLEurwxvu5xicQlW++jUwYO6xphrgDlblwvJdYJ+ePuXTUG f4+9+ByQ2PAAKbJ/yg1fgXeXe/WM3f9TtozOKHk8eKQdq6iFpFOtOlHqmZNfXnoAlEev XT/xhyk9H7muEpl3F2Ep+K8dSEfaPMy51ytNuqAc0UjEpaFtgGpRIQ8l8vTfZVxYt99/ cdlyCxrNJvL9qIuRhuSgx/Z8Cr+vRcSw/MBb48dvoN5Rq01JOLy5DsWBLkRXdyVInHj0 8xpQ== X-Gm-Message-State: AFuF++ldMWSP2ngRZUxJe7ANSbCx7dKjl8Z+vjkw8fccVRNgMed12A5Y sk0b7DpB6aU2DfbKYVOpGsgeDHRUQwQ+K+toT8nUIA4YHVMbAD7sV8+paSRRQtEulNI= X-Gm-Gg: AYBFou0qTdFEhZlKgc6Xs3Itd1NP44yg0/HwJzJ6RKc/p9B3QyMsUJgk2NkRsBlrRBe ImxH9SXu3ZZzgZoT4YPOn03HqRtZRese6EAKLkPh1KJ9hvBa+DTwJdPvOeRp0UjnC2C2fHpUqG+ i0F2DE+czpubxKeB2kZxUmEV9eP8SPyVxXRYUF/t3rnse4qQF4L1pWQTc+GxV6zu3EPUD5YGxu1 unXIaNLNGuRFwf/p4s5/iH+2vf9K4sepRLg3VfU/MObeFp5AMkWNxJwNsEsLcz7Jw0TwVpeDIfz ReP4X9sPrCtGgALUpa7eEPoyObMLD/cr0vp1ozpvsOO+vfE/Muo8Q0pk6V3qv5c/ubXZ8zfi7d7 9q3ahpUSPz1NFPj6zwVfmTVL4R+s7yQso2xLOBS2xgOYQDR4Tzs5GDkLuDNRicJfUYUvaenUeCn uvPDgTesERkGgdATdEahO7rgDw3wKA7XNWwBDiiuIjA0scWqMDSIWJNCoBVvx7iXg1O49TGnckY iwLX6dELqMp581UYEFVRub5AznWh7B5J0YF/wPQLK8zjJ/2XRv95693qSuG X-Received: by 2002:a05:620a:4112:b0:939:fbe0:9abc with SMTP id af79cd13be357-93c17e2edd3mr739670285a.9.1790130549781; Tue, 22 Sep 2026 19:29:09 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2482302fsm122626785a.12.2026.09.22.19.29.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 19:29:09 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org, liam@infradead.org, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, brendan.jackman@linux.dev, hannes@cmpxchg.org, "Zenghui Yu (Huawei)" Subject: [PATCH v3 2/2] mm/page_alloc: refactor build_node_zonelist() out of build_zonelists() Date: Tue, 22 Sep 2026 22:29:02 -0400 Message-ID: <20260923022902.2433614-3-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923022902.2433614-1-gourry@gourry.net> References: <20260923022902.2433614-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Extract per-node fallback-list construction into build_node_zonelist(). Build each selected node directly into the destination zonelist so no intermediate node_order array or node count is needed. Print the fallback order as each node is added. This lets us build new zonelists from candidate nodemasks instead of just the default N_MEMORY node state list. Add a zlidx argument so callers explicitly select the destination zonelist. The existing caller continues to use ZONELIST_FALLBACK. No functional change: build_zonelists() builds and prints the same FALLBACK list over N_MEMORY with node_load updates as before. Reviewed-by: Zi Yan Reviewed-by: Vlastimil Babka (SUSE) Reviewed-by: Zenghui Yu (Huawei) Signed-off-by: Gregory Price --- mm/page_alloc.c | 63 ++++++++++++++++++------------------------------- 1 file changed, 23 insertions(+), 40 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7f706e3d7e50..07e03c7c7771 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5866,31 +5866,6 @@ int find_next_best_node_in(int node, nodemask_t *use= d_node_mask, } =20 =20 -/* - * Build zonelists ordered by node and zones within node. - * This results in maximum locality--normal zone overflows into local - * DMA zone, if any--but risks exhausting DMA zone. - */ -static void build_zonelists_in_node_order(pg_data_t *pgdat, int *node_orde= r, - unsigned nr_nodes) -{ - struct zoneref *zonerefs; - int i; - - zonerefs =3D pgdat->node_zonelists[ZONELIST_FALLBACK]._zonerefs; - - for (i =3D 0; i < nr_nodes; i++) { - int nr_zones; - - pg_data_t *node =3D NODE_DATA(node_order[i]); - - nr_zones =3D build_zonerefs_node(node, zonerefs); - zonerefs +=3D nr_zones; - } - zonerefs->zone =3D NULL; - zonerefs->zone_idx =3D 0; -} - /* * Build __GFP_THISNODE zonelists */ @@ -5906,20 +5881,24 @@ static void build_thisnode_zonelists(pg_data_t *pgd= at) zonerefs->zone_idx =3D 0; } =20 -static void build_zonelists(pg_data_t *pgdat) +/* + * Build one zonelist ordered by node and zones within node. This results = in + * maximum locality--normal zone overflows into local DMA zone, if any--but + * risks exhausting DMA zone. + */ +static void build_node_zonelist(pg_data_t *pgdat, const nodemask_t *candid= ates, + int zlidx) { - static int node_order[MAX_NUMNODES]; - int node, nr_nodes =3D 0; + struct zoneref *zonerefs =3D pgdat->node_zonelists[zlidx]._zonerefs; nodemask_t used_mask =3D NODE_MASK_NONE; - int local_node, prev_node; + int local_node =3D pgdat->node_id; + int prev_node =3D local_node; + int node; =20 - /* NUMA-aware ordering of nodes */ - local_node =3D pgdat->node_id; - prev_node =3D local_node; + pr_info("Fallback order for Node %d: ", local_node); =20 - memset(node_order, 0, sizeof(node_order)); while ((node =3D find_next_best_node_in(local_node, &used_mask, - &node_states[N_MEMORY])) >=3D 0) { + candidates)) >=3D 0) { /* * We don't want to pressure a particular node. * So adding penalty to the first node in same @@ -5929,18 +5908,22 @@ static void build_zonelists(pg_data_t *pgdat) node_distance(local_node, prev_node)) node_load[node] +=3D 1; =20 - node_order[nr_nodes++] =3D node; + zonerefs +=3D build_zonerefs_node(NODE_DATA(node), zonerefs); + pr_cont("%d ", node); prev_node =3D node; } =20 - build_zonelists_in_node_order(pgdat, node_order, nr_nodes); - build_thisnode_zonelists(pgdat); - pr_info("Fallback order for Node %d: ", local_node); - for (node =3D 0; node < nr_nodes; node++) - pr_cont("%d ", node_order[node]); + zonerefs->zone =3D NULL; + zonerefs->zone_idx =3D 0; pr_cont("\n"); } =20 +static void build_zonelists(pg_data_t *pgdat) +{ + build_node_zonelist(pgdat, &node_states[N_MEMORY], ZONELIST_FALLBACK); + build_thisnode_zonelists(pgdat); +} + #ifdef CONFIG_HAVE_MEMORYLESS_NODES /* * Return node id of node used for "local" allocations. --=20 2.55.0