From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 600A02F23 for ; Thu, 4 Jul 2024 04:31:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067514; cv=none; b=oKnhvx9wM34Ll97px3+8mhqSovx4PESpamhCA9JT/KvXhJ2kEiT+ZyqK6InqMsjBmpEWO2xnNp9k9p7Z7Hqn2IZ+ZVbiTNR29GaAh6qFK4plpZ1hD6MNcKhTamxdocnrJeyB36OPVBZiY8gocSkYQzo/T7rtF+BVcLiFKsEi/yo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067514; c=relaxed/simple; bh=KFdArfnfqEUxgBAxz6IkehnNwcNlGS1yjQG75QAse0k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X7xfrlhe1+QxkdP7oIcGypjJTbc9j8N+vysPa1Ug0N022UVLt/cvMu2Nv6dw9eHgu2bENv/bTW88W2t/1SAmQBBOfclrSNc2sjmJ4OWX65aZbdfuFlgZSQqGo+9UUWKsYV/zszRe5urDcL/bSw0KhAbwdFNcmgTL2q/7saS5cKc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=TnzHIbUf; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=bF+fGAUR; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=TnzHIbUf; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=bF+fGAUR; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="TnzHIbUf"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="bF+fGAUR"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="TnzHIbUf"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="bF+fGAUR" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 97BAB1FCF7; Thu, 4 Jul 2024 04:31:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=TnzHIbUfH5DwSZglMNw4Qi9u/goipKZZo3b5tCXWSNzXHcGWuTfIuyoCPpBaZ7EQWJZ4xP xc8s51VaseURhWTMdpAiOLKejN3kTYWph/K3arXExfBqJilkikxqtS1oEa7zsCdFYmuN0K Qf9du4CansKgid+LQOtm3331g59SMt8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=bF+fGAURxq2QVQDGIfnETCWk/28DcKQ+0DPeiRUpZJTGyNdki5AfAv3X8A2SjYr3HMCp0b 41nDiQe78mUbZ7Aw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=TnzHIbUfH5DwSZglMNw4Qi9u/goipKZZo3b5tCXWSNzXHcGWuTfIuyoCPpBaZ7EQWJZ4xP xc8s51VaseURhWTMdpAiOLKejN3kTYWph/K3arXExfBqJilkikxqtS1oEa7zsCdFYmuN0K Qf9du4CansKgid+LQOtm3331g59SMt8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067510; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bJqkPqmkpON5Zxkai/QS1QclW4CGF+6WU8c6arXYcis=; b=bF+fGAURxq2QVQDGIfnETCWk/28DcKQ+0DPeiRUpZJTGyNdki5AfAv3X8A2SjYr3HMCp0b 41nDiQe78mUbZ7Aw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7C25713A82; Thu, 4 Jul 2024 04:31:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YDqrGbUlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:49 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 01/45] arch/x86: Drop own definition of pgd,p4d_leaf Date: Thu, 4 Jul 2024 06:30:48 +0200 Message-ID: <20240704043132.28501-2-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_TWELVE(0.00)[12]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" We provide generic definitions of pXd_leaf in pgtable.h when the arch do not define their own, where the generic pXd_leaf always return false. Although x86 defines {pgd,p4d}_leaf, they end up being a no-op, so drop them and make them fallback to the generic one. Signed-off-by: Oscar Salvador --- arch/x86/include/asm/pgtable.h | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 65b8e5bb902c..772f778bac06 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -252,13 +252,6 @@ static inline unsigned long pgd_pfn(pgd_t pgd) return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT; } =20 -#define p4d_leaf p4d_leaf -static inline bool p4d_leaf(p4d_t p4d) -{ - /* No 512 GiB pages yet */ - return 0; -} - #define pte_page(pte) pfn_to_page(pte_pfn(pte)) =20 #define pmd_leaf pmd_leaf @@ -1396,9 +1389,6 @@ static inline bool pgdp_maps_userspace(void *__ptr) return (((ptr & ~PAGE_MASK) / sizeof(pgd_t)) < PGD_KERNEL_START); } =20 -#define pgd_leaf pgd_leaf -static inline bool pgd_leaf(pgd_t pgd) { return false; } - #ifdef CONFIG_MITIGATION_PAGE_TABLE_ISOLATION /* * All top-level MITIGATION_PAGE_TABLE_ISOLATION page tables are order-1 p= ages --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D0A7E576 for ; Thu, 4 Jul 2024 04:31:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067515; cv=none; b=PyDQOYefUq0+U4V/aEGntMBlr7gosVxu78kjDcar7/g3l18VDIuelPWRzMmPGgeVzwkZWThhy+Igici1VLjM73aogebEc8mTframT+G3z32et9Kdkh8m5pLV2vnUqd5forOuWdkrRXcIgFZDz/RFVx8XsAPtSQwFOsfMtNqbgt8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067515; c=relaxed/simple; bh=X+DjluLDjuDVWuMonHQsdMe5Vi1xaH24OGLhrKHXLUM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vygf2ViCZRjLW2emMODwAxSPz2msa48DFEYUI9KECT52lg60vuZ0uMvszHjBFbRI1TDuIlG37r0HcUosfEkuqG1LPXKt57X5w6YTAx9eirHM4usALR7Ff+qms9DjPveXhzPzMJXqRWCg8ikz6JHdv0xJmr9LlmMOPKvXnxFSOmY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D75C521C0D; Thu, 4 Jul 2024 04:31:51 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B7C9513889; Thu, 4 Jul 2024 04:31:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MGmJKLYlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:50 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 02/45] mm: Add {pmd,pud}_huge_lock helper Date: Thu, 4 Jul 2024 06:30:49 +0200 Message-ID: <20240704043132.28501-3-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: D75C521C0D X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" Deep down hugetlb and thp use the same lock for pud and pmd. Create two helpers that can be directly used by both of them, as they will be used in the generic pagewalkers. Signed-off-by: Oscar Salvador --- include/linux/mm_inline.h | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f4fe593c1400..93e3eb86ef4e 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -9,6 +9,7 @@ #include #include #include +#include =20 /** * folio_is_file_lru - Should the folio be on a file LRU or anon LRU? @@ -590,4 +591,30 @@ static inline bool vma_has_recency(struct vm_area_stru= ct *vma) return true; } =20 +static inline spinlock_t *pmd_huge_lock(pmd_t *pmd, struct vm_area_struct = *vma) +{ + spinlock_t *ptl; + + if (pmd_leaf(*pmd)) { + ptl =3D pmd_lock(vma->vm_mm, pmd); + if (pmd_leaf(*pmd)) + return ptl; + spin_unlock(ptl); + } + return NULL; +} + +static inline spinlock_t *pud_huge_lock(pud_t *pud, struct vm_area_struct = *vma) +{ + spinlock_t *ptl =3D pud_lock(vma->vm_mm, pud); + + if (pud_leaf(*pud)) { + ptl =3D pud_lock(vma->vm_mm, pud); + if (pud_leaf(*pud)) + return ptl; + spin_unlock(ptl); + } + return NULL; +} + #endif --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA2CD2110E for ; Thu, 4 Jul 2024 04:31:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067516; cv=none; b=PtUR4ntNL+w4htnsVO3lh0GGY7c+PHIbEag3816kBkBrvIdku11dOVG3tnksMqGbiKPo6ZhFhkAOYrKYL8Gatb9jJO2nO8ND9dR5JDDaOV2qgAW6EQNzDUU30+1SpHEwCA9EkLmROxBJpVpNDWMkB/WpbiV0J2L3Er3HMeaYtSQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067516; c=relaxed/simple; bh=m9zPXLXTm/mgl/dYIUFDktPz6b3OWubcd5U7cMa8rxw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G+7MxQViVxsi6H9i+fEN1oho/w5dwc4tcid6RhpP7YsB+xjAIJWJ2o96zhj7VnqIg2zmA7ZtXuKaRAxGwaZCb7ig90YTshBgPTHa5vXUuMeCfLMWvEf47Y24JH+CnMf/J9Zn94EF4Uj0WdTwdv9RrhzXKRNd7WuKonBOcQDcpZI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 359EE21180; Thu, 4 Jul 2024 04:31:53 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 044A513889; Thu, 4 Jul 2024 04:31:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4D3mN7clhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:51 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 03/45] mm/pagewalk: Move vma_pgtable_walk_begin and vma_pgtable_walk_end upfront Date: Thu, 4 Jul 2024 06:30:50 +0200 Message-ID: <20240704043132.28501-4-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 359EE21180 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" In order to prepare walk_pgd_range for handling hugetlb pages, move the hugetlb vma locking into __walk_page_range. Signed-off-by: Oscar Salvador --- mm/pagewalk.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index ae2f08ce991b..eba705def9a0 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -269,7 +269,6 @@ static int walk_hugetlb_range(unsigned long addr, unsig= ned long end, const struct mm_walk_ops *ops =3D walk->ops; int err =3D 0; =20 - hugetlb_vma_lock_read(vma); do { next =3D hugetlb_entry_end(h, addr, end); pte =3D hugetlb_walk(vma, addr & hmask, sz); @@ -280,7 +279,6 @@ static int walk_hugetlb_range(unsigned long addr, unsig= ned long end, if (err) break; } while (addr =3D next, addr !=3D end); - hugetlb_vma_unlock_read(vma); =20 return err; } @@ -339,11 +337,13 @@ static int __walk_page_range(unsigned long start, uns= igned long end, return err; } =20 + vma_pgtable_walk_begin(vma); if (is_vm_hugetlb_page(vma)) { if (ops->hugetlb_entry) err =3D walk_hugetlb_range(start, end, walk); } else err =3D walk_pgd_range(start, end, walk); + vma_pgtable_walk_end(vma); =20 if (ops->post_vma) ops->post_vma(walk); --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 586853B782 for ; Thu, 4 Jul 2024 04:31:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067517; cv=none; b=B9sZRkurmvs3fI2zmgNJtdOhL0N+n2bZFCo3PZrGwONpDU/t0H69arUC+AB6gH3J5REw3YxK2JYRLjElL9WG9aW9A3XAyJL0/q7MvFMsku/65tXg4JFYU7Bg8J11DLSMVoeu5Yiw0rJA8iH1YK8TRjfJz6oIXMKEv0CfBao5Lv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067517; c=relaxed/simple; bh=vY33weUn+fFr/i0vts8InUHV8Fv/v1C7qSHT8gRsQDI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s+susJ8D2+oFag6kfPSIFbXqida0n0gukmidit13V0CXsnKcIdwok0OIRsFNtGn6/tKEoqy5ZSGYSz0ZYIRl2JJVtc545EeHfiwFApJYJmSmTdW+4KR/61NetBTlS/sGpWlu8R0dVnZ/PprduAfT6APulvUxDOhJX7cengFgSxI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8546721C10; Thu, 4 Jul 2024 04:31:54 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5593A13889; Thu, 4 Jul 2024 04:31:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SMuLELklhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:53 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 04/45] mm/pagewalk: Only call pud_entry when we have a pud leaf Date: Thu, 4 Jul 2024 06:30:51 +0200 Message-ID: <20240704043132.28501-5-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 8546721C10 X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" Check first whether the pud is leaf one before trying to call pud_entry. Signed-off-by: Oscar Salvador --- mm/pagewalk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index eba705def9a0..d93e77411482 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -155,7 +155,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long add= r, unsigned long end, =20 walk->action =3D ACTION_SUBTREE; =20 - if (ops->pud_entry) + if (ops->pud_entry && pud_leaf(*pud)) err =3D ops->pud_entry(pud, addr, next, walk); if (err) break; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69E5C45979 for ; Thu, 4 Jul 2024 04:31:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067518; cv=none; b=InmsHhwaIObXJtDRBN4IDO45QJKRl2cQ1Pszs4xa5PD4jIKEVtqIergWybJ0ES+MgcghY4eRHjzCM4dgng2pI8VQDZ92q4W/R8CaZPIOCB+xA+kM1AfNu4QDVYM4IiK+QAZiAJY6tVsHc7ub2Je9QakSxePXpzZvGC4c/gwpDMI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067518; c=relaxed/simple; bh=c/HemdA2aDxHy0Qr3edOwoMNWIqOMDdImC7HbbxLkp0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h6XPWnz/ewjFFHKnjvXk3rYQBipMc5NousrhSiGISKUVf0H2qnIlTH18Fkhrm3vXwFbw1Gx70u2crLBD8CLz+H5FOG089TvVhKGX9OS+Kq7+wc47DHSWICmPun6kd52uUWi5W0TmohRzLRUar2iNF+ebEFG5oAJ7hLFZRkPNNQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=h+c3YldW; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=frfAxxwF; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=h+c3YldW; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=frfAxxwF; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="h+c3YldW"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="frfAxxwF"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="h+c3YldW"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="frfAxxwF" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BD80C1FCFB; Thu, 4 Jul 2024 04:31:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=h+c3YldWsDCWou0FH1uqAUtnGxvDkmkrw5MQS+eAbm6nc4xHjoC7SiMmO1KgT1ogxMucRW JsZAInX2H9tHjZJ29ry01mlBA/1iNay0rlLOoCB6DCVmON6lUqsmtsj4w0Z5P4bSRXrmdd E1ywbhdn4+EiM/LJbQhJoM39l9zXjNI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=frfAxxwFBvqVUDaANiozJD91Zhfd1mCsF2hzZSdarkYr3nve6JYZaO/PFTJ73t684ex8JF l+vQvnp2Om9NkABQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=h+c3YldWsDCWou0FH1uqAUtnGxvDkmkrw5MQS+eAbm6nc4xHjoC7SiMmO1KgT1ogxMucRW JsZAInX2H9tHjZJ29ry01mlBA/1iNay0rlLOoCB6DCVmON6lUqsmtsj4w0Z5P4bSRXrmdd E1ywbhdn4+EiM/LJbQhJoM39l9zXjNI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067515; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCzlv35dbgXloc5Cmstin2CCiJG29oVOef/hhtOkA+o=; b=frfAxxwFBvqVUDaANiozJD91Zhfd1mCsF2hzZSdarkYr3nve6JYZaO/PFTJ73t684ex8JF l+vQvnp2Om9NkABQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A7EBF13889; Thu, 4 Jul 2024 04:31:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id eHJfJLolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:54 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 05/45] mm/pagewalk: Enable walk_pmd_range to handle cont-pmds Date: Thu, 4 Jul 2024 06:30:52 +0200 Message-ID: <20240704043132.28501-6-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" HugeTLB pages can be cont-pmd mapped, so teach walk_pmd_range to handle those. This will save us some cycles as we do it in one-shot instead of calling in multiple times. Signed-off-by: Oscar Salvador --- include/linux/pgtable.h | 12 ++++++++++++ mm/pagewalk.c | 12 +++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 2a6a3cccfc36..3a7b8751747e 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1914,6 +1914,18 @@ typedef unsigned int pgtbl_mod_mask; #define __pte_leaf_size(x,y) pte_leaf_size(y) #endif =20 +#ifndef pmd_cont +#define pmd_cont(x) false +#endif + +#ifndef CONT_PMD_SIZE +#define CONT_PMD_SIZE 0 +#endif + +#ifndef CONT_PMDS +#define CONT_PMDS 0 +#endif + /* * We always define pmd_pfn for all archs as it's used in lots of generic * code. Now it happens too for pud_pfn (and can happen for larger diff --git a/mm/pagewalk.c b/mm/pagewalk.c index d93e77411482..a9c36f9e9820 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -81,11 +81,18 @@ static int walk_pmd_range(pud_t *pud, unsigned long add= r, unsigned long end, const struct mm_walk_ops *ops =3D walk->ops; int err =3D 0; int depth =3D real_depth(3); + int cont_pmds; =20 pmd =3D pmd_offset(pud, addr); do { again: - next =3D pmd_addr_end(addr, end); + if (pmd_cont(*pmd)) { + cont_pmds =3D CONT_PMDS; + next =3D pmd_cont_addr_end(addr, end); + } else { + cont_pmds =3D 1; + next =3D pmd_addr_end(addr, end); + } if (pmd_none(*pmd)) { if (ops->pte_hole) err =3D ops->pte_hole(addr, next, depth, walk); @@ -126,8 +133,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long add= r, unsigned long end, =20 if (walk->action =3D=3D ACTION_AGAIN) goto again; - - } while (pmd++, addr =3D next, addr !=3D end); + } while (pmd +=3D cont_pmds, addr =3D next, addr !=3D end); =20 return err; } --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6C9949650 for ; Thu, 4 Jul 2024 04:31:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067520; cv=none; b=juPMAvOf5C6mRHnN1tLR59KrKcXFNA0nsi4IT62IYwhbmTDTi0KMAXjImYQy35+QucBgK18UyChBxzGYDb4eGi2dv0S1YO3ts9CgZs3WVi2PAl6zCgKrmI06fWbuTIdPUMKWuX1AKQ4e63yR++h1pLcivsslEDnvTRP+12ZBicE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067520; c=relaxed/simple; bh=kngb8nlng11G16IWBCLNBURCZpdeETcwn2HiGHKXpo4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hltFak9xqg7ZDdmomn02IJnfU5Sdpthl95mYLKhViy8m1bencEzfmlOz/eISmD20s0t0TfepQTAXZPtiasKsZQ7CUrkFPGcxOlYUMbuMu5lKv4PyT33XjfbiG3sDHdUrx8CTSUfBl7y3y9jpfcXodtakqgQ/DgRYPMcXBrwyxs4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=JZiVO/eg; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=yNbA1i/f; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=JZiVO/eg; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=yNbA1i/f; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="JZiVO/eg"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="yNbA1i/f"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="JZiVO/eg"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="yNbA1i/f" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0726C1FCF9; Thu, 4 Jul 2024 04:31:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=JZiVO/egdO1NVZ9h89++ifx1zQVaAaRFVXTNxdvP7+Eqp6m0SFCnQDqtDBNPLeRwhOr/3k RIAR/120aKOEkaNyNgfpVNaziAWXWsQJTL2b+KTKfAdROuN3yyhKk9oEtfYU7aWFSE8j9d IVYd41OVz8gn1ZHBdLdnAJCiolhzEpA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=yNbA1i/f3g+Uk1+/g4XzzNXwTBw31YbefcoyfvVe9XSdiq9mU3Rgm3u5T2CICi+Xi0VqWq O+llDAuqdN+YiJDA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=JZiVO/egdO1NVZ9h89++ifx1zQVaAaRFVXTNxdvP7+Eqp6m0SFCnQDqtDBNPLeRwhOr/3k RIAR/120aKOEkaNyNgfpVNaziAWXWsQJTL2b+KTKfAdROuN3yyhKk9oEtfYU7aWFSE8j9d IVYd41OVz8gn1ZHBdLdnAJCiolhzEpA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067517; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R/rRNjZ7G0L51JRh0RrYbiNiKS2AE7CnLW4VygCa50U=; b=yNbA1i/f3g+Uk1+/g4XzzNXwTBw31YbefcoyfvVe9XSdiq9mU3Rgm3u5T2CICi+Xi0VqWq O+llDAuqdN+YiJDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DE2E513A82; Thu, 4 Jul 2024 04:31:55 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WPQSMrslhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:55 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 06/45] mm/pagewalk: Do not try to split non-thp pud or pmd leafs Date: Thu, 4 Jul 2024 06:30:53 +0200 Message-ID: <20240704043132.28501-7-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Hugetlb pages will be handled in the generic path, so do not try to split pud or pmd leafs if they are not thp. Signed-off-by: Oscar Salvador --- mm/pagewalk.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index a9c36f9e9820..78d45f1450aa 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -119,7 +119,8 @@ static int walk_pmd_range(pud_t *pud, unsigned long add= r, unsigned long end, * Check this here so we only break down trans_huge * pages when we _need_ to */ - if ((!walk->vma && (pmd_leaf(*pmd) || !pmd_present(*pmd))) || + if (((!walk->vma || is_vm_hugetlb_page(walk->vma)) && + (pmd_leaf(*pmd) || !pmd_present(*pmd))) || walk->action =3D=3D ACTION_CONTINUE || !(ops->pte_entry)) continue; @@ -169,7 +170,8 @@ static int walk_pud_range(p4d_t *p4d, unsigned long add= r, unsigned long end, if (walk->action =3D=3D ACTION_AGAIN) goto again; =20 - if ((!walk->vma && (pud_leaf(*pud) || !pud_present(*pud))) || + if (((!walk->vma || is_vm_hugetlb_page(walk->vma)) && + (pud_leaf(*pud) || !pud_present(*pud))) || walk->action =3D=3D ACTION_CONTINUE || !(ops->pmd_entry || ops->pte_entry)) continue; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8B5E3B782 for ; Thu, 4 Jul 2024 04:31:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067521; cv=none; b=Qql6nqiR2AtuY2bNf9Oh0pX7ZOeWL1crdfRiQqVZuVFpcndb0snUK11IvakKb8toulTU6D1Of8Rq+6KsW2zYf6Bza7HEprzkIOBHXsNMrysgpeMuCQMmrU2Y+dX+9f+VPsw7KAVZHF8z+fbIwFD6OpzylXM3P5KoE1krJt5jAd8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067521; c=relaxed/simple; bh=2G6muL+WZqZUb4vEJ8WHsrXsy6VoC9mI/aYzlVAX5uU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=b5LCjTkRK4sSHavt3vyZbrOKehXO5MJehq5VCySyKxg4D6EaZH8CJ2J06JDjepVIUB62SmIVR/n+ufr+AS3u2s9TkJr13lmwQM18H8DTGd5cdQ93K5BKehOK/8+5us8UtE9yOUldCP++PJ3g1KifxNuM3hT+dXapxiSg3ofFBnY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 3F26421C12; Thu, 4 Jul 2024 04:31:58 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 270FD13889; Thu, 4 Jul 2024 04:31:57 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0IRfBb0lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:57 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 07/45] arch/s390: Enable __s390_enable_skey_pmd to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:30:54 +0200 Message-ID: <20240704043132.28501-8-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 3F26421C12 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" __s390_enable_skey_pmd does nothing for THP, but will handle pmd-leafs hugetlb vmas from now onwards, so teach it how to handle those. Signed-off-by: Oscar Salvador --- arch/s390/mm/gmap.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/s390/mm/gmap.c b/arch/s390/mm/gmap.c index 474a25ca5c48..e1d098dc7f07 100644 --- a/arch/s390/mm/gmap.c +++ b/arch/s390/mm/gmap.c @@ -2723,6 +2723,20 @@ static int __s390_enable_skey_pte(pte_t *pte, unsign= ed long addr, static int __s390_enable_skey_pmd(pmd_t *pmd, unsigned long addr, unsigned long next, struct mm_walk *walk) { + if (pmd_leaf(*pmd) && is_vm_hugetlb_page(vma)) + unsigned long start, end; + struct page *page =3D pmd_page(*pmd); + + if (pmd_val(*pmd) & _SEGMENT_ENTRY_INVALID || + !(pmd_val(*pmd) & _SEGMENT_ENTRY_WRITE)) + return 0; + + start =3D pmd_val(*pmd) & HPAGE_MASK; + end =3D start + HPAGE_SIZE; + __storage_key_init_range(start, end); + set_bit(PG_arch_1, &page->flags); + } + cond_resched(); return 0; } --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3770F6DD0D for ; Thu, 4 Jul 2024 04:32:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067522; cv=none; b=olfN9O35vIM/rVB6kn2ZqCL7RW/aubssWC0PfdLrhuqeSceNnubW1VE2N6XOJlK3Fv931dg85zAaaTVGDuRrAMfSUnMV/C2RNifUZsG5U3t7U7ceqkh8pqiAczqMo+4IW5++PEXm0wJkuIHdwwWfWzBAHs+loPNdz2/0ASMRcw0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067522; c=relaxed/simple; bh=y25WqAsEYH7fwKRx+uFvYqGe8nKyVWi2BDSC44TtPHo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vl5JI8jDfBqTtZiCTWwfgMXLPJSFQ4kSGBv4vJrsRQROGdzsAPF7VZmCYs3Qh/BSvlgiLrCKtCaT7kiuCPyig3kCV5cXukDk5h9pQddTiYZPXsIBcyyn8tuYDdyBHaldZFYCKp4AFE6Z2FXlzWdpjRFBEVJC4MaUgOF1sB8X1b0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 824D71FCF7; Thu, 4 Jul 2024 04:31:59 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5F5B113889; Thu, 4 Jul 2024 04:31:58 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AKLwEr4lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:58 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 08/45] fs/proc: Enable smaps_pmd_entry to handle PMD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:30:55 +0200 Message-ID: <20240704043132.28501-9-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 824D71FCF7 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach smaps_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 775a2e8d600c..78d84d7e2353 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -804,6 +804,18 @@ static void smaps_pte_hole_lookup(unsigned long addr, = struct mm_walk *walk) #endif } =20 +#ifdef CONFIG_HUGETLB_PAGE +static void mss_hugetlb_update(struct mem_size_stats *mss, struct folio *f= olio, + struct vm_area_struct *vma, pte_t *pte) +{ + if (folio_likely_mapped_shared(folio) || + hugetlb_pmd_shared(pte)) + mss->shared_hugetlb +=3D huge_page_size(hstate_vma(vma)); + else + mss->private_hugetlb +=3D huge_page_size(hstate_vma(vma)); +} +#endif + static void smaps_pte_entry(pte_t *pte, unsigned long addr, struct mm_walk *walk) { @@ -851,12 +863,13 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, smaps_account(mss, page, false, young, dirty, locked, present); } =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, struct mm_walk *walk) { struct mem_size_stats *mss =3D walk->private; struct vm_area_struct *vma =3D walk->vma; + bool hugetlb =3D is_vm_hugetlb_page(vma); bool locked =3D !!(vma->vm_flags & VM_LOCKED); struct page *page =3D NULL; bool present =3D false; @@ -865,7 +878,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long a= ddr, if (pmd_present(*pmd)) { page =3D vm_normal_page_pmd(vma, addr, *pmd); present =3D true; - } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) { + } else if (is_swap_pmd(*pmd)) { swp_entry_t entry =3D pmd_to_swp_entry(*pmd); =20 if (is_pfn_swap_entry(entry)) @@ -874,6 +887,12 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long = addr, if (IS_ERR_OR_NULL(page)) return; folio =3D page_folio(page); + + if (hugetlb) { + mss_hugetlb_update(mss, folio, vma, (pte_t *)pmd); + return; + } + if (folio_test_anon(folio)) mss->anonymous_thp +=3D HPAGE_PMD_SIZE; else if (folio_test_swapbacked(folio)) @@ -900,7 +919,7 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long ad= dr, unsigned long end, pte_t *pte; spinlock_t *ptl; =20 - ptl =3D pmd_trans_huge_lock(pmd, vma); + ptl =3D pmd_huge_lock(pmd, vma); if (ptl) { smaps_pmd_entry(pmd, addr, walk); spin_unlock(ptl); --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D2D3757E7 for ; Thu, 4 Jul 2024 04:32:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067524; cv=none; b=sXQYLHBqi7tyGBKd15heIDlV19aMZfhJojGMq7GbooxH7yDy5JPLIF0Zb45QD2y/y3tJIo2w01kf0qRxpPIF5jvuQrzv6TDe6jFxwPyytyqjfa/QBQYoQteeWTpXfZgxVbsKZYaNMwm/nIf7U4URzEKRFs1rQej/WFTucQg6Hu4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067524; c=relaxed/simple; bh=cjk/JLRgvQpbw0r3DIONPGQ7F/AxQIIt+ehoTwPy9T0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=isv4pxPuzOnVf5aU00UYcXPLxvLngiulGAXE8MD+O6M+lQSoSlbop9AOoYkl/O5ljRJ/X8vMXLujr9UpueoyEgpWVjqnrm7CGyL98LtpjgCOmjcDJTBD50I9UB900UnyD8sY2B9Z0YEVJ4nygUmslg8f0d8rsrJ3u8kR0TJxw24= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=mvxA8c0w; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=gtiiSyL8; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=mvxA8c0w; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=gtiiSyL8; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="mvxA8c0w"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="gtiiSyL8"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="mvxA8c0w"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="gtiiSyL8" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D4A8C1FCF8; Thu, 4 Jul 2024 04:32:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=mvxA8c0wNNkkavEyFQzOp2mf8378ze4FIH4m4ZfgOZRJ7zpzcoSPW950dQZsa3S/CK+ttv iZlMjg840j6JYRueSYvhI7YBduf5ZmQglN6zSGHvYGjJBa7Jy2q7URNYkz9x4o2JnliPCw zchyR+/MljAZfrr+nxrpTvg2ZDweQk4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=gtiiSyL8IoxKC27iK+u4Mo4MByRQvcwll4qDeKH5vMoKVQYazRAVgxQtYJlScwNErbS0UE MtZSHFG7y/+A7DCQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=mvxA8c0wNNkkavEyFQzOp2mf8378ze4FIH4m4ZfgOZRJ7zpzcoSPW950dQZsa3S/CK+ttv iZlMjg840j6JYRueSYvhI7YBduf5ZmQglN6zSGHvYGjJBa7Jy2q7URNYkz9x4o2JnliPCw zchyR+/MljAZfrr+nxrpTvg2ZDweQk4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067520; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=759iJlxKnytdAlee8rRqvYsRUqK1GzIE0jiP12kfaog=; b=gtiiSyL8IoxKC27iK+u4Mo4MByRQvcwll4qDeKH5vMoKVQYazRAVgxQtYJlScwNErbS0UE MtZSHFG7y/+A7DCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A5A4213889; Thu, 4 Jul 2024 04:31:59 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uIG7I78lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:31:59 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 09/45] mm: Implement pud-version functions for swap and vm_normal_page_pud Date: Thu, 4 Jul 2024 06:30:56 +0200 Message-ID: <20240704043132.28501-10-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" HugeTLB pages will be handled on pud level as well, so we need to implement pud-version of vm_normal_page_pud and swp-pud functions. Signed-off-by: Oscar Salvador --- arch/powerpc/include/asm/book3s/64/pgtable.h | 1 + include/linux/mm.h | 4 ++ include/linux/pgtable.h | 6 +++ include/linux/swapops.h | 15 ++++++ mm/memory.c | 51 ++++++++++++++++++++ 5 files changed, 77 insertions(+) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/in= clude/asm/book3s/64/pgtable.h index 519b1743a0f4..fa4bb8d6356f 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -687,6 +687,7 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newp= rot) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val((pte)) & ~_PAGE_P= TE }) #define __swp_entry_to_pte(x) __pte((x).val | _PAGE_PTE) #define __pmd_to_swp_entry(pmd) (__pte_to_swp_entry(pmd_pte(pmd))) +#define __pud_to_swp_entry(pud) (__pte_to_swp_entry(pud_pte(pud))) #define __swp_entry_to_pmd(x) (pte_pmd(__swp_entry_to_pte(x))) =20 #ifdef CONFIG_MEM_SOFT_DIRTY diff --git a/include/linux/mm.h b/include/linux/mm.h index 5f1075d19600..baade06b159b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2371,6 +2371,10 @@ struct folio *vm_normal_folio_pmd(struct vm_area_str= uct *vma, unsigned long addr, pmd_t pmd); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long = addr, pmd_t pmd); +struct folio *vm_normal_folio_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t pud); +struct page *vm_normal_page_pud(struct vm_area_struct *vma, unsigned long = addr, + pud_t pud); =20 void zap_vma_ptes(struct vm_area_struct *vma, unsigned long address, unsigned long size); diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 3a7b8751747e..a9edeb86b7fe 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1990,4 +1990,10 @@ pgprot_t vm_get_page_prot(unsigned long vm_flags) \ } \ EXPORT_SYMBOL(vm_get_page_prot); =20 +#ifdef CONFIG_HUGETLB_PAGE +#ifndef __pud_to_swp_entry +#define __pud_to_swp_entry(pud) ((swp_entry_t) { pud_val(pud) }) +#endif +#endif + #endif /* _LINUX_PGTABLE_H */ diff --git a/include/linux/swapops.h b/include/linux/swapops.h index cb468e418ea1..182957f0d013 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -126,6 +126,21 @@ static inline int is_swap_pte(pte_t pte) return !pte_none(pte) && !pte_present(pte); } =20 +#ifdef CONFIG_HUGETLB_PAGE +static inline int is_swap_pud(pud_t pud) +{ + return !pud_none(pud) && !pud_present(pud); +} + +static inline swp_entry_t pud_to_swp_entry(pud_t pud) +{ + swp_entry_t arch_entry; + + arch_entry =3D __pud_to_swp_entry(pud); + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); +} +#endif + /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. diff --git a/mm/memory.c b/mm/memory.c index 0a769f34bbb2..90c5dfac35c6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -718,6 +718,57 @@ struct folio *vm_normal_folio_pmd(struct vm_area_struc= t *vma, } #endif =20 +#ifdef CONFIG_HUGETLB_PAGE +struct page *vm_normal_page_pud(struct vm_area_struct *vma, unsigned long = addr, + pud_t pud) +{ + unsigned long pfn =3D pud_pfn(pud); + + /* + * There is no pmd_special() but there may be special pmds, e.g. + * in a direct-access (dax) mapping, so let's just replicate the + * !CONFIG_ARCH_HAS_PTE_SPECIAL case from vm_normal_page() here. + */ + if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) { + if (vma->vm_flags & VM_MIXEDMAP) { + if (!pfn_valid(pfn)) + return NULL; + goto out; + } else { + unsigned long off; + + off =3D (addr - vma->vm_start) >> PAGE_SHIFT; + if (pfn =3D=3D vma->vm_pgoff + off) + return NULL; + if (!is_cow_mapping(vma->vm_flags)) + return NULL; + } + } + + if (pud_devmap(pud)) + return NULL; + if (unlikely(pfn > highest_memmap_pfn)) + return NULL; + + /* + * NOTE! We still have PageReserved() pages in the page tables. + * eg. VDSO mappings can cause them to exist. + */ +out: + return pfn_to_page(pfn); +} + +struct folio *vm_normal_folio_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t pud) +{ + struct page *page =3D vm_normal_page_pud(vma, addr, pud); + + if (page) + return page_folio(page); + return NULL; +} +#endif + static void restore_exclusive_pte(struct vm_area_struct *vma, struct page *page, unsigned long address, pte_t *ptep) --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C23B413698E for ; Thu, 4 Jul 2024 04:32:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067525; cv=none; b=om1ONVPrZeGxzVQ02bVKn1Vpygw40rBb3lNb5Jf4KyVaIj47BNJtntrucT745TK+XUgX0lG2U10Ux+dk3atp+r0U9hECuuhSQNgfhDBJRj0y8thpj5WFHfpzPDZbPP/7XgtGrDdwTKivRw51GJ2VrUhZtwR1DPZMDs/turP6NJU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067525; c=relaxed/simple; bh=7J11hfigJ61V9ZKKSCxcuGfKdKkC/8ZzJ7ODULGmkWc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=e6Jc7eM00EpXrnDqA9xGqMeoulsooj3y9T5cu64BlqPHriKtzyn4KP3TVbFML1H/LXRx/cDddm6sntDSBoRWHCrb78HK4EoPVtMiTt3JrSgcB914XBZs/itni5Z+aHPjoWBGjBirudp83/UBZdorsOetYTEv44ZgMusQVPsNAW4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 26D7421C0D; Thu, 4 Jul 2024 04:32:02 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 01D7013889; Thu, 4 Jul 2024 04:32:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MCZPN8AlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:00 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 10/45] fs/proc: Create smaps_pud_range to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:30:57 +0200 Message-ID: <20240704043132.28501-11-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 26D7421C0D X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create smaps_pud_range in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 78d84d7e2353..3f3460ff03b0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -912,6 +912,40 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long = addr, } #endif =20 +#ifdef CONFIG_HUGETLB_PAGE +static int smaps_pud_range(pud_t *pudp, unsigned long addr, unsigned long = end, + struct mm_walk *walk) +{ + spinlock_t *ptl; + struct folio *folio; + struct vm_area_struct *vma =3D walk->vma; + struct mem_size_stats *mss =3D walk->private; + + ptl =3D pud_huge_lock(pudp, vma); + if (!ptl) + return 0; + + if (pud_present(*pudp)) { + folio =3D vm_normal_folio_pud(vma, addr, *pudp); + } else if (is_swap_pud(*pudp)) { + /* PUD-hugetlbs can have swap entries */ + swp_entry_t swpent =3D pud_to_swp_entry(*pudp); + + if (is_pfn_swap_entry(swpent)) + folio =3D pfn_swap_entry_folio(swpent); + } + + if (folio) + /* Only for now hugetlb pages can end up here */ + mss_hugetlb_update(mss, folio, vma, (pte_t *)pudp); + + spin_unlock(ptl); + return 0; +} +#else +#define smaps_pud_range NULL +#endif + static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long e= nd, struct mm_walk *walk) { @@ -1061,12 +1095,14 @@ static int smaps_hugetlb_range(pte_t *pte, unsigned= long hmask, #endif /* HUGETLB_PAGE */ =20 static const struct mm_walk_ops smaps_walk_ops =3D { + .pud_entry =3D smaps_pud_range, .pmd_entry =3D smaps_pte_range, .hugetlb_entry =3D smaps_hugetlb_range, .walk_lock =3D PGWALK_RDLOCK, }; =20 static const struct mm_walk_ops smaps_shmem_walk_ops =3D { + .pud_entry =3D smaps_pud_range, .pmd_entry =3D smaps_pte_range, .hugetlb_entry =3D smaps_hugetlb_range, .pte_hole =3D smaps_pte_hole, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEACC13958F for ; Thu, 4 Jul 2024 04:32:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067526; cv=none; b=TNONC98AKPJl8TfHs5IQWEXp3y9wAUycqBtv/KV5izKL4+2umTXBqLMnjJDkG+N7I2wIVWxvNpWzOIG5lu4PXeXfXsjq8LkDiE6TNyhxjvUp9A4aupybikjLzQhwZHaoRvnTiLOdVL0i+3huZSqdkWHsoy9uAlqAEToMC64JRk4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067526; c=relaxed/simple; bh=tGhPdhSralDuwi+M9bpSezY1Kw7wQDFoR77boM1eqjA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TBacm6j4tOsKs1m8sn7g8/qr4FvAZ19chNTo23pzg3ftn2v4sdqGYuZ1buLutot4nF1p4xhHt1tfmZ9Fk7CynIFqHMqW/qQiCbquqpq5dezH88uS1+GSYTZFSgm+iLY6POrAekqUWDXi9av6aPuricW32lexCYtHXZY898MXefc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 770A51FCFD; Thu, 4 Jul 2024 04:32:03 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4D9D013889; Thu, 4 Jul 2024 04:32:02 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IPAUDcIlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:02 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 11/45] fs/proc: Enable smaps_pte_entry to handle cont-pte mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:30:58 +0200 Message-ID: <20240704043132.28501-12-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 770A51FCFD X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" HugeTLB pages can be cont-pte mapped, so teach smaps_pte_entry to handle them. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 19 +++++++++++++------ include/linux/pgtable.h | 12 ++++++++++++ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 3f3460ff03b0..4d94b6ce58dd 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -825,6 +825,7 @@ static void smaps_pte_entry(pte_t *pte, unsigned long a= ddr, struct page *page =3D NULL; bool present =3D false, young =3D false, dirty =3D false; pte_t ptent =3D ptep_get(pte); + unsigned long size =3D pte_cont(ptent) ? PAGE_SIZE * CONT_PTES : PAGE_SIZ= E; =20 if (pte_present(ptent)) { page =3D vm_normal_page(vma, addr, ptent); @@ -834,18 +835,18 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, } else if (is_swap_pte(ptent)) { swp_entry_t swpent =3D pte_to_swp_entry(ptent); =20 - if (!non_swap_entry(swpent)) { + if (!is_vm_hugetlb_page(vma) && !non_swap_entry(swpent)) { int mapcount; =20 - mss->swap +=3D PAGE_SIZE; + mss->swap +=3D size; mapcount =3D swp_swapcount(swpent); if (mapcount >=3D 2) { - u64 pss_delta =3D (u64)PAGE_SIZE << PSS_SHIFT; + u64 pss_delta =3D (u64)size << PSS_SHIFT; =20 do_div(pss_delta, mapcount); mss->swap_pss +=3D pss_delta; } else { - mss->swap_pss +=3D (u64)PAGE_SIZE << PSS_SHIFT; + mss->swap_pss +=3D (u64)size << PSS_SHIFT; } } else if (is_pfn_swap_entry(swpent)) { if (is_device_private_entry(swpent)) @@ -860,7 +861,10 @@ static void smaps_pte_entry(pte_t *pte, unsigned long = addr, if (!page) return; =20 - smaps_account(mss, page, false, young, dirty, locked, present); + if (is_vm_hugetlb_page(vma)) + mss_hugetlb_update(mss, page_folio(page), vma, pte); + else + smaps_account(mss, page, false, young, dirty, locked, present); } =20 #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES @@ -952,6 +956,7 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long ad= dr, unsigned long end, struct vm_area_struct *vma =3D walk->vma; pte_t *pte; spinlock_t *ptl; + unsigned long size, cont_ptes; =20 ptl =3D pmd_huge_lock(pmd, vma); if (ptl) { @@ -965,7 +970,9 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long ad= dr, unsigned long end, walk->action =3D ACTION_AGAIN; return 0; } - for (; addr !=3D end; pte++, addr +=3D PAGE_SIZE) + size =3D pte_cont(ptep_get(pte)) ? PAGE_SIZE * CONT_PTES : PAGE_SIZE; + cont_ptes =3D pte_cont(ptep_get(pte)) ? CONT_PTES : 1; + for (; addr !=3D end; pte +=3D cont_ptes, addr +=3D size) smaps_pte_entry(pte, addr, walk); pte_unmap_unlock(pte - 1, ptl); out: diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index a9edeb86b7fe..991137dab87e 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1926,6 +1926,18 @@ typedef unsigned int pgtbl_mod_mask; #define CONT_PMDS 0 #endif =20 +#ifndef pte_cont +#define pte_cont(x) false +#endif + +#ifndef CONT_PTE_SIZE +#define CONT_PTE_SIZE 0 +#endif + +#ifndef CONT_PTES +#define CONT_PTES 0 +#endif + /* * We always define pmd_pfn for all archs as it's used in lots of generic * code. Now it happens too for pud_pfn (and can happen for larger --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92D7613B7BC for ; Thu, 4 Jul 2024 04:32:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067528; cv=none; b=QbhvJQ4RobbrM6mzh1RJ1cH1N6hXDinlhI66MOKAuJypcAcR8KwHDg6u1yFdbOhbzVW/DIWDHiJwQTno+drUzpCmFgMy2F4+8lAYZHofZMCRN7yXJPMMv4CMMowhRwKbjX4cipxn6GnGkziQb7qgDJixvBGblfC93LiVgWb5nV8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067528; c=relaxed/simple; bh=5GkIn/zV+sv8ou4yiI1OchGUrKPM0/dydgHojNwLtfA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c/ch7Uch8ugvwkbSzyYc0z6YVu9p5Qi+JV7Zj4twQ+bYJlam+9xgBLtJkfeS/e3KmNsCOtuVnuj6bGdAWPj2gFbxhV9r36AXO7W3aX1OZo0y155eGYH+sVzDUJrHPLqRsnLj+aQZLHuK1wMV9Lz/60cUWLOB5vwhIjuTugOi9pU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=RIxxEKxG; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=qELKKw+j; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=RIxxEKxG; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=qELKKw+j; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="RIxxEKxG"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="qELKKw+j"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="RIxxEKxG"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="qELKKw+j" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C84641FCFC; Thu, 4 Jul 2024 04:32:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=RIxxEKxG5cR+JdvOXGvScHx2quxv+jhuEkgOckg4pFfPpwOj9VOZl1vh5qJmR4/JuTNxQ4 D/UknVe0GOar65hwsjkxTWr72/C44FARudhSihdPho6fBNUUoiPDaqE3fz2zwhDes4iDuP OItI5Fg3f+Kf/XMNa7u2ylOqzbGXysA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=qELKKw+j6f1vh0ZBQrKxzDcNKdWTqCngnE0ElYT6ZMBZDJuWGRwB7g2w0Ut5FIQYasv9A8 JRnzseqZKYp7iOAQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=RIxxEKxG5cR+JdvOXGvScHx2quxv+jhuEkgOckg4pFfPpwOj9VOZl1vh5qJmR4/JuTNxQ4 D/UknVe0GOar65hwsjkxTWr72/C44FARudhSihdPho6fBNUUoiPDaqE3fz2zwhDes4iDuP OItI5Fg3f+Kf/XMNa7u2ylOqzbGXysA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TzRTJDegzctm0o/zod38K6uM5eCrhph5+Nc4qITcL0E=; b=qELKKw+j6f1vh0ZBQrKxzDcNKdWTqCngnE0ElYT6ZMBZDJuWGRwB7g2w0Ut5FIQYasv9A8 JRnzseqZKYp7iOAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9BDC913889; Thu, 4 Jul 2024 04:32:03 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SDs9IcMlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:03 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 12/45] fs/proc: Enable pagemap_pmd_range to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:30:59 +0200 Message-ID: <20240704043132.28501-13-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach pagemap_pmd_range. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 4d94b6ce58dd..ec429d82b921 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1824,9 +1824,9 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned lo= ng addr, unsigned long end, spinlock_t *ptl; pte_t *pte, *orig_pte; int err =3D 0; -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES =20 - ptl =3D pmd_trans_huge_lock(pmdp, vma); + ptl =3D pmd_huge_lock(pmdp, vma); if (ptl) { unsigned int idx =3D (addr & ~PMD_MASK) >> PAGE_SHIFT; u64 flags =3D 0, frame =3D 0; @@ -1848,7 +1848,6 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned lo= ng addr, unsigned long end, if (pm->show_pfn) frame =3D pmd_pfn(pmd) + idx; } -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION else if (is_swap_pmd(pmd)) { swp_entry_t entry =3D pmd_to_swp_entry(pmd); unsigned long offset; @@ -1861,7 +1860,8 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned lo= ng addr, unsigned long end, frame =3D swp_type(entry) | (offset << MAX_SWAPFILES_SHIFT); } - flags |=3D PM_SWAP; + if (!is_vm_hugetlb_page(vma)) + flags |=3D PM_SWAP; if (pmd_swp_soft_dirty(pmd)) flags |=3D PM_SOFT_DIRTY; if (pmd_swp_uffd_wp(pmd)) @@ -1869,7 +1869,6 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned lo= ng addr, unsigned long end, VM_BUG_ON(!is_pmd_migration_entry(pmd)); page =3D pfn_swap_entry_to_page(entry); } -#endif =20 if (page) { folio =3D page_folio(page); @@ -1899,7 +1898,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned lo= ng addr, unsigned long end, spin_unlock(ptl); return err; } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ =20 /* * We can assume that @vma always points to a valid one and @end never --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E47F413C8FE for ; Thu, 4 Jul 2024 04:32:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067529; cv=none; b=KlTlWwrnf/rDQhM1xpPBGB+lbkwMvrsXo9FMUEzbOHJ6+ikYFZ3rZpaFkx2azVOfw8wdrRw202rgPDw9Okf/j+9P2pjOrCKZ8Jj3XzGP6Lyl2qQTMhe0HL+8iLVl78Ebn7uSr9FsISg9Um6h8lKUSh/B5y3uXd4Fkfk2Jef+at0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067529; c=relaxed/simple; bh=Y2eKlpTRqw2tQSuejatAyQ/lGCV+ROCKXY1uSbuxfuM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bP2qiV1UpbL2IrRU6EsX4laQGm4apAhD7oB69chWx9eb63jcVs+mhI+SU1ctPlpxoFYH6nxVwWR7vJ71IqHmkqMx/nxhFm+k1mr10zSi+p0Ti6tuYfiThAG3vc12sY3HKozwnVJSbGT4dnxWH+CGQ0AfOWfvLQvFbaZyP4a63/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 2DFBF1FCF9; Thu, 4 Jul 2024 04:32:06 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EC78E13889; Thu, 4 Jul 2024 04:32:04 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MJ1iNMQlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:04 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 13/45] mm: Implement pud-version uffd functions Date: Thu, 4 Jul 2024 06:31:00 +0200 Message-ID: <20240704043132.28501-14-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 2DFBF1FCF9 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" HugeTLB pages will be handled on pud level as well, so we need to implement pud-versions of uffd functions in order to properly handle it. Signed-off-by: Oscar Salvador --- arch/arm64/include/asm/pgtable.h | 7 ++ arch/x86/include/asm/pgtable.h | 158 +++++++++++++++++------------ include/asm-generic/pgtable_uffd.h | 30 ++++++ include/linux/pgtable.h | 5 + 4 files changed, 136 insertions(+), 64 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index f8efbc128446..936ed3a915a3 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -556,6 +556,9 @@ static inline int pmd_trans_huge(pmd_t pmd) #define pmd_mkyoung(pmd) pte_pmd(pte_mkyoung(pmd_pte(pmd))) #define pmd_mkinvalid(pmd) pte_pmd(pte_mkinvalid(pmd_pte(pmd))) #ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP +#define pud_uffd_wp(pud) pte_uffd_wp(pud_pte(pud)) +#define pud_mkuffd_wp(pud) pte_pud(pte_mkuffd_wp(pud_pte(pud))) +#define pud_clear_uffd_wp(pud) pte_pud(pte_clear_uffd_wp(pud_pte(pud))) #define pmd_uffd_wp(pmd) pte_uffd_wp(pmd_pte(pmd)) #define pmd_mkuffd_wp(pmd) pte_pmd(pte_mkuffd_wp(pmd_pte(pmd))) #define pmd_clear_uffd_wp(pmd) pte_pmd(pte_clear_uffd_wp(pmd_pte(pmd))) @@ -563,6 +566,10 @@ static inline int pmd_trans_huge(pmd_t pmd) #define pmd_swp_mkuffd_wp(pmd) pte_pmd(pte_swp_mkuffd_wp(pmd_pte(pmd))) #define pmd_swp_clear_uffd_wp(pmd) \ pte_pmd(pte_swp_clear_uffd_wp(pmd_pte(pmd))) +#define pud_swp_uffd_wp(pud) pte_swp_uffd_wp(pud_pte(pud)) +#define pud_swp_mkuffd_wp(pud) pte_pud(pte_swp_mkuffd_wp(pud_pte(pud))) +#define pud_swp_clear_uffd_wp(pud) \ + pte_pud(pte_swp_clear_uffd_wp(pud_pte(pud))) #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ =20 #define pmd_write(pmd) pte_write(pmd_pte(pmd)) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 772f778bac06..640edc31962f 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -512,70 +512,6 @@ static inline pmd_t pmd_wrprotect(pmd_t pmd) return pmd_mksaveddirty(pmd); } =20 -#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP -static inline int pmd_uffd_wp(pmd_t pmd) -{ - return pmd_flags(pmd) & _PAGE_UFFD_WP; -} - -static inline pmd_t pmd_mkuffd_wp(pmd_t pmd) -{ - return pmd_wrprotect(pmd_set_flags(pmd, _PAGE_UFFD_WP)); -} - -static inline pmd_t pmd_clear_uffd_wp(pmd_t pmd) -{ - return pmd_clear_flags(pmd, _PAGE_UFFD_WP); -} -#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ - -static inline pmd_t pmd_mkold(pmd_t pmd) -{ - return pmd_clear_flags(pmd, _PAGE_ACCESSED); -} - -static inline pmd_t pmd_mkclean(pmd_t pmd) -{ - return pmd_clear_flags(pmd, _PAGE_DIRTY_BITS); -} - -static inline pmd_t pmd_mkdirty(pmd_t pmd) -{ - pmd =3D pmd_set_flags(pmd, _PAGE_DIRTY | _PAGE_SOFT_DIRTY); - - return pmd_mksaveddirty(pmd); -} - -static inline pmd_t pmd_mkwrite_shstk(pmd_t pmd) -{ - pmd =3D pmd_clear_flags(pmd, _PAGE_RW); - - return pmd_set_flags(pmd, _PAGE_DIRTY); -} - -static inline pmd_t pmd_mkdevmap(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_DEVMAP); -} - -static inline pmd_t pmd_mkhuge(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_PSE); -} - -static inline pmd_t pmd_mkyoung(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_ACCESSED); -} - -static inline pmd_t pmd_mkwrite_novma(pmd_t pmd) -{ - return pmd_set_flags(pmd, _PAGE_RW); -} - -pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma); -#define pmd_mkwrite pmd_mkwrite - static inline pud_t pud_set_flags(pud_t pud, pudval_t set) { pudval_t v =3D native_pud_val(pud); @@ -659,6 +595,85 @@ static inline pud_t pud_mkwrite(pud_t pud) return pud_clear_saveddirty(pud); } =20 +#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP +static inline int pud_uffd_wp(pud_t pud) +{ + return pud_flags(pud) & _PAGE_UFFD_WP; +} + +static inline pud_t pud_mkuffd_wp(pud_t pud) +{ + return pud_wrprotect(pud_set_flags(pud, _PAGE_UFFD_WP)); +} + +static inline pud_t pud_clear_uffd_wp(pud_t pud) +{ + return pud_clear_flags(pud, _PAGE_UFFD_WP); +} + +static inline int pmd_uffd_wp(pmd_t pmd) +{ + return pmd_flags(pmd) & _PAGE_UFFD_WP; +} + +static inline pmd_t pmd_mkuffd_wp(pmd_t pmd) +{ + return pmd_wrprotect(pmd_set_flags(pmd, _PAGE_UFFD_WP)); +} + +static inline pmd_t pmd_clear_uffd_wp(pmd_t pmd) +{ + return pmd_clear_flags(pmd, _PAGE_UFFD_WP); +} +#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ + +static inline pmd_t pmd_mkold(pmd_t pmd) +{ + return pmd_clear_flags(pmd, _PAGE_ACCESSED); +} + +static inline pmd_t pmd_mkclean(pmd_t pmd) +{ + return pmd_clear_flags(pmd, _PAGE_DIRTY_BITS); +} + +static inline pmd_t pmd_mkdirty(pmd_t pmd) +{ + pmd =3D pmd_set_flags(pmd, _PAGE_DIRTY | _PAGE_SOFT_DIRTY); + + return pmd_mksaveddirty(pmd); +} + +static inline pmd_t pmd_mkwrite_shstk(pmd_t pmd) +{ + pmd =3D pmd_clear_flags(pmd, _PAGE_RW); + + return pmd_set_flags(pmd, _PAGE_DIRTY); +} + +static inline pmd_t pmd_mkdevmap(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_DEVMAP); +} + +static inline pmd_t pmd_mkhuge(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_PSE); +} + +static inline pmd_t pmd_mkyoung(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_ACCESSED); +} + +static inline pmd_t pmd_mkwrite_novma(pmd_t pmd) +{ + return pmd_set_flags(pmd, _PAGE_RW); +} + +pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma); +#define pmd_mkwrite pmd_mkwrite + #ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY static inline int pte_soft_dirty(pte_t pte) { @@ -1574,6 +1589,21 @@ static inline pmd_t pmd_swp_clear_uffd_wp(pmd_t pmd) { return pmd_clear_flags(pmd, _PAGE_SWP_UFFD_WP); } + +static inline pud_t pud_swp_mkuffd_wp(pud_t pud) +{ + return pud_set_flags(pud, _PAGE_SWP_UFFD_WP); +} + +static inline int pud_swp_uffd_wp(pud_t pud) +{ + return pud_flags(pud) & _PAGE_SWP_UFFD_WP; +} + +static inline pud_t pud_swp_clear_uffd_wp(pud_t pud) +{ + return pud_clear_flags(pud, _PAGE_SWP_UFFD_WP); +} #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ =20 static inline u16 pte_flags_pkey(unsigned long pte_flags) diff --git a/include/asm-generic/pgtable_uffd.h b/include/asm-generic/pgtab= le_uffd.h index 828966d4c281..453118eb2161 100644 --- a/include/asm-generic/pgtable_uffd.h +++ b/include/asm-generic/pgtable_uffd.h @@ -12,6 +12,11 @@ static __always_inline int pmd_uffd_wp(pmd_t pmd) return 0; } =20 +static __always_inline int pud_uffd_wp(pud_t pud) +{ + return 0; +} + static __always_inline pte_t pte_mkuffd_wp(pte_t pte) { return pte; @@ -22,6 +27,11 @@ static __always_inline pmd_t pmd_mkuffd_wp(pmd_t pmd) return pmd; } =20 +static __always_inline pud_t pud_mkuffd_wp(pud_t pud) +{ + return pmd; +} + static __always_inline pte_t pte_clear_uffd_wp(pte_t pte) { return pte; @@ -32,6 +42,11 @@ static __always_inline pmd_t pmd_clear_uffd_wp(pmd_t pmd) return pmd; } =20 +static __always_inline pud_t pud_clear_uffd_wp(pud_t pud) +{ + return pmd; +} + static __always_inline pte_t pte_swp_mkuffd_wp(pte_t pte) { return pte; @@ -61,6 +76,21 @@ static inline pmd_t pmd_swp_clear_uffd_wp(pmd_t pmd) { return pmd; } + +static inline pud_t pud_swp_mkuffd_wp(pud_t pud) +{ + return pud; +} + +static inline int pud_swp_uffd_wp(pud_t pud) +{ + return 0; +} + +static inline pud_t pud_swp_clear_uffd_wp(pud_t pud) +{ + return pud; +} #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ =20 #endif /* _ASM_GENERIC_PGTABLE_UFFD_H */ diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 991137dab87e..a2e2ebb93f21 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1433,6 +1433,11 @@ static inline int pmd_soft_dirty(pmd_t pmd) return 0; } =20 +static inline int pud_soft_dirty(pud_t pud) +{ + return 0; +} + static inline pte_t pte_mksoft_dirty(pte_t pte) { return pte; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0920F13C9DE for ; Thu, 4 Jul 2024 04:32:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067530; cv=none; b=b4DmDS1OKdYmZUM1tw1cszEj0BVn8A/zbSLFiAK9avS40iCNPYrShvV8ly9oWzSikutx1LM5rBn2rUs4iBvkBEsLF3TM0z8dZwoIeDEAc+Z/r1nu17JN1QJrrcybUICVpj8BxC9vv1uhlpeoXF1Bwau8d8YI5zADNCAYtMgnA+w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067530; c=relaxed/simple; bh=6i0AByUNPycn/ZqdJjY0fr8AnD7wB3gAtGhx/AMwE5U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JaSeI2EherWVMzkPd9dJBcf/VI1ztBIbkCM0udI7Cq6oEk4bKgtWBNMxqK/OTLSzZ4DkRjQdYRzui5sGWFuPcXJGsHsJht1PddcY3gcXJxQSmrb7MxSAKNd5NElCEIzQ6/VxAGctRNa+sZOIm9COOx+0lfS6Jk4BZWyTi3qptPI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 820001FCF7; Thu, 4 Jul 2024 04:32:07 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5043013889; Thu, 4 Jul 2024 04:32:06 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cPLADsYlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:06 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 14/45] fs/proc: Create pagemap_pud_range to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:01 +0200 Message-ID: <20240704043132.28501-15-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 820001FCF7 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap) but hugetlb can, so create pagemap_pud_range in order to handle PUD-mapped hugetlb vmas Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index ec429d82b921..5965a074467e 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1924,6 +1924,65 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned l= ong addr, unsigned long end, return err; } =20 +#ifdef CONFIG_HUGETLB_PAGE +static int pagemap_pud_range(pud_t *pudp, unsigned long addr, unsigned lon= g end, + struct mm_walk *walk) +{ + pud_t pud; + int err =3D 0; + spinlock_t *ptl; + u64 flags =3D 0, frame =3D 0; + struct pagemapread *pm =3D walk->private; + struct vm_area_struct *vma =3D walk->vma; + + ptl =3D pud_huge_lock(pudp, vma); + if (!ptl) + return err; + + pud =3D *pudp; + + if (vma->vm_flags & VM_SOFTDIRTY) + flags |=3D PM_SOFT_DIRTY; + if (pud_present(pud)) { + struct folio *folio =3D page_folio(pud_page(pud)); + + flags |=3D PM_PRESENT; + if (!folio_test_anon(folio)) + flags |=3D PM_FILE; + + if (!folio_likely_mapped_shared(folio)) + flags |=3D PM_MMAP_EXCLUSIVE; + + if (pud_soft_dirty(pud)) + flags |=3D PM_SOFT_DIRTY; + if (pud_uffd_wp(pud)) + flags |=3D PM_UFFD_WP; + if (pm->show_pfn) + frame =3D pud_pfn(pud) + + ((addr & ~PUD_MASK) >> PAGE_SHIFT); + } else if (pud_swp_uffd_wp(pud)) { + /* Only hugetlb can have swap entries at PUD level*/ + flags |=3D PM_UFFD_WP; + } + + for (; addr !=3D end; addr +=3D PAGE_SIZE) { + pagemap_entry_t pme =3D make_pme(frame, flags); + + err =3D add_to_pagemap(&pme, pm); + if (err) + return err; + if (pm->show_pfn && (flags & PM_PRESENT)) + frame++; + } + spin_unlock(ptl); + + cond_resched(); + return err; +} +#else +#define pagemap_pud_range NULL +#endif + #ifdef CONFIG_HUGETLB_PAGE /* This function walks within one hugetlb entry in the single call */ static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask, @@ -1980,6 +2039,7 @@ static int pagemap_hugetlb_range(pte_t *ptep, unsigne= d long hmask, #endif /* HUGETLB_PAGE */ =20 static const struct mm_walk_ops pagemap_ops =3D { + .pud_entry =3D pagemap_pud_range, .pmd_entry =3D pagemap_pmd_range, .pte_hole =3D pagemap_pte_hole, .hugetlb_entry =3D pagemap_hugetlb_range, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9218913D244 for ; Thu, 4 Jul 2024 04:32:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067532; cv=none; b=EbzTnSXTylcgz7BsLtuC3Mp6Xveqrz48fh6LHvGmATUwozCznbT6/W62JU5OLxoAvcPd29BfW2nyOFCMGWBc8eGWD+tWZWT+aPEt+X7KXDx8DEGeGIFFmKGrD6gTMSXQQrqbXgcOKWyZmPFNLDyQ/nRyK5jCw8ndfGhEqmNjerE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067532; c=relaxed/simple; bh=eyJY9NWoZ4hgVK9bByr04uTn+2RTeLXcPqA2C30/qTI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gNhVHK2iQoVyG6xrH0TCiIhBGuUqlOdUct54iCWKFMgKBXhHc0aKAFD5vtCkrlCJKkl0AOrsrLPpbNIgxYMwsfHXa5S91sA4enw2IyxVQO8JEs/EgpcmuddlM6mZb1AwGat05gsesudYtcqUpd9yiD2M5Xgk0j1W4J8LZEG9G/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=reCsEU+x; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=cWSSs/Dz; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=reCsEU+x; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=cWSSs/Dz; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="reCsEU+x"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="cWSSs/Dz"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="reCsEU+x"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="cWSSs/Dz" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E2CC31FCFB; Thu, 4 Jul 2024 04:32:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=reCsEU+xOf0IC135atdv1qtTW7GwE3KuG5U1k4v+SluCIY6H3D2FUi8FAWIyXYCZIx84yW 2P3PP6oHEwHguqWCpYJ2Wi1pGYR41wpaVnU4D4lDnskSWrRP5eI4/8eoAU1cZhaAgG1ovS /ddxekibmWnrHYkaVCjv2+cJtRQFDJs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=cWSSs/Dz6GKtXty5+7bttcX9oK91QygswkqdEf4rRjRzvfkLm/fnE40h99EAYCwuc44cs1 Y9XuXXYLWDeZrCBA== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=reCsEU+xOf0IC135atdv1qtTW7GwE3KuG5U1k4v+SluCIY6H3D2FUi8FAWIyXYCZIx84yW 2P3PP6oHEwHguqWCpYJ2Wi1pGYR41wpaVnU4D4lDnskSWrRP5eI4/8eoAU1cZhaAgG1ovS /ddxekibmWnrHYkaVCjv2+cJtRQFDJs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067528; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fM4emcZ2cxXRGcFoZk72IXVFIB/7cc45xsWWyiF4u+4=; b=cWSSs/Dz6GKtXty5+7bttcX9oK91QygswkqdEf4rRjRzvfkLm/fnE40h99EAYCwuc44cs1 Y9XuXXYLWDeZrCBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A7B8713889; Thu, 4 Jul 2024 04:32:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cGPCI8clhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:07 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 15/45] fs/proc: Adjust pte_to_pagemap_entry for hugetlb vmas Date: Thu, 4 Jul 2024 06:31:02 +0200 Message-ID: <20240704043132.28501-16-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" HugeTLB pages cannot be swapped , so do not mark them as that. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5965a074467e..22200018371d 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1795,7 +1795,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, frame =3D swp_type(entry) | (offset << MAX_SWAPFILES_SHIFT); } - flags |=3D PM_SWAP; + if (!is_vm_hugetlb_page(vma)) + flags |=3D PM_SWAP; if (is_pfn_swap_entry(entry)) page =3D pfn_swap_entry_to_page(entry); if (pte_marker_entry_uffd_wp(entry)) --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 030231422CC for ; Thu, 4 Jul 2024 04:32:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067533; cv=none; b=V5FXSgYyJgm9dwirVUVAPQoCZvYDSyVbOOXa6q14fMtXnwbMVkd+sFELNu7bypdGn/FqmwjWzRkoljwoZ4hKfghsBdfBTjyuYsNce8Jz+VGBfmVomzG5UudgSjGxYIXw4ftg78/yl9WG1ilccruYO6f37Wd9wUASJz2Px9moI0E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067533; c=relaxed/simple; bh=JplJGugUueYsKXtwReqO01y88Jur3wKRsdxtgS9HNfg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gIvkKJthlcERjdJdDP1WtvPQaU4OJD6tKa/vGqeQ/FRBCTgYMi4HyKdG1N+G6OzlKUP+X9KTXKwaxSMOTg8kfZq+q4h3HUF2QOM+sz6XXvwW9PJFnaAjI63jycypsf3UoZNUGrHDxzN6ewLEHPZOjJSJJRwJxjqvitLBIl+QA18= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=GTzwuWvk; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=jvDVW3J6; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=GTzwuWvk; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=jvDVW3J6; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="GTzwuWvk"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="jvDVW3J6"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="GTzwuWvk"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="jvDVW3J6" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2A18B21180; Thu, 4 Jul 2024 04:32:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=GTzwuWvkW4352FKSC60QDraIO7eJOmAixIJz5RFRrudJmWfKJkpiuKZHVlx8nO0wQbcFzt qLNYDqW+shNAbNnPmhFgaxr/xkRqxsQJ9oLNgCTLnZXKEUR5ZknhTrQbnFUU6RkjXXQiol jNr+qp+D+BH1yGYXIesILMJvxw273IE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=jvDVW3J69xcP45jkzhmWbnnV1lDl4uY2pXNQ4DMnh1no9iNhknQuhbQaJO2boD0G3/hFKq b91zyh46S6CvurBg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=GTzwuWvkW4352FKSC60QDraIO7eJOmAixIJz5RFRrudJmWfKJkpiuKZHVlx8nO0wQbcFzt qLNYDqW+shNAbNnPmhFgaxr/xkRqxsQJ9oLNgCTLnZXKEUR5ZknhTrQbnFUU6RkjXXQiol jNr+qp+D+BH1yGYXIesILMJvxw273IE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ywx5qYincblCns1hUo2a7V32kYl58scbVMBY9LF3T58=; b=jvDVW3J69xcP45jkzhmWbnnV1lDl4uY2pXNQ4DMnh1no9iNhknQuhbQaJO2boD0G3/hFKq b91zyh46S6CvurBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1083C13889; Thu, 4 Jul 2024 04:32:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QIUEO8glhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:08 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 16/45] fs/proc: Enable pagemap_scan_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:03 +0200 Message-ID: <20240704043132.28501-17-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach pagemap_scan_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 41 ++++++++++++++++++++++++++--------------- 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 22200018371d..df649f69ea2c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2263,8 +2263,8 @@ static void make_uffd_wp_pte(struct vm_area_struct *v= ma, } } =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE -static unsigned long pagemap_thp_category(struct pagemap_scan_private *p, +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES +static unsigned long pagemap_pmd_category(struct pagemap_scan_private *p, struct vm_area_struct *vma, unsigned long addr, pmd_t pmd) { @@ -2296,7 +2296,8 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, if (pmd_swp_soft_dirty(pmd)) categories |=3D PAGE_IS_SOFT_DIRTY; =20 - if (p->masks_of_interest & PAGE_IS_FILE) { + if ((p->masks_of_interest & PAGE_IS_FILE) && + !is_vm_hugetlb_page(vma)) { swp =3D pmd_to_swp_entry(pmd); if (is_pfn_swap_entry(swp) && !folio_test_anon(pfn_swap_entry_folio(swp))) @@ -2321,7 +2322,7 @@ static void make_uffd_wp_pmd(struct vm_area_struct *v= ma, set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ =20 #ifdef CONFIG_HUGETLB_PAGE static unsigned long pagemap_hugetlb_category(pte_t pte) @@ -2522,22 +2523,22 @@ static int pagemap_scan_output(unsigned long catego= ries, return ret; } =20 -static int pagemap_scan_thp_entry(pmd_t *pmd, unsigned long start, +static int pagemap_scan_huge_entry(pmd_t *pmd, unsigned long start, unsigned long end, struct mm_walk *walk) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES struct pagemap_scan_private *p =3D walk->private; struct vm_area_struct *vma =3D walk->vma; unsigned long categories; spinlock_t *ptl; int ret =3D 0; =20 - ptl =3D pmd_trans_huge_lock(pmd, vma); + ptl =3D pmd_huge_lock(pmd, vma); if (!ptl) return -ENOENT; =20 categories =3D p->cur_vma_category | - pagemap_thp_category(p, vma, start, *pmd); + pagemap_pmd_category(p, vma, start, *pmd); =20 if (!pagemap_scan_is_interesting_page(categories, p)) goto out_unlock; @@ -2556,19 +2557,29 @@ static int pagemap_scan_thp_entry(pmd_t *pmd, unsig= ned long start, * needs to be performed on a portion of the huge page. */ if (end !=3D start + HPAGE_SIZE) { - spin_unlock(ptl); - split_huge_pmd(vma, pmd, start); pagemap_scan_backout_range(p, start, end); - /* Report as if there was no THP */ - return -ENOENT; + if (!is_vm_hugetlb_page(vma)) { + /* Report as if there was no THP */ + spin_unlock(ptl); + split_huge_pmd(vma, pmd, start); + ret =3D -ENOENT; + goto out; + } + ret =3D 0; + p->arg.walk_end =3D start; + goto out_unlock; } =20 make_uffd_wp_pmd(vma, start, pmd); - flush_tlb_range(vma, start, end); + if (is_vm_hugetlb_page(vma)) + flush_hugetlb_tlb_range(vma, start, end); + else + flush_tlb_range(vma, start, end); out_unlock: spin_unlock(ptl); +out: return ret; -#else /* !CONFIG_TRANSPARENT_HUGEPAGE */ +#else /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ return -ENOENT; #endif } @@ -2585,7 +2596,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigne= d long start, =20 arch_enter_lazy_mmu_mode(); =20 - ret =3D pagemap_scan_thp_entry(pmd, start, end, walk); + ret =3D pagemap_scan_huge_entry(pmd, start, end, walk); if (ret !=3D -ENOENT) { arch_leave_lazy_mmu_mode(); return ret; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06327142E7A for ; Thu, 4 Jul 2024 04:32:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067535; cv=none; b=ssEAhSckkGkL2MjHn+zwE8qriMJIEM0GGNvS6hb4+1OmXHPyHR9kKSG2PCymqzVJETUt3eg/rMITGORhKuhDhatFftv0S6zTPPcX/mIeX88flEbvak9Aj9KNgSqxc8PjTOp8xnYnPAggxtIEZVBMbmM66/B/EBQv/PpoQfzYSXU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067535; c=relaxed/simple; bh=sfs9op56n+Ymo/NUKk/B9a68bdkBPRf5wMfkaL74S/w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=saA6KXtgX5luHaZ5xtAzomqdf03E5p3kuvezUAEthK1LlU3vV2c1151hzBgvyCfsget90fLPJ5vV9tqGNMBOIuYXWVb8KfcpiEClP8aOdya5WpoXK32oSoaqGq2X9jSyuk7EtiS0h0usaaI2LNLU3xAxDWekSDcHVw7+XhntSOs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=rGzyLD7L; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=8nHTN6MV; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=rGzyLD7L; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=8nHTN6MV; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="rGzyLD7L"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="8nHTN6MV"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="rGzyLD7L"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="8nHTN6MV" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 717E31FCF8; Thu, 4 Jul 2024 04:32:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=rGzyLD7LkYyMAvmIw4xFUDH+buX9d/II8oeFi+WRfNzUSU6mBH1Xd+8r3gYOEPGL9dYDCy IlLqdjWxRy5mraTCwA+0ut4zZnng0xAJIvDVvopn+3dMXHd6APSdUZmCaNvfWzCqwkjiNn u/St7F29ojB/6wl0e44bDnr1D1/7TB8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=8nHTN6MVi8gMxqJWpwXDGmQUc1P5i9EMQ+Rwyw40Sbfz/xQvdeEiZOiulE9osEkcCpeY4k 3YaniTexv02tM6Cg== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=rGzyLD7LkYyMAvmIw4xFUDH+buX9d/II8oeFi+WRfNzUSU6mBH1Xd+8r3gYOEPGL9dYDCy IlLqdjWxRy5mraTCwA+0ut4zZnng0xAJIvDVvopn+3dMXHd6APSdUZmCaNvfWzCqwkjiNn u/St7F29ojB/6wl0e44bDnr1D1/7TB8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067531; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKLC6fxewOrLf0i98Rd25YFQam44kkf2M+DOutpQYdQ=; b=8nHTN6MVi8gMxqJWpwXDGmQUc1P5i9EMQ+Rwyw40Sbfz/xQvdeEiZOiulE9osEkcCpeY4k 3YaniTexv02tM6Cg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4E2E313889; Thu, 4 Jul 2024 04:32:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UP40DsolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:10 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 17/45] mm: Implement pud-version for pud_mkinvalid and pudp_establish Date: Thu, 4 Jul 2024 06:31:04 +0200 Message-ID: <20240704043132.28501-18-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" HugeTLB pages will be handled on pud level as well, so we need to implement pud-versions of pud_mkinvalid and pudp_establish. Signed-off-by: Oscar Salvador --- arch/arm64/include/asm/pgtable.h | 11 ++++++ arch/loongarch/include/asm/pgtable.h | 8 ++++ arch/mips/include/asm/pgtable.h | 7 ++++ arch/powerpc/include/asm/book3s/64/pgtable.h | 7 +++- arch/powerpc/mm/book3s64/pgtable.c | 15 ++++++- arch/riscv/include/asm/pgtable.h | 15 +++++++ arch/x86/include/asm/pgtable.h | 31 ++++++++++++++- include/linux/pgtable.h | 41 +++++++++++++++++++- mm/pgtable-generic.c | 21 ++++++++++ 9 files changed, 150 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 936ed3a915a3..5e26e63b1012 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -595,6 +595,7 @@ static inline pmd_t pmd_mkdevmap(pmd_t pmd) #define pud_write(pud) pte_write(pud_pte(pud)) =20 #define pud_mkhuge(pud) (__pud(pud_val(pud) & ~PUD_TABLE_BIT)) +#define pud_mkinvalid(pud) pte_pud(pte_mkinvalid(pud_pte(pud))) =20 #define __pud_to_phys(pud) __pte_to_phys(pud_pte(pud)) #define __phys_to_pud_val(phys) __phys_to_pte_val(phys) @@ -1344,6 +1345,16 @@ static inline pmd_t pmdp_establish(struct vm_area_st= ruct *vma, } #endif =20 +#ifdef CONFIG_HUGETLB_PAGE +#define pudp_establish pudp_establish +static inline pud_t pudp_establish(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, pud_t pud) +{ + page_table_check_pud_set(vma->vm_mm, pudp, pud); + return __pud(xchg_relaxed(&pud_val(*pudp), pud_val(pud))); +} +#endif + /* * Encode and decode a swap entry: * bits 0-1: present (must be zero) diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/= asm/pgtable.h index 161dd6e10479..cf73c2f2da2c 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -581,6 +581,14 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) return pmd; } =20 +static inline pud_t pud_mkinvalid(pud_t pud) +{ + pud_val(pud) |=3D _PAGE_PRESENT_INVALID; + pud_val(pud) &=3D ~(_PAGE_PRESENT | _PAGE_VALID | _PAGE_DIRTY | _PAGE_PRO= TNONE); + + return pud; +} + /* * The generic version pmdp_huge_get_and_clear uses a version of pmd_clear= () with a * different prototype. diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtabl= e.h index c29a551eb0ca..390a2f022147 100644 --- a/arch/mips/include/asm/pgtable.h +++ b/arch/mips/include/asm/pgtable.h @@ -736,6 +736,13 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) return pmd; } =20 +static inline pud_t pud_mkinvalid(pud_t pud) +{ + pud_val(pud) &=3D ~(_PAGE_PRESENT | _PAGE_VALID | _PAGE_DIRTY); + + return pud; +} + /* * The generic version pmdp_huge_get_and_clear uses a version of pmd_clear= () with a * different prototype. diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/in= clude/asm/book3s/64/pgtable.h index fa4bb8d6356f..f95ac2a87548 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1085,7 +1085,8 @@ static inline pte_t *pmdp_ptep(pmd_t *pmd) #define pmd_mksoft_dirty(pmd) pte_pmd(pte_mksoft_dirty(pmd_pte(pmd))) #define pmd_clear_soft_dirty(pmd) pte_pmd(pte_clear_soft_dirty(pmd_pte(pmd= ))) =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION +#if defined(CONFIG_ARCH_ENABLE_THP_MIGRATION) || defined(CONFIG_HUGETLB_PA= GE) +#define pud_swp_soft_dirty(pud) pte_swp_soft_dirty(pud_pte(pud)) #define pmd_swp_mksoft_dirty(pmd) pte_pmd(pte_swp_mksoft_dirty(pmd_pte(pmd= ))) #define pmd_swp_soft_dirty(pmd) pte_swp_soft_dirty(pmd_pte(pmd)) #define pmd_swp_clear_soft_dirty(pmd) pte_pmd(pte_swp_clear_soft_dirty(pmd= _pte(pmd))) @@ -1386,6 +1387,10 @@ static inline pgtable_t pgtable_trans_huge_withdraw(= struct mm_struct *mm, extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long add= ress, pmd_t *pmdp); =20 +#define __HAVE_ARCH_PUDP_INVALIDATE +extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long add= ress, + pud_t *pudp); + #define pmd_move_must_withdraw pmd_move_must_withdraw struct spinlock; extern int pmd_move_must_withdraw(struct spinlock *new_pmd_ptl, diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/= pgtable.c index f4d8d3c40e5c..1b6ae7898f99 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -37,7 +37,7 @@ EXPORT_SYMBOL(__pmd_frag_nr); unsigned long __pmd_frag_size_shift; EXPORT_SYMBOL(__pmd_frag_size_shift); =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES /* * This is called when relaxing access to a hugepage. It's also called in = the page * fault path when we don't hit any of the major fault cases, ie, a minor @@ -259,7 +259,18 @@ pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot) pmdv &=3D _HPAGE_CHG_MASK; return pmd_set_protbits(__pmd(pmdv), newprot); } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + unsigned long old_pud; + + VM_WARN_ON_ONCE(!pud_present(*pudp)); + old_pud =3D pud_hugepage_update(vma->vm_mm, address, pudp, _PAGE_PRESENT,= _PAGE_INVALID); + flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE); + return __pud(old_pud); +} +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ =20 /* For use by kexec, called with MMU off */ notrace void mmu_cleanup_all(void) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgta= ble.h index ebfe8faafb79..51600afa203c 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -657,6 +657,11 @@ static inline unsigned long pud_pfn(pud_t pud) return ((__pud_to_phys(pud) & PUD_MASK) >> PAGE_SHIFT); } =20 +static inline pud_t pud_mkinvalid(pud_t pud) +{ + return __pud(pud_val(pud) & ~(_PAGE_PRESENT|_PAGE_PROT_NONE)); +} + static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot) { return pte_pmd(pte_modify(pmd_pte(pmd), newprot)); @@ -804,6 +809,16 @@ extern pmd_t pmdp_collapse_flush(struct vm_area_struct= *vma, unsigned long address, pmd_t *pmdp); #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 +#ifdef CONFIG_HUGETLB_PAGE +#define pudp_establish pudp_establish +static inline pud_t pudp_establish(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, pud_t pud) +{ + page_table_check_pud_set(vma->vm_mm, pudp, pud); + return __pud(atomic_long_xchg((atomic_long_t *)pudp, pud_val(pud))); +} +#endif + /* * Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that * are !pte_none() && !pte_present(). diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 640edc31962f..572458a106e9 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -783,6 +783,12 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) __pgprot(pmd_flags(pmd) & ~(_PAGE_PRESENT|_PAGE_PROTNONE))); } =20 +static inline pud_t pud_mkinvalid(pud_t pud) +{ + return pfn_pud(pud_pfn(pud), + __pgprot(pud_flags(pud) & ~(_PAGE_PRESENT|_PAGE_PROTNONE))); +} + static inline u64 flip_protnone_guard(u64 oldval, u64 val, u64 mask); =20 static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) @@ -1353,6 +1359,23 @@ static inline pud_t pudp_huge_get_and_clear(struct m= m_struct *mm, return pud; } =20 +#ifndef pudp_establish +#define pudp_establish pudp_establish +static inline pud_t pudp_establish(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp, pud_t pud) +{ + page_table_check_pud_set(vma->vm_mm, pudp, pud); + if (IS_ENABLED(CONFIG_SMP)) { + return xchg(pudp, pud); + } else { + pud_t old =3D *pudp; + + WRITE_ONCE(*pudp, pud); + return old; + } +} +#endif + #define __HAVE_ARCH_PMDP_SET_WRPROTECT static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp) @@ -1389,7 +1412,6 @@ static inline pmd_t pmdp_establish(struct vm_area_str= uct *vma, #define __HAVE_ARCH_PMDP_INVALIDATE_AD extern pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); - /* * Page table pages are page-aligned. The lower half of the top * level is used for userspace and the top half for the kernel. @@ -1541,7 +1563,12 @@ static inline pte_t pte_swp_clear_soft_dirty(pte_t p= te) return pte_clear_flags(pte, _PAGE_SWP_SOFT_DIRTY); } =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION +#if defined(CONFIG_ARCH_ENABLE_THP_MIGRATION) || defined(CONFIG_HUGETLB_PA= GE) +static inline int pud_swp_soft_dirty(pud_t pud) +{ + return pud_flags(pud) & _PAGE_SWP_SOFT_DIRTY; +} + static inline pmd_t pmd_swp_mksoft_dirty(pmd_t pmd) { return pmd_set_flags(pmd, _PAGE_SWP_SOFT_DIRTY); diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index a2e2ebb93f21..458e3cbc96b2 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -956,6 +956,11 @@ extern pmd_t pmdp_invalidate(struct vm_area_struct *vm= a, unsigned long address, pmd_t *pmdp); #endif =20 +#ifndef __HAVE_ARCH_PUDP_INVALIDATE +extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long add= ress, + pud_t *pudp); +#endif + #ifndef __HAVE_ARCH_PMDP_INVALIDATE_AD =20 /* @@ -976,6 +981,26 @@ extern pmd_t pmdp_invalidate_ad(struct vm_area_struct = *vma, unsigned long address, pmd_t *pmdp); #endif =20 +#ifndef __HAVE_ARCH_PUDP_INVALIDATE_AD + +/* + * pudp_invalidate_ad() invalidates the PMD while changing a hugetlb mappi= ng in + * the page tables. This function is similar to pudp_invalidate(), but sho= uld + * only be used if the access and dirty bits would not be cleared by the = software + * in the new PUD value. The function ensures that hardware changes of th= e access + * and dirty bits updates would not be lost. + * + * Doing so can allow in certain architectures to avoid a TLB flush in most + * cases. Yet, another TLB flush might be necessary later if the PUD update + * itself requires such flush (e.g., if protection was set to be stricter)= . Yet, + * even when a TLB flush is needed because of the update, the caller may b= e able + * to batch these TLB flushing operations, so fewer TLB flush operations a= re + * needed. + */ +extern pud_t pudp_invalidate_ad(struct vm_area_struct *vma, + unsigned long address, pud_t *pudp); +#endif + #ifndef __HAVE_ARCH_PTE_SAME static inline int pte_same(pte_t pte_a, pte_t pte_b) { @@ -1406,7 +1431,16 @@ static inline pgprot_t pgprot_modify(pgprot_t oldpro= t, pgprot_t newprot) #endif =20 #ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY -#ifndef CONFIG_ARCH_ENABLE_THP_MIGRATION +static inline int pud_soft_dirty(pud_t pud) +{ + return 0; +} +#if !defined(CONFIG_ARCH_ENABLE_THP_MIGRATION) && !defined(CONFIG_HUGETLB_= PAGE) +static inline int pud_swp_soft_dirty(pud_t pud) +{ + return 0; +} + static inline pmd_t pmd_swp_mksoft_dirty(pmd_t pmd) { return pmd; @@ -1487,6 +1521,11 @@ static inline pmd_t pmd_swp_clear_soft_dirty(pmd_t p= md) { return pmd; } + +static inline int pud_swp_soft_dirty(pud_t pud) +{ + return 0; +} #endif =20 #ifndef __HAVE_PFNMAP_TRACKING diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711a..e11ad8663903 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -194,6 +194,27 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct= *mm, pmd_t *pmdp) } #endif =20 +#ifndef __HAVE_ARCH_PUDP_INVALIDATE +pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + VM_WARN_ON_ONCE(!pud_present(*pudp)); + pud_t old =3D pudp_establish(vma, address, pudp, pud_mkinvalid(*pudp)); + + flush_pud_tlb_range(vma, address, address + PUD_SIZE); + return old; +} +#endif + +#ifndef __HAVE_ARCH_PUDP_INVALIDATE_AD +pud_t pudp_invalidate_ad(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + VM_WARN_ON_ONCE(!pud_present(*pudp)); + return pudp_invalidate(vma, address, pudp); +} +#endif + #ifndef __HAVE_ARCH_PMDP_INVALIDATE pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp) --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68698142E9D for ; Thu, 4 Jul 2024 04:32:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067536; cv=none; b=pkHRRWekrEAkYaHr7g3hAzGIFScIk+Mq4lmLgqDmt5RhYG8o9rYkAwcXpRRBp96r7p0rDmn7Kc2T4OhacvotLBY13AYz11lv3KV97xfpZPj72bZm+LnrKYY9h4hcWZDGjRhX+nCAzDA6c8sLFCpA2BMQyyJ/QobISMVEVEWeHcE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067536; c=relaxed/simple; bh=VjG0ahYT9ZkoRgpfIDg2JJhLkIe9bwBPt86aq3YnsiA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ihoHwKlgM+t7GrZveX2NOu3DwzATLv9wbLNYVdXNkUmxDv5xkK9St4yTxsTxHk4WDpT06pFgn+46j4je5oYR5ClMM49NlJ9q1BcUxOmQCm4BkYAsJu8+G3v3ZX87WehKWN/tm8wymBnBfPEey1Fi8N8hRQ9YRF2wKc7QS7UBw0E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BB4731FCFA; Thu, 4 Jul 2024 04:32:12 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9738313889; Thu, 4 Jul 2024 04:32:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id gC+BH8slhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:11 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 18/45] fs/proc: Create pagemap_scan_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:05 +0200 Message-ID: <20240704043132.28501-19-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: BB4731FCFA X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap) but hugetlb can, so create pagemap_scan_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 104 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index df649f69ea2c..3785a44b97fa 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1925,7 +1925,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned lo= ng addr, unsigned long end, return err; } =20 -#ifdef CONFIG_HUGETLB_PAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int pagemap_pud_range(pud_t *pudp, unsigned long addr, unsigned lon= g end, struct mm_walk *walk) { @@ -2324,6 +2324,59 @@ static void make_uffd_wp_pmd(struct vm_area_struct *= vma, } #endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ =20 +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES +static unsigned long pagemap_pud_category(struct pagemap_scan_private *p, + struct vm_area_struct *vma, + unsigned long addr, pud_t pud) +{ + unsigned long categories =3D PAGE_IS_HUGE; + + if (pud_present(pud)) { + struct page *page; + + categories |=3D PAGE_IS_PRESENT; + if (!pud_uffd_wp(pud)) + categories |=3D PAGE_IS_WRITTEN; + + if (p->masks_of_interest & PAGE_IS_FILE) { + page =3D vm_normal_page_pud(vma, addr, pud); + if (page && !PageAnon(page)) + categories |=3D PAGE_IS_FILE; + } + + if (is_zero_pfn(pud_pfn(pud))) + categories |=3D PAGE_IS_PFNZERO; + if (pud_soft_dirty(pud)) + categories |=3D PAGE_IS_SOFT_DIRTY; + } else if (is_swap_pud(pud)) { + swp_entry_t swp; + + categories |=3D PAGE_IS_SWAPPED; + if (!pud_swp_uffd_wp(pud)) + categories |=3D PAGE_IS_WRITTEN; + if (pud_swp_soft_dirty(pud)) + categories |=3D PAGE_IS_SOFT_DIRTY; + } + + return categories; +} + +static void make_uffd_wp_pud(struct vm_area_struct *vma, + unsigned long addr, pud_t *pudp) +{ + pud_t old, pud =3D *pudp; + + if (pud_present(pud)) { + old =3D pudp_invalidate_ad(vma, addr, pudp); + pud =3D pud_mkuffd_wp(old); + set_pud_at(vma->vm_mm, addr, pudp, pud); + } else if (is_migration_entry(pud_to_swp_entry(pud))) { + pud =3D pud_swp_mkuffd_wp(pud); + set_pud_at(vma->vm_mm, addr, pudp, pud); + } +} +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ + #ifdef CONFIG_HUGETLB_PAGE static unsigned long pagemap_hugetlb_category(pte_t pte) { @@ -2685,6 +2738,54 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsign= ed long start, return ret; } =20 +#ifdef CONFIG_HUGETLB_PAGE +static int pagemap_scan_pud_entry(pud_t *pud, unsigned long start, + unsigned long end, struct mm_walk *walk) +{ + int ret =3D 0; + spinlock_t *ptl; + unsigned long categories; + struct vm_area_struct *vma =3D walk->vma; + struct pagemap_scan_private *p =3D walk->private; + + /* Only PUD-mapped hugetlb can reach here at this moment */ + ptl =3D pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + categories =3D p->cur_vma_category | + pagemap_pud_category(p, vma, start, *pud); + + if (!pagemap_scan_is_interesting_page(categories, p)) + goto out_unlock; + + ret =3D pagemap_scan_output(categories, p, start, &end); + if (start =3D=3D end) + goto out_unlock; + + if (~p->arg.flags & PM_SCAN_WP_MATCHING) + goto out_unlock; + if (~categories & PAGE_IS_WRITTEN) + goto out_unlock; + + if (end !=3D start + PUD_SIZE) { + ret =3D 0; + pagemap_scan_backout_range(p, start, end); + p->arg.walk_end =3D start; + goto out_unlock; + } + + make_uffd_wp_pud(vma, start, pud); + flush_hugetlb_tlb_range(vma, start, end); + +out_unlock: + spin_unlock(ptl); + return ret; +} +#else +#define pagemap_scan_pud_entry NULL +#endif + #ifdef CONFIG_HUGETLB_PAGE static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask, unsigned long start, unsigned long end, @@ -2772,6 +2873,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, = unsigned long end, =20 static const struct mm_walk_ops pagemap_scan_ops =3D { .test_walk =3D pagemap_scan_test_walk, + .pud_entry =3D pagemap_scan_pud_entry, .pmd_entry =3D pagemap_scan_pmd_entry, .pte_hole =3D pagemap_scan_pte_hole, .hugetlb_entry =3D pagemap_scan_hugetlb_entry, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86C98143759 for ; Thu, 4 Jul 2024 04:32:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067537; cv=none; b=A/yLQouyhxfQ7ybzBwCPuDTP0I4fpBNVyqYHJzaj9PrCg0YJBQn5u1hR3qrUyRbRvXRc3BarzBxXitzD0trk4qaDtIRt1mZwi8volmdy6V4gy4ofxNvAttN2nTF096Jyej81HUUDQbiOmOEsaHCMtS2C17Z0YTAcf3GmHdKPIhM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067537; c=relaxed/simple; bh=3uhKxR7Ge9aSjaaOQuskOvkTvX/X/MJEsg8TVZDJwGM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fNWrbjo2T5S/sW4/PVUpXhfTyUcnTcFkYmq/EVCAO7vJ8w7i2NJZGgqHehu0NdQrQPdZRkXzDxiXW++2ZHUdOkleKLsMYKo/JmMzZ+HuUcTYqSg1ba6xit5YqJv0B4jJxNX5g2oh7uS0jK9DBmoWhKE1zQYbnNfyQitVdtIi/g0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=j2uGj66F; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=aF1AIMEQ; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=j2uGj66F; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=aF1AIMEQ; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="j2uGj66F"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="aF1AIMEQ"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="j2uGj66F"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="aF1AIMEQ" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 053031FCFC; Thu, 4 Jul 2024 04:32:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=j2uGj66FrtGxFBiwFctbDQ4OTG+w9M73dH62olOmsLGZIhnPfy8i6BSi1M1uU7Z995dm0M rDy3fOSDZiLRMBRFvYSwMC3MWiBIZt60dh4cyu70ITZlu4RYz92tsp3lGRkhC2d9EZ2fgM G1/oiZiEwG0JggCZhXJlACHwMP0ZVFI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=aF1AIMEQj6zRqtFUbpMf5dtjXnB1Crkf8BYEYuKhtpIYT+dX14+NYfft0LEejATVkvxiN5 /oB961ArpbntQiAw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=j2uGj66FrtGxFBiwFctbDQ4OTG+w9M73dH62olOmsLGZIhnPfy8i6BSi1M1uU7Z995dm0M rDy3fOSDZiLRMBRFvYSwMC3MWiBIZt60dh4cyu70ITZlu4RYz92tsp3lGRkhC2d9EZ2fgM G1/oiZiEwG0JggCZhXJlACHwMP0ZVFI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067534; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wUAXkDKulvUPvsNZqyUfVRpX873YTDqHqTEgZWFzyt4=; b=aF1AIMEQj6zRqtFUbpMf5dtjXnB1Crkf8BYEYuKhtpIYT+dX14+NYfft0LEejATVkvxiN5 /oB961ArpbntQiAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DD98813889; Thu, 4 Jul 2024 04:32:12 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WFxbMcwlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:12 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 19/45] fs/proc: Enable gather_pte_stats to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:06 +0200 Message-ID: <20240704043132.28501-20-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach gather_pte_stats. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 3785a44b97fa..e13754d3246e 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3141,7 +3141,7 @@ static struct page *can_gather_numa_stats(pte_t pte, = struct vm_area_struct *vma, return page; } =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static struct page *can_gather_numa_stats_pmd(pmd_t pmd, struct vm_area_struct *vma, unsigned long addr) @@ -3176,15 +3176,21 @@ static int gather_pte_stats(pmd_t *pmd, unsigned lo= ng addr, pte_t *orig_pte; pte_t *pte; =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - ptl =3D pmd_trans_huge_lock(pmd, vma); +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + ptl =3D pmd_huge_lock(pmd, vma); if (ptl) { + unsigned long nr_pages; struct page *page; =20 + if (is_vm_hugetlb_page(vma)) + nr_pages =3D 1; + else + nr_pages =3D HPAGE_PMD_SIZE / PAGE_SIZE; + page =3D can_gather_numa_stats_pmd(*pmd, vma, addr); if (page) gather_stats(page, md, pmd_dirty(*pmd), - HPAGE_PMD_SIZE/PAGE_SIZE); + nr_pages); spin_unlock(ptl); return 0; } --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC0C8143883 for ; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067538; cv=none; b=GQOxL5mmWZidn2CDhR//Hito573PYqzMVrlLkV/6HgG31JcOkNJZ0q1CAEmzosjzRoTfKDT/1hMi356dT92D7IFqUh9r1uzHO0MBeDpu3UhHmVYfuZ95KKB4i6dRsESqVSMA8oArRK8+3kqf7S5btNGn24GE9qZFlXt4LLITM88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067538; c=relaxed/simple; bh=ZlNbxiR3YiE76on6qozqMsygXT/rN9zht/ZciTsCxQE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G6RcMDtDV1QxfHg7skw19xmCKFKhVtvzXROcDxVmemk8tnk7mYOEMawTtIzjQhDVl40kMKbm/9clqHWAnrNryV3GHpZhSnYN3w3xOYIGW+tBeAhLrjA3fc40krmzMPfVBaXiUxbQCv7N+H7Vr0fkIgRB37W2ODkfZpE4wY3QXek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=Sp75Awte; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Mj+n3kpv; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=Sp75Awte; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Mj+n3kpv; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="Sp75Awte"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Mj+n3kpv"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="Sp75Awte"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Mj+n3kpv" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 487F31FCF9; Thu, 4 Jul 2024 04:32:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Sp75Awtev0kxk3LPqK9XxrdT8mVKHhhRuRIBMOFdY8fEHNVHNNZ06aTT5CglZTRF8npqNz 2Yy3uyOucP4OPr8Nv3JT4zfkut2bNZa/kPSEfP3yjJrNw6aJH7PjRQ35FXbRRBUM/Hdhb5 qpJvjTmjFlI704iRP7Ya2VX/BQVd3/k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Mj+n3kpvgdn1kni7UbDFyJvbmlmLCXttzKSC9JGDxhxjtdRvE8XYhxKLUr9QMKGLWWztko A652jlHgiyBk0NDg== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Sp75Awtev0kxk3LPqK9XxrdT8mVKHhhRuRIBMOFdY8fEHNVHNNZ06aTT5CglZTRF8npqNz 2Yy3uyOucP4OPr8Nv3JT4zfkut2bNZa/kPSEfP3yjJrNw6aJH7PjRQ35FXbRRBUM/Hdhb5 qpJvjTmjFlI704iRP7Ya2VX/BQVd3/k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSKmwPU7BkbGtKq84iiMlBZLjbm/8YgbxNgKPRpjw3o=; b=Mj+n3kpvgdn1kni7UbDFyJvbmlmLCXttzKSC9JGDxhxjtdRvE8XYhxKLUr9QMKGLWWztko A652jlHgiyBk0NDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 28FA913889; Thu, 4 Jul 2024 04:32:14 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uClaBc4lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:14 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 20/45] fs/proc: Enable gather_pte_stats to handle cont-pte mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:07 +0200 Message-ID: <20240704043132.28501-21-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" HugeTLB pages can be cont-pte mapped, so teach smaps_pte_entry to handle them. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index e13754d3246e..98dd03c26e68 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3175,6 +3175,7 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long= addr, spinlock_t *ptl; pte_t *orig_pte; pte_t *pte; + unsigned long size =3D PAGE_SIZE, cont_ptes =3D 1; =20 #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES ptl =3D pmd_huge_lock(pmd, vma); @@ -3200,6 +3201,10 @@ static int gather_pte_stats(pmd_t *pmd, unsigned lon= g addr, walk->action =3D ACTION_AGAIN; return 0; } + if (pte_cont(ptep_get(pte))) { + size =3D PAGE_SIZE * CONT_PTES; + cont_ptes =3D CONT_PTES; + } do { pte_t ptent =3D ptep_get(pte); struct page *page =3D can_gather_numa_stats(ptent, vma, addr); @@ -3207,7 +3212,7 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long= addr, continue; gather_stats(page, md, pte_dirty(ptent), 1); =20 - } while (pte++, addr +=3D PAGE_SIZE, addr !=3D end); + } while (pte +=3D cont_ptes, addr +=3D size, addr !=3D end); pte_unmap_unlock(orig_pte, ptl); cond_resched(); return 0; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A5EC143C6A for ; Thu, 4 Jul 2024 04:32:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067540; cv=none; b=kGbNlHsQ004bGOeEzWvJVDTPr4M5jYqew2ujTRAm1LPYAG7/Z8GNKfVY5D8Ayk7wQNXeAXTgY1g62nfyEpCMDfg3mFZf9OKWLy5Lw52vlxtLEd8BKDAxvnKW8Lhh9V1Ov90V4dzS6Y4HITMzkwzAZ6maOoTsHSd22rjkt11KRjU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067540; c=relaxed/simple; bh=BwkWUXeKM8j1YJaQTaCDSmN0oO/MPcXx5cwB7kNu+5s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TyqggYfDZDSYxB9b4ml2KE1xVhSNzZTvTqymVVYD4aLSkfuY7ppu920Ux8VLabLzAqs5C+jO7HdznBD/7ZIaLaSzCYxmfxODoOvs45AzBi4wVc52oaIoII0CBJxq8kOtm9jVmgEhwveCLuXPUxokqxqkk16Ajq7kX8VJArGjXrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A8ED61FCF7; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6D23B13889; Thu, 4 Jul 2024 04:32:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2KYBFs8lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:15 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 21/45] fs/proc: Create gather_pud_stats to handle PUD-mapped hugetlb pages Date: Thu, 4 Jul 2024 06:31:08 +0200 Message-ID: <20240704043132.28501-22-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: A8ED61FCF7 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create gather_pud_stats in order to handle PUD-mapped hugetlb vmas. Also implement can_gather_numa_stats_pud which is the pud version of can_gather_numa_stats_pmd. Signed-off-by: Oscar Salvador --- arch/arm64/include/asm/pgtable.h | 1 + fs/proc/task_mmu.c | 56 ++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 5e26e63b1012..1a6b8be2f0d0 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -590,6 +590,7 @@ static inline pmd_t pmd_mkdevmap(pmd_t pmd) #define pfn_pmd(pfn,prot) __pmd(__phys_to_pmd_val((phys_addr_t)(pfn) << PA= GE_SHIFT) | pgprot_val(prot)) #define mk_pmd(page,prot) pfn_pmd(page_to_pfn(page),prot) =20 +#define pud_dirty(pud) pte_dirty(pud_pte(pud)) #define pud_young(pud) pte_young(pud_pte(pud)) #define pud_mkyoung(pud) pte_pud(pte_mkyoung(pud_pte(pud))) #define pud_write(pud) pte_write(pud_pte(pud)) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 98dd03c26e68..5df17b7cfe6c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3141,6 +3141,61 @@ static struct page *can_gather_numa_stats(pte_t pte,= struct vm_area_struct *vma, return page; } =20 +#ifdef CONFIG_HUGETLB_PAGE +static struct page *can_gather_numa_stats_pud(pud_t pud, + struct vm_area_struct *vma, + unsigned long addr) +{ + struct page *page; + int nid; + + if (!pud_present(pud)) + return NULL; + + page =3D pud_page(pud); + if (!page) + return NULL; + + if (PageReserved(page)) + return NULL; + + nid =3D page_to_nid(page); + if (!node_isset(nid, node_states[N_MEMORY])) + return NULL; + + return page; +} + +static int gather_pud_stats(pud_t *pud, unsigned long addr, + unsigned long end, struct mm_walk *walk) +{ + spinlock_t *ptl; + struct page *page; + unsigned long nr_pages; + struct numa_maps *md =3D walk->private; + struct vm_area_struct *vma =3D walk->vma; + + ptl =3D pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + if (is_vm_hugetlb_page(vma)) + nr_pages =3D 1; + else + nr_pages =3D HPAGE_PUD_SIZE / PAGE_SIZE; + + page =3D can_gather_numa_stats_pud(*pud, vma, addr); + if (page) + gather_stats(page, md, pud_dirty(*pud), + nr_pages); + + spin_unlock(ptl); + return 0; +} +#else +#define gather_pud_stats NULL +#endif + #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static struct page *can_gather_numa_stats_pmd(pmd_t pmd, struct vm_area_struct *vma, @@ -3245,6 +3300,7 @@ static int gather_hugetlb_stats(pte_t *pte, unsigned = long hmask, =20 static const struct mm_walk_ops show_numa_ops =3D { .hugetlb_entry =3D gather_hugetlb_stats, + .pud_entry =3D gather_pud_stats, .pmd_entry =3D gather_pte_stats, .walk_lock =3D PGWALK_RDLOCK, }; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F2D1144303 for ; Thu, 4 Jul 2024 04:32:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067541; cv=none; b=lUcvwTPkXUyq89LDlXTVgfm5NFxzlrORcYERI3wzjpF7kW3XQtxGi4byHnFNkFaAgt42kh4EnzQVAXj1215ElJvNWnj6PeiQzhtESTNZ5zzHIRvwBGUg0HJ5hPnhIltbumgW8sDlElpqRyiFTRxI4sYeAvjMfylwbYCmEa2H6M4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067541; c=relaxed/simple; bh=ciNhVhOKdrAaNI83ufvC/pkkbqW7liVHKSpNFuH+6Dg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IXoz1MyqY4x8eV+VXFE4i1st1JqOxSy7rKwbV6MxJCiKCf9HaeqTNzqvT8S9DpXotY/MLF50Y1f476XVxudO1O0HUzOnIb4avPRKi/ez0T+dKoaVivyg3Alb+NwluceIaIUSHZpFBZ9y/mUvO4FBIwyZbkzzsw579tMGY6Ckyi0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DB2C41FCFD; Thu, 4 Jul 2024 04:32:17 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C2D9D13889; Thu, 4 Jul 2024 04:32:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iBXOKtAlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:16 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 22/45] mm/mempolicy: Enable queue_folios_pmd to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:09 +0200 Message-ID: <20240704043132.28501-23-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: DB2C41FCFD X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach smaps_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- include/linux/mm_inline.h | 7 +++++++ mm/mempolicy.c | 42 ++++++++++++++++++++++++--------------- 2 files changed, 33 insertions(+), 16 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index 93e3eb86ef4e..521a001429d2 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -591,6 +591,13 @@ static inline bool vma_has_recency(struct vm_area_stru= ct *vma) return true; } =20 +static inline bool is_shared_pmd(pmd_t *pmd, struct vm_area_struct *vma) +{ + if (!is_vm_hugetlb_page(vma)) + return false; + return hugetlb_pmd_shared((pte_t *)pmd); +} + static inline spinlock_t *pmd_huge_lock(pmd_t *pmd, struct vm_area_struct = *vma) { spinlock_t *ptl; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index f8703feb68b7..5baf29da198c 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -455,7 +455,8 @@ static const struct mempolicy_operations mpol_ops[MPOL_= MAX] =3D { }; =20 static bool migrate_folio_add(struct folio *folio, struct list_head *folio= list, - unsigned long flags); + unsigned long flags, struct vm_area_struct *vma, + bool shared); static nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *pol, pgoff_t ilx, int *nid); =20 @@ -518,7 +519,8 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk= *walk) return; if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || !vma_migratable(walk->vma) || - !migrate_folio_add(folio, qp->pagelist, qp->flags)) + !migrate_folio_add(folio, qp->pagelist, qp->flags, walk->vma, + is_shared_pmd(pmd, walk->vma))) qp->nr_failed++; } =20 @@ -543,7 +545,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned = long addr, pte_t ptent; spinlock_t *ptl; =20 - ptl =3D pmd_trans_huge_lock(pmd, vma); + ptl =3D pmd_huge_lock(pmd, vma); if (ptl) { queue_folios_pmd(pmd, walk); spin_unlock(ptl); @@ -598,7 +600,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned = long addr, } if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || !vma_migratable(vma) || - !migrate_folio_add(folio, qp->pagelist, flags)) { + !migrate_folio_add(folio, qp->pagelist, flags, vma, false)) { qp->nr_failed++; if (strictly_unmovable(flags)) break; @@ -1025,8 +1027,11 @@ static long do_get_mempolicy(int *policy, nodemask_t= *nmask, =20 #ifdef CONFIG_MIGRATION static bool migrate_folio_add(struct folio *folio, struct list_head *folio= list, - unsigned long flags) + unsigned long flags, struct vm_area_struct *vma, + bool shared) { + bool ret =3D true; + bool is_hugetlb =3D is_vm_hugetlb_page(vma); /* * Unless MPOL_MF_MOVE_ALL, we try to avoid migrating a shared folio. * Choosing not to migrate a shared folio is not counted as a failure. @@ -1034,23 +1039,27 @@ static bool migrate_folio_add(struct folio *folio, = struct list_head *foliolist, * See folio_likely_mapped_shared() on possible imprecision when we * cannot easily detect if a folio is shared. */ - if ((flags & MPOL_MF_MOVE_ALL) || !folio_likely_mapped_shared(folio)) { - if (folio_isolate_lru(folio)) { - list_add_tail(&folio->lru, foliolist); - node_stat_mod_folio(folio, - NR_ISOLATED_ANON + folio_is_file_lru(folio), - folio_nr_pages(folio)); - } else { + if ((flags & MPOL_MF_MOVE_ALL) || + (!folio_likely_mapped_shared(folio) && !shared)) { + if (is_hugetlb) + return isolate_hugetlb(folio, foliolist); + + ret =3D folio_isolate_lru(folio); + if (!ret) /* * Non-movable folio may reach here. And, there may be * temporary off LRU folios or non-LRU movable folios. * Treat them as unmovable folios since they can't be * isolated, so they can't be moved at the moment. */ - return false; - } + return ret; + + list_add_tail(&folio->lru, foliolist); + node_stat_mod_folio(folio, + NR_ISOLATED_ANON + folio_is_file_lru(folio), + folio_nr_pages(folio)); } - return true; + return ret; } =20 /* @@ -1239,7 +1248,8 @@ static struct folio *alloc_migration_target_by_mpol(s= truct folio *src, #else =20 static bool migrate_folio_add(struct folio *folio, struct list_head *folio= list, - unsigned long flags) + unsigned long flags, struct vm_area_struct *vma, + bool shared) { return false; } --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B33291448D2 for ; Thu, 4 Jul 2024 04:32:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067542; cv=none; b=dIW5E9Q88mH/BAK4/abKimmOaz+i065YJGGmWFJ3GoFzaDVGUToIR+2aOSu/Iaz+tO4c+RAHmUj2J6o/sxMs2a81fywTY9/dlEpEohtbjZQtwuqXQB6Xz3Z0C890EQMNE4ir/65DrN40LYDWpnRiFH3Em7ZsnWtFX5YKJON+608= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067542; c=relaxed/simple; bh=n8CwSRarq4ActNGgFQhYTPVt1+MvFBSEx/o9wxAsuUM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iOeY6lPaZPwL5ZTr3coj5j4UVIgUBjsV4H/z+sfpiEAw1LGaLuBIb12BIwLjO/Iw23E/thA6mcmUVnu8hduCGZ3RmJsKZk0DXBDJMk5FhCOvxCDFc6CQ0i1AiwwH7NIDteOCqUnjattU6AiI0klqbVaORpFSclkEanQcXPdqeag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2B07D21C0D; Thu, 4 Jul 2024 04:32:19 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 06B6F13889; Thu, 4 Jul 2024 04:32:17 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id cMqbONElhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:17 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 23/45] mm/mempolicy: Create queue_folios_pud to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:10 +0200 Message-ID: <20240704043132.28501-24-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 2B07D21C0D X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create queue_folios_pud in order to handle PUD-mapped hugetlb vmas. Also implement is_pud_migration_entry and pud_folio, as they will be used i= n this patch. Signed-off-by: Oscar Salvador --- include/linux/pgtable.h | 1 + include/linux/swapops.h | 12 ++++++++++++ mm/mempolicy.c | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 458e3cbc96b2..23d51fec81ac 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -51,6 +51,7 @@ #endif =20 #define pmd_folio(pmd) page_folio(pmd_page(pmd)) +#define pud_folio(pud) page_folio(pud_page(pud)) =20 /* * A page table page can be thought of an array like this: pXd_t[PTRS_PER_= PxD] diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 182957f0d013..a23900961d11 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -542,6 +542,18 @@ static inline bool is_pfn_swap_entry(swp_entry_t entry) =20 struct page_vma_mapped_walk; =20 +#ifdef CONFIG_HUGETLB_PAGE +static inline int is_pud_migration_entry(pud_t pud) +{ + return is_swap_pud(pud) && is_migration_entry(pud_to_swp_entry(pud)); +} +#else +static inline int is_pud_migration_entry(pud_t pud) +{ + return 0; +} +#endif + #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION extern int set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw, struct page *page); diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 5baf29da198c..93b14090d484 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -501,6 +501,37 @@ static inline bool queue_folio_required(struct folio *= folio, return node_isset(nid, *qp->nmask) =3D=3D !(flags & MPOL_MF_INVERT); } =20 +static int queue_folios_pud(pud_t *pud, unsigned long addr, unsigned long = end, + struct mm_walk *walk) +{ + spinlock_t *ptl; + struct folio *folio; + struct vm_area_struct *vma =3D walk->vma; + struct queue_pages *qp =3D walk->private; + + ptl =3D pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + if (unlikely(is_pud_migration_entry(*pud))) { + qp->nr_failed++; + goto out; + } + folio =3D pud_folio(*pud); + if (!queue_folio_required(folio, qp)) + goto out; + if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || + !vma_migratable(walk->vma) || + !migrate_folio_add(folio, qp->pagelist, qp->flags, walk->vma, false)) + qp->nr_failed++; + + spin_unlock(ptl); +out: + if (qp->nr_failed && strictly_unmovable(qp->flags)) + return -EIO; + return 0; +} + static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) { struct folio *folio; @@ -730,6 +761,7 @@ static int queue_pages_test_walk(unsigned long start, u= nsigned long end, =20 static const struct mm_walk_ops queue_pages_walk_ops =3D { .hugetlb_entry =3D queue_folios_hugetlb, + .pud_entry =3D queue_folios_pud, .pmd_entry =3D queue_folios_pte_range, .test_walk =3D queue_pages_test_walk, .walk_lock =3D PGWALK_RDLOCK, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32747144D15 for ; Thu, 4 Jul 2024 04:32:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067543; cv=none; b=leO7vEf4Ljb7X5acrfZrrUVvMDfBqbSGiyNsWfuZVioBCysOS8ja35P9cvKOZ4lQkaY/hBQGTPV17chlJhff5e2xcEG/n30ckrFqocAJxJE++/fQMMzo7Oe5ra3kl5MgLUOYMdIiEJ32c6q1YiJ6YqB+M4gWtpgh0SPAoS9XM5k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067543; c=relaxed/simple; bh=EXDhJ09O2OJ9GtxQDINsaKUIPFQVmSM1oh/3/rS4he4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OKEZuhScsS5tVplstysaP1qBX1D3z+XMfz1IaZhEzg14Ij7OTHVbpns2qNP0N33EsaJ79jCRjcrvjkJuGbFq9YZEIfNDcJzGhKQ2mRso5RHQFaW7CIPx5+r+nNcl4/ebgwDr/FcJ+7Ta0pfPijQj+bDRm/iuGjUlcUhNZzSCGwA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=ZZlPRFX8; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=DZydrxp6; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=ZZlPRFX8; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=DZydrxp6; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="ZZlPRFX8"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="DZydrxp6"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="ZZlPRFX8"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="DZydrxp6" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 6CA131FCFE; Thu, 4 Jul 2024 04:32:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=ZZlPRFX8HPVeyLyltxoPjntrB3vyrMGorhnSIvawEhprDCtQiJ8kvY1aBNeBJMEnFNoka7 zEhX5cg2k/xkhsjTOrqeSfEe1lZe8CG3PQEh3+8UdhDAt9ff/QXS9LBS82y+sKrbT8fyYn t7/3YM8I8df36X7okK3RGPJBJuuuMoQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=DZydrxp6It5HlKS3mNh//5MUdSN05oCnIEplA26v4hntyU+xxNFeMDdkBpvi66dQmJHnJM /gkrpKtXKANlwKBg== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=ZZlPRFX8HPVeyLyltxoPjntrB3vyrMGorhnSIvawEhprDCtQiJ8kvY1aBNeBJMEnFNoka7 zEhX5cg2k/xkhsjTOrqeSfEe1lZe8CG3PQEh3+8UdhDAt9ff/QXS9LBS82y+sKrbT8fyYn t7/3YM8I8df36X7okK3RGPJBJuuuMoQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067540; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbU3NixA+ORa4pbwAalk6/qaoqPhUeSqtYDCOzcSsu0=; b=DZydrxp6It5HlKS3mNh//5MUdSN05oCnIEplA26v4hntyU+xxNFeMDdkBpvi66dQmJHnJM /gkrpKtXKANlwKBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5045F13889; Thu, 4 Jul 2024 04:32:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +B4rDtMlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:19 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 24/45] mm/memory_failure: Enable check_hwpoisoned_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:11 +0200 Message-ID: <20240704043132.28501-25-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also check_hwpoisoned_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/memory-failure.c | 44 ++++++++++++++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 0cb1b7bea9a5..8cae95e36365 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -771,27 +771,43 @@ static int check_hwpoisoned_entry(pte_t pte, unsigned= long addr, short shift, return 1; } =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int check_hwpoisoned_pmd_entry(pmd_t *pmdp, unsigned long addr, - struct hwpoison_walk *hwp) + struct hwpoison_walk *hwp, + struct vm_area_struct *vma) { pmd_t pmd =3D *pmdp; unsigned long pfn; - unsigned long hwpoison_vaddr; + unsigned short shift; + unsigned long hwpoison_vaddr =3D addr; =20 - if (!pmd_present(pmd)) - return 0; - pfn =3D pmd_pfn(pmd); - if (pfn <=3D hwp->pfn && hwp->pfn < pfn + HPAGE_PMD_NR) { - hwpoison_vaddr =3D addr + ((hwp->pfn - pfn) << PAGE_SHIFT); - set_to_kill(&hwp->tk, hwpoison_vaddr, PAGE_SHIFT); - return 1; + if (pmd_present(pmd)) { + pfn =3D pmd_pfn(pmd); + } else { + swp_entry_t swp =3D pmd_to_swp_entry(pmd); + + if (!is_hwpoison_entry(swp)) + return 0; + pfn =3D swp_offset_pfn(swp); } - return 0; + + shift =3D is_vm_hugetlb_page(vma) ? huge_page_shift(hstate_vma(vma)) + : PAGE_SHIFT; + + if (pfn > hwp->pfn || hwp->pfn > pfn + HPAGE_PMD_NR) + return 0; + + if (!is_vm_hugetlb_page(vma)) + hwpoison_vaddr +=3D (hwp->pfn - pfn) << PAGE_SHIFT; + + set_to_kill(&hwp->tk, hwpoison_vaddr, shift); + + return 1; } #else static int check_hwpoisoned_pmd_entry(pmd_t *pmdp, unsigned long addr, - struct hwpoison_walk *hwp) + struct hwpoison_walk *hwp, + struct vm_area_struct *vma) { return 0; } @@ -805,9 +821,9 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned lon= g addr, pte_t *ptep, *mapped_pte; spinlock_t *ptl; =20 - ptl =3D pmd_trans_huge_lock(pmdp, walk->vma); + ptl =3D pmd_huge_lock(pmdp, walk->vma); if (ptl) { - ret =3D check_hwpoisoned_pmd_entry(pmdp, addr, hwp); + ret =3D check_hwpoisoned_pmd_entry(pmdp, addr, hwp, walk->vma); spin_unlock(ptl); goto out; } --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A008145321 for ; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067544; cv=none; b=se85nRSlXCcCatiV407qPoRrrb6skKxT2N2STrQgQbC1MUkVWJ97MbsUX5qWGPHN3FPalbL/2PI3kBj9U2dKsEqQcw1wDDwbh93KTyObJqQBgyLZqB3uDlvAE6FBuFsOzdFjyFhFEKNxUJ/dxJSB8uykjCQtK3UepvFk65dFP8o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067544; c=relaxed/simple; bh=bGPa2HcHFWxQQ0CcFTQOgWln/PaFjOUQol7Z0orXRkM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eSNnjYVc2x3ozlUeg7PPdN41uuikUg4LfqF1MOSSbSAXEEvEHjcXtEQNU4yuvfhmOqoxMsidPVDd3vJ9X7R46JX4KXnhiCx05vlJix5i5YraCqPVVXj8wURky6dEXVBI6RlMS29df73jcOUly13r+dTX4+UeqtJZcUMuV7yXi7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id ABF721FD00; Thu, 4 Jul 2024 04:32:21 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8D48513889; Thu, 4 Jul 2024 04:32:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id sKv0HdQlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:20 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 25/45] mm/memory-failure: Create check_hwpoisoned_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:12 +0200 Message-ID: <20240704043132.28501-26-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: ABF721FD00 X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create check_hwpoisoned_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/memory-failure.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 8cae95e36365..622862c4c300 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -771,6 +771,43 @@ static int check_hwpoisoned_entry(pte_t pte, unsigned = long addr, short shift, return 1; } =20 +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES +static int hwpoison_pud_range(pud_t *pudp, unsigned long addr, + unsigned long end, struct mm_walk *walk) +{ + int ret =3D 0; + spinlock_t *ptl; + pud_t pud =3D *pudp; + unsigned long pfn; + struct hwpoison_walk *hwp =3D walk->private; + + ptl =3D pud_huge_lock(pudp, walk->vma); + if (!ptl) + return 0; + + if (pud_present(pud)) { + pfn =3D pud_pfn(pud); + } else { + swp_entry_t swp =3D pud_to_swp_entry(pud); + + if (!is_hwpoison_entry(swp)) + goto out_unlock; + pfn =3D swp_offset_pfn(swp); + } + + if (!pfn || pfn !=3D hwp->pfn) + goto out_unlock; + + set_to_kill(&hwp->tk, addr, huge_page_shift(hstate_vma(walk->vma))); + ret =3D 1; +out_unlock: + spin_unlock(ptl); + return ret; +} +#else +hwpoison_pud_range NULL +#endif + #ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int check_hwpoisoned_pmd_entry(pmd_t *pmdp, unsigned long addr, struct hwpoison_walk *hwp, @@ -862,6 +899,7 @@ static int hwpoison_hugetlb_range(pte_t *ptep, unsigned= long hmask, #endif =20 static const struct mm_walk_ops hwpoison_walk_ops =3D { + .pud_entry =3D hwpoison_pud_range, .pmd_entry =3D hwpoison_pte_range, .hugetlb_entry =3D hwpoison_hugetlb_range, .walk_lock =3D PGWALK_RDLOCK, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBB03145347 for ; Thu, 4 Jul 2024 04:32:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067546; cv=none; b=t/0P+V1tpeTem7PbHjDRv1SDI3hm/zIkIDz7bApBWbOvEBJH1vZxPtvsBvSewduZ2oVLNI8bSJrlmpy4O8Lr7FQdPtoSl9xnFvTMcc5txTJLnIayIDPj9Fgkl38FCJiIazpWHfkQYiI2W3odbvie+ypQB3iT6lbIfrMr337mqBI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067546; c=relaxed/simple; bh=tdPuf/5twvbRvm8RjGKcz4r5SkWuHK/Bp9g/YU26wy8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=o9zVvzes+thPiMIhUQYzvfbLqaIniXfeje0Z3qhzCnpM5fp2hApwMjCa7EUmj3DDZjJruBEorceeZS3fhyKku2cIbuq4CTSer4mAsmNKztPKQDcMNax+fAteteQHL/ChRm3yowO0SL18WHRrCZkXnmtObfcMKPLVbD51/xSgxRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 09ECD21180; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CD1EE13889; Thu, 4 Jul 2024 04:32:21 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UADILdUlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:21 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 26/45] mm/damon: Enable damon_young_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:13 +0200 Message-ID: <20240704043132.28501-27-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 09ECD21180 X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach damon_young_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/damon/vaddr.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 58829baf8b5d..00d32beffe38 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -443,30 +443,35 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned= long addr, struct folio *folio; struct damon_young_walk_private *priv =3D walk->private; =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (pmd_trans_huge(pmdp_get(pmd))) { - pmd_t pmde; - - ptl =3D pmd_lock(walk->mm, pmd); - pmde =3D pmdp_get(pmd); +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + ptl =3D pmd_huge_lock(vma, pmd); + if (ptl) { + unsigned long pfn; =20 - if (!pmd_present(pmde)) { + if (!pmd_present(*pmd)) { spin_unlock(ptl); return 0; } =20 - if (!pmd_trans_huge(pmde)) { - spin_unlock(ptl); - goto regular_page; + pfn =3D pmd_pfn(*pmd); + if (is_vm_hugetlb_page(walk->vma)) { + folio =3D pfn_folio(pfn); + if (folio) + folio_get(folio); + } else { + folio =3D damon_get_folio(pfn); } - folio =3D damon_get_folio(pmd_pfn(pmde)); if (!folio) goto huge_out; if (pmd_young(pmde) || !folio_test_idle(folio) || mmu_notifier_test_young(walk->mm, addr)) priv->young =3D true; - *priv->folio_sz =3D HPAGE_PMD_SIZE; + + if (is_vm_hugetlb_page(walk->vma)) + *priv->folio_sz =3D huge_page_size(h); + else + *priv->folio_sz =3D HPAGE_PMD_SIZE; folio_put(folio); huge_out: spin_unlock(ptl); @@ -474,7 +479,7 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned l= ong addr, } =20 regular_page: -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ =20 pte =3D pte_offset_map_lock(walk->mm, pmd, addr, &ptl); if (!pte) { --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABF9F1459E6 for ; Thu, 4 Jul 2024 04:32:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067547; cv=none; b=k0PuzpWUDI3SwesMdwVDehaHD3KlH1rFXWEe6nxptwzrNLass+HNMTru3SPnbMO8oWFlL/iBIZoyxU5pn7EM0CeqsFN5Dc2ev69GJKaAC0a6mQXwLEUlGe3UpwgxVvBqbACQ6ylSknVbSunrN4JHdpX0nV31RRUAkqsrL5KjrGQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067547; c=relaxed/simple; bh=DDM1CyCr7sw5ilryaMX/F+7wIWAeswYRZs0JW6CsuOk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=joUxN9BIxbOel/k6OzjgP5cg/LYTLjC3gmhYzBxoneh7tD/azqw9EnjMbD899Pb9dWN7EYWQZO3Ju0/YpH9iVcQh8zl1FbyDrKaot2/MVbgB771iUSdfrtvpj8WJyCAsLOyAMdJJtbbUr3hH2EBHHNO4hut6Wp5k7rnVeQACe4o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 41E2C1FCF8; Thu, 4 Jul 2024 04:32:24 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 22FA613889; Thu, 4 Jul 2024 04:32:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YJuNA9clhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:23 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 27/45] mm/damon: Create damon_young_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:14 +0200 Message-ID: <20240704043132.28501-28-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 41E2C1FCF8 X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create damon_young_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/damon/vaddr.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 00d32beffe38..2d5ad47b9dae 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -434,6 +434,39 @@ struct damon_young_walk_private { bool young; }; =20 +static int damon_young_pud_entry(pmd_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + spinlock_t *ptl; + struct folio *folio; + struct damon_young_walk_private *priv =3D walk->private; + + ptl =3D pud_huge_lock(vma, pud); + if (!ptl) + return 0; + + if (!pud_present(*pud)) + goto out; + + folio =3D pfn_folio(pud_pfn(*pud)); + if (!folio) + goto out; + folio_get(folio); + + if (pud_young(pmde) || !folio_test_idle(folio) || + mmu_notifier_test_young(walk->mm, addr)) + priv->young =3D true; + + *priv->folio_sz =3D huge_page_size(h); + folio_put(folio); +out: + spin_unlock(ptl); +#endif + return 0; +} + + static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long next, struct mm_walk *walk) { @@ -537,6 +570,7 @@ static int damon_young_hugetlb_entry(pte_t *pte, unsign= ed long hmask, #endif /* CONFIG_HUGETLB_PAGE */ =20 static const struct mm_walk_ops damon_young_ops =3D { + .pud_entry =3D damon_young_pud_entry, .pmd_entry =3D damon_young_pmd_entry, .hugetlb_entry =3D damon_young_hugetlb_entry, .walk_lock =3D PGWALK_RDLOCK, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 537BE145B01 for ; Thu, 4 Jul 2024 04:32:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067548; cv=none; b=emcTr2karXgXeuZ6ugx4Kt9F3k3N39loN7MsTXrny2uJid4tWETfMs4ppUnSOgiVkno/SWkVZATzpcu82guRqIFNAMoXp23KzAeu0cNsbfDQpFlIBgxbKhnZFgrlHJSRAzTiBqrm2QPmAFso8ctRh4KUSKUY2xkGIiZNH0GUbzs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067548; c=relaxed/simple; bh=IjVgFFhoVqDdp8QfTsBGLnFIesfrSQbxw+JNIYiTa5o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T7cwLkmE+7YykkxZ5MWryZMc3VoQAQi4nKp0p+/eeBuX2SVBh36Lv0b4OdEYMYc7HvKMzp0COZAvrmHf0kM9AfcAENH2MwffesJXhLjadCe1ZRYdwOpv2sReAirQNJ1cm2PevgDqQ4zAhlPbOMCbxKSNAdYEU9tQ+eFrc/+07EA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 94F3A21C10; Thu, 4 Jul 2024 04:32:25 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6893D13889; Thu, 4 Jul 2024 04:32:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id EK8YFNglhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:24 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 28/45] mm/damon: Enable damon_mkold_pmd_entry to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:15 +0200 Message-ID: <20240704043132.28501-29-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 94F3A21C10 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach damon_mkold_pmd_entry. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/damon/ops-common.c | 21 ++++++++++++++++----- mm/damon/vaddr.c | 15 +++++---------- 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index d25d99cb5f2b..6727658a3ef5 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -53,18 +53,29 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct= *vma, unsigned long addr =20 void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned lon= g addr) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - struct folio *folio =3D damon_get_folio(pmd_pfn(pmdp_get(pmd))); +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + struct folio *folio; + unsigned long size; + + if (is_vm_hugetlb_page(vma)) { + folio =3D pfn_folio(pdm_pfn(*pmd)) + folio_get(folio); + size =3D huge_page_size(hstate_vma(vma)); + } else { + folio =3D damon_get_folio(pmd_pfn(*pmd)); + size =3D PMD_SIZE; + } =20 if (!folio) - return; + return 0; =20 - if (pmdp_clear_young_notify(vma, addr, pmd)) + if (pmdp_test_and_clear_young(vma, addr, pmd) || + mmu_notifier_clear_young(mm, addr, addr + size)) folio_set_young(folio); =20 folio_set_idle(folio); folio_put(folio); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /*CONFIG_PGTABLE_HAS_HUGE_LEAVES */ } =20 #define DAMON_MAX_SUBSCORE (100) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 2d5ad47b9dae..47c84cdda32c 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -304,21 +304,16 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned= long addr, pmd_t pmde; spinlock_t *ptl; =20 - if (pmd_trans_huge(pmdp_get(pmd))) { - ptl =3D pmd_lock(walk->mm, pmd); - pmde =3D pmdp_get(pmd); - - if (!pmd_present(pmde)) { + ptl =3D pmd_huge_lock(walk->vma, pmd); + if (ptl) { + if (!pmd_present(*pmd)) { spin_unlock(ptl); return 0; } =20 - if (pmd_trans_huge(pmde)) { - damon_pmdp_mkold(pmd, walk->vma, addr); - spin_unlock(ptl); - return 0; - } + damon_pmdp_mkold(pmd, walk->vma, addr); spin_unlock(ptl); + return 0; } =20 pte =3D pte_offset_map_lock(walk->mm, pmd, addr, &ptl); --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86443145B27 for ; Thu, 4 Jul 2024 04:32:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067550; cv=none; b=ryBi5CZygoC6V4mmmvBPktk6jtTIZ4aZJdsqeNa5adnN9Hp+qaSZABwOfheKPqoU32RCFK61+WkAQ17bNPinrsEVBUzd1eluCaLwgRNeTNMOo53uySscsENCmA0E6OewPYFqnvcoFQytY7/tzdICuGXahalRL2IqTbcVyh5XOIc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067550; c=relaxed/simple; bh=yGVKT+2M/8fGKrJ24xpO8G0WwuoeWGAspio7/eAfg+8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IBDefR2KSfm4eQu1kxuCS2GFW3B4NWih94nMc6vz1biWpJgRxYyNELPyz2ehar/q//CnCJ5l7HGO9cHCzFFOOywacGFwyXQdhQeD6UJrNoC2DYtmxp4cT3DnxGeuuvlXq1YScAN9+e56/TFmOHjWaXURlP3vW0fnuKK16BuaUHQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=vi5KLgdE; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=PNL9w7HV; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=vi5KLgdE; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=PNL9w7HV; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="vi5KLgdE"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="PNL9w7HV"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="vi5KLgdE"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="PNL9w7HV" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id BBF8521C0D; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=vi5KLgdENdkrnyeebmRYr1GTUMnsDQhLSyx0si8djR9aeKoZAFXDbChioD9oLU5n2WEp19 sZ/1YXnUx4zsqmQaqBABY+2OXBM8pE0ZclE7Xl4ENMKN82fsQA0jjs9APgzard6WS/Yweg oOQsVALT38rdOWZXcSnlqZBgxaDhVAo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=PNL9w7HV1zFYjASnMOA/fooIqae95dSIt4v71VfIDGS0e9g8E4lsal0N9CyLMaXb/wifCJ M4zsefGx9x+meuCg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=vi5KLgdENdkrnyeebmRYr1GTUMnsDQhLSyx0si8djR9aeKoZAFXDbChioD9oLU5n2WEp19 sZ/1YXnUx4zsqmQaqBABY+2OXBM8pE0ZclE7Xl4ENMKN82fsQA0jjs9APgzard6WS/Yweg oOQsVALT38rdOWZXcSnlqZBgxaDhVAo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067546; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+Eaq4YS+I2JMNj2PlLYmITmw1dUtdxAYyfPBt6d+VGg=; b=PNL9w7HV1zFYjASnMOA/fooIqae95dSIt4v71VfIDGS0e9g8E4lsal0N9CyLMaXb/wifCJ M4zsefGx9x+meuCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id ACDBD13889; Thu, 4 Jul 2024 04:32:25 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WC9sJdklhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:25 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 29/45] mm/damon: Create damon_mkold_pud_entry to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:16 +0200 Message-ID: <20240704043132.28501-30-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create damon_mkold_pud_entry in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/damon/vaddr.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 47c84cdda32c..6a383ce5a775 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -329,6 +329,37 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned = long addr, return 0; } =20 +static int damon_mkold_pud_entry(pmd_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + spinlock_t *ptl; + struct folio *folio; + struct vm_area_struct *vma =3D walk->vma; + unsigned long size =3D huge_page_size(hstate_vma(vma)); + + ptl =3D pud_huge_lock(vma, pud); + if (!ptl) + return 0; + + if (!pud_present(*pud)) + goto out; + + folio =3D pfn_folio(pud_pfn(*pud)); + folio_get(folio); + + if (pudp_test_and_clear_young(vma, addr, pud) || + mmu_notifier_clear_young(mm, addr, addr + size)) + folio_set_young(folio); + + folio_set_idle(folio); + folio_put(folio); +out: + spin_unlock(ptl); +#endif + return 0; +} + #ifdef CONFIG_HUGETLB_PAGE static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr) @@ -383,6 +414,7 @@ static int damon_mkold_hugetlb_entry(pte_t *pte, unsign= ed long hmask, #endif /* CONFIG_HUGETLB_PAGE */ =20 static const struct mm_walk_ops damon_mkold_ops =3D { + .pud_entry =3D damon_mkold_pud_entry, .pmd_entry =3D damon_mkold_pmd_entry, .hugetlb_entry =3D damon_mkold_hugetlb_entry, .walk_lock =3D PGWALK_RDLOCK, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5516145FE4 for ; Thu, 4 Jul 2024 04:32:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067551; cv=none; b=HgUdtnSxelfWlEkrOADXzzkgpPaxgx+kDa4GYY7i3Nacg+SEwn2FDladquXvfJG2Lqqv7oziXi2eNoI9yiN61gOWyms6SrNJeDomM7F02l9ypmAk49Io+OwgeypyUSmkbhbzxPniZ0e9y6xg1q7XSQVTx5VyYAj9ddTxozYBiBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067551; c=relaxed/simple; bh=8hDZyNTIFcoLrIplaNZzSjI6izOqLgxUhvo8Qa79pnc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bh2LUTjMk7dwmdJLGV2huW5diwa2V4e20jKL49caSzEF7PRrN/RYJ+ywhNIF6tW0dvgAMhh/XrmlTLrWAJ3Od8qS43vpglxnOOHAj8KMVHSFBrJLp2EzMygVCdMSksYG7ofGcmahC7WNF+1IiA15oR3zduQBKILQG32inHAub5Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0FE1121C11; Thu, 4 Jul 2024 04:32:28 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DDC9613889; Thu, 4 Jul 2024 04:32:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qGv2MdolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:26 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 30/45] mm,mincore: Enable mincore_pte_range to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:17 +0200 Message-ID: <20240704043132.28501-31-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 0FE1121C11 X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also mincore_pte_range. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/mincore.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/mincore.c b/mm/mincore.c index d6bd19e520fc..5154bc705f60 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 #include #include "swap.h" @@ -106,8 +107,9 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, unsigned char *vec =3D walk->private; int nr =3D (end - addr) >> PAGE_SHIFT; =20 - ptl =3D pmd_trans_huge_lock(pmd, vma); + ptl =3D pmd_huge_lock(pmd, vma); if (ptl) { + /* Better handling of hugetlb is required (pte marker etc.) */ memset(vec, 1, nr); spin_unlock(ptl); goto out; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0C6F14658A for ; Thu, 4 Jul 2024 04:32:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067553; cv=none; b=kXlNGguLhShlhraBkPvdsXd+Fl6rEBmmq8SLN66+4kk6PPLCBC/gENjlH1EPpJknIsiIjl4AxdKf3/uct2s2PYcuPExpfT3UeCCR+CBjNkfViiR0X/kdDXqOI++8VlRnjdtBvO52C/TWrSsNwfk/XCFK/fdwT5+Y9yr1u3zyBms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067553; c=relaxed/simple; bh=IuTB4L1ipAkwfZoHCTo5/wLkUzpL+5baXkSV6FyuYB4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hee+n9qf6i7KtQxR1ERse5uFgnI7QoyRlH9Mp44lzNkTrv+kExchHe5aJ/4/tIFLiYLcW4OK7Hx1qqcuTggPf8IsGCPKHU661n9dtNReLzbHGDXyz1DJMNih71U1Q2xmPlUSSYXOHCLMKLLEwLfxzA6OnWm6nEWgH46lvvFnOQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=QpIyFU+S; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=udDCKM8V; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=QpIyFU+S; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=udDCKM8V; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="QpIyFU+S"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="udDCKM8V"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="QpIyFU+S"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="udDCKM8V" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5024D21C12; Thu, 4 Jul 2024 04:32:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=QpIyFU+SNEDWpoiZMvudIRFmIbkNPlPQAdHO8szaEcTZHDv7mQap7dxr0MNsVTThx2cHcx jhq/PHrBd4eT+7TOlVKBNgQLFqsLoSzcymO1iD3cWeXTiS0+9wXL7guAJDapk88hwYPbsN UXGfJEukOZwgFWGHbc1yAGUFbMmlXQI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=udDCKM8VSh5fm40PS9NUBV30hL3Hc21Fph1pBda42Ie9OAKZOIyY5YZKKy4Vasv/sNi9T4 b+SyyOfTIIs5xDDg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=QpIyFU+SNEDWpoiZMvudIRFmIbkNPlPQAdHO8szaEcTZHDv7mQap7dxr0MNsVTThx2cHcx jhq/PHrBd4eT+7TOlVKBNgQLFqsLoSzcymO1iD3cWeXTiS0+9wXL7guAJDapk88hwYPbsN UXGfJEukOZwgFWGHbc1yAGUFbMmlXQI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hlyc02hswOFBPpFjPMKLH2SPKevIA4btQnEnfz8PzX8=; b=udDCKM8VSh5fm40PS9NUBV30hL3Hc21Fph1pBda42Ie9OAKZOIyY5YZKKy4Vasv/sNi9T4 b+SyyOfTIIs5xDDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3048813889; Thu, 4 Jul 2024 04:32:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kBsMB9wlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:28 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 31/45] mm/mincore: Create mincore_pud_range to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:18 +0200 Message-ID: <20240704043132.28501-32-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so create mincore_pud_range in order to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/mincore.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/mm/mincore.c b/mm/mincore.c index 5154bc705f60..786df7246899 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -98,6 +98,25 @@ static int mincore_unmapped_range(unsigned long addr, un= signed long end, return 0; } =20 +static int mincore_pud_range(pud_t *pud, unsigned long addr, unsigned long= end, + struct mm_walk *walk) +{ +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES + spinlock_t *ptl; + unsigned char *vec =3D walk->private; + int nr =3D (end - addr) >> PAGE_SHIFT; + struct vm_area_struct *vma =3D walk->vma; + + ptl =3D pud_huge_lock(pud, vma); + if (!ptl) + return 0; + + memset(vec, 1, nr); + spin_unlock(ptl); +#endif + return 0; +} + static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long= end, struct mm_walk *walk) { @@ -175,6 +194,7 @@ static inline bool can_do_mincore(struct vm_area_struct= *vma) } =20 static const struct mm_walk_ops mincore_walk_ops =3D { + .pud_entry =3D mincore_pud_range, .pmd_entry =3D mincore_pte_range, .pte_hole =3D mincore_unmapped_range, .hugetlb_entry =3D mincore_hugetlb, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2415D146A60 for ; Thu, 4 Jul 2024 04:32:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067554; cv=none; b=A9nintOcaeqiBaZDG1nmkCmchdnnY644KtjCG31ySJWShtj5gqywfA6BGmxLEkoMH1+XlwEB+PvzfvEmLH7JO3OBmt/Q9ILo5Nr5zytouopdjEQY0615FyI8ge1Iw9+zPti7+Tv0pPKD1X/qVdd1IBL4doZWYJNCDTtGC9P1E2A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067554; c=relaxed/simple; bh=r9CxWPDak7WYAGHDmIZDqEsshTEdomdt6C4UzsZPCIg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DSuHXjmLRHigsRYoQC8+lMz7vHXNOqoqgb2bgBbOek8F7XVZas59ce2SNsD4garDI6tOfpGAEP2PmBZWcfBF8pXmGXGqhC7UuqJghtRfO8fH2FLo5cs5fiaTzyvoFqpTpjo2FV/I9HFDUxtDKWCPHGHx2nSK5QobqyRJ8MWT9/I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8B3D621180; Thu, 4 Jul 2024 04:32:30 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7235D13889; Thu, 4 Jul 2024 04:32:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0HweF90lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:29 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 32/45] mm/hmm: Enable hmm_vma_walk_pmd, to handle hugetlb vmas Date: Thu, 4 Jul 2024 06:31:19 +0200 Message-ID: <20240704043132.28501-33-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 8B3D621180 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" PMD-mapped hugetlb vmas will also reach hmm_vma_walk_pmd. Add the required code so it knows how to handle those there. Signed-off-by: Oscar Salvador --- mm/hmm.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 7e0229ae4a5a..fbee08973544 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -183,7 +183,7 @@ static inline unsigned long pmd_to_hmm_pfn_flags(struct= hmm_range *range, hmm_pfn_flags_order(PMD_SHIFT - PAGE_SHIFT); } =20 -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, unsigned long end, unsigned long hmm_pfns[], pmd_t pmd) @@ -206,11 +206,11 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, u= nsigned long addr, hmm_pfns[i] =3D pfn | cpu_flags; return 0; } -#else /* CONFIG_TRANSPARENT_HUGEPAGE */ +#else /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ /* stub to allow the code below to compile */ int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, unsigned long end, unsigned long hmm_pfns[], pmd_t pmd); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */ =20 static inline unsigned long pte_to_hmm_pfn_flags(struct hmm_range *range, pte_t pte) @@ -336,7 +336,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, -1, walk); =20 - if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { + if (is_pmd_migration_entry(pmd)) { if (hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0)) { hmm_vma_walk->last =3D addr; pmd_migration_entry_wait(walk->mm, pmdp); @@ -351,7 +351,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return hmm_pfns_fill(start, end, range, HMM_PFN_ERROR); } =20 - if (pmd_devmap(pmd) || pmd_trans_huge(pmd)) { + if (pmd_devmap(pmd) || pmd_leaf(pmd)) { /* * No need to take pmd_lock here, even if some other thread * is splitting the huge pmd we will get that event through --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BDB745000 for ; Thu, 4 Jul 2024 04:32:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067555; cv=none; b=USfFBnTCr/7GvtSue3lImn8TKPVpWB9+kyD/JDFmMzfs8JxMifGr29yxdCTJoWN9IcCQktvEkx0yZfaCfham8PETUDjulxZ4SGUX1js84TyTl8fIRKelYz88Kfx7nSAKoCVt2ghGqHZpHd+QhxEJhDuUf0GaiBllKF3NOmAhbLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067555; c=relaxed/simple; bh=b9RoC8BUyBgraGZurHs62mAW314Bvdf9N/tMXnfaIpA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p4sJHlLurCBtvP901lhly2e9pKIzLkNzeE7KxAA+n70Iuohx+itHSXTI5hoc+hYqpnYGPfqmjvjamQN8S3Okr8/oKElJqvs91rU+m6dbGoGQjMw4UpcMjzttq+4TvHkH4F4QAD51mU3Rr8ckwxx0eswZUTmvop844kw3HoY/H7M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D6F111FCF7; Thu, 4 Jul 2024 04:32:31 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B441C13889; Thu, 4 Jul 2024 04:32:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 8OBhJ94lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:30 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 33/45] mm/hmm: Enable hmm_vma_walk_pud to handle PUD-mapped hugetlb vmas Date: Thu, 4 Jul 2024 06:31:20 +0200 Message-ID: <20240704043132.28501-34-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: D6F111FCF7 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Normal THP cannot be PUD-mapped (besides devmap), but hugetlb can, so enable hmm_vma_walk_pud to handle PUD-mapped hugetlb vmas. Signed-off-by: Oscar Salvador --- mm/hmm.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index fbee08973544..2b752f703b6d 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -396,8 +396,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, return 0; } =20 -#if defined(CONFIG_ARCH_HAS_PTE_DEVMAP) && \ - defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) +#if (defined(CONFIG_ARCH_HAS_PTE_DEVMAP) || defined (CONFIG_PGTABLE_HAS_HU= GE_LEAVES)) static inline unsigned long pud_to_hmm_pfn_flags(struct hmm_range *range, pud_t pud) { @@ -429,7 +428,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long = start, unsigned long end, return hmm_vma_walk_hole(start, end, -1, walk); } =20 - if (pud_leaf(pud) && pud_devmap(pud)) { + if (pud_leaf(pud)) { unsigned long i, npages, pfn; unsigned int required_fault; unsigned long *hmm_pfns; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC889146A99 for ; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067557; cv=none; b=umKpN7aV4KXslCKKT145tRrDiZrxpSkhj6hABI97MPP70OnSfUlXlj7lY2yNL3SfFrwrCf4A+0oe8s/h24VMuG1KmXKQfZdD4LjBM9RdesMwBXZHy1IaM+EwdO1mODddK2F/mXOZB//Cu6QAHqGoaUDWcYhkv/gnVM/kv4DV6Ns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067557; c=relaxed/simple; bh=EGWKGVBtZJssI002yVffnY9LfU+kgVhFuG/CZwPEkcU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WS1vV/OgTpgHmSjfQw11JGgT91joiqt5gRjBMoNZlbIKJ8zDpIjt+CHOigX+vTkyNj2uIkqMOG2Ecn42UAJrGeHDVoubp54ChX0gd5xXX7h8rW0yHK6kbbEA/amEqNHU4s9hIwV70FgwCmEOELeow9tbhdhGK8vpGAjng0F7yrk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=sTaFfZGs; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=CWK1uG79; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=bqsFGOa2; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=B6iVvmiy; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="sTaFfZGs"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="CWK1uG79"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="bqsFGOa2"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="B6iVvmiy" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 215491FCFB; Thu, 4 Jul 2024 04:32:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067554; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=sTaFfZGsh0QN0KmSZGZupmvc9+i8aT6yh6wqcxWRjw+XHu1zEisZrGN4NNNyCZDSvyJGEU PFjK4MlwkHT8GncglykUIdk+8ISfhp13f+z8nTbaYj1t2I52ze+OrMapZLKDq37mK/EAY8 up94b5BLuEXKnEEwf9Dw0eVYsTwPEig= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067554; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=CWK1uG79vcWta/RwK11oIoeDmAvTnhOeCFGg5+Xlvmv5A/bL50Ok8Y/E9WOG3clC5ufnFC Qi1EyHmHHAYPxtAw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067553; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=bqsFGOa2yGJrOu6R6Wz97cBOkJkF3/uec0u9bnCDleCJ/vApP0GE8p/TCajg1w1b3lZVJQ Wb9dzSwUbpFaSXKXl4Q6eXjlUVVBl4JNvifi+kag3LCsXlCHFKonxim9J8IEGBEX9C5QV0 GiEpgBU7SWbf0a+5Jsh7OpzPXIYc/BU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067553; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh4f2O4+wRTS3lNqvGwH972LquLk6Sxiod9/OijqLHw=; b=B6iVvmiyIQfFPvvBsrtHLZj8Lm+q7MF8LvHMT8cMfsKfCmslqOG4WdKgUVsuxWbCfOn648 Wa2GOQPr90SaqfAw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F0D7D13889; Thu, 4 Jul 2024 04:32:31 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id OPOFNd8lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:31 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 34/45] arch/powerpc: Skip hugetlb vmas in subpage_mark_vma_nohuge Date: Thu, 4 Jul 2024 06:31:21 +0200 Message-ID: <20240704043132.28501-35-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- arch/powerpc/mm/book3s64/subpage_prot.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/powerpc/mm/book3s64/subpage_prot.c b/arch/powerpc/mm/book= 3s64/subpage_prot.c index ec98e526167e..dd529adda87f 100644 --- a/arch/powerpc/mm/book3s64/subpage_prot.c +++ b/arch/powerpc/mm/book3s64/subpage_prot.c @@ -159,6 +159,8 @@ static void subpage_mark_vma_nohuge(struct mm_struct *m= m, unsigned long addr, * VM_NOHUGEPAGE and split them. */ for_each_vma_range(vmi, vma, addr + len) { + if (is_vm_hugetlb_page(vma)) + continue; vm_flags_set(vma, VM_NOHUGEPAGE); walk_page_vma(vma, &subpage_walk_ops, NULL); } --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0C53146A9A for ; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067558; cv=none; b=kja/6eUX63YQnCoWGWKd/LvhtJZZBvpCOhGR9ZcDHuH0J7BhlfTGUJFB92LZaFs0Ht2Q6GNFNmViz2tFcI6HQu9wK8d7gPK8lmM3Dc7A+JN40OY73r58ExlwVGxb9xtyxzo99z7cAd9opq3JIqRteSaDaa83v63tYPGT0y6lAKk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067558; c=relaxed/simple; bh=SydXxW3j7Kjs4d6Qt7kl9XyeIkPU3a9GngnSohteojQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BAM6t9k24m7+ep/+ZmAjZou/mC2yfUrDYyD4IV/7EFBVPXrOmcQqS8ClrgfuLEFGwb5liOMubUfnIil/dSo1YiiGS/ORaZFQjX27QUbc2Eq0b7CeKezR96QwNlhv8b6+XA5ShL+nO3t0goLQWS9uhZOuCk058JNB2sw9/nCEKHE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 6F06E1FCFA; Thu, 4 Jul 2024 04:32:34 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3F79513A82; Thu, 4 Jul 2024 04:32:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id eMdVC+ElhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:33 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 35/45] arch/s390: Skip hugetlb vmas in thp_split_mm Date: Thu, 4 Jul 2024 06:31:22 +0200 Message-ID: <20240704043132.28501-36-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 6F06E1FCFA X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- arch/s390/mm/gmap.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/s390/mm/gmap.c b/arch/s390/mm/gmap.c index e1d098dc7f07..580e4ab6f018 100644 --- a/arch/s390/mm/gmap.c +++ b/arch/s390/mm/gmap.c @@ -2538,6 +2538,8 @@ static inline void thp_split_mm(struct mm_struct *mm) VMA_ITERATOR(vmi, mm, 0); =20 for_each_vma(vmi, vma) { + if (is_vm_hugetlb_page(vma)) + continue; vm_flags_mod(vma, VM_NOHUGEPAGE, VM_HUGEPAGE); walk_page_vma(vma, &thp_split_walk_ops, NULL); } --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 759531474B5 for ; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067558; cv=none; b=jOwWxv3UY9luCsgKstxkOUqcTltsxIeMwf/MyVs5HeAafFGucCORXOqWapR7dqNLK5k78a/pUg0rhxHHBKy0lk+F4Gyvltby+r2+s1e5cqRlD6mcnGXaXasN90nlP1c4OMePItaEC0mlUX+eWkYzHOxsEHwrH+keOTjA1UMv4vU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067558; c=relaxed/simple; bh=3U59ZP18lkNdZ7l489KUIyawpNPZopBBsX003E0BKzM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uWTC70yMn+mIsPVxq7hSd11/skG+viX34Il519CvoaCn0CDS71GfuMsdZ8udoI4cJClp4ULrG119YmGzZPxUr3Usy402pmbhat8rMNTTo2eb3+5GQaUAVnyWWkrvsaEJUFH4OCGMkB0EniNSWel6d2a8Zky4RfM+EQbkbX+j7mk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=MBgo3NSc; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=sCT0CMUz; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=MBgo3NSc; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=sCT0CMUz; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="MBgo3NSc"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="sCT0CMUz"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="MBgo3NSc"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="sCT0CMUz" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B4E2A21180; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=MBgo3NSczt8U7EEAXcD2SQA8wQ51zLbu6/4GLxv3/acFo/O9aeAjJinYHrahN+wHh6WN6E nJWgfUf/qMJFBilRyvUHpRCrHv3QH2RC9j0K0L3OAEZimxeISUardGz2TZ7zET0RJVlsf9 vPaxOvkBTgVVPFz9W64mTDrSkfrzns0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=sCT0CMUzMRt8XjozOQiz7eH0jI8RX27HURNG1GIXpzDYx1fhHI5xooLjdVheihL1p8vQfG 66YxnUMPC011JyDw== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=MBgo3NSczt8U7EEAXcD2SQA8wQ51zLbu6/4GLxv3/acFo/O9aeAjJinYHrahN+wHh6WN6E nJWgfUf/qMJFBilRyvUHpRCrHv3QH2RC9j0K0L3OAEZimxeISUardGz2TZ7zET0RJVlsf9 vPaxOvkBTgVVPFz9W64mTDrSkfrzns0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067555; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKNX/My9OkoTrzpvFpT5zCUWuebklZgLLB1xxXIoHYo=; b=sCT0CMUzMRt8XjozOQiz7eH0jI8RX27HURNG1GIXpzDYx1fhHI5xooLjdVheihL1p8vQfG 66YxnUMPC011JyDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 93F1113889; Thu, 4 Jul 2024 04:32:34 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QADdHuIlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:34 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 36/45] fs/proc: Make clear_refs_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:23 +0200 Message-ID: <20240704043132.28501-37-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,imap1.dmz-prg2.suse.org:helo]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- fs/proc/task_mmu.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5df17b7cfe6c..df94f2093588 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1579,6 +1579,9 @@ static int clear_refs_test_walk(unsigned long start, = unsigned long end, struct clear_refs_private *cp =3D walk->private; struct vm_area_struct *vma =3D walk->vma; =20 + if (is_vm_hugetlb_page(vma)) + return 1; + if (vma->vm_flags & VM_PFNMAP) return 1; =20 --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF12B1474D9 for ; Thu, 4 Jul 2024 04:32:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067560; cv=none; b=VWVi4Rlzinsz7M3qof2CWV98gqBMhE1pzWPuUw7yQJGq/jjJF9FmO3dWR0y22KOqAGLwLR2Av0RsmryPrji4JhwgacNH2FCDRuzNYTShWGOK+zgQEsnGd8YaYVkNQLzolG/QH8WhWM0Yt3pnfOkyYMPQ5dXKqKgGrq7h4iPMPnM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067560; c=relaxed/simple; bh=sQejekRUyVvof6FFs+sGP7fR828TdGKbEu0/e+tUYR4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I3cIaMnaADCl5OAqqhs+dMjzBGBImc6jeIrpSdO8fn6ddymYgZys9RqSbFwk0sRYHb/NEd5d9VaCZ5rTz+/J9RcxmPYmAa0hlYlQ3g4Ox8M/TXMg3N2aHA+E1Rh28H0jdPlrzf/rBaznQwfAm2YUqayVbBH+SLRNhXgWdUF5+fg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=FLLFrGlR; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Xn+qmriL; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=FLLFrGlR; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=Xn+qmriL; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="FLLFrGlR"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Xn+qmriL"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="FLLFrGlR"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="Xn+qmriL" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 166C121C0D; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=FLLFrGlRYQvTCYxVLiVDlOvcqdI03sK7tsIHp0UaVD2y3rWv0sfPku+5j1Sb+TGxATnbIj 2SvrEeBnu35COLNGzRhuWQiuv+d1fMu1ZxIzeBld3MHcaPV5255bX31dpcaYpS7sxPB+yj Wh3sG0PCXGnABuZy8kGWiY/fHIKMyn0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=Xn+qmriLdnNRRO+exHfJ7IQkKp73CpsQ6YRsMlGxYx0y8gyUntE4TOjbftU2SpDj9j99UM 4IbKv/L2PFMFs+CA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=FLLFrGlRYQvTCYxVLiVDlOvcqdI03sK7tsIHp0UaVD2y3rWv0sfPku+5j1Sb+TGxATnbIj 2SvrEeBnu35COLNGzRhuWQiuv+d1fMu1ZxIzeBld3MHcaPV5255bX31dpcaYpS7sxPB+yj Wh3sG0PCXGnABuZy8kGWiY/fHIKMyn0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067557; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nyp1GchWN9j9DCiswMlchcq+0OOi/ydZqtxsOmg3fI=; b=Xn+qmriLdnNRRO+exHfJ7IQkKp73CpsQ6YRsMlGxYx0y8gyUntE4TOjbftU2SpDj9j99UM 4IbKv/L2PFMFs+CA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DDB1513889; Thu, 4 Jul 2024 04:32:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QI6DMOMlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:35 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 37/45] mm/lock: Make mlock_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:24 +0200 Message-ID: <20240704043132.28501-38-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,imap1.dmz-prg2.suse.org:helo]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/mlock.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/mlock.c b/mm/mlock.c index 52d6e401ad67..b37079b3505f 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -409,6 +409,17 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long a= ddr, return 0; } =20 +static int mlock_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + /* * mlock_vma_pages_range() - mlock any pages already in the range, * or munlock all pages in the range. @@ -425,6 +436,7 @@ static void mlock_vma_pages_range(struct vm_area_struct= *vma, { static const struct mm_walk_ops mlock_walk_ops =3D { .pmd_entry =3D mlock_pte_range, + .test_walk =3D mlock_test_walk, .walk_lock =3D PGWALK_WRLOCK_VERIFY, }; =20 --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF48F47A64 for ; Thu, 4 Jul 2024 04:32:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067561; cv=none; b=BWqRKVBqzrb9CbhaNfmvO33zGYPAfJYs9BP2+YkAhKxZy60WNkK9yLH9QdaPf1nHbgXjh3Yyft5pc9oJY4uenVCrqR0pYe/BWLHFL5WsoiQpUlclL5nwJPQB1X0+jJ4gybSUfZJ1Bst5mKCIuRwtNjEI+YP8hld9Yy7ahPpn1Wg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067561; c=relaxed/simple; bh=kqCSDZ05ilm/0TUzR8rJaPdyK3biGq/s90mBx1aESYo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WQ5iP3udpGfn/9eQ4cgXVzzZFvNW3Z92HroxofqAcRTlmL/jXkz94d9bnmVVoPWMAl/ypSQZHTxXtJDJuIJhFuYkxTt6HaDU/F8ukuUtFYpN6AEhrHSJRro0Ub6N5c1ari2LWNpPBX6jNptdWKoyK86OYPPTJGsExthJs27tLNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=wu38pd2U; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=3vLpc8uy; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=wu38pd2U; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=3vLpc8uy; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="wu38pd2U"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="3vLpc8uy"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="wu38pd2U"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="3vLpc8uy" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4E4831FCF7; Thu, 4 Jul 2024 04:32:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=wu38pd2Ur04WQggmlopkDml4wHinDJSRhnDrkpQt+PtRYgYO79418P+xobCSITpMNuUGJr N+r3hCiv4ga0bzDDG6Hf5y+011wgZlCP9rDrAInPoWk4nEgL/L2Ryqm/Fs6j9r+hmIMepL jQdzslJTtqTqyHqPiP2tBLeURFlt62k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=3vLpc8uy3DMDbdMeN7hpDSFZj9ThV9qGHZSesmupuyJqz3PIjoXBK+9NW6fTTC/h3zsQFl 1ol0a9rc5zXXd3CQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=wu38pd2Ur04WQggmlopkDml4wHinDJSRhnDrkpQt+PtRYgYO79418P+xobCSITpMNuUGJr N+r3hCiv4ga0bzDDG6Hf5y+011wgZlCP9rDrAInPoWk4nEgL/L2Ryqm/Fs6j9r+hmIMepL jQdzslJTtqTqyHqPiP2tBLeURFlt62k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067558; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lrt/KcsLV9mhSUC002LaYTg+7UaG25OX7K0/IX8flZ0=; b=3vLpc8uy3DMDbdMeN7hpDSFZj9ThV9qGHZSesmupuyJqz3PIjoXBK+9NW6fTTC/h3zsQFl 1ol0a9rc5zXXd3CQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 370FC13889; Thu, 4 Jul 2024 04:32:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4F+tCOUlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:37 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 38/45] mm/madvise: Make swapin_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:25 +0200 Message-ID: <20240704043132.28501-39-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/madvise.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index 96c026fe0c99..4c7c409e8b4a 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -212,8 +212,20 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned = long start, return 0; } =20 +static int swapin_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops swapin_walk_ops =3D { .pmd_entry =3D swapin_walk_pmd_entry, + .test_walk =3D swapin_test_walk, .walk_lock =3D PGWALK_RDLOCK, }; =20 --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5EA5E1487E1 for ; Thu, 4 Jul 2024 04:32:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067562; cv=none; b=npJeGzK8vTQMnCPzFJ6JcXiQjUIJHrVwHoqfszDeSTpIVtWhIrlF0GqplXJvfvm/uaBsB0VhBBJrfcYgQvTCMq0bi8rw8NADeFuCoz4KJjjM9ZaVkZXcArW8iyTnrO8K2mISbmpJrCH5Oh0VdpeRtFs/o6kBQi4aOTrTXbhAchU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067562; c=relaxed/simple; bh=8gai68bXl2uiZ/cXNMkqDGWXNRoE9vEx1D9cFzbOn7w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c8xevVpa02vnh9/5leWZf5Xu5a27OFD0d+SAOZ2r71OG9/7Wu0K4wq52jC6F7R09NEQbgTpiE1SIPDpLU3NqY3dS3R1WPLj6iATQieA7T6sE272lHX8PhzcdiUu0xNmUZmfaKuEqa8Soap1X8a43NCTdJbuBLPih5cSL+MiCYfk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=hwqDJNVK; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=LSnnQJXR; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=hwqDJNVK; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=LSnnQJXR; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="hwqDJNVK"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="LSnnQJXR"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="hwqDJNVK"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="LSnnQJXR" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9876421C10; Thu, 4 Jul 2024 04:32:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=hwqDJNVK/CCOylYSLTwWXUVICQyx83G/knpDrCzGjHxZsYrpDDVMSCB321NNI98Hf7c5Fe fliGr2rF4bdqNm4A5diMwERQGXRj6hZTmNR2Bkz1zFSjM3T+0O/7G2l+AXTUcqMyvnhuqU fHH0ZeRfPHMYS1U4b1g+F9jlmYWgAqw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=LSnnQJXRLmy1Y92nnMR+gbafnAKL4iJ1SAKvsbEP0jdZUQgWseUiTP/YO4XvsimDx5/+i6 QWKNbAFvKiCbi8AA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=hwqDJNVK/CCOylYSLTwWXUVICQyx83G/knpDrCzGjHxZsYrpDDVMSCB321NNI98Hf7c5Fe fliGr2rF4bdqNm4A5diMwERQGXRj6hZTmNR2Bkz1zFSjM3T+0O/7G2l+AXTUcqMyvnhuqU fHH0ZeRfPHMYS1U4b1g+F9jlmYWgAqw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067559; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OwRMEygtt9hYpWc9+hDgMaQcaULG8MqCO3fZe7sMmEA=; b=LSnnQJXRLmy1Y92nnMR+gbafnAKL4iJ1SAKvsbEP0jdZUQgWseUiTP/YO4XvsimDx5/+i6 QWKNbAFvKiCbi8AA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6F2E913889; Thu, 4 Jul 2024 04:32:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SEcAFuYlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:38 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 39/45] mm/madvise: Make madvise_cold_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:26 +0200 Message-ID: <20240704043132.28501-40-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/madvise.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index 4c7c409e8b4a..e60311636c4c 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -565,8 +565,20 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pm= d, return 0; } =20 +static int madvise_cold_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops cold_walk_ops =3D { .pmd_entry =3D madvise_cold_or_pageout_pte_range, + .test_walk =3D madvise_cold_test_walk, .walk_lock =3D PGWALK_RDLOCK, }; =20 --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D205148848 for ; Thu, 4 Jul 2024 04:32:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067564; cv=none; b=pXJ5LZwH4RfIKOLZlbXa+AyCSJqmJ+BCDkmsQIFy1lrFIRnhmLyXJ3AdNcdDzXi+hodv/S6EsIrotRm87E9L6c/lkJ+V3k24b1tZAdsyL/4CkO4qybmDSHl7immywvwK+c2002fb9TVgHIQfHzspRH4FC/R7FCfo1LhPUJ4kqWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067564; c=relaxed/simple; bh=LA4Bt0HatbsR7tMBIK5rYy4CCuaafzjwktEMfkTXsI4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=o4C9CaG6kQrlGnyTRWiVExyiod0WOLHXn8Eboa31oe6fxiImlcZ5qQiFMOtSO6hlepXlojfg7JQeSFv1lHOwUZ9WsTLRgo6Kth/hkDIBpg+JkhGQlciY4t31V7PXOsoOeePIuyMvV7UNt8S1Jg+Y7V3yj1sYNZ+SWUZjsv2aSP0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=j6iEK8Pq; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=vQ4JL4C6; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=j6iEK8Pq; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=vQ4JL4C6; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="j6iEK8Pq"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="vQ4JL4C6"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="j6iEK8Pq"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="vQ4JL4C6" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D03B721C11; Thu, 4 Jul 2024 04:32:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=j6iEK8PqgI9QRFImbu/UVsJysu4KND3ZF8Hf9UZxL94XKAaxTl/7gm9F4NIZVKQ4NnS873 TQNyiXjMCFq8XNcKsBsPNmyqfm2qouwQCrbcFcFAGdwMNdfwK24xrwVgkLwP2jbuyhvLUH BPd9xM4HZ98MSq7QMtepwwtCtWt/SOs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=vQ4JL4C6UPipOlC2cZ+CsrSqGZzo+7etIj0YNYzIdJOh1eq9XU9yhO8jX+7cuSFjbBdfjf W0zSPHZ7LidSpVCg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=j6iEK8PqgI9QRFImbu/UVsJysu4KND3ZF8Hf9UZxL94XKAaxTl/7gm9F4NIZVKQ4NnS873 TQNyiXjMCFq8XNcKsBsPNmyqfm2qouwQCrbcFcFAGdwMNdfwK24xrwVgkLwP2jbuyhvLUH BPd9xM4HZ98MSq7QMtepwwtCtWt/SOs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067560; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvSHNTt8bLluhGl3df5sQFwppsiw5zo4fh71urj/PzI=; b=vQ4JL4C6UPipOlC2cZ+CsrSqGZzo+7etIj0YNYzIdJOh1eq9XU9yhO8jX+7cuSFjbBdfjf W0zSPHZ7LidSpVCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AF3B513A82; Thu, 4 Jul 2024 04:32:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2CQKJuclhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:39 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 40/45] mm/madvise: Make madvise_free_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:27 +0200 Message-ID: <20240704043132.28501-41-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,imap1.dmz-prg2.suse.org:helo]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/madvise.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index e60311636c4c..08f72622913f 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -805,8 +805,20 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned= long addr, return 0; } =20 +static int madvise_free_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops madvise_free_walk_ops =3D { .pmd_entry =3D madvise_free_pte_range, + .test_walk =3D madvise_free_test_walk, .walk_lock =3D PGWALK_RDLOCK, }; =20 --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 951F4148FFB for ; Thu, 4 Jul 2024 04:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067565; cv=none; b=T5PpIrwF1Q4ku9zmZOcwb/1huwjYsDoNuw2K6YHCLK8k62pK4Ev+WYpXiV/Z3GDuwG1OIT3ZWvRtUKCysBjP6vSN848sRL0TlSfpgx3hE2VvgEdRjSZ/gOk6X7F5PeIPU/wtPJ41lh9nrnH0QXQR+8kLGUUT6MGVUZXMmWAofCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067565; c=relaxed/simple; bh=9GYDzTTbrab85piNyP7RffzjbFiX+hGhrD3+y5lm5SQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bzq7mfg9VUOXfKXSMCAOQM5B3lXws4TV3B6WSR+QdwNKAwRxoZ1G5HzKK55lIug5pSxZ1QgHPUrmOCnfECrJubuM73e5DBE+tRos/zJk6Yqib0Tk9DiuqoORIIyExuOQRl2oOSV4pZIAIVER2QEvwZ9uGDjosHPbM633u93Uino= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 27A961FCF8; Thu, 4 Jul 2024 04:32:42 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0274313889; Thu, 4 Jul 2024 04:32:40 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SAXCNuglhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:40 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 41/45] mm/migrate_device: Make migrate_vma_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:28 +0200 Message-ID: <20240704043132.28501-42-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 27A961FCF8 X-Rspamd-Action: no action X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/migrate_device.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 6d66dc1c6ffa..c44ac45b207d 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -279,8 +279,20 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, return 0; } =20 +static int migrate_vma_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops migrate_vma_walk_ops =3D { .pmd_entry =3D migrate_vma_collect_pmd, + .test_walk =3D migrate_vma_test_walk, .pte_hole =3D migrate_vma_collect_hole, .walk_lock =3D PGWALK_RDLOCK, }; --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1B9D14901B for ; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067566; cv=none; b=k7hwlktjpX+XsPXGd/G9eLoIea0V1E89MS3QNCwagJZi9R/JmpIyXQCADcbJ2WnEygvqIxp0Nys/gRSq7JOQG6eEuK99MUUvSO6ScZDPNWoUpujizbUU2cQg3CAVyGiUzZVzFz/yewC2M60u8n8OhHSVlkFfvUxys1n/7KyU7v8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067566; c=relaxed/simple; bh=SvWtT8eXdxiInxt0lWb3Rj5G0POa5eYV4UnyF6GLYLo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c3+pXD3Ql530GO5v/ZvM/EMdiARqPOhw2wMgHLOipzErPWL3Q4Dxhe5HwdPPlCKJ4/wPf4VVqc0fld0qkiPjfC3dL9coSDsBHyny0x0f1zrjKRRQ3YFtyNCptHe6KwOWFT/ovyIkv9+IDjeSr3qBlzNmDhDCGW8Hdn7jIWNqm3s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=f6TMrMH1; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=7wTQx6i+; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=f6TMrMH1; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=7wTQx6i+; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="f6TMrMH1"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="7wTQx6i+"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="f6TMrMH1"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="7wTQx6i+" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 69E391FCFA; Thu, 4 Jul 2024 04:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=f6TMrMH1l0DF/S8bfNE80GfcQ/4KU9eJ27sxLCy3LzyHrHWU4yVZdFy7kYhwfc7mVnZVdu xsRX3Nb1px3A9NnzgBOX5ai81npXNqEkbHs8Yk6Z2158wI9WwdgLuclp+kv3Leuji+b1kc vOC5FehQO0GHOMoxqJ/B9oV+I3yygyc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=7wTQx6i+oEK8SbN8wor1yxW+lVrtevHmwonbwTOp+kRSZvWPS1cw79vzxMRpqgGlJBhIHR U7Fz/q4+qA9HUzCQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=f6TMrMH1l0DF/S8bfNE80GfcQ/4KU9eJ27sxLCy3LzyHrHWU4yVZdFy7kYhwfc7mVnZVdu xsRX3Nb1px3A9NnzgBOX5ai81npXNqEkbHs8Yk6Z2158wI9WwdgLuclp+kv3Leuji+b1kc vOC5FehQO0GHOMoxqJ/B9oV+I3yygyc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067563; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OvwrckhnGQ1ia89v5nuOKOB6N7ZG52TVWWVE7z3ysd0=; b=7wTQx6i+oEK8SbN8wor1yxW+lVrtevHmwonbwTOp+kRSZvWPS1cw79vzxMRpqgGlJBhIHR U7Fz/q4+qA9HUzCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4B28813889; Thu, 4 Jul 2024 04:32:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WEYNDuolhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:42 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 42/45] mm/memcontrol: Make mem_cgroup_move_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:29 +0200 Message-ID: <20240704043132.28501-43-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/memcontrol-v1.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 99cc9501eec1..542922562cf9 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -1319,8 +1319,20 @@ static int mem_cgroup_move_charge_pte_range(pmd_t *p= md, return ret; } =20 +static int mem_cgroup_move_test_walk(unsigned long start, unsigned long en= d, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops charge_walk_ops =3D { .pmd_entry =3D mem_cgroup_move_charge_pte_range, + .test_walk =3D mem_cgroup_move_test_walk, .walk_lock =3D PGWALK_RDLOCK, }; =20 --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5739F149C54 for ; Thu, 4 Jul 2024 04:32:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067567; cv=none; b=JrIRXh96zJlzHKuUOLDs9HYGyXZrLoyPBM7CueNlU7I6rJHWppT2AtfhursZkt12bsx+R+uLbObixMt4mFXjMWPp72qOEyZsEuIpY6qO+gnKagPDH5cCOuQkOYrrY/vSr6ZyEkmUpPX1XcQ/M7HvzvV85JwyCFp1Gxin72sYkI0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067567; c=relaxed/simple; bh=xMdRQ0+0gaRKpuFYIzRmduSg0i6X8rCX6I1uxKX4eOc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oVAjNtGRxSFzSLikBEX58CmVJp80ZFm8CSzKX5EHeJlgwNcIj+5nQH9n5vOhB2JgiLU2xSO6Y5g5415UFyiRTwkb1vYxQ7FFyKzJbAjcKhQPDmcSwYY/V7Vj1xbnOy1XHDauUe2EWnZZnclrbxy0uWueJdbdYn3b+sz+FNU5/Z0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A41351FCFB; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8D0C513889; Thu, 4 Jul 2024 04:32:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2IqsHeslhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:43 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 43/45] mm/memcontrol: Make mem_cgroup_count_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:30 +0200 Message-ID: <20240704043132.28501-44-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: A41351FCFB X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/memcontrol-v1.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 542922562cf9..cd8ad1a7f170 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -1039,8 +1039,20 @@ static int mem_cgroup_count_precharge_pte_range(pmd_= t *pmd, return 0; } =20 +static int mem_cgroup_count_test_walk(unsigned long start, unsigned long e= nd, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops precharge_walk_ops =3D { .pmd_entry =3D mem_cgroup_count_precharge_pte_range, + .test_walk =3D mem_cgroup_count_test_walk, .walk_lock =3D PGWALK_RDLOCK, }; =20 --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B4E6149DE9 for ; Thu, 4 Jul 2024 04:32:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067568; cv=none; b=PdbY9P/X8fUovlVC37M4PpdQkJ+ZlYWjGhlPugHxups8QEuXkxwD/DcqIiMk7ir0V7BysmLLxh4C2bCrtaHvPmk2UsMmdPl6DQuEO6n1q9+LQwkfVx5hioChu1teo1dXI6+WTeNhVIZNR03z3JWKeAeAV0wx6N1D95OUeQi2UAE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067568; c=relaxed/simple; bh=RcwFClZPGTFoXt0drfryl0gzPaFgfKFSVQ5t/DvqB4o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cPJZkq80HPfQGs4wVdZs0DHy7oN6KVG8NqkSM14BugIEuYzepruNAmi1iZ3QntzGaaUtSgRi+kHJCmYlWnnd0I9TgQpgX8lFN3ZO6rFIqf5a0jpFv3BRy+MvU/hFVDPCmVlwQlFnUNE4gmLjiyF/tNm2xQE+W6oZmdMcPKxSXh4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=qWQLNf4E; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=9PS3LOnT; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=qWQLNf4E; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=9PS3LOnT; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="qWQLNf4E"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="9PS3LOnT"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="qWQLNf4E"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="9PS3LOnT" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D89E621180; Thu, 4 Jul 2024 04:32:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=qWQLNf4EDHAPFOcSiw8o9yIVaDWy3wN7EvJ2sANX/QaSm9/dzdpfyxkS+tGCPT67FTj67/ qxSBubrEfCFMg2meOU9ONT/982yfM0wORqljjkCK0CIkPk/MmyWkBuIbujyg+alQu42Nox +3A8UPaXHPIjQsi6zuoCjBr/Vxx+ZLY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=9PS3LOnTWX4tc9eF256ZXVJczKFHze/Ne7gNltkr5CcTePNcTuvg5Xdekx74hNiZwcr4HX nEMGuKsw4KpydzBA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=qWQLNf4EDHAPFOcSiw8o9yIVaDWy3wN7EvJ2sANX/QaSm9/dzdpfyxkS+tGCPT67FTj67/ qxSBubrEfCFMg2meOU9ONT/982yfM0wORqljjkCK0CIkPk/MmyWkBuIbujyg+alQu42Nox +3A8UPaXHPIjQsi6zuoCjBr/Vxx+ZLY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720067565; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9eR5Nhbb0ANiKXQprN8VC6tLG959+ERIV2i0QqHsYD4=; b=9PS3LOnTWX4tc9eF256ZXVJczKFHze/Ne7gNltkr5CcTePNcTuvg5Xdekx74hNiZwcr4HX nEMGuKsw4KpydzBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C103213889; Thu, 4 Jul 2024 04:32:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id +JDnKuwlhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:44 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 44/45] mm/hugetlb_vmemmap: Make vmemmap_test_walk skip hugetlb vmas Date: Thu, 4 Jul 2024 06:31:31 +0200 Message-ID: <20240704043132.28501-45-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -6.80 X-Spam-Level: X-Spam-Flag: NO X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; R_RATELIMIT(0.00)[to_ip_from(RLghcz9uomn9ay1xnuhrww8uur)]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="utf-8" Skip hugetlb vmas as we are not interested in those. Signed-off-by: Oscar Salvador --- mm/hugetlb_vmemmap.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 829112b0a914..3e6fd5ae27bd 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -151,9 +151,21 @@ static int vmemmap_pte_entry(pte_t *pte, unsigned long= addr, return 0; } =20 +static int vmemmap_test_walk(unsigned long start, unsigned long end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + + if (is_vm_hugetlb_page(vma)) + return 1; + + return 0; +} + static const struct mm_walk_ops vmemmap_remap_ops =3D { .pmd_entry =3D vmemmap_pmd_entry, .pte_entry =3D vmemmap_pte_entry, + .test_walk =3D vmemmap_test_walk, }; =20 static int vmemmap_remap_range(unsigned long start, unsigned long end, --=20 2.26.2 From nobody Sun Feb 8 02:20:48 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D55E714A091 for ; Thu, 4 Jul 2024 04:32:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067571; cv=none; b=VzFHtC7XQHPeDRUgX5V5LwuBex1hW4PFVgXw2vnrqcBrSGO5ejihPE5Hffke5D5wThbL7M4/dv2ZRzTNsWEQHpHRkiamPsFqaD73sYrmyHlleuqmH7t1kFkCIYiU8vyILCWMUlYCj4z+++wKqfWAOvA/qDk7Z9357GqQq9hrTNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720067571; c=relaxed/simple; bh=QshJOkmDi71DB8T1DtsDIKnVd/jaoOR9uj57jxAL840=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EKxjP0wMY77xn2xgM+36tDauEBzgDJvgvuTY0C6TWVeWr14j6FqZsQ2ZLDIhVW6s/4xugHdiAXImK67lSz6Yudh/IKkuEKbC5XXguL3TuC2jAgElFM9flVqCBFv4BAkIV08uiXOGDMDG351NpbKHEkwSIIjieApzAP0jZJ6NO/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1983A1FCF7; Thu, 4 Jul 2024 04:32:47 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 06B7013889; Thu, 4 Jul 2024 04:32:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0LqbOO0lhmbnDAAAD6G6ig (envelope-from ); Thu, 04 Jul 2024 04:32:45 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , SeongJae Park , Miaohe Lin , Michal Hocko , Matthew Wilcox , Christophe Leroy , Oscar Salvador Subject: [PATCH 45/45] mm: Delete all hugetlb_entry entries Date: Thu, 4 Jul 2024 06:31:32 +0200 Message-ID: <20240704043132.28501-46-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240704043132.28501-1-osalvador@suse.de> References: <20240704043132.28501-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 1983A1FCF7 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Flag: NO X-Spam-Score: -4.00 X-Spam-Level: Content-Type: text/plain; charset="utf-8" Generic pagewalker can now deal with hugetlb pages as well, so there is no need for specific hugetlb_entry functions. Drop them all. Signed-off-by: Oscar Salvador --- arch/s390/mm/gmap.c | 27 ------ fs/proc/task_mmu.c | 181 --------------------------------------- include/linux/pagewalk.h | 10 --- mm/damon/vaddr.c | 89 ------------------- mm/hmm.c | 54 ------------ mm/memory-failure.c | 17 ---- mm/mempolicy.c | 47 ---------- mm/mincore.c | 22 ----- mm/mprotect.c | 10 --- mm/pagewalk.c | 49 +---------- 10 files changed, 1 insertion(+), 505 deletions(-) diff --git a/arch/s390/mm/gmap.c b/arch/s390/mm/gmap.c index 580e4ab6f018..3307f0ec505c 100644 --- a/arch/s390/mm/gmap.c +++ b/arch/s390/mm/gmap.c @@ -2743,34 +2743,7 @@ static int __s390_enable_skey_pmd(pmd_t *pmd, unsign= ed long addr, return 0; } =20 -static int __s390_enable_skey_hugetlb(pte_t *pte, unsigned long addr, - unsigned long hmask, unsigned long next, - struct mm_walk *walk) -{ - pmd_t *pmd =3D (pmd_t *)pte; - unsigned long start, end; - struct page *page =3D pmd_page(*pmd); - - /* - * The write check makes sure we do not set a key on shared - * memory. This is needed as the walker does not differentiate - * between actual guest memory and the process executable or - * shared libraries. - */ - if (pmd_val(*pmd) & _SEGMENT_ENTRY_INVALID || - !(pmd_val(*pmd) & _SEGMENT_ENTRY_WRITE)) - return 0; - - start =3D pmd_val(*pmd) & HPAGE_MASK; - end =3D start + HPAGE_SIZE; - __storage_key_init_range(start, end); - set_bit(PG_arch_1, &page->flags); - cond_resched(); - return 0; -} - static const struct mm_walk_ops enable_skey_walk_ops =3D { - .hugetlb_entry =3D __s390_enable_skey_hugetlb, .pte_entry =3D __s390_enable_skey_pte, .pmd_entry =3D __s390_enable_skey_pmd, .walk_lock =3D PGWALK_WRLOCK, diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index df94f2093588..52fa82336825 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1066,52 +1066,15 @@ static void show_smap_vma_flags(struct seq_file *m,= struct vm_area_struct *vma) seq_putc(m, '\n'); } =20 -#ifdef CONFIG_HUGETLB_PAGE -static int smaps_hugetlb_range(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct mem_size_stats *mss =3D walk->private; - struct vm_area_struct *vma =3D walk->vma; - pte_t ptent =3D huge_ptep_get(walk->mm, addr, pte); - struct folio *folio =3D NULL; - bool present =3D false; - - if (pte_present(ptent)) { - folio =3D page_folio(pte_page(ptent)); - present =3D true; - } else if (is_swap_pte(ptent)) { - swp_entry_t swpent =3D pte_to_swp_entry(ptent); - - if (is_pfn_swap_entry(swpent)) - folio =3D pfn_swap_entry_folio(swpent); - } - - if (folio) { - /* We treat non-present entries as "maybe shared". */ - if (!present || folio_likely_mapped_shared(folio) || - hugetlb_pmd_shared(pte)) - mss->shared_hugetlb +=3D huge_page_size(hstate_vma(vma)); - else - mss->private_hugetlb +=3D huge_page_size(hstate_vma(vma)); - } - return 0; -} -#else -#define smaps_hugetlb_range NULL -#endif /* HUGETLB_PAGE */ - static const struct mm_walk_ops smaps_walk_ops =3D { .pud_entry =3D smaps_pud_range, .pmd_entry =3D smaps_pte_range, - .hugetlb_entry =3D smaps_hugetlb_range, .walk_lock =3D PGWALK_RDLOCK, }; =20 static const struct mm_walk_ops smaps_shmem_walk_ops =3D { .pud_entry =3D smaps_pud_range, .pmd_entry =3D smaps_pte_range, - .hugetlb_entry =3D smaps_hugetlb_range, .pte_hole =3D smaps_pte_hole, .walk_lock =3D PGWALK_RDLOCK, }; @@ -1987,66 +1950,10 @@ static int pagemap_pud_range(pud_t *pudp, unsigned = long addr, unsigned long end, #define pagemap_pud_range NULL #endif =20 -#ifdef CONFIG_HUGETLB_PAGE -/* This function walks within one hugetlb entry in the single call */ -static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct pagemapread *pm =3D walk->private; - struct vm_area_struct *vma =3D walk->vma; - u64 flags =3D 0, frame =3D 0; - int err =3D 0; - pte_t pte; - - if (vma->vm_flags & VM_SOFTDIRTY) - flags |=3D PM_SOFT_DIRTY; - - pte =3D huge_ptep_get(walk->mm, addr, ptep); - if (pte_present(pte)) { - struct folio *folio =3D page_folio(pte_page(pte)); - - if (!folio_test_anon(folio)) - flags |=3D PM_FILE; - - if (!folio_likely_mapped_shared(folio) && - !hugetlb_pmd_shared(ptep)) - flags |=3D PM_MMAP_EXCLUSIVE; - - if (huge_pte_uffd_wp(pte)) - flags |=3D PM_UFFD_WP; - - flags |=3D PM_PRESENT; - if (pm->show_pfn) - frame =3D pte_pfn(pte) + - ((addr & ~hmask) >> PAGE_SHIFT); - } else if (pte_swp_uffd_wp_any(pte)) { - flags |=3D PM_UFFD_WP; - } - - for (; addr !=3D end; addr +=3D PAGE_SIZE) { - pagemap_entry_t pme =3D make_pme(frame, flags); - - err =3D add_to_pagemap(&pme, pm); - if (err) - return err; - if (pm->show_pfn && (flags & PM_PRESENT)) - frame++; - } - - cond_resched(); - - return err; -} -#else -#define pagemap_hugetlb_range NULL -#endif /* HUGETLB_PAGE */ - static const struct mm_walk_ops pagemap_ops =3D { .pud_entry =3D pagemap_pud_range, .pmd_entry =3D pagemap_pmd_range, .pte_hole =3D pagemap_pte_hole, - .hugetlb_entry =3D pagemap_hugetlb_range, .walk_lock =3D PGWALK_RDLOCK, }; =20 @@ -2789,67 +2696,6 @@ static int pagemap_scan_pud_entry(pud_t *pud, unsign= ed long start, #define pagemap_scan_pud_entry NULL #endif =20 -#ifdef CONFIG_HUGETLB_PAGE -static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask, - unsigned long start, unsigned long end, - struct mm_walk *walk) -{ - struct pagemap_scan_private *p =3D walk->private; - struct vm_area_struct *vma =3D walk->vma; - unsigned long categories; - spinlock_t *ptl; - int ret =3D 0; - pte_t pte; - - if (~p->arg.flags & PM_SCAN_WP_MATCHING) { - /* Go the short route when not write-protecting pages. */ - - pte =3D huge_ptep_get(walk->mm, start, ptep); - categories =3D p->cur_vma_category | pagemap_hugetlb_category(pte); - - if (!pagemap_scan_is_interesting_page(categories, p)) - return 0; - - return pagemap_scan_output(categories, p, start, &end); - } - - i_mmap_lock_write(vma->vm_file->f_mapping); - ptl =3D huge_pte_lock(hstate_vma(vma), vma->vm_mm, ptep); - - pte =3D huge_ptep_get(walk->mm, start, ptep); - categories =3D p->cur_vma_category | pagemap_hugetlb_category(pte); - - if (!pagemap_scan_is_interesting_page(categories, p)) - goto out_unlock; - - ret =3D pagemap_scan_output(categories, p, start, &end); - if (start =3D=3D end) - goto out_unlock; - - if (~categories & PAGE_IS_WRITTEN) - goto out_unlock; - - if (end !=3D start + HPAGE_SIZE) { - /* Partial HugeTLB page WP isn't possible. */ - pagemap_scan_backout_range(p, start, end); - p->arg.walk_end =3D start; - ret =3D 0; - goto out_unlock; - } - - make_uffd_wp_huge_pte(vma, start, ptep, pte); - flush_hugetlb_tlb_range(vma, start, end); - -out_unlock: - spin_unlock(ptl); - i_mmap_unlock_write(vma->vm_file->f_mapping); - - return ret; -} -#else -#define pagemap_scan_hugetlb_entry NULL -#endif - static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end, int depth, struct mm_walk *walk) { @@ -2879,7 +2725,6 @@ static const struct mm_walk_ops pagemap_scan_ops =3D { .pud_entry =3D pagemap_scan_pud_entry, .pmd_entry =3D pagemap_scan_pmd_entry, .pte_hole =3D pagemap_scan_pte_hole, - .hugetlb_entry =3D pagemap_scan_hugetlb_entry, }; =20 static int pagemap_scan_get_args(struct pm_scan_arg *arg, @@ -3275,34 +3120,8 @@ static int gather_pte_stats(pmd_t *pmd, unsigned lon= g addr, cond_resched(); return 0; } -#ifdef CONFIG_HUGETLB_PAGE -static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, struct mm_walk *walk) -{ - pte_t huge_pte =3D huge_ptep_get(walk->mm, addr, pte); - struct numa_maps *md; - struct page *page; - - if (!pte_present(huge_pte)) - return 0; - - page =3D pte_page(huge_pte); - - md =3D walk->private; - gather_stats(page, md, pte_dirty(huge_pte), 1); - return 0; -} - -#else -static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, struct mm_walk *walk) -{ - return 0; -} -#endif =20 static const struct mm_walk_ops show_numa_ops =3D { - .hugetlb_entry =3D gather_hugetlb_stats, .pud_entry =3D gather_pud_stats, .pmd_entry =3D gather_pte_stats, .walk_lock =3D PGWALK_RDLOCK, diff --git a/include/linux/pagewalk.h b/include/linux/pagewalk.h index 27cd1e59ccf7..6df0726eecb6 100644 --- a/include/linux/pagewalk.h +++ b/include/linux/pagewalk.h @@ -31,16 +31,6 @@ enum page_walk_lock { * depth is -1 if not known, 0:PGD, 1:P4D, 2:PUD, 3:PMD. * Any folded depths (where PTRS_PER_P?D is equal to 1) * are skipped. - * @hugetlb_entry: if set, called for each hugetlb entry. This hook - * function is called with the vma lock held, in order to - * protect against a concurrent freeing of the pte_t* or - * the ptl. In some cases, the hook function needs to drop - * and retake the vma lock in order to avoid deadlocks - * while calling other functions. In such cases the hook - * function must either refrain from accessing the pte or - * ptl after dropping the vma lock, or else revalidate - * those items after re-acquiring the vma lock and before - * accessing them. * @test_walk: caller specific callback function to determine whether * we walk over the current vma or not. Returning 0 means * "do page table walk over the current vma", returning diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 6a383ce5a775..82a8d3146f05 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -360,63 +360,9 @@ static int damon_mkold_pud_entry(pmd_t *pud, unsigned = long addr, return 0; } =20 -#ifdef CONFIG_HUGETLB_PAGE -static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm, - struct vm_area_struct *vma, unsigned long addr) -{ - bool referenced =3D false; - pte_t entry =3D huge_ptep_get(mm, addr, pte); - struct folio *folio =3D pfn_folio(pte_pfn(entry)); - unsigned long psize =3D huge_page_size(hstate_vma(vma)); - - folio_get(folio); - - if (pte_young(entry)) { - referenced =3D true; - entry =3D pte_mkold(entry); - set_huge_pte_at(mm, addr, pte, entry, psize); - } - -#ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(mm, addr, - addr + huge_page_size(hstate_vma(vma)))) - referenced =3D true; -#endif /* CONFIG_MMU_NOTIFIER */ - - if (referenced) - folio_set_young(folio); - - folio_set_idle(folio); - folio_put(folio); -} - -static int damon_mkold_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct hstate *h =3D hstate_vma(walk->vma); - spinlock_t *ptl; - pte_t entry; - - ptl =3D huge_pte_lock(h, walk->mm, pte); - entry =3D huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) - goto out; - - damon_hugetlb_mkold(pte, walk->mm, walk->vma, addr); - -out: - spin_unlock(ptl); - return 0; -} -#else -#define damon_mkold_hugetlb_entry NULL -#endif /* CONFIG_HUGETLB_PAGE */ - static const struct mm_walk_ops damon_mkold_ops =3D { .pud_entry =3D damon_mkold_pud_entry, .pmd_entry =3D damon_mkold_pmd_entry, - .hugetlb_entry =3D damon_mkold_hugetlb_entry, .walk_lock =3D PGWALK_RDLOCK, }; =20 @@ -562,44 +508,9 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned = long addr, return 0; } =20 -#ifdef CONFIG_HUGETLB_PAGE -static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct damon_young_walk_private *priv =3D walk->private; - struct hstate *h =3D hstate_vma(walk->vma); - struct folio *folio; - spinlock_t *ptl; - pte_t entry; - - ptl =3D huge_pte_lock(h, walk->mm, pte); - entry =3D huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) - goto out; - - folio =3D pfn_folio(pte_pfn(entry)); - folio_get(folio); - - if (pte_young(entry) || !folio_test_idle(folio) || - mmu_notifier_test_young(walk->mm, addr)) - priv->young =3D true; - *priv->folio_sz =3D huge_page_size(h); - - folio_put(folio); - -out: - spin_unlock(ptl); - return 0; -} -#else -#define damon_young_hugetlb_entry NULL -#endif /* CONFIG_HUGETLB_PAGE */ - static const struct mm_walk_ops damon_young_ops =3D { .pud_entry =3D damon_young_pud_entry, .pmd_entry =3D damon_young_pmd_entry, - .hugetlb_entry =3D damon_young_hugetlb_entry, .walk_lock =3D PGWALK_RDLOCK, }; =20 diff --git a/mm/hmm.c b/mm/hmm.c index 2b752f703b6d..fccde5dae818 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -463,59 +463,6 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long= start, unsigned long end, #define hmm_vma_walk_pud NULL #endif =20 -#ifdef CONFIG_HUGETLB_PAGE -static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long start, unsigned long end, - struct mm_walk *walk) -{ - unsigned long addr =3D start, i, pfn; - struct hmm_vma_walk *hmm_vma_walk =3D walk->private; - struct hmm_range *range =3D hmm_vma_walk->range; - struct vm_area_struct *vma =3D walk->vma; - unsigned int required_fault; - unsigned long pfn_req_flags; - unsigned long cpu_flags; - spinlock_t *ptl; - pte_t entry; - - ptl =3D huge_pte_lock(hstate_vma(vma), walk->mm, pte); - entry =3D huge_ptep_get(walk->mm, addr, pte); - - i =3D (start - range->start) >> PAGE_SHIFT; - pfn_req_flags =3D range->hmm_pfns[i]; - cpu_flags =3D pte_to_hmm_pfn_flags(range, entry) | - hmm_pfn_flags_order(huge_page_order(hstate_vma(vma))); - required_fault =3D - hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, cpu_flags); - if (required_fault) { - int ret; - - spin_unlock(ptl); - hugetlb_vma_unlock_read(vma); - /* - * Avoid deadlock: drop the vma lock before calling - * hmm_vma_fault(), which will itself potentially take and - * drop the vma lock. This is also correct from a - * protection point of view, because there is no further - * use here of either pte or ptl after dropping the vma - * lock. - */ - ret =3D hmm_vma_fault(addr, end, required_fault, walk); - hugetlb_vma_lock_read(vma); - return ret; - } - - pfn =3D pte_pfn(entry) + ((start & ~hmask) >> PAGE_SHIFT); - for (; addr < end; addr +=3D PAGE_SIZE, i++, pfn++) - range->hmm_pfns[i] =3D pfn | cpu_flags; - - spin_unlock(ptl); - return 0; -} -#else -#define hmm_vma_walk_hugetlb_entry NULL -#endif /* CONFIG_HUGETLB_PAGE */ - static int hmm_vma_walk_test(unsigned long start, unsigned long end, struct mm_walk *walk) { @@ -554,7 +501,6 @@ static const struct mm_walk_ops hmm_walk_ops =3D { .pud_entry =3D hmm_vma_walk_pud, .pmd_entry =3D hmm_vma_walk_pmd, .pte_hole =3D hmm_vma_walk_hole, - .hugetlb_entry =3D hmm_vma_walk_hugetlb_entry, .test_walk =3D hmm_vma_walk_test, .walk_lock =3D PGWALK_RDLOCK, }; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 622862c4c300..c4ce4cf16651 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -882,26 +882,9 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned lo= ng addr, return ret; } =20 -#ifdef CONFIG_HUGETLB_PAGE -static int hwpoison_hugetlb_range(pte_t *ptep, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct hwpoison_walk *hwp =3D walk->private; - pte_t pte =3D huge_ptep_get(walk->mm, addr, ptep); - struct hstate *h =3D hstate_vma(walk->vma); - - return check_hwpoisoned_entry(pte, addr, huge_page_shift(h), - hwp->pfn, &hwp->tk); -} -#else -#define hwpoison_hugetlb_range NULL -#endif - static const struct mm_walk_ops hwpoison_walk_ops =3D { .pud_entry =3D hwpoison_pud_range, .pmd_entry =3D hwpoison_pte_range, - .hugetlb_entry =3D hwpoison_hugetlb_range, .walk_lock =3D PGWALK_RDLOCK, }; =20 diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 93b14090d484..8b5ca719193c 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -645,51 +645,6 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned= long addr, return 0; } =20 -static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ -#ifdef CONFIG_HUGETLB_PAGE - struct queue_pages *qp =3D walk->private; - unsigned long flags =3D qp->flags; - struct folio *folio; - spinlock_t *ptl; - pte_t entry; - - ptl =3D huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); - entry =3D huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) { - if (unlikely(is_hugetlb_entry_migration(entry))) - qp->nr_failed++; - goto unlock; - } - folio =3D pfn_folio(pte_pfn(entry)); - if (!queue_folio_required(folio, qp)) - goto unlock; - if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || - !vma_migratable(walk->vma)) { - qp->nr_failed++; - goto unlock; - } - /* - * Unless MPOL_MF_MOVE_ALL, we try to avoid migrating a shared folio. - * Choosing not to migrate a shared folio is not counted as a failure. - * - * See folio_likely_mapped_shared() on possible imprecision when we - * cannot easily detect if a folio is shared. - */ - if ((flags & MPOL_MF_MOVE_ALL) || - (!folio_likely_mapped_shared(folio) && !hugetlb_pmd_shared(pte))) - if (!isolate_hugetlb(folio, qp->pagelist)) - qp->nr_failed++; -unlock: - spin_unlock(ptl); - if (qp->nr_failed && strictly_unmovable(flags)) - return -EIO; -#endif - return 0; -} - #ifdef CONFIG_NUMA_BALANCING /* * This is used to mark a range of virtual addresses to be inaccessible. @@ -760,7 +715,6 @@ static int queue_pages_test_walk(unsigned long start, u= nsigned long end, } =20 static const struct mm_walk_ops queue_pages_walk_ops =3D { - .hugetlb_entry =3D queue_folios_hugetlb, .pud_entry =3D queue_folios_pud, .pmd_entry =3D queue_folios_pte_range, .test_walk =3D queue_pages_test_walk, @@ -768,7 +722,6 @@ static const struct mm_walk_ops queue_pages_walk_ops = =3D { }; =20 static const struct mm_walk_ops queue_pages_lock_vma_walk_ops =3D { - .hugetlb_entry =3D queue_folios_hugetlb, .pmd_entry =3D queue_folios_pte_range, .test_walk =3D queue_pages_test_walk, .walk_lock =3D PGWALK_WRLOCK, diff --git a/mm/mincore.c b/mm/mincore.c index 786df7246899..26f699a47371 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -23,27 +23,6 @@ #include #include "swap.h" =20 -static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long = addr, - unsigned long end, struct mm_walk *walk) -{ -#ifdef CONFIG_HUGETLB_PAGE - unsigned char present; - unsigned char *vec =3D walk->private; - - /* - * Hugepages under user process are always in RAM and never - * swapped out, but theoretically it needs to be checked. - */ - present =3D pte && !huge_pte_none_mostly(huge_ptep_get(walk->mm, addr, pt= e)); - for (; addr !=3D end; vec++, addr +=3D PAGE_SIZE) - *vec =3D present; - walk->private =3D vec; -#else - BUG(); -#endif - return 0; -} - /* * Later we can get more picky about what "in core" means precisely. * For now, simply check to see if the page is in the page cache, @@ -197,7 +176,6 @@ static const struct mm_walk_ops mincore_walk_ops =3D { .pud_entry =3D mincore_pud_range, .pmd_entry =3D mincore_pte_range, .pte_hole =3D mincore_unmapped_range, - .hugetlb_entry =3D mincore_hugetlb, .walk_lock =3D PGWALK_RDLOCK, }; =20 diff --git a/mm/mprotect.c b/mm/mprotect.c index 222ab434da54..ca1962d5cb95 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -555,15 +555,6 @@ static int prot_none_pte_entry(pte_t *pte, unsigned lo= ng addr, 0 : -EACCES; } =20 -static int prot_none_hugetlb_entry(pte_t *pte, unsigned long hmask, - unsigned long addr, unsigned long next, - struct mm_walk *walk) -{ - return pfn_modify_allowed(pte_pfn(ptep_get(pte)), - *(pgprot_t *)(walk->private)) ? - 0 : -EACCES; -} - static int prot_none_test(unsigned long addr, unsigned long next, struct mm_walk *walk) { @@ -572,7 +563,6 @@ static int prot_none_test(unsigned long addr, unsigned = long next, =20 static const struct mm_walk_ops prot_none_walk_ops =3D { .pte_entry =3D prot_none_pte_entry, - .hugetlb_entry =3D prot_none_hugetlb_entry, .test_walk =3D prot_none_test, .walk_lock =3D PGWALK_WRLOCK, }; diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 78d45f1450aa..7e2721f49e68 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -257,49 +257,6 @@ static int walk_pgd_range(unsigned long addr, unsigned= long end, return err; } =20 -#ifdef CONFIG_HUGETLB_PAGE -static unsigned long hugetlb_entry_end(struct hstate *h, unsigned long add= r, - unsigned long end) -{ - unsigned long boundary =3D (addr & huge_page_mask(h)) + huge_page_size(h); - return boundary < end ? boundary : end; -} - -static int walk_hugetlb_range(unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - struct vm_area_struct *vma =3D walk->vma; - struct hstate *h =3D hstate_vma(vma); - unsigned long next; - unsigned long hmask =3D huge_page_mask(h); - unsigned long sz =3D huge_page_size(h); - pte_t *pte; - const struct mm_walk_ops *ops =3D walk->ops; - int err =3D 0; - - do { - next =3D hugetlb_entry_end(h, addr, end); - pte =3D hugetlb_walk(vma, addr & hmask, sz); - if (pte) - err =3D ops->hugetlb_entry(pte, hmask, addr, next, walk); - else if (ops->pte_hole) - err =3D ops->pte_hole(addr, next, -1, walk); - if (err) - break; - } while (addr =3D next, addr !=3D end); - - return err; -} - -#else /* CONFIG_HUGETLB_PAGE */ -static int walk_hugetlb_range(unsigned long addr, unsigned long end, - struct mm_walk *walk) -{ - return 0; -} - -#endif /* CONFIG_HUGETLB_PAGE */ - /* * Decide whether we really walk over the current vma on [@start, @end) * or skip it via the returned value. Return 0 if we do walk over the @@ -346,11 +303,7 @@ static int __walk_page_range(unsigned long start, unsi= gned long end, } =20 vma_pgtable_walk_begin(vma); - if (is_vm_hugetlb_page(vma)) { - if (ops->hugetlb_entry) - err =3D walk_hugetlb_range(start, end, walk); - } else - err =3D walk_pgd_range(start, end, walk); + err =3D walk_pgd_range(start, end, walk); vma_pgtable_walk_end(vma); =20 if (ops->post_vma) --=20 2.26.2