From nobody Thu Oct 9 02:56:38 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B92FE1C5D7B for ; Fri, 20 Jun 2025 12:30:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422639; cv=none; b=RRQeFrQfJUnRqp3qBJaTo7DVp3HasqNgqMt2Au//W2I9XoPVilubV5D34DbBPOtIwDqx7hE+sFHfBOhmZDjemkn5BMU7NAxyD6C1bh0367RMwW73KpYUIEEijCgyzRVemuSvCeq7YW8+yZaa1PamJJZg86X7PVirS215l46X+fc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422639; c=relaxed/simple; bh=DYyPW92qnthPMo2Nw/18u31telCzzahE5NRBoDa8YhA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pEpwu3gqjW6vY8sPbeMy6FzwGgQvBAFBplmyq6+K0Cn/7CWXSetcrZUrnlMK24hguCEguOU2UJbFqfAr8imSuzB1PxyXZWA41yhrUH88nHHYf3Ws51+iyrkSVbnejBpvuy7NjioFMU9ccUWP+GHonwRrAmNf0ekUhjmRHHNbHiA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 41FE421222; Fri, 20 Jun 2025 12:30:28 +0000 (UTC) Authentication-Results: smtp-out1.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C3EBF13A99; Fri, 20 Jun 2025 12:30:27 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UJ0cLWNUVWjNKAAAD6G6ig (envelope-from ); Fri, 20 Jun 2025 12:30:27 +0000 From: Oscar Salvador To: Andrew Morton Cc: David Hildenbrand , Muchun Song , Peter Xu , Gavin Guo , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH v2 1/5] mm,hugetlb: Change mechanism to detect a COW on private mapping Date: Fri, 20 Jun 2025 14:30:10 +0200 Message-ID: <20250620123014.29748-2-osalvador@suse.de> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250620123014.29748-1-osalvador@suse.de> References: <20250620123014.29748-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spam-Level: X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Spam-Flag: NO X-Rspamd-Queue-Id: 41FE421222 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Score: -4.00 Content-Type: text/plain; charset="utf-8" hugetlb_wp() checks whether the process is trying to COW on a private mappi= ng in order to know whether the reservation for that address was already consu= med or not. If it was consumed and we are the ownner of the mapping, the folio will hav= e to be unmapped from the other processes. Currently, that check is done by looking up the folio in the pagecache and compare it to the folio which is mapped in our pagetables. If it differs, it means we already mapped it privately before, consuming a reservation on the way. All we are interested in is whether the mapped folio is anonymous, so we can simplify and check for that instead. Also, we transition from a trylock to a folio_lock, since the former was on= ly needed when hugetlb_fault() had to lock both folios, in order to avoid dead= lock. Fixes: 40549ba8f8e0 ("hugetlb: use new vma_lock for pmd sharing synchroniza= tion") Reported-by: Gavin Guo Closes: https://lore.kernel.org/lkml/20250513093448.592150-1-gavinguo@igali= a.com/ Signed-off-by: Oscar Salvador Suggested-by: Peter Xu Acked-by: David Hildenbrand --- mm/hugetlb.c | 70 +++++++++++++--------------------------------------- 1 file changed, 17 insertions(+), 53 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 8746ed2fec13..175edafeec67 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6152,8 +6152,7 @@ static void unmap_ref_private(struct mm_struct *mm, s= truct vm_area_struct *vma, * cannot race with other handlers or page migration. * Keep the pte_same checks anyway to make transition from the mutex easie= r. */ -static vm_fault_t hugetlb_wp(struct folio *pagecache_folio, - struct vm_fault *vmf) +static vm_fault_t hugetlb_wp(struct vm_fault *vmf) { struct vm_area_struct *vma =3D vmf->vma; struct mm_struct *mm =3D vma->vm_mm; @@ -6215,16 +6214,17 @@ static vm_fault_t hugetlb_wp(struct folio *pagecach= e_folio, PageAnonExclusive(&old_folio->page), &old_folio->page); =20 /* - * If the process that created a MAP_PRIVATE mapping is about to - * perform a COW due to a shared page count, attempt to satisfy - * the allocation without using the existing reserves. The pagecache - * page is used to determine if the reserve at this address was - * consumed or not. If reserves were used, a partial faulted mapping - * at the time of fork() could consume its reserves on COW instead - * of the full address range. + * If the process that created a MAP_PRIVATE mapping is about to perform + * a COW due to a shared page count, attempt to satisfy the allocation + * without using the existing reserves. + * In order to determine where this is a COW on a MAP_PRIVATE mapping it + * is enough to check whether the old_folio is anonymous. This means that + * the reserve for this address was consumed. If reserves were used, a + * partial faulted mapping at the fime of fork() could consume its reserv= es + * on COW instead of the full address range. */ if (is_vma_resv_set(vma, HPAGE_RESV_OWNER) && - old_folio !=3D pagecache_folio) + folio_test_anon(old_folio)) cow_from_owner =3D true; =20 folio_get(old_folio); @@ -6603,7 +6603,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, hugetlb_count_add(pages_per_huge_page(h), mm); if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { /* Optimization, do the COW without a second fault */ - ret =3D hugetlb_wp(folio, vmf); + ret =3D hugetlb_wp(vmf); } =20 spin_unlock(vmf->ptl); @@ -6670,11 +6670,9 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, { vm_fault_t ret; u32 hash; - struct folio *folio =3D NULL; - struct folio *pagecache_folio =3D NULL; + struct folio *folio; struct hstate *h =3D hstate_vma(vma); struct address_space *mapping; - int need_wait_lock =3D 0; struct vm_fault vmf =3D { .vma =3D vma, .address =3D address & huge_page_mask(h), @@ -6769,8 +6767,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, * If we are going to COW/unshare the mapping later, we examine the * pending reservations for this page now. This will ensure that any * allocations necessary to record that reservation occur outside the - * spinlock. Also lookup the pagecache page now as it is used to - * determine if a reservation has been consumed. + * spinlock. */ if ((flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) && !(vma->vm_flags & VM_MAYSHARE) && !huge_pte_write(vmf.orig_pte)) { @@ -6780,11 +6777,6 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, } /* Just decrements count, does not deallocate */ vma_end_reservation(h, vma, vmf.address); - - pagecache_folio =3D filemap_lock_hugetlb_folio(h, mapping, - vmf.pgoff); - if (IS_ERR(pagecache_folio)) - pagecache_folio =3D NULL; } =20 vmf.ptl =3D huge_pte_lock(h, mm, vmf.pte); @@ -6798,10 +6790,6 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, (flags & FAULT_FLAG_WRITE) && !huge_pte_write(vmf.orig_pte)) { if (!userfaultfd_wp_async(vma)) { spin_unlock(vmf.ptl); - if (pagecache_folio) { - folio_unlock(pagecache_folio); - folio_put(pagecache_folio); - } hugetlb_vma_unlock_read(vma); mutex_unlock(&hugetlb_fault_mutex_table[hash]); return handle_userfault(&vmf, VM_UFFD_WP); @@ -6813,23 +6801,14 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, stru= ct vm_area_struct *vma, /* Fallthrough to CoW */ } =20 - /* - * hugetlb_wp() requires page locks of pte_page(vmf.orig_pte) and - * pagecache_folio, so here we need take the former one - * when folio !=3D pagecache_folio or !pagecache_folio. - */ + /* hugetlb_wp() requires page locks of pte_page(vmf.orig_pte) */ folio =3D page_folio(pte_page(vmf.orig_pte)); - if (folio !=3D pagecache_folio) - if (!folio_trylock(folio)) { - need_wait_lock =3D 1; - goto out_ptl; - } - + folio_lock(folio); folio_get(folio); =20 if (flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) { if (!huge_pte_write(vmf.orig_pte)) { - ret =3D hugetlb_wp(pagecache_folio, &vmf); + ret =3D hugetlb_wp(&vmf); goto out_put_page; } else if (likely(flags & FAULT_FLAG_WRITE)) { vmf.orig_pte =3D huge_pte_mkdirty(vmf.orig_pte); @@ -6840,16 +6819,10 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, stru= ct vm_area_struct *vma, flags & FAULT_FLAG_WRITE)) update_mmu_cache(vma, vmf.address, vmf.pte); out_put_page: - if (folio !=3D pagecache_folio) - folio_unlock(folio); + folio_unlock(folio); folio_put(folio); out_ptl: spin_unlock(vmf.ptl); - - if (pagecache_folio) { - folio_unlock(pagecache_folio); - folio_put(pagecache_folio); - } out_mutex: hugetlb_vma_unlock_read(vma); =20 @@ -6861,15 +6834,6 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, vma_end_read(vma); =20 mutex_unlock(&hugetlb_fault_mutex_table[hash]); - /* - * Generally it's safe to hold refcount during waiting page lock. But - * here we just wait to defer the next page fault to avoid busy loop and - * the page is not used after unlocked before returning from the current - * page fault. So we are safe from accessing freed page, even if we wait - * here without taking refcount. - */ - if (need_wait_lock) - folio_wait_locked(folio); return ret; } =20 --=20 2.50.0 From nobody Thu Oct 9 02:56:38 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A69B628A41E for ; Fri, 20 Jun 2025 12:30:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422645; cv=none; b=u+p+s7nIInXPikt5emrFh7e4F5hY2G5lzVU6MJyozcu1NdNnrnUjMGiba5VQYvSa8Bu7JzmUpaBw9dJ50/oUx+5gGZ1lueOMlnwxOQ8i3ZjfrD/cVUm3d9ArU4qCHkHilLHvzKkPyQ6iZtEs38XqYXTulQIk1kMwmlLALXK1rDM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422645; c=relaxed/simple; bh=4C18bo8F6uv6nuTgpqPUCETw/chARsIy06IarRrNeUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M2q/Mea/lBmjiNcDC3sf9HAK4YtY5Vcbc88q5yXe1BfwTohGDhvW3C/0YDkpjlnJX6MwUaAQeFeEgBHohQ1eG112WgASv8Qe/2cb+7bu41leFel7Ng+emIeAz7ve41sdBOQX8D5usMO4XiHH77K3kKV5I/idvuH3mZ+zzZ7GEgc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=TDzkiczE; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=VF4FnFI/; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=TDzkiczE; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=VF4FnFI/; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="TDzkiczE"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="VF4FnFI/"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="TDzkiczE"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="VF4FnFI/" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C911221247; Fri, 20 Jun 2025 12:30:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=TDzkiczET41mkTCZ8P+9YHuEwGuADw2CzQx/o1hn7ns7w7yMTnNbNVgXif+1m/jDTERJqn G3GiLN9f8mKKauSQ8/BTgjwf6NFK0RWMKCBrqm1hDego/dqtEjUK2Veimpi3+uJmnx6gVh qwVUnB4gui8K52rX8iqbhigQoZanmyY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=VF4FnFI/duTAeAi8flp4qiCCCYV7DqbpWOX4wEkg1rPGe9TofoTw9rJkCllRMjGsTe0Frd ZTD2YeS7XkEqFrBA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=TDzkiczET41mkTCZ8P+9YHuEwGuADw2CzQx/o1hn7ns7w7yMTnNbNVgXif+1m/jDTERJqn G3GiLN9f8mKKauSQ8/BTgjwf6NFK0RWMKCBrqm1hDego/dqtEjUK2Veimpi3+uJmnx6gVh qwVUnB4gui8K52rX8iqbhigQoZanmyY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=VF4FnFI/duTAeAi8flp4qiCCCYV7DqbpWOX4wEkg1rPGe9TofoTw9rJkCllRMjGsTe0Frd ZTD2YeS7XkEqFrBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 549D4136BA; Fri, 20 Jun 2025 12:30:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id GLbyEWRUVWjNKAAAD6G6ig (envelope-from ); Fri, 20 Jun 2025 12:30:28 +0000 From: Oscar Salvador To: Andrew Morton Cc: David Hildenbrand , Muchun Song , Peter Xu , Gavin Guo , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH v2 2/5] mm,hugetlb: Sort out folio locking in the faulting path Date: Fri, 20 Jun 2025 14:30:11 +0200 Message-ID: <20250620123014.29748-3-osalvador@suse.de> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250620123014.29748-1-osalvador@suse.de> References: <20250620123014.29748-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_SEVEN(0.00)[8]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Level: X-Spam-Flag: NO X-Spam-Score: -6.80 Content-Type: text/plain; charset="utf-8" Recent conversations showed that there was a misunderstanding about why we were locking the folio prior to call in hugetlb_wp(). In fact, as soon as we have the folio mapped into the pagetables, we no lon= ger need to hold it locked, because we know that no concurrent truncation could= have happened. There is only one case where the folio needs to be locked, and that is when= we are handling an anonymous folio, because hugetlb_wp() will check whether it= can re-use it exclusively for the process that is faulting it in. So, pass the folio locked to hugetlb_wp() when that is the case. Suggested-by: David Hildenbrand Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 43 +++++++++++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 10 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 175edafeec67..1a5f713c1e4c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6437,6 +6437,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, pte_t new_pte; bool new_folio, new_pagecache_folio =3D false; u32 hash =3D hugetlb_fault_mutex_hash(mapping, vmf->pgoff); + bool folio_locked =3D true; =20 /* * Currently, we are forced to kill the process in the event the @@ -6602,6 +6603,11 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, =20 hugetlb_count_add(pages_per_huge_page(h), mm); if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { + /* No need to lock file folios. See comment in hugetlb_fault() */ + if (!anon_rmap) { + folio_locked =3D false; + folio_unlock(folio); + } /* Optimization, do the COW without a second fault */ ret =3D hugetlb_wp(vmf); } @@ -6616,7 +6622,8 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, if (new_folio) folio_set_hugetlb_migratable(folio); =20 - folio_unlock(folio); + if (folio_locked) + folio_unlock(folio); out: hugetlb_vma_unlock_read(vma); =20 @@ -6636,7 +6643,8 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, if (new_folio && !new_pagecache_folio) restore_reserve_on_error(h, vma, vmf->address, folio); =20 - folio_unlock(folio); + if (folio_locked) + folio_unlock(folio); folio_put(folio); goto out; } @@ -6670,7 +6678,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, { vm_fault_t ret; u32 hash; - struct folio *folio; + struct folio *folio =3D NULL; struct hstate *h =3D hstate_vma(vma); struct address_space *mapping; struct vm_fault vmf =3D { @@ -6687,6 +6695,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, * be hard to debug if called functions make assumptions */ }; + bool folio_locked =3D false; =20 /* * Serialize hugepage allocation and instantiation, so that we don't @@ -6801,13 +6810,24 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, stru= ct vm_area_struct *vma, /* Fallthrough to CoW */ } =20 - /* hugetlb_wp() requires page locks of pte_page(vmf.orig_pte) */ - folio =3D page_folio(pte_page(vmf.orig_pte)); - folio_lock(folio); - folio_get(folio); - if (flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) { if (!huge_pte_write(vmf.orig_pte)) { + /* + * Anonymous folios need to be lock since hugetlb_wp() + * checks whether we can re-use the folio exclusively + * for us in case we are the only user of it. + */ + folio =3D page_folio(pte_page(vmf.orig_pte)); + folio_get(folio); + if (folio_test_anon(folio)) { + spin_unlock(vmf.ptl); + folio_lock(folio); + folio_locked =3D true; + spin_lock(vmf.ptl); + if (unlikely(!pte_same(vmf.orig_pte, huge_ptep_get(mm, + vmf.address, vmf.pte)))) + goto out_put_page; + } ret =3D hugetlb_wp(&vmf); goto out_put_page; } else if (likely(flags & FAULT_FLAG_WRITE)) { @@ -6819,8 +6839,11 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, flags & FAULT_FLAG_WRITE)) update_mmu_cache(vma, vmf.address, vmf.pte); out_put_page: - folio_unlock(folio); - folio_put(folio); + if (folio) { + if (folio_locked) + folio_unlock(folio); + folio_put(folio); + } out_ptl: spin_unlock(vmf.ptl); out_mutex: --=20 2.50.0 From nobody Thu Oct 9 02:56:38 2025 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEE6B1DA3D for ; Fri, 20 Jun 2025 12:30:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422632; cv=none; b=gvp0TSJehrVMzT4GszZgiH7mNYQapUonI8yd/G5xnbHjCP50WYsi7EMYHgC5K6iS+Q6n93xm25zkqjzqJoNSt2CCN2T/d7lEZqpVFeDkKZswf5pIRR/eL2xa/A29akMzzQnM0rZ9jKbBKB3YnuduV7iWKr2Iu/SOQp2Ce1ZMGTQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422632; c=relaxed/simple; bh=S8ywCHIrPv1TFxfhggKzpeG1YWigLuz2M8Wv8Ahtddo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TWRwXsYJgbdyz1bmEYcV+lJIHC8AwhAANJcqrlyckm1z8NJLeh0YfPcWTzqZQqAotuXn016PhwdGtviBE/MNVreLWKyaqt5vDPqmg7KZGq4RhWLjTd4wq2QI35lna+t+xkrlOIOB+35ICYG+zxe+BQbul8UkQyBDX1J2R2vsbnw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 665F31F7CF; Fri, 20 Jun 2025 12:30:29 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DE45513A99; Fri, 20 Jun 2025 12:30:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id eH2VM2RUVWjNKAAAD6G6ig (envelope-from ); Fri, 20 Jun 2025 12:30:28 +0000 From: Oscar Salvador To: Andrew Morton Cc: David Hildenbrand , Muchun Song , Peter Xu , Gavin Guo , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH v2 3/5] mm,hugetlb: Rename anon_rmap to new_anon_folio and make it boolean Date: Fri, 20 Jun 2025 14:30:12 +0200 Message-ID: <20250620123014.29748-4-osalvador@suse.de> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250620123014.29748-1-osalvador@suse.de> References: <20250620123014.29748-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spam-Level: X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Spam-Flag: NO X-Rspamd-Queue-Id: 665F31F7CF X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Score: -4.00 Content-Type: text/plain; charset="utf-8" anon_rmap is used to determine whether the new allocated folio is anonymous. Rename it to something more meaningul like new_anon_folio and make it boole= an, as we use it like that. While we are at it, drop 'new_pagecache_folio' as 'new_anon_folio' is enoug= h to check whether we need to restore the consumed reservation. Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1a5f713c1e4c..57bb8b2dce21 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6427,17 +6427,16 @@ static bool hugetlb_pte_stable(struct hstate *h, st= ruct mm_struct *mm, unsigned static vm_fault_t hugetlb_no_page(struct address_space *mapping, struct vm_fault *vmf) { + u32 hash =3D hugetlb_fault_mutex_hash(mapping, vmf->pgoff); + bool new_folio, new_anon_folio =3D false; struct vm_area_struct *vma =3D vmf->vma; struct mm_struct *mm =3D vma->vm_mm; struct hstate *h =3D hstate_vma(vma); vm_fault_t ret =3D VM_FAULT_SIGBUS; - int anon_rmap =3D 0; - unsigned long size; + bool folio_locked =3D true; struct folio *folio; + unsigned long size; pte_t new_pte; - bool new_folio, new_pagecache_folio =3D false; - u32 hash =3D hugetlb_fault_mutex_hash(mapping, vmf->pgoff); - bool folio_locked =3D true; =20 /* * Currently, we are forced to kill the process in the event the @@ -6518,6 +6517,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio =3D true; + new_anon_folio =3D !(vma->vm_flags & VM_MAYSHARE); =20 if (vma->vm_flags & VM_MAYSHARE) { int err =3D hugetlb_add_to_page_cache(folio, mapping, @@ -6536,10 +6536,8 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, ret =3D VM_FAULT_SIGBUS; goto out; } - new_pagecache_folio =3D true; } else { folio_lock(folio); - anon_rmap =3D 1; } } else { /* @@ -6588,7 +6586,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, if (!pte_same(huge_ptep_get(mm, vmf->address, vmf->pte), vmf->orig_pte)) goto backout; =20 - if (anon_rmap) + if (new_anon_folio) hugetlb_add_new_anon_rmap(folio, vma, vmf->address); else hugetlb_add_file_rmap(folio); @@ -6604,7 +6602,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, hugetlb_count_add(pages_per_huge_page(h), mm); if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { /* No need to lock file folios. See comment in hugetlb_fault() */ - if (!anon_rmap) { + if (!new_anon_folio) { folio_locked =3D false; folio_unlock(folio); } @@ -6640,7 +6638,8 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, backout: spin_unlock(vmf->ptl); backout_unlocked: - if (new_folio && !new_pagecache_folio) + /* We only need to restore reservations for private mappings */ + if (new_folio && new_anon_folio) restore_reserve_on_error(h, vma, vmf->address, folio); =20 if (folio_locked) --=20 2.50.0 From nobody Thu Oct 9 02:56:38 2025 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A12831DA3D for ; Fri, 20 Jun 2025 12:30:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422638; cv=none; b=X0NGPu/3o1ielCWpy00TNlCH8nKjnVoDa8Tfa30HKhKwTI/as5chPyIjqWQoZsBnUYLrXURhS+15TtFvQ2Oni8XHAkY6qIykZiNQQkhhaKKmO6o+JNYucK+I9LcwihDaSAlRQfkg8qaJGlvtFcf/Z9NBaeYhEhBpN/hVHqUuOds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422638; c=relaxed/simple; bh=PeZJy0irop+IqFUTm3kwAzdkFYJc3Owr/GcP3sRc0lI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eMpE0faLL6noexchkhMzjniy33PXdl9MKfTrs4BV0p0mEsMh52J8lz1D1/c/4wqDhxQl5mEKqIZT8UyAhGBaxNwQbUwO3TmxKeasZj6Mf9ZDAsUBL0tS3ZCPwsELLPG37dD4zrymTe88dSws+5ZORXjJSj3owsIrhHzgKZoAgUk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=1G+OesWg; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=MKuKDAP2; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=1G+OesWg; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=MKuKDAP2; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="1G+OesWg"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="MKuKDAP2"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="1G+OesWg"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="MKuKDAP2" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id F349F1F7D1; Fri, 20 Jun 2025 12:30:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1750422630; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pqj2rggzXteZzAUJN8tx9Q855H+tvUXgTa4vK7ebx5c=; b=1G+OesWgkHDerbnxpS/4O0VE493I5+0vzf8Aq9NuP5NbsNk6vI1nOXnJJkWdLYPbOzRztZ M1CnlnBUPRvzuXDva4UssDsRfpZ61O4EvyRpeBOp5vOOZeBeDa5h0V0PDGTFtjPmkCIAGZ 69QzSNITI8SqjStx9h20MCLEC9bkyUg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1750422630; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pqj2rggzXteZzAUJN8tx9Q855H+tvUXgTa4vK7ebx5c=; b=MKuKDAP2NL8oZHqlEBFsKROK4xnQRY4cSAvbwZK/4Dn8lt9SLzNgNM1mm+o5wUg8Okg08z u6JbW/BzU+jv4WBQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1750422630; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pqj2rggzXteZzAUJN8tx9Q855H+tvUXgTa4vK7ebx5c=; b=1G+OesWgkHDerbnxpS/4O0VE493I5+0vzf8Aq9NuP5NbsNk6vI1nOXnJJkWdLYPbOzRztZ M1CnlnBUPRvzuXDva4UssDsRfpZ61O4EvyRpeBOp5vOOZeBeDa5h0V0PDGTFtjPmkCIAGZ 69QzSNITI8SqjStx9h20MCLEC9bkyUg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1750422630; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pqj2rggzXteZzAUJN8tx9Q855H+tvUXgTa4vK7ebx5c=; b=MKuKDAP2NL8oZHqlEBFsKROK4xnQRY4cSAvbwZK/4Dn8lt9SLzNgNM1mm+o5wUg8Okg08z u6JbW/BzU+jv4WBQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7962B136BA; Fri, 20 Jun 2025 12:30:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id EGLmGmVUVWjNKAAAD6G6ig (envelope-from ); Fri, 20 Jun 2025 12:30:29 +0000 From: Oscar Salvador To: Andrew Morton Cc: David Hildenbrand , Muchun Song , Peter Xu , Gavin Guo , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH v2 4/5] mm,hugetlb: Drop obsolete comment about non-present pte and second faults Date: Fri, 20 Jun 2025 14:30:13 +0200 Message-ID: <20250620123014.29748-5-osalvador@suse.de> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250620123014.29748-1-osalvador@suse.de> References: <20250620123014.29748-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Flag: NO X-Spam-Score: -6.80 X-Spamd-Result: default: False [-6.80 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_SEVEN(0.00)[8]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:mid,suse.de:email]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Level: Content-Type: text/plain; charset="utf-8" There is a comment in hugetlb_fault() that does not hold anymore. This one: /* * vmf.orig_pte could be a migration/hwpoison vmf.orig_pte at this * point, so this check prevents the kernel from going below assuming * that we have an active hugepage in pagecache. This goto expects * the 2nd page fault, and is_hugetlb_entry_(migration|hwpoisoned) * check will properly handle it. */ This was written because back in the day we used to do: hugetlb_fault () { ptep =3D huge_pte_offset(...) if (ptep) { entry =3D huge_ptep_get(ptep) if (unlikely(is_hugetlb_entry_migration(entry)) ... else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) ... } ... ... /* * entry could be a migration/hwpoison entry at this point, so this * check prevents the kernel from going below assuming that we have * a active hugepage in pagecache. This goto expects the 2nd page fault, * and is_hugetlb_entry_(migration|hwpoisoned) check will properly * handle it. */ if (!pte_present(entry)) goto out_mutex; ... } The code was designed to check for hwpoisoned/migration entries upfront, and then bail out if further down the pte was not present anymore, relying on the second fault to properly handle migration/hwpoison entries t= hat time around. The way we handle this is different nowadays, so drop the misleading commen= t. Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 57bb8b2dce21..868ee8ed45c0 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6746,13 +6746,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, =20 ret =3D 0; =20 - /* - * vmf.orig_pte could be a migration/hwpoison vmf.orig_pte at this - * point, so this check prevents the kernel from going below assuming - * that we have an active hugepage in pagecache. This goto expects - * the 2nd page fault, and is_hugetlb_entry_(migration|hwpoisoned) - * check will properly handle it. - */ + /* Not present, either a migration or a hwpoisoned entry */ if (!pte_present(vmf.orig_pte)) { if (unlikely(is_hugetlb_entry_migration(vmf.orig_pte))) { /* --=20 2.50.0 From nobody Thu Oct 9 02:56:38 2025 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B12D91C5D7B for ; Fri, 20 Jun 2025 12:30:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422644; cv=none; b=k50qY1Y73g2gyvnhoHVVZwswv+q/nCE17bGBUL7MuxH6gReks24LjCeTCTxkOf5327HJpF6s3WrD2jD0+rzSYu+DvTT3zuZgPXGtmZb2pUZAvR3XNPDVd0pSYRJ/FSXgBuxUkAzVh3dZWfJmgweXbYIB1iJ/E3RlsBlBKqJtV7Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750422644; c=relaxed/simple; bh=07/pZfCDAJ9okFUghNJ3W/JUw1tdKSQ0ShoQtA0MFrI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KzMkybh1sYerxMBVBkrm6oKn5GfBoVomsIhxKBFPubPrBzvhtcgrzbTGUsmNXvUSV9k2Hhz67/N5vCd66qDN4XNhiK9zd4zBHNLU7Z0ypEi6dC/Iugehili7YFtsXb94AG0SyKELSoYDWX34Ph81eZSxA3QP3WR0tSBbpaNebNA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 877541F38D; Fri, 20 Jun 2025 12:30:30 +0000 (UTC) Authentication-Results: smtp-out2.suse.de; none Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1077613A99; Fri, 20 Jun 2025 12:30:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id sH9WAWZUVWjNKAAAD6G6ig (envelope-from ); Fri, 20 Jun 2025 12:30:30 +0000 From: Oscar Salvador To: Andrew Morton Cc: David Hildenbrand , Muchun Song , Peter Xu , Gavin Guo , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH v2 5/5] mm,hugetlb: Drop unlikelys from hugetlb_fault Date: Fri, 20 Jun 2025 14:30:14 +0200 Message-ID: <20250620123014.29748-6-osalvador@suse.de> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250620123014.29748-1-osalvador@suse.de> References: <20250620123014.29748-1-osalvador@suse.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 50.00]; REPLY(-4.00)[] X-Rspamd-Queue-Id: 877541F38D X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Spam-Level: X-Spam-Flag: NO X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spam-Score: -4.00 Content-Type: text/plain; charset="utf-8" The unlikely predates an era where we were checking for hwpoisoned/migration entries prior to checking whether the pte was present. Currently, we check for the pte to be a migration/hwpoison entry after we have checked that is not present, so it must be either one or the other. Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 868ee8ed45c0..1ef554b6934a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6748,7 +6748,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, =20 /* Not present, either a migration or a hwpoisoned entry */ if (!pte_present(vmf.orig_pte)) { - if (unlikely(is_hugetlb_entry_migration(vmf.orig_pte))) { + if (is_hugetlb_entry_migration(vmf.orig_pte)) { /* * Release the hugetlb fault lock now, but retain * the vma lock, because it is needed to guard the @@ -6759,7 +6759,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, mutex_unlock(&hugetlb_fault_mutex_table[hash]); migration_entry_wait_huge(vma, vmf.address, vmf.pte); return 0; - } else if (unlikely(is_hugetlb_entry_hwpoisoned(vmf.orig_pte))) + } else if (is_hugetlb_entry_hwpoisoned(vmf.orig_pte)) ret =3D VM_FAULT_HWPOISON_LARGE | VM_FAULT_SET_HINDEX(hstate_index(h)); goto out_mutex; --=20 2.50.0