From nobody Wed Dec 17 07:06:56 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E6777CF30 for ; Wed, 22 May 2024 21:03:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411832; cv=none; b=Q6+HfNW933qcBVcZHtRNNwRw5HcyENBPzwCqiZjoUiGX1Fs8lfQbe+JssWLHDv8AVMGRV4dOwgc6tsl3/kXtPn42bxcQclPU87IEN9DLWxX8GVEixGNDQaPys0KhvqGac06ItE0Gtk92hpYKa9LuYciyFCsHuOPD3eKVmnJm62o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411832; c=relaxed/simple; bh=yHbV7bClt89198/dDEq89L+L/KeiofrkagrD4RuOaeY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rkCts/2qOHmJhQtCbn4lhJ1ijH7TYTH9Jc8gILPe9W9iWkEStAyIc8/UlDxK1ecWyZxlZfXPUzBvWt3N0vvvXccMIQPpLXs84ewublWGR/7g0S4E99J6FR/HmTb5toObeTxFC3mFNMdcjvvzm6V4E2ZiEJbconkFNtADdFb9KIA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=S7hw2T1l; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="S7hw2T1l" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716411830; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=02LYfptPkP3NVWI/Y1/skwE/AqJ4JQacKiY+62xb+tg=; b=S7hw2T1lzXSjzjaAOfJSljMeC3dixA0KUyjXQWYPIRZeoi/7HQWyq7BHAHkCnf0o1Ca0Lu igQ/yzmFhyDkaGcE+uP+P8UrG6n2mkh3bQhB9KVZqfdSr73XL45XPiq4p2w8LGpCjhmsOs gFwfVJYytBXsiyEshoH8djsd+KqrxM4= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-30-cKasN1UEODSQMPyfVtjrrA-1; Wed, 22 May 2024 17:03:48 -0400 X-MC-Unique: cKasN1UEODSQMPyfVtjrrA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BC5A729AA3B8; Wed, 22 May 2024 21:03:47 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 61F1F28E2; Wed, 22 May 2024 21:03:45 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Mike Rapoport , Minchan Kim , Sergey Senozhatsky , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH RFC 1/6] mm: update _mapcount and page_type documentation Date: Wed, 22 May 2024 23:03:36 +0200 Message-ID: <20240522210341.1030552-2-david@redhat.com> In-Reply-To: <20240522210341.1030552-1-david@redhat.com> References: <20240522210341.1030552-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's make it clearer that _mapcount must no longer be used for own purposes, and how _mapcount and page_type behaves nowadays (also in the context of hugetlb folios, which are typed folios that will be mapped to user space). Move the documentation regarding "-1" over from page_mapcount_reset(), which we will remove next. Signed-off-by: David Hildenbrand --- include/linux/mm.h | 5 ----- include/linux/mm_types.h | 24 +++++++++++++++--------- 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 9849dfda44d4..018e7c0265ca 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1206,11 +1206,6 @@ static inline int folio_entire_mapcount(const struct= folio *folio) return atomic_read(&folio->_entire_mapcount) + 1; } =20 -/* - * The atomic page->_mapcount, starts from -1: so that transitions - * both from it and to it can be tracked, using atomic_inc_and_test - * and atomic_add_negative(-1). - */ static inline void page_mapcount_reset(struct page *page) { atomic_set(&(page)->_mapcount, -1); diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 24323c7d0bd4..532a3030405d 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -46,9 +46,7 @@ struct mem_cgroup; * which is guaranteed to be aligned. If you use the same storage as * page->mapping, you must restore it to NULL before freeing the page. * - * If your page will not be mapped to userspace, you can also use the four - * bytes in the mapcount union, but you must call page_mapcount_reset() - * before freeing it. + * The mapcount field must not be used for own purposes. * * If you want to use the refcount field, it must be used in such a way * that other CPUs temporarily incrementing and then decrementing the @@ -152,16 +150,24 @@ struct page { =20 union { /* This union is 4 bytes in size. */ /* - * If the page can be mapped to userspace, encodes the number - * of times this page is referenced by a page table. + * For pages part of non-typed folios for which mappings are + * tracked via the RMAP, encodes the number of times this page + * is directly referenced by a page table. + * + * Note that the mapcount is always initialized to -1, so that + * transitions both from it and to it can be tracked, using + * atomic_inc_and_test() and atomic_add_negative(-1). */ atomic_t _mapcount; =20 /* - * If the page is neither PageSlab nor mappable to userspace, - * the value stored here may help determine what this page - * is used for. See page-flags.h for a list of page types - * which are currently stored here. + * For head pages of typed folios, the value stored here + * allows for determining what this page is used for. The + * tail pages of typed folios will not store a type + * (mapcount =3D=3D -1). + * + * See page-flags.h for a list of page types which are currently + * stored here. */ unsigned int page_type; }; --=20 2.45.0 From nobody Wed Dec 17 07:06:56 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E706146D46 for ; Wed, 22 May 2024 21:04:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411853; cv=none; b=PQHVKPJZRWGIVaH2cscB4H4uk3+tus6CSS9r5hFvkxKYhtRRBwLE4VD6CuGGMMzO43iVhEXE45RPGQ0FbyXX2BuevQqG5grpU+iliv/W0l8sOWNj+YK938Vb3sSHhxDkwXXMvCFiEsHCYIl2oa5RcSKrZ/jfFvdS7hqxq4Wq+54= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411853; c=relaxed/simple; bh=2+I4E7+1vmTeq+NzomJw1rLPkZCo+eWCc5UNkyJyVFg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cX/oMRJ83YD4L1MxVsSoQg8s/iryLposDP8Rhd3a8A5kdLPOXdiXfjvQYfYKMxelIMu5ScxofThL6UAMJuvrP+4fRz4PI27tb9iJg1+Z7szUHdL9bXqbsEDydfv86v8Dctm1i895LB/7i+zYWkeS6TApWw1SopfpQTPiycD2Hyk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LhsJd9Qb; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LhsJd9Qb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716411850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bs73/DhQAjKCEOqgI5WjSruY0e+O/eQEazhESJihTfc=; b=LhsJd9QbkCpmzUyoTqYWnVRq1BtoeiUn1vI6cDCN2pN0DrCXLZP/nZ6HDwzWzPGULu22XI kHPPh3ZZbbZJWNW62Y/x4Z30oPpmfT1vJcYr7vTbA+5/m3W7ocr6bFFYMWaTceSAxbSaMX 1QrsoPLy1hdt1UXGu6LNGilq6wEZSLY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-515-HtzN_wyHOzWPGa337eP5gw-1; Wed, 22 May 2024 17:03:50 -0400 X-MC-Unique: HtzN_wyHOzWPGa337eP5gw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6B0D6185A78E; Wed, 22 May 2024 21:03:50 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2949B7414; Wed, 22 May 2024 21:03:47 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Mike Rapoport , Minchan Kim , Sergey Senozhatsky , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH RFC 2/6] mm: allow reuse of the lower 16bit of the page type with an actual type Date: Wed, 22 May 2024 23:03:37 +0200 Message-ID: <20240522210341.1030552-3-david@redhat.com> In-Reply-To: <20240522210341.1030552-1-david@redhat.com> References: <20240522210341.1030552-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" As long as the owner sets a page type first, we can allow reuse of the lower 16bit! Restrict it to the head page. We'll use that for zsmalloc next, to set a proper type while still reusing that field to store information that cannot go elsewhere for now. Fear of running out of bits for storing the actual type? Actually, we don't need one bit per type, we could store a single value instead. Signed-off-by: David Hildenbrand --- include/linux/mm_types.h | 5 +++++ include/linux/page-flags.h | 22 ++++++++++++++-------- 2 files changed, 19 insertions(+), 8 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 532a3030405d..6dae8e15037b 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -168,6 +168,11 @@ struct page { * * See page-flags.h for a list of page types which are currently * stored here. + * + * Owners of typed folios may reuse the lower 16bit of the + * head page page_type field after setting the page type, + * but must reset these 16 bit to -1 before clearing the + * page type. */ unsigned int page_type; }; diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 104078afe0b1..ed9ac4b5233d 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -942,17 +942,23 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) * __ClearPageFoo *sets* the bit used for PageFoo. We reserve a few high = and * low bits so that an underflow or overflow of _mapcount won't be * mistaken for a page type value. + * + * The highest bit must always be 1, to make page_has_type() work as expec= ted. */ =20 #define PAGE_TYPE_BASE 0xf0000000 -/* Reserve 0x0000007f to catch underflows of _mapcount */ -#define PAGE_MAPCOUNT_RESERVE -128 -#define PG_buddy 0x00000080 -#define PG_offline 0x00000100 -#define PG_table 0x00000200 -#define PG_guard 0x00000400 -#define PG_hugetlb 0x00000800 -#define PG_slab 0x00001000 +/* + * Reserve 0x0000ffff to catch underflows of _mapcount and + * allow owners that set a type to reuse the lower 16 bit for their own + * purposes. + */ +#define PAGE_MAPCOUNT_RESERVE -65536 +#define PG_buddy 0x00010080 +#define PG_offline 0x00020000 +#define PG_table 0x00040000 +#define PG_guard 0x00080000 +#define PG_hugetlb 0x00100800 +#define PG_slab 0x00200000 =20 #define PageType(page, flag) \ ((page->page_type & (PAGE_TYPE_BASE | flag)) =3D=3D PAGE_TYPE_BASE) --=20 2.45.0 From nobody Wed Dec 17 07:06:56 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D0B013D61A for ; Wed, 22 May 2024 21:04:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411842; cv=none; b=Vhyvly6pBVH4cIvTS8M2mSEFcNzjhEOgv8p1eP9w6pVYfEo9EarPeKQtw+sJEUkZ93vPwNA8VI81p/TSJCs46BToZLl+2VYdy9D+O7YFLECNvIwuYB86yO+e9F+d750duwT7nBHOJWtxT3JjAQiQ8JHncwrsAzqeK9N/88W+mWU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411842; c=relaxed/simple; bh=tLBgD71UiTZxC+6huT5kb+pDjgDdZYuOpWPLdflh9G4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sWefEYdf8b6UO4c5NAAm6b3+tPL6azrciC0gc/0Iv4sBm0Ul6mZr4xPx3dGjRCJSHCFXMVA3TdQLLO0Cd0ndQUFjCPT78bb7HmxMTkglh1UQsk+atBm0iR84XYbWTteNIhB9N0quYVEg5OIL4y79sC7XOcD3w/L0GQHRJIxCzAM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Sr+555z1; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Sr+555z1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716411839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DITBJncVvL2hCWefig3RBM7cSj7OZ/WRaZeUNOkaevI=; b=Sr+555z1A23qqRNM2wQ7+vzaVbwPn/A6Ncl3EumejsLNnHWYuFVRaAYN42JRmdFLjS39+u +1B2+M31qxqskxx9NJ/FkvGdFrnvCfx/j9fdE8nYGxscOFHJ5jUK/e2zGkKm2etDDJ+rBE Fao/v2xt3p3CwliUuaR3q4OdxW5a5i4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-688-dYd8dj26PQesLwmRv2SXfA-1; Wed, 22 May 2024 17:03:53 -0400 X-MC-Unique: dYd8dj26PQesLwmRv2SXfA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F3C5780028D; Wed, 22 May 2024 21:03:52 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id C6A5628E3; Wed, 22 May 2024 21:03:50 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Mike Rapoport , Minchan Kim , Sergey Senozhatsky , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH RFC 3/6] mm/zsmalloc: use a proper page type Date: Wed, 22 May 2024 23:03:38 +0200 Message-ID: <20240522210341.1030552-4-david@redhat.com> In-Reply-To: <20240522210341.1030552-1-david@redhat.com> References: <20240522210341.1030552-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's clean it up: use a proper page type and store our data (offset into a page) in the lower 16 bit as documented. We'll have to restrict ourselves to <=3D 64KB base page size (so the offset fits into 16 bit), which sounds reasonable. Unfortunately, we don't have any space to store it elsewhere for now. Based on this, we should do a proper "struct zsdesc" conversion, as proposed in [1]. This removes the last _mapcount/page_type offender. [1] https://lore.kernel.org/all/20231130101242.2590384-1-42.hyeyoo@gmail.co= m/ Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 3 +++ mm/Kconfig | 1 + mm/zsmalloc.c | 23 +++++++++++++++++++---- 3 files changed, 23 insertions(+), 4 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index ed9ac4b5233d..ccaf16656de9 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -959,6 +959,7 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) #define PG_guard 0x00080000 #define PG_hugetlb 0x00100800 #define PG_slab 0x00200000 +#define PG_zsmalloc 0x00400000 =20 #define PageType(page, flag) \ ((page->page_type & (PAGE_TYPE_BASE | flag)) =3D=3D PAGE_TYPE_BASE) @@ -1073,6 +1074,8 @@ FOLIO_TYPE_OPS(hugetlb, hugetlb) FOLIO_TEST_FLAG_FALSE(hugetlb) #endif =20 +PAGE_TYPE_OPS(Zsmalloc, zsmalloc, zsmalloc) + /** * PageHuge - Determine if the page belongs to hugetlbfs * @page: The page to test. diff --git a/mm/Kconfig b/mm/Kconfig index b4cb45255a54..0371d79b1b75 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -190,6 +190,7 @@ config ZSMALLOC tristate prompt "N:1 compression allocator (zsmalloc)" if ZSWAP depends on MMU + depends on PAGE_SIZE_LESS_THAN_256KB # we want <=3D 64KB help zsmalloc is a slab-based memory allocator designed to store pages of various compression levels efficiently. It achieves diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index b42d3545ca85..6f0032e06242 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -20,7 +20,8 @@ * page->index: links together all component pages of a zspage * For the huge page, this is always 0, so we use this field * to store handle. - * page->page_type: first object offset in a subpage of zspage + * page->page_type: PG_zsmalloc, lower 16 bit locate the first object + * offset in a subpage of a zspage * * Usage of struct page flags: * PG_private: identifies the first component page @@ -450,14 +451,22 @@ static inline struct page *get_first_page(struct zspa= ge *zspage) return first_page; } =20 +static inline void reset_first_obj_offset(struct page *page) +{ + page->page_type |=3D 0xffff; +} + static inline unsigned int get_first_obj_offset(struct page *page) { - return page->page_type; + return page->page_type & 0xffff; } =20 static inline void set_first_obj_offset(struct page *page, unsigned int of= fset) { - page->page_type =3D offset; + BUILD_BUG_ON(PAGE_SIZE & ~0xffff); + VM_WARN_ON_ONCE(offset & ~0xffff); + page->page_type &=3D ~0xffff; + page->page_type |=3D offset & 0xffff; } =20 static inline unsigned int get_freeobj(struct zspage *zspage) @@ -791,8 +800,9 @@ static void reset_page(struct page *page) __ClearPageMovable(page); ClearPagePrivate(page); set_page_private(page, 0); - page_mapcount_reset(page); page->index =3D 0; + reset_first_obj_offset(page); + __ClearPageZsmalloc(page); } =20 static int trylock_zspage(struct zspage *zspage) @@ -965,11 +975,13 @@ static struct zspage *alloc_zspage(struct zs_pool *po= ol, if (!page) { while (--i >=3D 0) { dec_zone_page_state(pages[i], NR_ZSPAGES); + __ClearPageZsmalloc(pages[i]); __free_page(pages[i]); } cache_free_zspage(pool, zspage); return NULL; } + __SetPageZsmalloc(page); =20 inc_zone_page_state(page, NR_ZSPAGES); pages[i] =3D page; @@ -1762,6 +1774,9 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 VM_BUG_ON_PAGE(!PageIsolated(page), page); =20 + /* We're committed, tell the world that this is a Zsmalloc page. */ + __SetPageZsmalloc(newpage); + /* The page is locked, so this pointer must remain valid */ zspage =3D get_zspage(page); pool =3D zspage->pool; --=20 2.45.0 From nobody Wed Dec 17 07:06:56 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44AED146586 for ; Wed, 22 May 2024 21:04:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411842; cv=none; b=k2LVx1661CNyEGYszx8K0c7LHJa4kIFPKVAf2Db0QWlCYZ6tE2vLyMmkgd6ngqymKgbKhyjb4rUnLku/FT7uZaRvZhg0+10WWNFeqsh2OjdfIurOPWCEJPpSlDOMeLmosYyhRdAIXAjTDQvJHqqZfNxuZiyDdROoGlXCioy2amc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411842; c=relaxed/simple; bh=0BfzNJgPHUIWX0iWLtLdfNYTXy+ODEs3tvBTu1w1dc8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GsiJWZq6YrLXO/OloADBDD55A0DdB+rq6BzPLDaWdITBNS8aoZpOm0xakmziy/wRs34Gvz6Zqz592ya6VrCO+JjpR/guv+LLONNfhrYK9FlgIk7pDvQ/hAabSk36wzQMbHi6ueTxQln4BZacShocnjo+ywGs8tV8Ri3R9N2pK5A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=iNnxP48H; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iNnxP48H" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716411840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V6qRfeG4CVJz5l48G3FsZPf4ap0/Ubwcpkx8trvSrMU=; b=iNnxP48H3m51no+v3npPEgp0jTguKrQqeasL9WXMVjZk88DASmU96yeAhIQvuVQp3E+pg3 Qz9wocw1SJxQvhcXPt17ndLlUvExzSSQr+3yM5iSH1I6i1qNc2UfwUo5BPtWa3Li3V22DV s/BjAovD9MZAKFoINnQQv91HPwlGzFc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-444-gaoAYFYkNC-gX_dN8vliYw-1; Wed, 22 May 2024 17:03:55 -0400 X-MC-Unique: gaoAYFYkNC-gX_dN8vliYw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 49F378008A4; Wed, 22 May 2024 21:03:55 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5F27B7412; Wed, 22 May 2024 21:03:53 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Mike Rapoport , Minchan Kim , Sergey Senozhatsky , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH RFC 4/6] mm/page_alloc: clear PageBuddy using __ClearPageBuddy() for bad pages Date: Wed, 22 May 2024 23:03:39 +0200 Message-ID: <20240522210341.1030552-5-david@redhat.com> In-Reply-To: <20240522210341.1030552-1-david@redhat.com> References: <20240522210341.1030552-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's stop using page_mapcount_reset() and clear PageBuddy using __ClearPageBuddy() instead. Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2e22ce5675ca..b595342e73c2 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -498,7 +498,8 @@ static void bad_page(struct page *page, const char *rea= son) dump_stack(); out: /* Leave bad fields for debug, except PageBuddy could make trouble */ - page_mapcount_reset(page); /* remove PageBuddy */ + if (PageBuddy(page)) + __ClearPageBuddy(page); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); } =20 @@ -1346,7 +1347,8 @@ static void check_new_page_bad(struct page *page) { if (unlikely(page->flags & __PG_HWPOISON)) { /* Don't complain about hwpoisoned pages */ - page_mapcount_reset(page); /* remove PageBuddy */ + if (PageBuddy(page)) + __ClearPageBuddy(page); return; } =20 --=20 2.45.0 From nobody Wed Dec 17 07:06:56 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF714146001 for ; Wed, 22 May 2024 21:04:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411842; cv=none; b=BbQjWA0Bw1kSZZoGbx3MiYyU/HbPV9cVEo7ukq3Frddoffby+QY+XgYx3oTJvEICRXVDvP4NMTeGuK8xLHc3bbpUttUy7jZmreSTuvZ1/JydZ88jxzSSQKvUXpGmvdJFeTBlzrjVHWCn4yPBOTD+T7/xdEpWzn6f51m/cQtChos= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411842; c=relaxed/simple; bh=AU3lGkdx+T9Jc19ZDNExTERbtu67kY/YvVysSe7kqEI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tn4PZ8rdlvgwMzvotFe9OMbzlzH0ptfH4jafdJ9VdXQhEWfBEzFo+k5i7HWa1+24+0N3old+7tzd7GWzKfXNRelel9p6zpEdzz5QIjiI8U3xQDPcOEUttC1ZV9DLIjJtAHknqaLKgNSVY8PJLgt9gBSydtWBtLPZ2jzM+DPsu64= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fDK8cwoi; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fDK8cwoi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716411839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UoCowHIZmwbvZhrcmlKDIE01w7msvc9uRa4jOe21Z9s=; b=fDK8cwoiQNoatJAyBKfnz+0bvyZsSWKkac4gb7gAS9GPUlN1FBTex9AaOXexKOf2l+PC56 NxQBLBorZSRODFf/pPrccHwyhW1IIEHJU3xzNkYMErdEmsQFtM2c5WyDDK7cuCrym55/Ht qqQiBWG/zwDUKMMFbVdcmMxW6Ahqt0I= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-194-SYtBQfzfNPKl0OuyCBwpzw-1; Wed, 22 May 2024 17:03:58 -0400 X-MC-Unique: SYtBQfzfNPKl0OuyCBwpzw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C62691C05142; Wed, 22 May 2024 21:03:57 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8409628E2; Wed, 22 May 2024 21:03:55 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Mike Rapoport , Minchan Kim , Sergey Senozhatsky , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH RFC 5/6] mm/filemap: reinitialize folio->_mapcount directly Date: Wed, 22 May 2024 23:03:40 +0200 Message-ID: <20240522210341.1030552-6-david@redhat.com> In-Reply-To: <20240522210341.1030552-1-david@redhat.com> References: <20240522210341.1030552-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's get rid of the page_mapcount_reset() call and simply reinitialize folio->_mapcount directly. Signed-off-by: David Hildenbrand --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index 382c3d06bfb1..c4ac7289e88a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -177,7 +177,7 @@ static void filemap_unaccount_folio(struct address_spac= e *mapping, * and we'd rather not leak it: if we're wrong, * another bad page check should catch it later. */ - page_mapcount_reset(&folio->page); + atomic_set(&folio->_mapcount, -1); folio_ref_sub(folio, mapcount); } } --=20 2.45.0 From nobody Wed Dec 17 07:06:56 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC3921482F2 for ; Wed, 22 May 2024 21:04:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411848; cv=none; b=JYQaMw60jGakkyVdWhXCH+my4JNnRip/uvwgfYzAoHZH2RGedqBj0dIN2IBDzBPEa2yWulD9UmWLbIF15WF+YetPh2qJK9rzLZVvTWFwnPV3GHTOjlSkUCG/eOetvKtSPlI88Px+eFCWxibSeO+01Dx0TaOeBN7/IS+wb2K2gpo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716411848; c=relaxed/simple; bh=VREeDSWzeQL6HFzOo4DCIAgPmrdV23oWLwOUQziKCX4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H300EcYEwJx4iWH+qz64QW8ScmFMTl5hSmr7NVksOl39whqA/rq8sFCZZ3i9EmfBHiJu0eFCV9E7fm6ggBVLxbMxnsZpSV4C/c5ttju3dFgl37xNTXqBJde7+191ap2PBvDVAFZqMjmkTlMCVb50+NKVYnmEGPXgqhwC/J6F+XM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=O4cb8Ppd; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="O4cb8Ppd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1716411845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pNNk1s4j6/1176F8MI21H7X7UFnb72xKHXVGsllS3eQ=; b=O4cb8Ppd/DC99x9KdH6ni1RdJOrw6k1pX6mdgQ/C5zAEODPlYObkpIHyCWqHp72YthmaDJ H+pQOrdw1Ud5lCxw4TNu57CXi5IdPGE6auCl69KwY3HG8DNeMz6epHdlGkirBBQ4Traq/f zMBRgCGydYo4w+PFItRf0gGZfgeuL0w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-422-ye0R7uQTPwaFaQc8HubRsQ-1; Wed, 22 May 2024 17:04:02 -0400 X-MC-Unique: ye0R7uQTPwaFaQc8HubRsQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A68A680118E; Wed, 22 May 2024 21:04:00 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 315BF28E3; Wed, 22 May 2024 21:03:58 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Mike Rapoport , Minchan Kim , Sergey Senozhatsky , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH RFC 6/6] mm/mm_init: initialize page->_mapcount directly in__init_single_page() Date: Wed, 22 May 2024 23:03:41 +0200 Message-ID: <20240522210341.1030552-7-david@redhat.com> In-Reply-To: <20240522210341.1030552-1-david@redhat.com> References: <20240522210341.1030552-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's simply reinitialize the page->_mapcount directly. We can now get rid of page_mapcount_reset(). Signed-off-by: David Hildenbrand --- include/linux/mm.h | 5 ----- include/linux/mm_types.h | 2 +- mm/mm_init.c | 2 +- 3 files changed, 2 insertions(+), 7 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 018e7c0265ca..3e1d3b0d545e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1206,11 +1206,6 @@ static inline int folio_entire_mapcount(const struct= folio *folio) return atomic_read(&folio->_entire_mapcount) + 1; } =20 -static inline void page_mapcount_reset(struct page *page) -{ - atomic_set(&(page)->_mapcount, -1); -} - /** * page_mapcount() - Number of times this precise page is mapped. * @page: The page. diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 6dae8e15037b..c377c040f87d 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -169,7 +169,7 @@ struct page { * See page-flags.h for a list of page types which are currently * stored here. * - * Owners of typed folios may reuse the lower 16bit of the + * Owners of typed folios may reuse the lower 16 bit of the * head page page_type field after setting the page type, * but must reset these 16 bit to -1 before clearing the * page type. diff --git a/mm/mm_init.c b/mm/mm_init.c index f72b852bd5b8..b4916751edce 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -568,7 +568,7 @@ void __meminit __init_single_page(struct page *page, un= signed long pfn, mm_zero_struct_page(page); set_page_links(page, zone, nid, pfn); init_page_count(page); - page_mapcount_reset(page); + atomic_set(&page->_mapcount, -1); page_cpupid_reset_last(page); page_kasan_tag_reset(page); =20 --=20 2.45.0