From nobody Thu Jun 11 00:04:38 2026 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A2FC388E50 for ; Tue, 9 Jun 2026 11:35:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004959; cv=none; b=ut2nxR+6s31rt3iecmzodqzO32zUAO3IcdYub21bXSigcmR6b8ng/lh257LSQxWcfujLrLQdVOmk+jb0yAi0kibmJ3+alYSVGPoSAp91A3DglMVeJ4mT6WmpJ7KkfXQ5rjrJ0sXuPveE5QKWRNQ2jApcuhROR2H+AE5sbee+KV8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004959; c=relaxed/simple; bh=hOTHzfv9Dw42L56/OwesQ5Xt6mIL60jUnFpFaT2TKnc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PLStrZTSB922n3XP2I47xpPWjel2Pqrr2iDwaoZ5GnSstiqEWAwRtXN3a1dbyhdBq5hANWcoMPpq+TWt+BfsBzqNdp0NEYSs8AzUrvWsdSQYtJ8fxQNWSRug8FNpC4/8V2xu568uGDt8Fqbn9D4HQefvoET9fj7fJO9+PJXVTLY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EGVH6R9E; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EGVH6R9E" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-36b9033d230so2806439a91.1 for ; Tue, 09 Jun 2026 04:35:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781004957; x=1781609757; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2a1iC93mF2361F98xckcBOo8JlNpMzrV7XxWUy9iDrk=; b=EGVH6R9E+EKkhljVMX61Xc+B/bdSVyu6yrvjpSpcjCErGj3vYqDJoRNYfNx54oPOxQ MYIcdLjZBhGfNRXvK6/gzcM+zLBpTqD+7FO6AfI9lD0eYS8xa4mJoW2NMC9HWfPKnXr/ TQTqlX1zO2c7Ax8Bs0ZQ20vi3rQ8u5fr2B978RuNm24vEKdeQWre4nCnXRkV7MHPG+xA thQVyGDbZYdx4wt1yWv57oKnkER7dcyuL2MqMfyoI5m/p+vCQy91Fo8Pk4qgg8Bp1lBp 1eeWCxVn/b/V7efO7DpTek+wTbEqXwM66avO/XXbsdWpz9U7+2UWJhtVR6W7ExJviPQk Hc2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781004958; x=1781609758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=2a1iC93mF2361F98xckcBOo8JlNpMzrV7XxWUy9iDrk=; b=MBtvOwMPHg1tQDbppvd02/sucRrA+n8nhKCKJW/nKoSp70h7vgDhBtwjeHpDxNKk/N lqEAi/HvPcBOeC2GOkcGuxW/4gvBJWST38ocWdU1sVGnV4WW64Ka+8gcd/NNByoeJvXB kDjtnvmNrm4AwZfO8r/UWPAjJg5SqEsTFN23uC0dYrjeavDIKkL7n2230NF2qCRjYhr5 3FALjOUeGfXudjkOnDC74O1Dbs94q/Glps05ta8iGocgiEh0KaqERR5SFaxmqo4OkYLa tffQaomNhKBTw/WdescUF90sNJJkehX5iUme+VQ0ZlxtBAHrZgLq8pxmkP4V0EjQkEaX K03A== X-Forwarded-Encrypted: i=1; AFNElJ8pqxb5YfqQaGBdm6L8bXmgt6llqsHPsmspchw2h/SrePGFGe8tzP9YcUFuTFseMzB1cwjtgVcAIoUY7eI=@vger.kernel.org X-Gm-Message-State: AOJu0YyYp7Dmso1Tv3hFQ+o5YI40BmQ9EFYhAiaVeHxf6hjC2KHjkHTM 0BElHbNdI1yQi/ltqLA5gJXwnGKRtI2v5s7LzlmDV7Vdqa9gAx75aVea X-Gm-Gg: Acq92OHhvZ9s87ztucsE62WQr6xR2ZljnVrJ4CTWrcQCzY035J1xBA2XWy1JlSuceO5 BOms1O9lc69fho+6YURuZf7q79aA5dGW6n8sp2RGkErDbjB0ok6igdr6bHmXqSKZ0YyNzSJLD7c tdrNlBACrldLkRP88yBjitmQiqP9jElPBRtWmYNpIDrXhdQy0mh3zLQAe+oEeFKoOrtaj+1dvA3 bVQQipVxFKXU+VOsN1gFhHpU7Mn498VamoPcJS8MmHL0ZySQP6FdzeQz+cUGbjhZVYQdUZ3rHeS ImXZF+kXNjOnNfJ+HP9YMinGfrsSmepRZKPfizdy0lb65DlQQo8vs99QYa2YrOiNLf4brRhJ7C5 mBm4Eak58N1O52NSjZVMbhc++eO01bWE1esm5xDsfhEzcP1Qhs7sxXwrVQcrx2nneRYJh7WMNwk nslKZ1Oli6CWqKxx5/3iR9ojGotxgz1dHFKLt6dO53yv6J9PKdSw== X-Received: by 2002:a17:90b:3f46:b0:35f:b9f1:fded with SMTP id 98e67ed59e1d1-3713336f91fmr14812189a91.12.1781004957417; Tue, 09 Jun 2026 04:35:57 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36f6c668e98sm19655292a91.5.2026.06.09.04.35.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 04:35:56 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v4 1/4] mm/zsmalloc: encode class index in obj value for lockless class lookup Date: Tue, 9 Jun 2026 19:35:17 +0800 Message-Id: <20260609113520.3507783-2-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260609113520.3507783-1-haowenchao22@gmail.com> References: <20260609113520.3507783-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao Encode the size_class index (class_idx) into the obj value so that zs_free() can determine the correct size_class without dereferencing the handle->obj->PFN->zpdesc->zspage->class chain under pool->lock. class_idx is invariant across page migration (only PFN is rewritten), so a lockless read of obj always yields a valid class_idx. Where obj has more bits below the PFN field than obj_idx alone needs, split that space into class_idx and obj_idx subfields: |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->| +-----------------+-------------------------+-----------------------+ | PFN | class_idx | obj_idx | +-----------------+-------------------------+-----------------------+ MSB ^ LSB | +-- ZS_OBJ_PFN_SHIFT The macro layout changes as follows: Before After Meaning ---------------- ------------------ ---------------------------- OBJ_INDEX_BITS ZS_OBJ_IDX_BITS width of obj_idx subfield OBJ_INDEX_MASK ZS_OBJ_IDX_MASK mask of obj_idx subfield (n/a) ZS_OBJ_CLASS_BITS width of class_idx subfield (n/a) ZS_OBJ_CLASS_MASK mask of class_idx subfield (n/a) ZS_OBJ_PFN_SHIFT bit offset of PFN in obj ZS_OBJ_CLASS_BITS folds to 0 (and the layout collapses to [PFN | obj_idx]) when obj has no spare bits, i.e. on 32-bit or on 64-bit fallback paths where MAX_POSSIBLE_PHYSMEM_BITS =3D=3D BITS_PER_LONG (e.g. UML); zs_free() then falls back to pool->lock. Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 99 +++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 85 insertions(+), 14 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 63128ddb7959..f84258d63917 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -67,8 +67,8 @@ #define MAX_POSSIBLE_PHYSMEM_BITS MAX_PHYSMEM_BITS #else /* - * If this definition of MAX_PHYSMEM_BITS is used, OBJ_INDEX_BITS will just - * be PAGE_SHIFT + * If this definition of MAX_PHYSMEM_BITS is used, ZS_OBJ_PFN_SHIFT will + * just be PAGE_SHIFT */ #define MAX_POSSIBLE_PHYSMEM_BITS BITS_PER_LONG #endif @@ -88,8 +88,23 @@ #define OBJ_TAG_BITS 1 #define OBJ_TAG_MASK OBJ_ALLOCATED_TAG =20 -#define OBJ_INDEX_BITS (BITS_PER_LONG - _PFN_BITS) -#define OBJ_INDEX_MASK ((_AC(1, UL) << OBJ_INDEX_BITS) - 1) +/* + * obj is encoded as [PFN | class_idx | obj_idx] within an unsigned long: + * + * |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->| + * +-----------------+-------------------------+-----------------------+ + * | PFN | class_idx | obj_idx | + * +-----------------+-------------------------+-----------------------+ + * MSB ^ LSB + * | + * +-- ZS_OBJ_PFN_SHIFT + * + * Encoding class_idx into obj lets zs_free() locate the size_class + * without holding pool->lock; class_idx is invariant across page + * migration (only PFN changes), so a lockless read of the obj value + * always yields a valid class_idx. + */ +#define ZS_OBJ_PFN_SHIFT (BITS_PER_LONG - _PFN_BITS) =20 #define HUGE_BITS 1 #define FULLNESS_BITS 4 @@ -98,9 +113,55 @@ =20 #define ZS_MAX_PAGES_PER_ZSPAGE (_AC(CONFIG_ZSMALLOC_CHAIN_SIZE, UL)) =20 +/* + * ceil(log2(ZS_MAX_PAGES_PER_ZSPAGE)) at preprocessor time, for use + * in #if below. Kconfig restricts ZSMALLOC_CHAIN_SIZE to [4, 16]. + */ +#if ZS_MAX_PAGES_PER_ZSPAGE <=3D 4 +#define ZS_CHAIN_LOG2 2 +#elif ZS_MAX_PAGES_PER_ZSPAGE <=3D 8 +#define ZS_CHAIN_LOG2 3 +#elif ZS_MAX_PAGES_PER_ZSPAGE <=3D 16 +#define ZS_CHAIN_LOG2 4 +#else +#error "ZSMALLOC_CHAIN_SIZE out of expected range [4,16]" +#endif + +/* PAGE_SHIFT - 5 =3D log2(PAGE_SIZE / 32); 32 =3D ZS_MIN_ALLOC_SIZE floor= . */ +#define ZS_MAX_OBJ_PER_PAGE_LOG2 (PAGE_SHIFT - 5) + +/* + * obj_idx width that keeps ZS_MIN_ALLOC_SIZE at its 32-byte floor. + * Below this, ZS_MIN_ALLOC_SIZE is auto-raised by the MAX(32, ...) + * formula -- still correct, but objects are coarser. + */ +#define ZS_OBJ_IDX_DENSE_BITS (ZS_CHAIN_LOG2 + ZS_MAX_OBJ_PER_PAGE_LOG2) + +/* + * Encode class_idx only when obj has spare bits; otherwise + * ZS_OBJ_CLASS_BITS folds to 0 (32-bit, or 64-bit UML/fallback). + */ +#if BITS_PER_LONG >=3D 64 && \ + ZS_OBJ_PFN_SHIFT >=3D (CLASS_BITS + 1) + ZS_OBJ_IDX_DENSE_BITS +#define ZS_OBJ_CLASS_BITS (CLASS_BITS + 1) +#else +#define ZS_OBJ_CLASS_BITS 0 +#endif +#define ZS_OBJ_CLASS_MASK ((_AC(1, UL) << ZS_OBJ_CLASS_BITS) - 1) + +#define ZS_OBJ_IDX_BITS (ZS_OBJ_PFN_SHIFT - ZS_OBJ_CLASS_BITS) +#define ZS_OBJ_IDX_MASK ((_AC(1, UL) << ZS_OBJ_IDX_BITS) - 1) + +/* + * Belt-and-suspenders: the #if above already guarantees this when + * class_idx is enabled. Catches future tweaks that bypass it. + */ +static_assert(ZS_OBJ_IDX_BITS >=3D ZS_CHAIN_LOG2, + "zsmalloc: ZS_MIN_ALLOC_SIZE would exceed ZS_MAX_ALLOC_SIZE"); + /* ZS_MIN_ALLOC_SIZE must be multiple of ZS_ALIGN */ #define ZS_MIN_ALLOC_SIZE \ - MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS)) + MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> ZS_OBJ_IDX_BITS)) /* each chunk includes extra space to keep handle */ #define ZS_MAX_ALLOC_SIZE PAGE_SIZE =20 @@ -721,26 +782,35 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *= zpdesc) static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); - *obj_idx =3D (obj & OBJ_INDEX_MASK); + *zpdesc =3D pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT); + *obj_idx =3D (obj & ZS_OBJ_IDX_MASK); } =20 static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT); +} + +/* Folds to 0 when ZS_OBJ_CLASS_BITS =3D=3D 0; no ifdef needed at callers.= */ +static unsigned int obj_to_class_idx(unsigned long obj) +{ + return (obj >> ZS_OBJ_IDX_BITS) & ZS_OBJ_CLASS_MASK; } =20 /** - * location_to_obj - get obj value encoded from (, ) + * location_to_obj - encode (, , ) into obj va= lue * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index + * @class_idx: size class index; ignored when ZS_OBJ_CLASS_BITS =3D=3D 0 */ -static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx, + unsigned int class_idx) { unsigned long obj; =20 - obj =3D zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; - obj |=3D obj_idx & OBJ_INDEX_MASK; + obj =3D zpdesc_pfn(zpdesc) << ZS_OBJ_PFN_SHIFT; + obj |=3D (unsigned long)(class_idx & ZS_OBJ_CLASS_MASK) << ZS_OBJ_IDX_BIT= S; + obj |=3D obj_idx & ZS_OBJ_IDX_MASK; =20 return obj; } @@ -1276,7 +1346,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_local(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(m_zpdesc, obj); + obj =3D location_to_obj(m_zpdesc, obj, zspage->class); record_obj(handle, obj); =20 return obj; @@ -1762,7 +1832,8 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(newzpdesc, obj_idx); + new_obj =3D location_to_obj(newzpdesc, obj_idx, + obj_to_class_idx(old_obj)); record_obj(handle, new_obj); } } --=20 2.34.1 From nobody Thu Jun 11 00:04:38 2026 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7EA3A36AB57 for ; Tue, 9 Jun 2026 11:36:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004962; cv=none; b=G303r/h2/Fn+Y52AwXZ/Ytmmna5UrCuBPExh/k80aFpKz9OFoE0/c5FfvsI6NsPg6DlGZ7WQUxCgI6HYmCpcu1igVoOdHboOHJE9oDQ0rCHIgVDz8fHIsVJzYzpX9rmAWzyyvzs85cHD/YTeFMApGSw0C3ppLWxQYBM8tZGwTEs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004962; c=relaxed/simple; bh=3sM15z4kj83iEQQiOMeVq7QQg+y5f9KWVZ6Jr9KmiFw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=D+w+1nAuXNMqtASKJW1/M1D9jzn9J36EI3txbUtAGRqgxcHxQMXGKQuWNOwxDi5wlp044rS8Qa22GXiJ/xe1ivJifdeDD0WBu36V672mczrzV+yi76xqrjNxmwEH9gEcXBE0ZjxVU/ph1skurJHJOHE0x046bXhBcDzZbXbbQV4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Alr2pubj; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Alr2pubj" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-36b9ec98144so4322631a91.1 for ; Tue, 09 Jun 2026 04:36:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781004961; x=1781609761; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AKDSLwDdGTSyq9QGc1Jp5iLqo9aL/jeA2DLoFosrfnI=; b=Alr2pubjLF3R8OCcoEJXmOg2oGoTDaDUro6Qr3/tUmh6mCsfiF1iu2l1+y5w93Bq0Z nNMa2isu1XzsXGDlXN/FrGjfmRq4ArtpSAE4RGrvXs6FQFvmPLJuDTemk7A2Vicxzaz0 y1azzODX7YAU9Ymw2pT+VDrJU2DnFWlTCBFZdk5MhXCM14d4yk6RDrfLkLUzY7jfrsr8 YCY8EDhJNfVfYQW+oX/3NOB17D8luf0/T4ay+onztQiN3BlKhcGFxbraA/qJ8nqovQie Q0nAlMoY9z9nMtZBlLh1FG7ig5K2pITI2k0ZA17lCeAdRV5lN33mLJQDfdvUFt/fsPGs ruHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781004961; x=1781609761; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=AKDSLwDdGTSyq9QGc1Jp5iLqo9aL/jeA2DLoFosrfnI=; b=XOCxJn1dHdHNwY+tIjMSZ+A3tS5vPEluqEV6HtLAI3t8jxk2ePyUR0n/bl6bA/EV+p GmEO84kFdBoVbXnLWOi4Co0D1HlB8z+xshumczM4T08PIvnjCAgwaL4idK8kwaVRW3sX FdG33aCH3tIMXgZM/6OZR0kCfj8MVQfon+LYrTa7s0yJWJL1HZDN7Z4p2kIVQk+KPwMv zuS/rXq7NuJTn0cIljMHZv86Kw1H75jtwBXsI9/7SLOuA+Sn0P6gKdpfB6uBEzAQXeT+ +MiNmvOqGg1N6I3vXuLyAkkW+pYSdZkz+Cw4FT9/9jHf6BnfXZKnxlj7WDDE9JAR7NfI YfWA== X-Forwarded-Encrypted: i=1; AFNElJ+LWBRcF+s98IlWvbGnGI1setSso3DP48arH9ZRI8hcxn6kupqp2GYfFuQDdRa1NVCX/Lm95fN5BvCVhCM=@vger.kernel.org X-Gm-Message-State: AOJu0YwnIg/Qgjog2DyM5yrr1CTLfua9eMWTDx0sksgCdanDu3l+tnMk kKb8d8j/9nZXYXbFkGrDBpZS9kmWX9HRnTLRpsxuWWMY2ndJcN5TNw/6 X-Gm-Gg: Acq92OHqGR+ETVc/Xa+ckxvoteohwqhRSoy4hC8DjaNbvphPU3eYT0POd2YVUY520HT 8feZB35L2uUmpjrzg9tBRxAbQHm9r5iC7XKSqGnVm53b6lwtTBunMncFMn7K5JPtdFeQjGoUdJi lSb3/B4w+0xZCvzTmnGb/cvaCRcwSxy2iDBP318BLA3q2usny+q6iV76nnBP5rsVje7Ky4Udn6p n59vDJmUJL88b1YEYyw8gj9scUyc3J2XginIIE/mNEk0DylStq6sdOIY5IzqQyz9gpmoT46pJ88 G8yGYjzqNEPyfJixXclax8n9faIXQKN4BUGdGJgMFSspoVzxJA3MNR8F4VQlfiBkZk9qwEVunKd HCh661mZZUOev0YsOag20PjlduZ1hqQ0VUuJXlxyxXEP2jLlUIOcjbzrnc8yeaQYLqKclcqRuuD jXN7flNld98rG8ZLnYY1HlUhUFtlfZ5vVfgdJ+LSoJ8cfxGxRV3g== X-Received: by 2002:a17:90b:2dc7:b0:36d:cf58:b79 with SMTP id 98e67ed59e1d1-375211ae901mr3093505a91.19.1781004960720; Tue, 09 Jun 2026 04:36:00 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36f6c668e98sm19655292a91.5.2026.06.09.04.35.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 04:36:00 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v4 2/4] mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems Date: Tue, 9 Jun 2026 19:35:18 +0800 Message-Id: <20260609113520.3507783-3-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260609113520.3507783-1-haowenchao22@gmail.com> References: <20260609113520.3507783-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao With class_idx encoded in obj, zs_free() can locate the size_class without holding pool->lock on 64-bit systems. Page migration also takes class->lock and only rewrites the PFN field of obj, so: 1. read obj locklessly, 2. lock the size_class derived from obj's class_idx, 3. re-read obj under class->lock to get a stable PFN. This eliminates the rwlock read-side cacheline bouncing between zs_free() and migration/compaction on multi-core systems. Annotate handle_to_obj()/record_obj() with READ_ONCE()/WRITE_ONCE() to prevent load/store tearing on the lockless read path and silence KCSAN data race reports. When ZS_OBJ_CLASS_BITS =3D=3D 0 (32-bit, or 64-bit with obj too narrow to hold class_idx), zs_free() keeps pool->lock. Signed-off-by: Wenchao Hao Reviewed-by: Nhat Pham --- mm/zsmalloc.c | 75 ++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 60 insertions(+), 15 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index f84258d63917..fe20ab297542 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -21,6 +21,10 @@ * pool->lock * class->lock * zspage->lock + * + * When ZS_OBJ_CLASS_BITS > 0, zs_free() skips pool->lock; it picks + * the size_class from obj's encoded class_idx and serializes against + * page migration via class->lock. */ =20 #include @@ -457,10 +461,13 @@ static void cache_free_zspage(struct zspage *zspage) kmem_cache_free(zspage_cachep, zspage); } =20 -/* class->lock(which owns the handle) synchronizes races */ +/* + * Pairs with READ_ONCE() in handle_to_obj(): zs_free() may read the + * handle locklessly, so prevent store tearing here. + */ static void record_obj(unsigned long handle, unsigned long obj) { - *(unsigned long *)handle =3D obj; + WRITE_ONCE(*(unsigned long *)handle, obj); } =20 static inline bool __maybe_unused is_first_zpdesc(struct zpdesc *zpdesc) @@ -817,7 +824,7 @@ static unsigned long location_to_obj(struct zpdesc *zpd= esc, unsigned int obj_idx =20 static unsigned long handle_to_obj(unsigned long handle) { - return *(unsigned long *)handle; + return READ_ONCE(*(unsigned long *)handle); } =20 static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, @@ -1451,10 +1458,58 @@ static void obj_free(int class_size, unsigned long = obj) mod_zspage_inuse(zspage, -1); } =20 +/* + * Resolve @handle to its zspage / size_class and acquire class->lock. + * + * When class_idx is encoded in obj (ZS_OBJ_CLASS_BITS > 0), it is + * invariant under page migration, so the handle can be read locklessly + * to pick the size_class. Once class->lock is held migration is + * blocked and the handle is re-read to obtain a stable PFN. + * + * Otherwise (32-bit, or 64-bit fallback paths like UML where the + * encoding is disabled), fall back to pool->lock for the lookup. + */ +#if ZS_OBJ_CLASS_BITS > 0 +static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned lo= ng handle, + unsigned long *objp, struct zspage **zspagep, + struct size_class **classp) + __acquires(&(*classp)->lock) +{ + struct zpdesc *f_zpdesc; + unsigned long obj; + + obj =3D handle_to_obj(handle); + *classp =3D pool->size_class[obj_to_class_idx(obj)]; + spin_lock(&(*classp)->lock); + /* Re-read under class->lock: PFN is now stable vs migration. */ + obj =3D handle_to_obj(handle); + obj_to_zpdesc(obj, &f_zpdesc); + *zspagep =3D get_zspage(f_zpdesc); + *objp =3D obj; +} +#else +static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned lo= ng handle, + unsigned long *objp, struct zspage **zspagep, + struct size_class **classp) + __acquires(&(*classp)->lock) +{ + struct zpdesc *f_zpdesc; + unsigned long obj; + + read_lock(&pool->lock); + obj =3D handle_to_obj(handle); + obj_to_zpdesc(obj, &f_zpdesc); + *zspagep =3D get_zspage(f_zpdesc); + *classp =3D zspage_class(pool, *zspagep); + spin_lock(&(*classp)->lock); + read_unlock(&pool->lock); + *objp =3D obj; +} +#endif + void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1462,17 +1517,7 @@ void zs_free(struct zs_pool *pool, unsigned long han= dle) if (IS_ERR_OR_NULL((void *)handle)) return; =20 - /* - * The pool->lock protects the race with zpage's migration - * so it's safe to get the page from handle. - */ - read_lock(&pool->lock); - obj =3D handle_to_obj(handle); - obj_to_zpdesc(obj, &f_zpdesc); - zspage =3D get_zspage(f_zpdesc); - class =3D zspage_class(pool, zspage); - spin_lock(&class->lock); - read_unlock(&pool->lock); + obj_handle_class_lock(pool, handle, &obj, &zspage, &class); =20 class_stat_sub(class, ZS_OBJS_INUSE, 1); obj_free(class->size, obj); --=20 2.34.1 From nobody Thu Jun 11 00:04:38 2026 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3168D3FA5D8 for ; Tue, 9 Jun 2026 11:36:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004966; cv=none; b=DqasEE1xcxdzAIO0iiZtPnuOGFES8iv7qPtWnbZEOGPmz+Lfn8+htTeAPmlkx+oJWEYE93NOwIdN8WCwnY8FbbQZUHT7uzwO5g8zFzVKa1fX6ZBwLHtivHIgDmeKNYRPHlUQbx1JR/6TQEW/z+O62+Os/KYIqhxxa+SMgFVBN9s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004966; c=relaxed/simple; bh=LTHsAjXU2tuHJhc4xIn70WBn3xyeFoMsOm5VDayBji8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=I23HAoHjp2cuSERWHdxlqsrYZAamTvOH22GPBYEEhAD0GF+9y9fn9LlXHu5JMDG1kpjB8s6i9JdekYRSgK5D7l0+s+A1k7ryt9ZI1drP31sko6KdQ49prqRhVBCNu+yHeletnp/T9nwBdCkNwoDiXL94Y244dvnGymePNVAvTTA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nXisYCDc; arc=none smtp.client-ip=209.85.216.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nXisYCDc" Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-37474afe908so759208a91.1 for ; Tue, 09 Jun 2026 04:36:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781004964; x=1781609764; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6/nTjntuZjnrvE8ZyTpIYFayYnVSKdUVLKXvluFBzrU=; b=nXisYCDc+sawcwatQh9bWEhfpy5oFg+OBNGNdv4y0mnkiNGnl1Z0hBqZPhcGqGY7kg y8JmTECDsRc/X5qX/2tMciLI2Mk/9zcSrO3doHRJZn9dMmKgjbacp2upSf3Nyln39a50 g1r0S8FeqvmxGyzah5uCb6CWqyBWV3IrpYNdd69PodvEjq8hYTFvuMWnleqADcuAT9tc dZP1TsgY1WbU3uY2alFc25w3Rxk7hmcwDHho6pD+GGtxXLDA/P5J4W925Np0aVMW1z76 bTaztRvkARJRrn5ZDJ6s+j1vo4hZUh2cBe+22cdjDBUD88DlqL1O6pJCNC0kwtP2PxSH xtHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781004964; x=1781609764; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6/nTjntuZjnrvE8ZyTpIYFayYnVSKdUVLKXvluFBzrU=; b=JYglL84MNGELDlE1zU9kO8msQBo1nTUaEnESGyotaNXGBEzycOTXfT8fBm9+blA/SB qbPHRokakcs9SzT0V6HhaezlzmIrbF43SW35nijiQgPrecZ2uawV6Ze8cNE6YExp11tH WaO61rDF0gsuv6d9iS9nKSvaisHbO89uT32tC20n6cFLq0SGct3wIpOwWFdycbrVKLXj Uk1PdkPLgrJ+7eFkMCbEVnMLle3hyEzVxe642barU6O04vJDHsJebi9+6lnLUjaxwRmj sDrgnpMzIjEJ1o0u/NIDr5hX5vBmYIWp8VAkVpkZJwfwlV8lQmt6qwOxmi0ZV7xmZwH9 TXYQ== X-Forwarded-Encrypted: i=1; AFNElJ/SiwHqJqKoZTWNL2YIKNy0FmgOM+kNrOxgzs+ciTcPsQMpfwdhSboUqXJ5oXl/Uv5s7qOHkgd0aKoRp5s=@vger.kernel.org X-Gm-Message-State: AOJu0YziXcc3wn1SYjeUwmMquWVkIFXWkPcwveo7A13tkyP/G1J52zfY 9hJBpfJesBNqVSf96JlKaeCVBxsHhbYQt8OtBkZd04a2I661Y6eAv8kV X-Gm-Gg: Acq92OEgWwpClodvzrGZpqzrbJcE8dBhS67v8h8DXWK/CQajMcg8B8bLJyhQfhgqvg+ dNTd2P+gEbmDrdZUkanNz+YxGJx9rsoHk0T1Nl6AM+vWC+pnCUuYk78OJkx1zdjT5ZKYaiq0qtL KyXU6IhEykkbdTE1iI0eIk9+qgJxB6VBf5TbV4iipvua5OJR682jmNZ+PJP03OxY/i9zlq5Uywb mG9CIn/7REXl15OMwBVzDXpDILWUCh7Q9PdyA2FCIW4VXQ77IyQmhwnLW/uFFMeZiOP5mktZRw+ 69s9pqoZJkQpGLGAq2HLT8eHG//Laetb5ImytI+qFVzFgV7yUREENNKY2ZteivYKg173bwMtwQV LX8HHNmv9HbzMs6iq4NxZL5tw9+SGhMmJx3sKMqzACg2bZ9MJnGmMdLcFJA/8Utma7mcNgxqpe5 M8gOjnKT1RDj+nM82xWaj5S9F1k7c4tPLtiPAvKLJ+tabra+7Zrw== X-Received: by 2002:a17:90b:2742:b0:36a:c4b:76cf with SMTP id 98e67ed59e1d1-370f0b55e45mr21925890a91.23.1781004964510; Tue, 09 Jun 2026 04:36:04 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36f6c668e98sm19655292a91.5.2026.06.09.04.36.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 04:36:03 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Xueyuan Chen , Wenchao Hao Subject: [PATCH v4 3/4] mm/zsmalloc: drop class lock before freeing zspage Date: Tue, 9 Jun 2026 19:35:19 +0800 Message-Id: <20260609113520.3507783-4-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260609113520.3507783-1-haowenchao22@gmail.com> References: <20260609113520.3507783-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Reviewed-by: Nhat Pham Reviewed-by: Joshua Hahn Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index fe20ab297542..1602ccb6d814 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -878,13 +878,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } =20 -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zs_pool *pool, struct zsp= age *zspage) { struct zpdesc *zpdesc, *next; =20 - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 @@ -900,7 +897,13 @@ static void __free_zspage(struct zs_pool *pool, struct= size_class *class, } while (zpdesc !=3D NULL); =20 cache_free_zspage(zspage); +} =20 +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(pool, zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1513,6 +1516,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free =3D NULL; =20 if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1523,10 +1527,23 @@ void zs_free(struct zs_pool *pool, unsigned long ha= ndle) obj_free(class->size, obj); =20 fullness =3D fix_fullness_group(class, zspage); - if (fullness =3D=3D ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness =3D=3D ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free =3D zspage; + } else { + kick_deferred_free(pool); + } + } =20 spin_unlock(&class->lock); + + if (zspage_to_free) { + __free_zspage_lockless(pool, zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); --=20 2.34.1 From nobody Thu Jun 11 00:04:38 2026 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FC823FFAD3 for ; Tue, 9 Jun 2026 11:36:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004969; cv=none; b=ALJCD0c3ZN6D+8k9NSuOggdPooQ5/1Pym8ZP1JL6gpiNr4jrfhQU/p6F/pkCfPC10Va+3e12d6vojcmTocyMeqQiWp0i08iV86oIoTSmCUcLQp3fkhP6tph5HKcB4K4ARv4oQptwwohqcwAKGCroL359AsY2xckbEiMD627W5eQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781004969; c=relaxed/simple; bh=OWl8XodKuHvtGBZB4JPH6bzMG/fxw0APweCIxIBRAv4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QUzarSd7qtXKXAvdq3uiMIPTq1HYuWsRJ9J9r+kEwSj7gSzZfUecA2QrjUS7M1VgJQDtd99YX2kKiWaoLtohlaRw6DN0oKW2/oz53amugN4jyThZKsU8VJUjq5UFyUvBpB2BYlwtqaHIOjuBc2wibGFMgt4UePjC5nN8Daa/2IU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=J5MlSYlS; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J5MlSYlS" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-36b9033d230so2806522a91.1 for ; Tue, 09 Jun 2026 04:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781004968; x=1781609768; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=78gCuAHbgodEfjkwRKqmw0oYajjv5lRliBK5sAWYFv4=; b=J5MlSYlSnAWCAsJqusmgtcKqLnt5dynwQz/vrdZmLjuifGiikTg/5n2Brs7CF5j9TI jmD4SRFM8nH82IzC4N5/6khphsxknz3tcdGN12NTiN/oL43nDowo4fCpFbhtTlBz3b9Q T/hQ/UPdrj+mhK6/oOLSqw0pEsILHZjAjq3/CuO/69w5ub4nv+ARGWhr8WhUrRixSib6 z/j0wKCluB5vPf1YJUFIHGCnyGIu2kO20a01iDZDPe2gQzzYzK7PGY2l/h6G9Yglt155 hw5Rye06CbzadvTvFmky68V7vwKID9jCRgJgNOrouDknGc/Q2LBXf77hxHNFWCZ92iat 8sYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781004968; x=1781609768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=78gCuAHbgodEfjkwRKqmw0oYajjv5lRliBK5sAWYFv4=; b=Xckltc8f1rlaHvHiVmfJmh9tIxSqmTiuW/NmZTc/59tL+XppgQmR3DW+FPZ+lWm5dK wW6O0EgCQlgKbqDFqL7KqLQceAa5E5acRmKJ9RgOXDowDaKC2NEPYRUACUHUVLBt2cr6 gMJ3jCViwpqiOSAKRIdy9QjhbBcnUb1NcUv1UpvbV9Yz15PYAvMJGagVhmAc5ykaoDe1 1eqj2MFrDy/5tdey6K7oIO1eQtFI1baL2XmfZyvqTZVkS4haz5m+gR8UXUSRfnVR+dum TOrpaNNGTr+uD7Fdk0pvSfMsEtgcnOB0S31bXsg7oUrQ8et+t7bORFpT5voq3juUV6DK BPbQ== X-Forwarded-Encrypted: i=1; AFNElJ9ptKwx3k6QZqQqIWAObgyT/6eKDN4eHbo9wsPBCb3qM9pPIOzGSjUqzmdvWXECVpF6sqPYVUYORa0G++g=@vger.kernel.org X-Gm-Message-State: AOJu0Yxv06vzhIIVWsPR/SDIiFGFrS/n/AJ4H0F9p9PeElZZpkU8Q4iO HzLwgCv3TdBi3dAZRlOPZ45v1zJc80vQyYYtc2kUgNA8r/nk+0zq9XaE X-Gm-Gg: Acq92OE30PGRA/qDIyTyooXWNQ+m6Euxjr91oo3BLnqvRF/aA5IQa4novEIXGGd6Gsn IuCm141GxIx3OTg5ezIwpf9KTYT4vscu4y6asK073wGUjwNQQBLTqCCO0bUJjlyA6WItRWdYoRL IfqAsA+Q5X8E/u/5jAisTrCkhAhGG1Alc4ltPOMdt95oo3AxXnAPyDpbsQAmGJjWLnpOQt4+HgE si/RWTOAGdir0JPlXWS3/IUyASlN+7513Cd99y5unEJi7b/dQHEkN0H40lVoehY9bfDZwS+XVYp dCJ9qEuMjYAmIsb7If8b1r+vNHuzcY/T1vL36kTWEQYXwQ0nnoj9h6kFjqIwoWoBJAcWCRzLcAn hWYMOqqvBjG78AvEuHqmn3/HwMlZVe6uhvJOGm+rW6zto5GEC24/Vun1b9+mUhUbEZJ7KnmxPgh cgULoF3Hohd13c8daOGGoFuuiDN6yV8eOL87c5dadj6X/nCl9ZRQ== X-Received: by 2002:a17:90b:544e:b0:35f:c493:cac5 with SMTP id 98e67ed59e1d1-3713337027emr15348877a91.16.1781004967777; Tue, 09 Jun 2026 04:36:07 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-36f6c668e98sm19655292a91.5.2026.06.09.04.36.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 04:36:07 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v4 4/4] mm/zsmalloc: document free_zspage helper variants Date: Tue, 9 Jun 2026 19:35:20 +0800 Message-Id: <20260609113520.3507783-5-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260609113520.3507783-1-haowenchao22@gmail.com> References: <20260609113520.3507783-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao After splitting __free_zspage() into a lockless core and a wrapper that does the class-stat bookkeeping, three similarly-named helpers coexist: free_zspage / __free_zspage / __free_zspage_lockless. Add a comment block above them describing what each does and where it is used, so the names are not easy to confuse. No functional change. Suggested-by: Nhat Pham Reviewed-by: Nhat Pham Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1602ccb6d814..c0edce381d20 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -878,6 +878,22 @@ static int trylock_zspage(struct zspage *zspage) return 0; } =20 +/* + * Three free helpers, kept apart here: + * + * __free_zspage_lockless(): bare core; walks zpdescs and returns pages + * to the buddy allocator. Caller owns all zpdesc locks and has + * removed the zspage from its class list. Used by zs_free() outside + * class->lock so the buddy-side work does not stall the class. + * + * __free_zspage(): __free_zspage_lockless() + per-class accounting, + * under class->lock. Used by async_free_zspage(), the worker for + * zspages whose trylock_zspage() failed. + * + * free_zspage(): full wrapper - trylock zpdescs, remove from class + * list, call __free_zspage(); kicks deferred free on contention. + * Used by compaction. + */ static inline void __free_zspage_lockless(struct zs_pool *pool, struct zsp= age *zspage) { struct zpdesc *zpdesc, *next; --=20 2.34.1