From nobody Fri Jun 19 22:16:45 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42CC43C4167 for ; Tue, 16 Jun 2026 03:13:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579616; cv=none; b=M6spKUgolremRcrUvAPvB/7a6jpJUgE3wZQrh/MBjtHAA2fNyZ2zuk4WpKOiBRx4quhEgqS/4NbIeN2RLqfVnwe86u20Tnyk6CDMx9qNFucr49gnHfdZHcMFOvO9AAdo8YxQxhqyS8HLIIoBboOSgeXDnJT6h3OPENhS0YJEvMQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579616; c=relaxed/simple; bh=ZtaCZnncQ5ThNUtcwBXnXE/MUa64Fm2RgkT7c98OmwA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FKdbSukpIzWQWfyPWiDYhOnPZE88uTjsp4fZY1regAAijn6U7pLlDT5HtzUAD/9H1lmLQTo8sdVPaQASE5QjB6u6i4KZO/ulY2XGQJ8u1kwRqDK5JTVczgPL9uPymIMdeRg0oR7zxKKyDPouLCG/lNwd9/he6sTY3Dva1fpC2cc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QFZRxUJ8; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QFZRxUJ8" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2bf237e1433so49342205ad.1 for ; Mon, 15 Jun 2026 20:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781579614; x=1782184414; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7CMxZGmSd9rDJg1W64dvSL1ed3tT8uDPMgraGKCi12o=; b=QFZRxUJ85vhjJbXV2Fl1UVzTMxJbCNOxsLOhj4csaEU8eeJh16fy9/ZycIOdltnZ8U Wv8XnWcuNlHteeGztjpMgHDIHSf6QimIFZyyM5UgJyKDmx4y0cFlpdUoLQkOdOynnlht AWWJTAHWhuG/wtxa40ti0/J91u03thsk7ESD5yFUalk6PVmDgxss2ZE5CQrpI3DUo4mS FIFEGV4HWWGY+B9BjBbh1dZSp0ofucJoN5Oagl/hcFr+UpT6x/kI/ocQyOnP0anxV6W2 wR7T9FhZBH+rYDsO32AbRoxanbtvjh0jprews2rgAKRUFSa72Gq4LZ+k0Rfc+F50BVBs 6c+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781579614; x=1782184414; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=7CMxZGmSd9rDJg1W64dvSL1ed3tT8uDPMgraGKCi12o=; b=jifYYKkpy2M8+x8OEf3csGbV9uJp+tPU40ICj2Sw2osgFFJwwBFY++/t5XB9VcsRvn LPundZIWcQkFF+Rv775YJ1TaExdoFTqVnF5fRqwX8EabECDKMAkGxTRbIVEiemroCHA4 H+FEHnvM8jt4YanlxaCanoY6NLgtnN5/8GdinfjbbvSCRb37kDAcmwdO5/Orbcgpt89t x+jjt2X/KKbYgd+P11I43P3HMCiLPr9yS/n+NqYdmg4tfQX9EiO77BMdGnTP0OfapXCU UuMvokbhZVpUS7BndbQfQtf5+t1W5dRB4p5D/R9oHqCZUxG0hjHYZmzWJbeg1CqgxShM gvww== X-Forwarded-Encrypted: i=1; AFNElJ8ALJSVWH0RViVCFZnWPaTXSxodFC3Usfl4xRJ9LG5IbTzi18QAVsjyLPw+U5tYcFeP3f6X3K0Uo1TgUuk=@vger.kernel.org X-Gm-Message-State: AOJu0YzYDVW/oUfHdn3TbnZOnby5UhiUY7j5z2Q29ara8gs1gDt1yzgf kW9ZdbCqKcxhrU00H30N/P+kvESCiHsp7++E6rq2Xz58VRpJ3zTEPPnX X-Gm-Gg: Acq92OEt9XGJGO33omkY+P1Bcpi89CIZGLX4ghAaMDkCrvg/gzLakuwAPIGN0w8PpJN O/6XpoOpTms7GUUW0yVf6EQFxpi+p9V1pOIIAHnuk4zY81EJ30u4cIi4K0DBsRfpLc+Un77RlLY 9HHmqjpN7+GjPPMJmG7dVhqcH6iRqNh3uilDcSN+ocBCziJ4ogcPHm8gp7AAYvmI1iEnKcuXuxx RsAOgHm2S87yQ3lbVwd32DtnSW4fLG1oQrkm0Lcs820Wh8B/j4OGRPoAge5GTrZe/M2Vxnf7m5I ZVSMLYFT+c02O8AZCRWDZmw9jfXWCIpJ2lAGCnp+4wtQ44pxXW0N351b4BTvO9R3dGn7n4usgEw ZA8fxV45s1pUJd6Go+aK7UcjrBWCC5qp/mG6TEFHbEZrZ+nx/RYCzytFo21X7RaEfEDeKAvmXrA k4CkxkQ61TKyk4ZGngPoV9qIQPXv6aCjxUnAeswVo= X-Received: by 2002:a17:902:c401:b0:2c2:27be:39aa with SMTP id d9443c01a7336-2c411d7b1efmr179165385ad.17.1781579614578; Mon, 15 Jun 2026 20:13:34 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c42f2e5644sm118923385ad.4.2026.06.15.20.13.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jun 2026 20:13:34 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v5 1/4] mm/zsmalloc: encode class index in obj value for lockless class lookup Date: Tue, 16 Jun 2026 11:13:06 +0800 Message-Id: <20260616031309.3036234-2-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260616031309.3036234-1-haowenchao22@gmail.com> References: <20260616031309.3036234-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao Encode the size_class index (class_idx) into the obj value so that zs_free() can determine the correct size_class without dereferencing the handle->obj->PFN->zpdesc->zspage->class chain under pool->lock. class_idx is invariant across page migration (only PFN is rewritten), so a lockless read of obj always yields a valid class_idx. Where obj has more bits below the PFN field than obj_idx alone needs, split that space into class_idx and obj_idx subfields: |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->| +-----------------+-------------------------+-----------------------+ | PFN | class_idx | obj_idx | +-----------------+-------------------------+-----------------------+ MSB ^ LSB | +-- ZS_OBJ_PFN_SHIFT The macro layout changes as follows: Before After Meaning ---------------- ------------------ ---------------------------- OBJ_INDEX_BITS ZS_OBJ_IDX_BITS width of obj_idx subfield OBJ_INDEX_MASK ZS_OBJ_IDX_MASK mask of obj_idx subfield (n/a) ZS_OBJ_CLASS_BITS width of class_idx subfield (n/a) ZS_OBJ_CLASS_MASK mask of class_idx subfield (n/a) ZS_OBJ_PFN_SHIFT bit offset of PFN in obj ZS_OBJ_CLASS_BITS folds to 0 (and the layout collapses to [PFN | obj_idx]) when obj has no spare bits, i.e. on 32-bit or on 64-bit fallback paths where MAX_POSSIBLE_PHYSMEM_BITS =3D=3D BITS_PER_LONG (e.g. UML); zs_free() then falls back to pool->lock. Reviewed-by: Nhat Pham Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 105 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 91 insertions(+), 14 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 63128ddb7959..c4fc06b259af 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -67,8 +67,8 @@ #define MAX_POSSIBLE_PHYSMEM_BITS MAX_PHYSMEM_BITS #else /* - * If this definition of MAX_PHYSMEM_BITS is used, OBJ_INDEX_BITS will just - * be PAGE_SHIFT + * If this definition of MAX_PHYSMEM_BITS is used, ZS_OBJ_PFN_SHIFT will + * just be PAGE_SHIFT */ #define MAX_POSSIBLE_PHYSMEM_BITS BITS_PER_LONG #endif @@ -88,8 +88,23 @@ #define OBJ_TAG_BITS 1 #define OBJ_TAG_MASK OBJ_ALLOCATED_TAG =20 -#define OBJ_INDEX_BITS (BITS_PER_LONG - _PFN_BITS) -#define OBJ_INDEX_MASK ((_AC(1, UL) << OBJ_INDEX_BITS) - 1) +/* + * obj is encoded as [PFN | class_idx | obj_idx] within an unsigned long: + * + * |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->| + * +-----------------+-------------------------+-----------------------+ + * | PFN | class_idx | obj_idx | + * +-----------------+-------------------------+-----------------------+ + * MSB ^ LSB + * | + * +-- ZS_OBJ_PFN_SHIFT + * + * Encoding class_idx into obj lets zs_free() locate the size_class + * without holding pool->lock; class_idx is invariant across page + * migration (only PFN changes), so a lockless read of the obj value + * always yields a valid class_idx. + */ +#define ZS_OBJ_PFN_SHIFT (BITS_PER_LONG - _PFN_BITS) =20 #define HUGE_BITS 1 #define FULLNESS_BITS 4 @@ -98,9 +113,61 @@ =20 #define ZS_MAX_PAGES_PER_ZSPAGE (_AC(CONFIG_ZSMALLOC_CHAIN_SIZE, UL)) =20 +/* + * Bits to index a page within a zspage =3D ceil(log2(ZS_MAX_PAGES_PER_ZSP= AGE)). + * Computed at preprocessor time, for use in #if below. Kconfig + * restricts ZSMALLOC_CHAIN_SIZE to [4, 16]. + */ +#if ZS_MAX_PAGES_PER_ZSPAGE <=3D 4 +#define ZS_PAGES_PER_ZSPAGE_BITS 2 +#elif ZS_MAX_PAGES_PER_ZSPAGE <=3D 8 +#define ZS_PAGES_PER_ZSPAGE_BITS 3 +#elif ZS_MAX_PAGES_PER_ZSPAGE <=3D 16 +#define ZS_PAGES_PER_ZSPAGE_BITS 4 +#else +#error "ZSMALLOC_CHAIN_SIZE out of expected range [4,16]" +#endif + +/* + * Bits to index an object within a single PAGE_SIZE at the smallest + * possible object size: log2(PAGE_SIZE / 32) =3D PAGE_SHIFT - 5. + * 32 is the hard floor of ZS_MIN_ALLOC_SIZE. + */ +#define ZS_OBJS_PER_PAGE_BITS (PAGE_SHIFT - 5) + +/* + * Bits to index any object in the densest possible zspage. Below this, + * ZS_MIN_ALLOC_SIZE is auto-raised by the MAX(32, ...) formula -- still + * correct, but objects are coarser. + */ +#define ZS_OBJS_PER_ZSPAGE_BITS \ + (ZS_PAGES_PER_ZSPAGE_BITS + ZS_OBJS_PER_PAGE_BITS) + +/* + * Encode class_idx only when obj has spare bits; otherwise + * ZS_OBJ_CLASS_BITS folds to 0 (32-bit, or 64-bit UML/fallback). + */ +#if BITS_PER_LONG >=3D 64 && \ + ZS_OBJ_PFN_SHIFT >=3D (CLASS_BITS + 1) + ZS_OBJS_PER_ZSPAGE_BITS +#define ZS_OBJ_CLASS_BITS (CLASS_BITS + 1) +#else +#define ZS_OBJ_CLASS_BITS 0 +#endif +#define ZS_OBJ_CLASS_MASK ((_AC(1, UL) << ZS_OBJ_CLASS_BITS) - 1) + +#define ZS_OBJ_IDX_BITS (ZS_OBJ_PFN_SHIFT - ZS_OBJ_CLASS_BITS) +#define ZS_OBJ_IDX_MASK ((_AC(1, UL) << ZS_OBJ_IDX_BITS) - 1) + +/* + * Belt-and-suspenders: the #if above already guarantees this when + * class_idx is enabled. Catches future tweaks that bypass it. + */ +static_assert(ZS_OBJ_IDX_BITS >=3D ZS_PAGES_PER_ZSPAGE_BITS, + "zsmalloc: ZS_MIN_ALLOC_SIZE would exceed ZS_MAX_ALLOC_SIZE"); + /* ZS_MIN_ALLOC_SIZE must be multiple of ZS_ALIGN */ #define ZS_MIN_ALLOC_SIZE \ - MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS)) + MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> ZS_OBJ_IDX_BITS)) /* each chunk includes extra space to keep handle */ #define ZS_MAX_ALLOC_SIZE PAGE_SIZE =20 @@ -721,26 +788,35 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *= zpdesc) static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); - *obj_idx =3D (obj & OBJ_INDEX_MASK); + *zpdesc =3D pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT); + *obj_idx =3D (obj & ZS_OBJ_IDX_MASK); } =20 static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT); +} + +/* Folds to 0 when ZS_OBJ_CLASS_BITS =3D=3D 0; no ifdef needed at callers.= */ +static unsigned int obj_to_class_idx(unsigned long obj) +{ + return (obj >> ZS_OBJ_IDX_BITS) & ZS_OBJ_CLASS_MASK; } =20 /** - * location_to_obj - get obj value encoded from (, ) + * location_to_obj - encode (, , ) into obj va= lue * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index + * @class_idx: size class index; ignored when ZS_OBJ_CLASS_BITS =3D=3D 0 */ -static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx, + unsigned int class_idx) { unsigned long obj; =20 - obj =3D zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; - obj |=3D obj_idx & OBJ_INDEX_MASK; + obj =3D zpdesc_pfn(zpdesc) << ZS_OBJ_PFN_SHIFT; + obj |=3D (unsigned long)(class_idx & ZS_OBJ_CLASS_MASK) << ZS_OBJ_IDX_BIT= S; + obj |=3D obj_idx & ZS_OBJ_IDX_MASK; =20 return obj; } @@ -1276,7 +1352,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_local(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(m_zpdesc, obj); + obj =3D location_to_obj(m_zpdesc, obj, zspage->class); record_obj(handle, obj); =20 return obj; @@ -1762,7 +1838,8 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(newzpdesc, obj_idx); + new_obj =3D location_to_obj(newzpdesc, obj_idx, + obj_to_class_idx(old_obj)); record_obj(handle, new_obj); } } --=20 2.34.1 From nobody Fri Jun 19 22:16:45 2026 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BC393C5546 for ; Tue, 16 Jun 2026 03:13:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579619; cv=none; b=sJUkBOaYNcmDLGg+TPd8Jy/I4tMKaVgGnQB/tvojuiAQQSIw+ee3RCzwvgO2+zUp+LGzJOz4V8cPc4Z0vIAaj2as6hsT0XjWvP3MOokQXrh0KxP8c6Ak93VzjG5Te0jTchHgGVGYgZ37ehCYY336pBuPDbyTYIOGTy153tmsSwI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579619; c=relaxed/simple; bh=grv8tCexJ8EpVkO9dVNEIW79tsUx22D7Iuf7JDMDVQA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Zz1TNhXxxWOINXp4PwdaYQ03EkVdR1Ra6HxzHgK049zFPqIddd3GtdB4WgbbMhp+cqGk+JiJaCKNfKDM7hcIWkROKzLhQm3KQS6DPEPFZ1h5yDCzOmNBaCYKcn1PVzSO0TGY7A78/kNGZ2R/bnPUm6Tz7nCWS6kqFyX/7t4UWk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SKfCXGe6; arc=none smtp.client-ip=209.85.216.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SKfCXGe6" Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-36da151a152so2826654a91.1 for ; Mon, 15 Jun 2026 20:13:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781579618; x=1782184418; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oAA0GvzR2uNOJ0BNgmtlA7kpJcS02qAJ4po9Qn4gun4=; b=SKfCXGe6SfzlC1l4/xONeH+NP9JjW2AFcEd6UGQ7gGTXt1DXb9151lwBN7lpc5Ajxb koLSpiT2xda/K/yAubzUvphZV3knOK/+ZeLSc12lKl82CiiR/bmkod9kjy5GdeJ7bOI7 FelzmYiWxjq6nL7bOM/ubiXU2t+UrubAq+OjCaozTu0Lopd5/Z6gtb4DL+XVBoDx+9J8 oF+upedBHZpBohhgEaMmjCZhkQmYMel680gBp8wIoiEDHq+eO5QfmF6UBeRm+y4FEKHK ++iQcma58mbBkmcLEGEGAE2+O1Ks0IqCfH5C2zCXBQzBCFwb/ypE8oUqe8PmEkw7tHeU EfZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781579618; x=1782184418; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=oAA0GvzR2uNOJ0BNgmtlA7kpJcS02qAJ4po9Qn4gun4=; b=L51ifql/JYkW1WkSdOeT8wJMOo48+LHJ1sQ+rfJ8iVG45mSQzjjhpx2jW5ifMVA8mF RroXGXoitjO0b6Q2721p6vljcfVRasPg2t6lYc3w//CkxLEHRQIZ6vwQmn9dMvBmm296 wGkJP1neoYZQvvSHXORjWwJyKk9JHUrc3eg9XI4lWZK3mbF9i5iyGNXea1nFUDwTlmZ+ 4XyGgDnGynLf0wM/uYwDK1zyqLWTvAmm7fn9qaZPL7F02G6QTG6NpSLby6XSvsHC/87n QDkr8LfxEmmyiNf5YELeAlvYquzZKF+CWdaOglCmmaBXQszC/WZj1AG1lzApnA2fxm3Z ZiHA== X-Forwarded-Encrypted: i=1; AFNElJ+/ClMjriWYBrMf+TKwlr0ICfN4xZVhD9PEkIGqMvQF9J3OhZR7eYSyN+hEGkKXg8lnQyVm2PuadKxf7qE=@vger.kernel.org X-Gm-Message-State: AOJu0Yyxc1nYMrpHYN4R8fagNoYLFMFz0CVCBe2yWrrzP5ax4aTTahgk N2h+EMGlkOGxrpabl8rmTXuHFVhWMGksg2bW5QaoLcbpYiPjTCm3gVNq X-Gm-Gg: Acq92OHoztQxIesQAjGMgu9qaUKWaGtqXGTpWDr9pnUXK229GjQP/RgKk22K6s9GkFf rYcxEiXKykvGZtHRgaAijmWa6R+y59Ewe2rGqFDSkBr9lkNpG7y0/13KCgu2h8yJFyZhlauBuki 7tIpbs9PJyA3a8wsxfbrxtCxjhlmSFkVzU/uIgYh1ZzQ7bHteI0Dpnz1XFkuiiY01g+pcXVBOgz WaxRZQ6/eUb2E4JjutvBe9vTYzMJoZIRNGz1Obh9oNX/SPQD1AxJ0IR/YcVSwH6YposnH2Ji7Qc kwdm8f+s16N+A5ZdTgDoN84A3kgqA4PIQXSLJedMZLv2IaMhazmBJGpaeAxKhKidZmFoZ/vJyEb VhGH+C9qEJKF/JbGCUjgmUhNFSgUdJLCOXoB4up9fskASSvipzkwqK0Or+c2jeZtLd8JiS/TgV7 /FGjrFOHGqOjV/lDqem10nHxBSAS2usAQUFf3F3eA= X-Received: by 2002:a17:90b:1a8b:b0:36a:5d1f:7b6 with SMTP id 98e67ed59e1d1-37c2bc5d322mr12991906a91.2.1781579617621; Mon, 15 Jun 2026 20:13:37 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c42f2e5644sm118923385ad.4.2026.06.15.20.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jun 2026 20:13:37 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v5 2/4] mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems Date: Tue, 16 Jun 2026 11:13:07 +0800 Message-Id: <20260616031309.3036234-3-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260616031309.3036234-1-haowenchao22@gmail.com> References: <20260616031309.3036234-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao With class_idx encoded in obj, zs_free() can locate the size_class without holding pool->lock on 64-bit systems. Page migration also takes class->lock and only rewrites the PFN field of obj, so: 1. read obj locklessly, 2. lock the size_class derived from obj's class_idx, 3. re-read obj under class->lock to get a stable PFN. This eliminates the rwlock read-side cacheline bouncing between zs_free() and migration/compaction on multi-core systems. Annotate handle_to_obj()/record_obj() with READ_ONCE()/WRITE_ONCE() to prevent load/store tearing on the lockless read path and silence KCSAN data race reports. When ZS_OBJ_CLASS_BITS =3D=3D 0 (32-bit, or 64-bit with obj too narrow to hold class_idx), zs_free() keeps pool->lock. Reviewed-by: Nhat Pham Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 75 ++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 60 insertions(+), 15 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index c4fc06b259af..1e9a150cd255 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -21,6 +21,10 @@ * pool->lock * class->lock * zspage->lock + * + * When ZS_OBJ_CLASS_BITS > 0, zs_free() skips pool->lock; it picks + * the size_class from obj's encoded class_idx and serializes against + * page migration via class->lock. */ =20 #include @@ -463,10 +467,13 @@ static void cache_free_zspage(struct zspage *zspage) kmem_cache_free(zspage_cachep, zspage); } =20 -/* class->lock(which owns the handle) synchronizes races */ +/* + * Pairs with READ_ONCE() in handle_to_obj(): zs_free() may read the + * handle locklessly, so prevent store tearing here. + */ static void record_obj(unsigned long handle, unsigned long obj) { - *(unsigned long *)handle =3D obj; + WRITE_ONCE(*(unsigned long *)handle, obj); } =20 static inline bool __maybe_unused is_first_zpdesc(struct zpdesc *zpdesc) @@ -823,7 +830,7 @@ static unsigned long location_to_obj(struct zpdesc *zpd= esc, unsigned int obj_idx =20 static unsigned long handle_to_obj(unsigned long handle) { - return *(unsigned long *)handle; + return READ_ONCE(*(unsigned long *)handle); } =20 static inline bool obj_allocated(struct zpdesc *zpdesc, void *obj, @@ -1457,10 +1464,58 @@ static void obj_free(int class_size, unsigned long = obj) mod_zspage_inuse(zspage, -1); } =20 +/* + * Resolve @handle to its zspage / size_class and acquire class->lock. + * + * When class_idx is encoded in obj (ZS_OBJ_CLASS_BITS > 0), it is + * invariant under page migration, so the handle can be read locklessly + * to pick the size_class. Once class->lock is held migration is + * blocked and the handle is re-read to obtain a stable PFN. + * + * Otherwise (32-bit, or 64-bit fallback paths like UML where the + * encoding is disabled), fall back to pool->lock for the lookup. + */ +#if ZS_OBJ_CLASS_BITS > 0 +static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned lo= ng handle, + unsigned long *objp, struct zspage **zspagep, + struct size_class **classp) + __acquires(&(*classp)->lock) +{ + struct zpdesc *f_zpdesc; + unsigned long obj; + + obj =3D handle_to_obj(handle); + *classp =3D pool->size_class[obj_to_class_idx(obj)]; + spin_lock(&(*classp)->lock); + /* Re-read under class->lock: PFN is now stable vs migration. */ + obj =3D handle_to_obj(handle); + obj_to_zpdesc(obj, &f_zpdesc); + *zspagep =3D get_zspage(f_zpdesc); + *objp =3D obj; +} +#else +static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned lo= ng handle, + unsigned long *objp, struct zspage **zspagep, + struct size_class **classp) + __acquires(&(*classp)->lock) +{ + struct zpdesc *f_zpdesc; + unsigned long obj; + + read_lock(&pool->lock); + obj =3D handle_to_obj(handle); + obj_to_zpdesc(obj, &f_zpdesc); + *zspagep =3D get_zspage(f_zpdesc); + *classp =3D zspage_class(pool, *zspagep); + spin_lock(&(*classp)->lock); + read_unlock(&pool->lock); + *objp =3D obj; +} +#endif + void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1468,17 +1523,7 @@ void zs_free(struct zs_pool *pool, unsigned long han= dle) if (IS_ERR_OR_NULL((void *)handle)) return; =20 - /* - * The pool->lock protects the race with zpage's migration - * so it's safe to get the page from handle. - */ - read_lock(&pool->lock); - obj =3D handle_to_obj(handle); - obj_to_zpdesc(obj, &f_zpdesc); - zspage =3D get_zspage(f_zpdesc); - class =3D zspage_class(pool, zspage); - spin_lock(&class->lock); - read_unlock(&pool->lock); + obj_handle_class_lock(pool, handle, &obj, &zspage, &class); =20 class_stat_sub(class, ZS_OBJS_INUSE, 1); obj_free(class->size, obj); --=20 2.34.1 From nobody Fri Jun 19 22:16:45 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88A0B3C5836 for ; Tue, 16 Jun 2026 03:13:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579622; cv=none; b=jVcOv858fIiV/dD6D+PhboQtfyl6SDsnJ0YIDM72MW/fHIwR7A1PSA6KP4znUCNDtKHC/6Zinndu4sY7EdzWI/LuU/JJymFckgfXgSARuuC2e57g+1H6eThYiTrBaA5JNFvQEsmwHc9CoZLemwZtg+Qq4fFUG2oohVKTUEf3z84= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579622; c=relaxed/simple; bh=FK1Bc1wL01z6MIDrydl0Y2wN14P2laIgAGGwrpJ/CQI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=diPpH6dUsg6PK9sZDNcKEDIK13KSorv//ZhIU/GECOgAyJ2g1ggACJkeSMX3bY0UpqdaSRcj7nkVYJYg1zl9YDTOsRsI5i3gdi8bYOq1j7vm3OlHQQG81+FwfSNMvGxLRq3TSOvC95k3CNHrVwyZvT7Vsg9TDVV/vToWkapqf3U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hW8JTjrH; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hW8JTjrH" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2bf30d530bdso40104285ad.3 for ; Mon, 15 Jun 2026 20:13:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781579621; x=1782184421; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YS1IcSMum0RF5BraEGhpud+NVT65eKWpTnhBO1AgKGw=; b=hW8JTjrHa0C/3C4FuVIms1TQ+0YdmqOtbaWCSacZeLNMmwllZtM2PN8qKx5+f6aQ/c 3XHTYSrlpWWhgOrsg9hFd9Jhs9bv9A7Ri13lhZSZ1QHlGo6cxErOwmQPKIOBdaYS5xzN k65W1vqG/9jim08ko/qrpFbe+FD6WrSpbzAdQwoPgacGqOwfJkVX4D3E1SjH0r3/B3BT 7BizDFXH4iFAaOCXcFt7nl5yUiG8NNde5d4NIiW1QM0rsgcQoUJ4STUGFhYEQ8byfYdf vWw5cjETigvzXxwuh5r5OsYbeTEy4mNKAbM8/4g3GA9wGg5SY3EZBsiFa8v1RjguCVE6 Hrmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781579621; x=1782184421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YS1IcSMum0RF5BraEGhpud+NVT65eKWpTnhBO1AgKGw=; b=QgrfUCmrMjGOUOC+h2tZGTFToGDoMrQsED/h6rWyK9yIWrMs5AquoLIsmDzm3mOwbI UEzByQKsGintYm7gUdKBWRzOsI8OeCR1/ipeQvhE+SMKnHtbywLaKROV4vt9vwfA5LPd idrzCIL986slQCAlIExrEyRHfqlaFX563rCoYC63wZZpn5VYHx6zu/eS50xqi/AVwa0Q aQ7/TKxGjztjdv8WK0GCLTzjA/dY4R5TYnDjQhV5y2z4w7yXEet3W176B/99yi5uTWfI HzziOjnXutJ850O3dsSHUUWSH399wEhOm8yl5OuW+4TAe0MAtDW+nwppZPvYP2Fjs78H ztmg== X-Forwarded-Encrypted: i=1; AFNElJ++aSjsDePkGhTmkwXY7apyO1oTop04xXNSx9vqmkX5+gXJrg11O6dxvRW/H6JAIG8+qida5xrnJTMHodY=@vger.kernel.org X-Gm-Message-State: AOJu0Yyg36t9nybp5ct3xRmcd5HOdLAG7ABiOQpY7V036KaLMlukNYY7 YZEyrTc/1kPQvCSdYbXiJ0GZvTnFaZzcf8dyYZfIo5Wv4nHboedBYk+8 X-Gm-Gg: Acq92OGAbmF9/oGrs06N6qM7HYMIZj8cfMMCe89eoN/fNIlPGYPaZzaSaHCVChLQ40S FLtLParzTfncP7UsEnIWrJc8NEcni6l/hys0Wzz7gjOOkSm/EUv8GdFQ3W6Gq6shBF24Lx4a1OI Wxt+FEndrQB2fsPDoFZ53crIh4Lv2HkyhqTCyJrlhHOBzMbPcC9phjdlR6LmtRUCVGBdQu4zygK Igyq8AQf/Kw/kylGMHh1HzVzXW9dloeOP3jvr28z/txEVN7imCLLBEPznfu+vMgjlVuzmzXeZiI YPSrTY0WKH9biKymREzxrpzy3EmlX39lrFzpSiNPxmFfJZy5CPbnwHcHQGD/4L7F3LmmM7TSupj EwMt6mgUfnIfB4xEdo7GC/db6PIWY0fma6wejU02A44qk/px2dxjkpGZkPY5AVH4lAyCxX0U7/c VBoVIaMuQna0FFqPn+iVHi9DJD/6ifS9MkAUnhvdY= X-Received: by 2002:a17:903:3bcf:b0:2be:39bd:8dd8 with SMTP id d9443c01a7336-2c66429433bmr142968875ad.33.1781579620838; Mon, 15 Jun 2026 20:13:40 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c42f2e5644sm118923385ad.4.2026.06.15.20.13.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jun 2026 20:13:40 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Xueyuan Chen , Wenchao Hao Subject: [PATCH v5 3/4] mm/zsmalloc: drop class lock before freeing zspage Date: Tue, 16 Jun 2026 11:13:08 +0800 Message-Id: <20260616031309.3036234-4-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260616031309.3036234-1-haowenchao22@gmail.com> References: <20260616031309.3036234-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Reviewed-by: Nhat Pham Reviewed-by: Joshua Hahn Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1e9a150cd255..8a0c0c68121d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -884,13 +884,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } =20 -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zspage *zspage) { struct zpdesc *zpdesc, *next; =20 - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 @@ -906,7 +903,13 @@ static void __free_zspage(struct zs_pool *pool, struct= size_class *class, } while (zpdesc !=3D NULL); =20 cache_free_zspage(zspage); +} =20 +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1519,6 +1522,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free =3D NULL; =20 if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1529,10 +1533,23 @@ void zs_free(struct zs_pool *pool, unsigned long ha= ndle) obj_free(class->size, obj); =20 fullness =3D fix_fullness_group(class, zspage); - if (fullness =3D=3D ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness =3D=3D ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free =3D zspage; + } else { + kick_deferred_free(pool); + } + } =20 spin_unlock(&class->lock); + + if (zspage_to_free) { + __free_zspage_lockless(zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); --=20 2.34.1 From nobody Fri Jun 19 22:16:45 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C8E13C6609 for ; Tue, 16 Jun 2026 03:13:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579625; cv=none; b=lKRaNEbAVRB53bnfgsBVVDeRLWleCdknM99W/GLV8e/JasiFI7zi8rB+QI+53CgrAEfUfJQjz1cXblIBJdDO7c2fC1tw5ft1HRbQ5yx2jBTi0rtU0gdbvqiOwgx9nFdBA/trOT1EIk92JKUhOOoUC2R2OSl3zLqyqG7gdc1lx1U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781579625; c=relaxed/simple; bh=e0ujycxjNvJRsGvIQTQvgLtWHHw+lb5+UEwvvtQB2po=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DhSO+sJC61xNYET6Lt6FpdipFekbg9lM1dLwrk2bP2+8f6r1c0Q4YB0ZMSquPtxqjWurnwhD1/LCbR2DGRXAnTMwaRY0mfFuvfx8UNiFp+EXw6pEhl6tAP0razFe1ldt1BBTID0qqk5/dxk1BvnqB8HwyKa7z3iFzlNtG2vCsqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HwHV+9Az; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HwHV+9Az" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2c0bb4a94b8so31688445ad.2 for ; Mon, 15 Jun 2026 20:13:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781579624; x=1782184424; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YPqw2XIMhMyn82jhf5fcsIJXemWf4yzevbAx5L3vlSY=; b=HwHV+9AzxnNUNZh3x1K7GWoO9sGriK02cd552oVcMY6kuLC/GxN6KFY+PTXcxITe+F saIXjZCd/JulbZFwYEjG/qiD9ZzOcdTX8aU10La/6vlPzGuGoZ3rr0VZKAZJM89I46b7 lwoRyhqwT+rtWQ612IOmW9AOf/NNY1CX4Va5015r19e5N6cqdgs1Xkix7Z1u48Y0yt0k Ox5C2cL2HlppEdJNDqohj8qF7tJ9JNpJd37/MDWVZVN9q4QAvVVk4spjZzSGegzf2Y04 xsJcygwGclgK0CrVBpoyhkGQKzGloVL4aOqdd+amKy46k12iR4R/xdQIu/UtXALE35D7 kdtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781579624; x=1782184424; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YPqw2XIMhMyn82jhf5fcsIJXemWf4yzevbAx5L3vlSY=; b=bf2N2YzJCyhiezovRu3hMU+By9ygsjQ0hHvJ+kjbf7Y/qJ7lBLkigvIbG3TbBrsHFw CeraIU6nQjlaYqil45YXJSi9i+kjp8YzdAryP5rhJM+D6+pQ7HfYQ27NjzfPKx/j44m3 VRdUOpn6hXrq5e/FDiUCHTXmYpLRVU+cwYVAX85tP/NDfmlEibaKQeWEbegEHnsvc33E LBv4eLmalPRs6J5CW3jn0FU1GpjSIWES4qiU+mUqujdoKmSkLYQivnLLwytMRVySmVMc hwtXgzEwkNtjkh8ZiN9W7zTfIeTU2jvGs8T72ookRtwZ7hm8l6J1j443xNd9V0+dRvrl N05g== X-Forwarded-Encrypted: i=1; AFNElJ+jZoPMCOrGJxBtNQiSOcr65QHH6uCdompUYwQh0oKVUpWD/gniprYGEJS/88S/mFZVifqLgJ/xV08FJjc=@vger.kernel.org X-Gm-Message-State: AOJu0YzdmAJzShPYTSA4iH/4yzpGDA1DMgZTpzJ4ZZ6u4wEPoCcvxyXw HDKE83y05eFbrwty/RtsU9EBAgvvCBWqAC8ZzS6bWKR0iP74TFBwYWYA X-Gm-Gg: Acq92OEY0fuixUIhE5GsiTj+AL9SXClqSJdlSuU/yqWmQfyJwEJxnRpyoft0TyJI7It nYXRqT//jDe4/ezRe60qGPbN5EfPtZYAlhbjToNBCriLsloiVIWWXXL7fS3ilfNa6frGX8ZbEe5 gAdX30wtteBTGSOX6czMevXAjH7iwKDC2qUToBCoai3Q0IDKUPU29w9Cf+KNwQU9DojaApgMG1x tqaLg6kAuHQ/NpRXOwxG7/fZINbzrMoLdFOZMycR2jP5FM00vxptj2VD6Sx2dhKigb6aEFpRYMG hIeRHrYzYNMyBmSNAxoQjn42au3K3WYIjp03DuhESgX5gdIuMzqC3BXKEQeJbSb3Sne127nQIR0 AkNOb9Bp9iREVkdXUT+qwLPK0SojnK5MnEoQQUPYmcMTBy9q088Dh5OF9ob+SJ+2pUQVM99yOeL /a80ZWZRWHTKDhBApx4mnrS+5uRQKJLrMr6CWfyWY= X-Received: by 2002:a17:902:f64f:b0:2c2:2a8a:af69 with SMTP id d9443c01a7336-2c4110b4879mr179489185ad.9.1781579623869; Mon, 15 Jun 2026 20:13:43 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c42f2e5644sm118923385ad.4.2026.06.15.20.13.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jun 2026 20:13:43 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v5 4/4] mm/zsmalloc: document free_zspage helper variants Date: Tue, 16 Jun 2026 11:13:09 +0800 Message-Id: <20260616031309.3036234-5-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260616031309.3036234-1-haowenchao22@gmail.com> References: <20260616031309.3036234-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao After splitting __free_zspage() into a lockless core and a wrapper that does the class-stat bookkeeping, three similarly-named helpers coexist: free_zspage / __free_zspage / __free_zspage_lockless. Add a comment block above them describing what each does and where it is used, so the names are not easy to confuse. No functional change. Suggested-by: Nhat Pham Reviewed-by: Nhat Pham Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 8a0c0c68121d..23fd01398d41 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -884,6 +884,22 @@ static int trylock_zspage(struct zspage *zspage) return 0; } =20 +/* + * Three free helpers, kept apart here: + * + * __free_zspage_lockless(): bare core; walks zpdescs and returns pages + * to the buddy allocator. Caller owns all zpdesc locks and has + * removed the zspage from its class list. Used by zs_free() outside + * class->lock so the buddy-side work does not stall the class. + * + * __free_zspage(): __free_zspage_lockless() + per-class accounting, + * under class->lock. Used by async_free_zspage(), the worker for + * zspages whose trylock_zspage() failed. + * + * free_zspage(): full wrapper - trylock zpdescs, remove from class + * list, call __free_zspage(); kicks deferred free on contention. + * Used by compaction. + */ static inline void __free_zspage_lockless(struct zspage *zspage) { struct zpdesc *zpdesc, *next; --=20 2.34.1