From nobody Mon Jun 8 06:36:55 2026 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C300A3DA5AF for ; Fri, 5 Jun 2026 08:43:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648991; cv=none; b=SzswDsUjWcsxPGeNYwPmxjz7vzdysXFjFqUAjHoMJ2USNSmKKtf9u2sG5//fcMrnMmFFHFd3vzeW6MW7iI0IEOEHHhvVSCks12nXmTXc7SI2SlQtSqn23KLNdDH58J8e36FvK/rkeQJYQltuzqy6JJSXznREhv0oKKp+KzIMj3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648991; c=relaxed/simple; bh=ylUxj8OGdLlBiqDfOwRXDS3h9yYALv05t16XWh+8C38=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JHAhk+brgv0+SiSd7vuBXATq6sQ7s2xduZ3RxgzrlNfmulDXJVxC8M+mSLkSS9Sd/0QpaYeA/GUZHZxLFV3lEXk+gWGw+n7S8FgCFV0wFR1q4qvcJGOYm+BQAWyEpYWUr2MtBga5F/IQLqjMKvTTl8pJP9r0u48j0NwUGlWc7Uw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=X8Wq7IUl; arc=none smtp.client-ip=209.85.216.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="X8Wq7IUl" Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-36dac5d5da0so830263a91.2 for ; Fri, 05 Jun 2026 01:43:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780648989; x=1781253789; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=quOGtfpkvwp2+8B7oBqQS3hulhtIRkQ3/ceADPVIKv0=; b=X8Wq7IUlifL2CqK3dfjNbGuH6Z8CHGigm3uNUWOkdFIcV9fU1Om94n/7bSuKeFE8c/ R+oKxjgGDcgVWlxl83rXaqZ8TROsI1SLpg/vNTtTwdUTali2qq+nC7zHATD451xrUU8r qdvAVBYvOKugxoN13fsflLOSbJB1AckcEkgDV+MO+OPEBI6IgtaKB92w3MIFLAwVN5Cp PKI5oNOIaI68nfhM6d55GPAKPhG2YgrgriIEw1Z3KMVpTZqK7krsYoyDrqAeOzYA2bwb mn/KspVA2qDl+HCK87M5xXKMEoflck9jSNU8j7Zjf+XrbCqcIS+e69pdvJTV27kSOQ4w SzTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780648989; x=1781253789; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=quOGtfpkvwp2+8B7oBqQS3hulhtIRkQ3/ceADPVIKv0=; b=PpvnN2LDqQG5ML/D3UMMlbUKjjhzfJNN8QLAKAcoEGjVunMFFZ3BLA8T71Sp6gdO+m XtL3D9wT6EbQfOZ9pnD6GMDxqMq62GojIbIVaN8Fv4Lbzvvvwk01rCHkgT05k+wgaZsV f0KAX23RhNm+6uCqEzb5iz5xtQW7YTlSjvyBHF+civf0vchhQVgOiYYMOE5L9WJu1dzY 4GRDFiqaaElu698bH+riAL365qu0Netkb3w6DIMnHjGTth7PaHLaIZdK6pYJvU7dU9t9 30WQVgoPpUq3JK8HcQ7tiu7D2s/q6NRE+VE9f3ALt1b1eGvOIjpNY2lOQrnGuc4hVCeR mkrg== X-Forwarded-Encrypted: i=1; AFNElJ8ChcwZ2zGRjUZTXG3p9Y18rrfP1lvRVsnlfGwH/KXjdF/hW24r3IUuVFs6R6DmUQ+RxtaahEGywi0BKbM=@vger.kernel.org X-Gm-Message-State: AOJu0Yx9vjCsrJh9DduLh1tRhcucSzULvG8a/v8lOyYBPlHA/qimtoCY f5eeD4JPKiEw39ya4UXGQ8WKhqlbReau74fA/0pVMZUJkqBlZBvFDD7p X-Gm-Gg: Acq92OEtjV23F22k97KBMWu7jj6ARngh6/AxyGfLI8Hgs32uNqOjWoFwBTwmOuihuDZ CT9ktn+lP/IApbx65gDCYxA/OKyQIlNtHy1FOlLZ+6EGqznEFEh1RLtbKCi3CfeV3StXopQj68N BQrJ21S7n/BSzVqHCh9OEcljGVTeyt0K0osFi4Rbb6rmByaH3mqafToa/JlieijQkshld3kvecK BWpphrUwNSafySEpGKN9gv27bzCcCxZrs00F0QvsZejSBDRPa+8rek4fL+ZNLZ9mnZaduo/t6dt etzza2aPB6YQNqRSvXHFF+oZqypHhgQWA5IcxdmZ8xGtpJFZSkoBYR0m3p1rdnR9sfJ4GJu5Sfg OgvNOqn5dYhlG6rHwMeB7BSfhpLFksjgZXRed8J0Cwtm8glBRYPjBkzRmiz5PnML09ghEn58mGf 67QBINh4feZLuYsM3AUiHoXc9OiDikYogBbYjW+yXj1D2Lu3t8F5rrp9Ir/cfJ X-Received: by 2002:a17:90b:38cc:b0:36d:f28b:72e0 with SMTP id 98e67ed59e1d1-370ee92adfdmr2924474a91.12.1780648988874; Fri, 05 Jun 2026 01:43:08 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3712fcb2607sm926632a91.0.2026.06.05.01.43.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2026 01:43:08 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v3 1/4] mm/zsmalloc: encode class index in obj value for lockless class lookup Date: Fri, 5 Jun 2026 16:42:39 +0800 Message-Id: <20260605084242.1549811-2-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260605084242.1549811-1-haowenchao22@gmail.com> References: <20260605084242.1549811-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao Encode the size_class index (class_idx) into the obj value so that zs_free() can determine the correct size_class without dereferencing the handle->obj->PFN->zpdesc->zspage->class chain under pool->lock. class_idx is invariant across page migration (only PFN is rewritten), so a lockless read of obj always yields a valid class_idx. The space below the PFN field in obj is over-provisioned on 64-bit systems, with more bits than obj_idx needs. Split that space into class_idx and obj_idx subfields: |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->| +-----------------+-------------------------+-----------------------+ | PFN | class_idx | obj_idx | +-----------------+-------------------------+-----------------------+ MSB ^ LSB | +-- ZS_OBJ_PFN_SHIFT The macro layout changes as follows: Before After Meaning ---------------- ------------------ ---------------------------- OBJ_INDEX_BITS ZS_OBJ_IDX_BITS width of obj_idx subfield OBJ_INDEX_MASK ZS_OBJ_IDX_MASK mask of obj_idx subfield (n/a) ZS_OBJ_CLASS_BITS width of class_idx subfield (n/a) ZS_OBJ_CLASS_MASK mask of class_idx subfield (n/a) ZS_OBJ_PFN_SHIFT bit offset of PFN in obj On 32-bit systems there is no spare room for class_idx, so the encoding is disabled (ZS_OBJ_CLASS_BITS =3D 0) and the obj layout remains [PFN | obj_idx]. Reviewed-by: Nhat Pham Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 80 ++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 66 insertions(+), 14 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 63128ddb7959..6b0014b43408 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -67,8 +67,8 @@ #define MAX_POSSIBLE_PHYSMEM_BITS MAX_PHYSMEM_BITS #else /* - * If this definition of MAX_PHYSMEM_BITS is used, OBJ_INDEX_BITS will just - * be PAGE_SHIFT + * If this definition of MAX_PHYSMEM_BITS is used, ZS_OBJ_PFN_SHIFT will + * just be PAGE_SHIFT */ #define MAX_POSSIBLE_PHYSMEM_BITS BITS_PER_LONG #endif @@ -88,8 +88,27 @@ #define OBJ_TAG_BITS 1 #define OBJ_TAG_MASK OBJ_ALLOCATED_TAG =20 -#define OBJ_INDEX_BITS (BITS_PER_LONG - _PFN_BITS) -#define OBJ_INDEX_MASK ((_AC(1, UL) << OBJ_INDEX_BITS) - 1) +/* + * obj is encoded as [PFN | class_idx | obj_idx] within an unsigned long: + * + * |<-- _PFN_BITS -->|<-- ZS_OBJ_CLASS_BITS -->|<-- ZS_OBJ_IDX_BITS -->| + * +-----------------+-------------------------+-----------------------+ + * | PFN | class_idx | obj_idx | + * +-----------------+-------------------------+-----------------------+ + * MSB ^ LSB + * | + * +-- ZS_OBJ_PFN_SHIFT + * + * Encoding class_idx into obj lets zs_free() locate the size_class + * without holding pool->lock; class_idx is invariant across page + * migration (only PFN changes), so a lockless read of the obj value + * always yields a valid class_idx. + * + * On 32-bit systems there is no spare room for class_idx, so + * ZS_OBJ_CLASS_BITS is 0 and the layout collapses to the original + * [PFN | obj_idx] without any ifdef in callers. + */ +#define ZS_OBJ_PFN_SHIFT (BITS_PER_LONG - _PFN_BITS) =20 #define HUGE_BITS 1 #define FULLNESS_BITS 4 @@ -98,9 +117,29 @@ =20 #define ZS_MAX_PAGES_PER_ZSPAGE (_AC(CONFIG_ZSMALLOC_CHAIN_SIZE, UL)) =20 +/* + * Reuse the width that struct zspage already reserves for its + * class field (zspage->class:CLASS_BITS + 1) for the class_idx + * field encoded in obj. On 32-bit there is no spare room, so set + * it to 0; the encoded class_idx then folds to a constant 0 and + * the layout collapses back to [PFN | obj_idx]. + */ +#if BITS_PER_LONG >=3D 64 +#define ZS_OBJ_CLASS_BITS (CLASS_BITS + 1) +#else +#define ZS_OBJ_CLASS_BITS 0 +#endif +#define ZS_OBJ_CLASS_MASK ((_AC(1, UL) << ZS_OBJ_CLASS_BITS) - 1) + +#define ZS_OBJ_IDX_BITS (ZS_OBJ_PFN_SHIFT - ZS_OBJ_CLASS_BITS) +#define ZS_OBJ_IDX_MASK ((_AC(1, UL) << ZS_OBJ_IDX_BITS) - 1) + +static_assert(ZS_OBJ_IDX_BITS > 0, + "zsmalloc: PFN + class_idx leave no room for obj_idx"); + /* ZS_MIN_ALLOC_SIZE must be multiple of ZS_ALIGN */ #define ZS_MIN_ALLOC_SIZE \ - MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS)) + MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> ZS_OBJ_IDX_BITS)) /* each chunk includes extra space to keep handle */ #define ZS_MAX_ALLOC_SIZE PAGE_SIZE =20 @@ -721,26 +760,38 @@ static struct zpdesc *get_next_zpdesc(struct zpdesc *= zpdesc) static void obj_to_location(unsigned long obj, struct zpdesc **zpdesc, unsigned int *obj_idx) { - *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); - *obj_idx =3D (obj & OBJ_INDEX_MASK); + *zpdesc =3D pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT); + *obj_idx =3D (obj & ZS_OBJ_IDX_MASK); } =20 static void obj_to_zpdesc(unsigned long obj, struct zpdesc **zpdesc) { - *zpdesc =3D pfn_zpdesc(obj >> OBJ_INDEX_BITS); + *zpdesc =3D pfn_zpdesc(obj >> ZS_OBJ_PFN_SHIFT); +} + +/* + * On 32-bit systems ZS_OBJ_CLASS_BITS is 0 and ZS_OBJ_CLASS_MASK is 0, + * so this collapses to a constant 0. No ifdef needed at the call site. + */ +static unsigned int obj_to_class_idx(unsigned long obj) +{ + return (obj >> ZS_OBJ_IDX_BITS) & ZS_OBJ_CLASS_MASK; } =20 /** - * location_to_obj - get obj value encoded from (, ) + * location_to_obj - encode (, , ) into obj va= lue * @zpdesc: zpdesc object resides in zspage * @obj_idx: object index + * @class_idx: size class index; ignored on 32-bit (ZS_OBJ_CLASS_BITS =3D= =3D 0) */ -static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx) +static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int o= bj_idx, + unsigned int class_idx) { unsigned long obj; =20 - obj =3D zpdesc_pfn(zpdesc) << OBJ_INDEX_BITS; - obj |=3D obj_idx & OBJ_INDEX_MASK; + obj =3D zpdesc_pfn(zpdesc) << ZS_OBJ_PFN_SHIFT; + obj |=3D (unsigned long)(class_idx & ZS_OBJ_CLASS_MASK) << ZS_OBJ_IDX_BIT= S; + obj |=3D obj_idx & ZS_OBJ_IDX_MASK; =20 return obj; } @@ -1276,7 +1327,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, kunmap_local(vaddr); mod_zspage_inuse(zspage, 1); =20 - obj =3D location_to_obj(m_zpdesc, obj); + obj =3D location_to_obj(m_zpdesc, obj, zspage->class); record_obj(handle, obj); =20 return obj; @@ -1762,7 +1813,8 @@ static int zs_page_migrate(struct page *newpage, stru= ct page *page, =20 old_obj =3D handle_to_obj(handle); obj_to_location(old_obj, &dummy, &obj_idx); - new_obj =3D (unsigned long)location_to_obj(newzpdesc, obj_idx); + new_obj =3D location_to_obj(newzpdesc, obj_idx, + obj_to_class_idx(old_obj)); record_obj(handle, new_obj); } } --=20 2.34.1 From nobody Mon Jun 8 06:36:55 2026 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 144DA3D9052 for ; Fri, 5 Jun 2026 08:43:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648994; cv=none; b=KiS5vbFTe6tjj9ozmiXTAuLPXhcyG2QjhhuSHLA49v+eVLsH417eSOy3KdYA6k0S5f+IZ6kBIjhQrvn9i6+1XjbClNiEutPNwzZDe8zAimFTldY1irrcZZxHgeWN3/YNsywkzLM3vf1p8ZHD/L+S07LlclQYPUXRMo1mKPTLfyw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648994; c=relaxed/simple; bh=0lGVAQZ5P+uA0ou7wGsBYz/mnzlFu7PDqqeo3PvK4Cw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tAPDdF/1FXPt1Yx02yT13TCZnLA82TjGLiw2AH3uiSlr9bWFlU2skkSS4qNEirOrXS75yn9AUj4m7UHwonXiSC+rZ3AqGxPTOTQbsQCHjPGs6UqIh+GEOM9KYf8aczrb7JbWfg77Igni2gIYeDwnQvPdSJayETJGI0lf/NYRnnA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QX2VBl7l; arc=none smtp.client-ip=209.85.216.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QX2VBl7l" Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-36d98b68d68so1178375a91.2 for ; Fri, 05 Jun 2026 01:43:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780648992; x=1781253792; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z1nQOtmsuSqcQUPuYJZAkaLb8MkxxRia+eZsD/iG1mU=; b=QX2VBl7lU5n44nxgeHROzSGEij8dYbbJPyypY3Xm6Faxj+Iz3InWFGj2mn3rqUm6AP CqwGZCAaZE6LPIR5lPyWmc+mDOrEqyRw3C/ukM5jkY0le0evID33UZMp0N2xDrxxrCRd icWIRKxMDvc5jaozLb+xegjZf9GnwTcY4ZCFAgh8VfgxAFuWybrmoQau7cUeE8eZ+leu s6B38n7uq/1xWF4+7HTzHUZsYAyrGesO1vQdU/lZawawgpN1PVgrmZs03n5q76LAccv3 4na7h9zz31dCmJGo27FO9lVOwclIG+W4cHDXd4KVdhFvkGCLGtYoQHQYtSEYbMmujmet ynUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780648992; x=1781253792; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Z1nQOtmsuSqcQUPuYJZAkaLb8MkxxRia+eZsD/iG1mU=; b=lu5R+KCpVfrMeL0t/sSbVT44aTjSwvCnbYKot3v7yiXO8iktvIedQFMJ1/paXTP0YY jbImEu4XCajkG6JrZ3B88tO8MLGAGN/P+Igda6CWp6WyMLIe0ss93Q5V3zUit/z/q4zb 4Kr7TE+LkeKRiYoE3A8Zfo+RhNoll4Dju0xAyUTcscyfXjKgWUKQKXsauTCBKMUUajBJ Q83sHQFSzqh0bQQ0Y7XUdpuJoVTSiwRbtLYmKvWynGkspvOvDY6ELOqtDu961ULRxhgi FoUCEg65IfNgB7jNqAGvCUJVGEXSXnOnEvcUVhRXG64Peqm68+HFRtuLn8fQksS9iQ7M jHQg== X-Forwarded-Encrypted: i=1; AFNElJ87cNGzE3jdG6iuzn4TBQycBwUxPy6dkLTufg1DwrvurEVfe7WnX6Pdpm7QR7T8o5m3YMQBRFRCN6jP6i8=@vger.kernel.org X-Gm-Message-State: AOJu0Yypd25j8MBsoDKcFGXOu3Gacx4glo49UkJu4fHlAfhNYhEJKsKN t2ngrbc+asmbk/7Urhi5gOjrT6MKXLNWViR0W/7ET7mCtk4N7MuDiV5V X-Gm-Gg: Acq92OHwXSY6Od1iq9I9rWql2zxpu6dgKf+dXcLtZ/SeSKZqliC9tCEzXYokWc1islJ awLsZ0EuCtEC3RdrdX0nbqOWtVTQv/d5kORt49jD+mJtKFuvIExDe1ZzucLhfpU7IxaLWGhFdwQ Yapps8xhov0AtTfwVy9SvMp8Ozk2ItTs8M/hHhKb1BkII5ZB65QTTF7iqTBBOVMYEAr0pybigrI IZSFPniNxWe5C+dh8p/LR+FnIR9vvHBnA3I71POHUUtuOlzy04MfZ4U35tP9Eb5K67z69hNyB67 bCOFgD7J5akDm8c2IjHdnBEyxQ3hiQOfZvNhgZnwSqdRHXYFP5BOwWynqALdEWzq2MZecYAdbVa nOq8Doz+SLLCnlgZM403w8MAkHyTmkbfamYdiqOBBkWrkyvao0YaG1PibXDlg2JiQiu/VcXFcqW r0rV5gaojrEJdozPAWjEVi6bnUvWLKRFosxGLb1iX2Yq1Z8jOWjafKs7pJWM7G X-Received: by 2002:a17:90b:38cc:b0:368:f179:ba07 with SMTP id 98e67ed59e1d1-370eec125f1mr2950361a91.9.1780648992235; Fri, 05 Jun 2026 01:43:12 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3712fcb2607sm926632a91.0.2026.06.05.01.43.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2026 01:43:11 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v3 2/4] mm/zsmalloc: drop pool->lock from zs_free on 64-bit systems Date: Fri, 5 Jun 2026 16:42:40 +0800 Message-Id: <20260605084242.1549811-3-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260605084242.1549811-1-haowenchao22@gmail.com> References: <20260605084242.1549811-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao With class_idx encoded in obj, zs_free() can locate the size_class without holding pool->lock on 64-bit systems. Page migration also takes class->lock and only rewrites the PFN field of obj, so: 1. read obj locklessly, 2. lock the size_class derived from obj's class_idx, 3. re-read obj under class->lock to get a stable PFN. This eliminates the rwlock read-side cacheline bouncing between zs_free() and migration/compaction on multi-core systems. On 32-bit systems pool->lock is preserved. Reviewed-by: Nhat Pham Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 67 ++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 55 insertions(+), 12 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6b0014b43408..88d10f814da9 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -21,6 +21,10 @@ * pool->lock * class->lock * zspage->lock + * + * On 64-bit systems zs_free() does not take pool->lock; it locates + * the size_class using class_idx encoded in obj and serializes against + * page migration via class->lock. */ =20 #include @@ -1432,10 +1436,59 @@ static void obj_free(int class_size, unsigned long = obj) mod_zspage_inuse(zspage, -1); } =20 +/* + * Resolve @handle to its zspage / size_class and acquire class->lock. + * + * 64-bit: class_idx is encoded in obj and is invariant under page + * migration, so the handle can be read locklessly to pick the + * size_class. Once class->lock is held migration is blocked and the + * handle is re-read to obtain a stable PFN. + * + * 32-bit: the unlocked load of *(unsigned long *)handle is not + * single-copy atomic and class_idx is not encoded (ZS_OBJ_CLASS_BITS + * =3D=3D 0), so fall back to pool->lock for the lookup. + */ +#if BITS_PER_LONG >=3D 64 +static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned lo= ng handle, + unsigned long *objp, struct zspage **zspagep, + struct size_class **classp) + __acquires(&(*classp)->lock) +{ + struct zpdesc *f_zpdesc; + unsigned long obj; + + obj =3D handle_to_obj(handle); + *classp =3D pool->size_class[obj_to_class_idx(obj)]; + spin_lock(&(*classp)->lock); + /* Re-read under class->lock: PFN is now stable vs migration. */ + obj =3D handle_to_obj(handle); + obj_to_zpdesc(obj, &f_zpdesc); + *zspagep =3D get_zspage(f_zpdesc); + *objp =3D obj; +} +#else +static inline void obj_handle_class_lock(struct zs_pool *pool, unsigned lo= ng handle, + unsigned long *objp, struct zspage **zspagep, + struct size_class **classp) + __acquires(&(*classp)->lock) +{ + struct zpdesc *f_zpdesc; + unsigned long obj; + + read_lock(&pool->lock); + obj =3D handle_to_obj(handle); + obj_to_zpdesc(obj, &f_zpdesc); + *zspagep =3D get_zspage(f_zpdesc); + *classp =3D zspage_class(pool, *zspagep); + spin_lock(&(*classp)->lock); + read_unlock(&pool->lock); + *objp =3D obj; +} +#endif + void zs_free(struct zs_pool *pool, unsigned long handle) { struct zspage *zspage; - struct zpdesc *f_zpdesc; unsigned long obj; struct size_class *class; int fullness; @@ -1443,17 +1496,7 @@ void zs_free(struct zs_pool *pool, unsigned long han= dle) if (IS_ERR_OR_NULL((void *)handle)) return; =20 - /* - * The pool->lock protects the race with zpage's migration - * so it's safe to get the page from handle. - */ - read_lock(&pool->lock); - obj =3D handle_to_obj(handle); - obj_to_zpdesc(obj, &f_zpdesc); - zspage =3D get_zspage(f_zpdesc); - class =3D zspage_class(pool, zspage); - spin_lock(&class->lock); - read_unlock(&pool->lock); + obj_handle_class_lock(pool, handle, &obj, &zspage, &class); =20 class_stat_sub(class, ZS_OBJS_INUSE, 1); obj_free(class->size, obj); --=20 2.34.1 From nobody Mon Jun 8 06:36:55 2026 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9057B40629E for ; Fri, 5 Jun 2026 08:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648998; cv=none; b=LUdt1rCRjI653/BIbrkuoTYgWSSd9QtNdkVV+GKHkGsM4llv+tyikqNK5KHFcO84qNyroWBIDZnu+VIe7dJsRZH8q03ZpofhC3EXNwO5Pc+cOft36whJWAv37/XCql1D4MsbytGdf5CcS3eoYrgQTemZzuRp7J+aLdLnhd0y4/Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780648998; c=relaxed/simple; bh=CUtHBCNi8z+cxNsqQp/7jAzIf9QD7guzxjv+fVd/asw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EJx3SmkOtOk9gg6STzuXPP1aSkm3ONtzXu+z05QeSiRPHTY9cB2NoHwkA3R4GlyrsUE7GjpHxdCbY+pH6CdIGfTSF1zP9irzJzv8jp1o7MJX26eXjcCCTSKT6MuvcPygsOirGIGFKNnt+uIUCTJeFWmdF7IvvigvVnJfvcb+3rE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PSdJMMjJ; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PSdJMMjJ" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-36ba3ea5c46so1019629a91.1 for ; Fri, 05 Jun 2026 01:43:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780648996; x=1781253796; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=n185DjbA+W+3OzU5ke/OSZXijWXnafvOkeAJL8VmlKA=; b=PSdJMMjJuHEEtsJ5weOMor17w933HhLuEGNJlEica8iOwZnt3IEBQqAp0oHAy+uH4n MId4dSgm1bs9eKouYxEBF/vrqHQP6bdQ7gOhpVyyfEDgMCsC0y3/tYd6Nn0M2Bwjk149 yl+AOWPjVkZ7ojnbgmvePy4wrre6G2crTcxSdafYlmqXlk+ZIqEN7Tqh3P7hT+DWFjzm 5R5TCDigUSALGcTu9Z+NHxC0E7yvd7Rn+UWnnJh8Jnn4+J6J/dvK1nhjdHgAbdWbTeBq DZgkckBShOlAnUATx4qQy0nNDoeYz+uqLS1J3nZo0iekbPTewlkQrDkaqMw1P3Jk1Rf8 mWxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780648996; x=1781253796; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=n185DjbA+W+3OzU5ke/OSZXijWXnafvOkeAJL8VmlKA=; b=bCdGJZsjKFhbPnNZgm6mtMS+rEu7UHRUwrDq1Ph6O6W9lge10AbbiiWtBdhxE2jwHj Vl2V7w81cB8JOp3eNMPFI5p8mY2z1sXNPVt8M2wIeV21g43htCQrZPvKiZ/z88zT33SP 4s7GYE5YmkH+iQb20XuL2TIETDYc3MxXJZIm6YzjG1aio3h/P68lWLQsoTGgY2ToUOUL gtkvqMxbYj80YMj0Uk6UQ4o8zfry3leaKuuwzbqNVytOBo4HpM1SjpNSAwH/hoeD3czM uOIHveHuCFsLtWhiEu28iCtscpqcHjsYDEy6KmTfLiRIe2uQVdll4buHOBLiSG4d8y0L Kz8g== X-Forwarded-Encrypted: i=1; AFNElJ/meJxZjYcggsDG2OKz4Vf2neTWh/RD4w39CkJhhcev+J+YBX+JLBLrnbwSQkyWzS4nmE4ZVqNrvXJn7nE=@vger.kernel.org X-Gm-Message-State: AOJu0YxoV6sGsf3khmn/WmXosUMSpXMFwdkmevM8ZcXK7eoVWpyLTl1B +gAMCx6vA27TqvALgz95SgG9/j6d9H8dQUKJ9xOClF3kiQPRpPRAhQtJ X-Gm-Gg: Acq92OEL8lBpkyi+rIAtZmKceH4Y3Ak7IdT53SNN7loC+E5revTF4tuTaRj2dozRX4p kipQREx8ZXgz86IdGaRaZbaAzEqNYDajdx/coomijwhNk/GxpxLyqOUL1Og8K962JPz9H26p5v/ o8RmDVQqbURfcPumk6YnY3isGauwUdceXd5EFukIH7tiN5QN4ttddApHg2VmVfcqPIA4F0sS+Cp XA44YQPFqNCAq3yW0O6aX8Y3elSB3VmBnjrimbPYVV4VWn1Q+hiLQZRCbMF/2WZlJWHo69L2BGs lLzOjkEffEunP8t6KeCt7NXHylpMK7K0RgCZmuTQKQWUOKhHn7WLYS7na4G+8Q0K1I/Acck84Xo 02WFVsKr9ghpngLAjXfqNsBildSWJ72Ji38OfH3QQ3gayhZ162nMY8+8/XABDdzeJaC89lvPzti 5lXxeag/3RQ475WMBGFZy4h71qzzT0g30hetUB/XKqYfdbNauhqQ== X-Received: by 2002:a17:90b:57c4:b0:35f:b647:d98a with SMTP id 98e67ed59e1d1-370ee64373dmr2739847a91.5.1780648995812; Fri, 05 Jun 2026 01:43:15 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3712fcb2607sm926632a91.0.2026.06.05.01.43.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2026 01:43:15 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Xueyuan Chen , Wenchao Hao Subject: [PATCH v3 3/4] mm/zsmalloc: drop class lock before freeing zspage Date: Fri, 5 Jun 2026 16:42:41 +0800 Message-Id: <20260605084242.1549811-4-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260605084242.1549811-1-haowenchao22@gmail.com> References: <20260605084242.1549811-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xueyuan Chen Currently in zs_free(), the class->lock is held until the zspage is completely freed and the counters are updated. However, freeing pages back to the buddy allocator requires acquiring the zone lock. Under heavy memory pressure, zone lock contention can be severe. When this happens, the CPU holding the class->lock will stall waiting for the zone lock, thereby blocking all other CPUs attempting to acquire the same class->lock. This patch shrinks the critical section of the class->lock to reduce lock contention. By moving the actual page freeing process outside the class->lock, we can improve the concurrency performance of zs_free(). Testing on the RADXA O6 platform shows that with 12 CPUs concurrently performing zs_free() operations, the execution time is reduced by 20%. Signed-off-by: Xueyuan Chen Reviewed-by: Nhat Pham Reviewed-by: Joshua Hahn Signed-off-by: Wenchao Hao --- mm/zsmalloc.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 88d10f814da9..2ecdf79cea03 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -856,13 +856,10 @@ static int trylock_zspage(struct zspage *zspage) return 0; } =20 -static void __free_zspage(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) +static inline void __free_zspage_lockless(struct zs_pool *pool, struct zsp= age *zspage) { struct zpdesc *zpdesc, *next; =20 - assert_spin_locked(&class->lock); - VM_BUG_ON(get_zspage_inuse(zspage)); VM_BUG_ON(zspage->fullness !=3D ZS_INUSE_RATIO_0); =20 @@ -878,7 +875,13 @@ static void __free_zspage(struct zs_pool *pool, struct= size_class *class, } while (zpdesc !=3D NULL); =20 cache_free_zspage(zspage); +} =20 +static void __free_zspage(struct zs_pool *pool, struct size_class *class, + struct zspage *zspage) +{ + assert_spin_locked(&class->lock); + __free_zspage_lockless(pool, zspage); class_stat_sub(class, ZS_OBJS_ALLOCATED, class->objs_per_zspage); atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); } @@ -1492,6 +1495,7 @@ void zs_free(struct zs_pool *pool, unsigned long hand= le) unsigned long obj; struct size_class *class; int fullness; + struct zspage *zspage_to_free =3D NULL; =20 if (IS_ERR_OR_NULL((void *)handle)) return; @@ -1502,10 +1506,23 @@ void zs_free(struct zs_pool *pool, unsigned long ha= ndle) obj_free(class->size, obj); =20 fullness =3D fix_fullness_group(class, zspage); - if (fullness =3D=3D ZS_INUSE_RATIO_0) - free_zspage(pool, class, zspage); + if (fullness =3D=3D ZS_INUSE_RATIO_0) { + if (trylock_zspage(zspage)) { + remove_zspage(class, zspage); + class_stat_sub(class, ZS_OBJS_ALLOCATED, + class->objs_per_zspage); + zspage_to_free =3D zspage; + } else { + kick_deferred_free(pool); + } + } =20 spin_unlock(&class->lock); + + if (zspage_to_free) { + __free_zspage_lockless(pool, zspage_to_free); + atomic_long_sub(class->pages_per_zspage, &pool->pages_allocated); + } cache_free_handle(handle); } EXPORT_SYMBOL_GPL(zs_free); --=20 2.34.1 From nobody Mon Jun 8 06:36:55 2026 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2209E3D75C0 for ; Fri, 5 Jun 2026 08:43:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780649001; cv=none; b=X3+gncDJHdua9U+MROAaiz5wBj0SVbRu/xjtzm04jahn4GUXx24K/BMrfdj/SvqKEk/XXjGAUOtbCxPTIHx31K7I4eYtHnqGbHjGiqPSKGJPDDjVg2TTvjzR0+5kdnFT5XlWikC+Hd6tnsLI6DRoetPHEzdacOXpYr42apdpLNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780649001; c=relaxed/simple; bh=wSDsiSkevtHZyspDLFv2g3e4ELgdNKkWBW9bCKoYiXU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=X7SGSaHRP76LUmEGSC6IktCuwTLkSEOdvL8Qh9o3e9Be7oeljTh9l8sdbUVWbXQOIFz/MAIHM3OwILwrf815DV6rxCfFn1juoljjA9mVVTMTg7tUIq84WI9WAvnJ1NxSemuKYY98rOGNUgUsanZGj589NusxR7Fhm2L/bhI0N+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GdrcYTAr; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GdrcYTAr" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-36bba9a1089so1087382a91.3 for ; Fri, 05 Jun 2026 01:43:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780648999; x=1781253799; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J5brAHyiBTWKYNL/XpyaHSOnMQ5/iB4MkIc268tnFsk=; b=GdrcYTArYoo+2rbjhaM9O1N3iu+PQc6U+QQUHeeiIA6OGp8OvlJ6CnrI/SIulkXcdh iSm3514V0iSIvxrq8JHBB0JF1XuklFsOsuv94NoBlo5+YzLVnbyPNy6GPTON3sH/TEmb 7zsPYqKd4UjbsTig2UEBSoXDMvEiStM+rmSp1RAYIkDuqDHDiqayIEsMQUTBoe8c9fIh 2GdWs97Anc0V6Fi65p11VfMR3ncc12Oa3iaj0W8GWJb6c0nU8sOpUoI+e+Vw6cVxIFdJ Yr9lh4TDndSWw3uw2kuraOlrCczo8WX+09ryQG1nlnMXQnEti4Nu8XH4PRYOYW0GE1/M FPbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780648999; x=1781253799; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=J5brAHyiBTWKYNL/XpyaHSOnMQ5/iB4MkIc268tnFsk=; b=VONAM44jf05s2c/aT4K1B+lHaL6OIgoiFhgY4ILogHqptmpL2fSTFEc9MyNeTYcx/S VZkWdwJKQHgTYPkyHrkZDFF0RViTtWI+ZymLMMHbsBNfcVOBSBaD6nstFcawg8zyKyag XUhp32kQZR0ZiFLqQCZDpqayvi6tab6x7xDlVXzPHc48moN2YCzmqbY33J66l8w3TOQn ZOIFWD4RykJ9hd0y5XSykX8sfBbtE4XGE4+sj4iVgyaX/jjGZsjiiwk+0imlhDAhU15l 7Hag11KC4QTSLaGxuvRejr1NSmmrMk4hQQu6Q+0XPfJkT8VumafgMBPbEZFW93dyamyj Fqiw== X-Forwarded-Encrypted: i=1; AFNElJ9Bjc/CWE3kbFtxhOb/IhlPQ3bnyrVODv+IvDTahIohJdaI0jGoj7lX+sgigr8Yqoq05hhY/BY5qCBjMhE=@vger.kernel.org X-Gm-Message-State: AOJu0YyawVS0MCGTk6rgXMFaAYKtgGzfep5ok+irPmwZKCJN4ec7M+H+ wW0a8FhPKC54xfjL0eST7Vc5DrnSYBIyzLJWcIFi0y3+UKVIaoqsGZvn X-Gm-Gg: Acq92OEiYI2UgSLi+QxwV1YasKoFrbSidQekgaV7NjO+8l7eZ76TL62wFYuZVRdyBw2 FO2FYvi90+DTa3DWWB0mji39uHTICLOVnlzvAZ0rmP+C6WwDXL/pHDM/jdHvmGAI4gtOZXdFnIt Sjqsyef67WkH3gce0f7as2gwQHw16LqUVyMNZlcDaZ4EnqH3nmURj8JYdft/W6++VavEziHbPSj XBhYfFWdpy6xri70Oh+vGjuPgHXjdtrSgQjfODRT9Xez8iID2B9j9CkDxEt+nDDx4uzKg09l1Z6 K/vS3iLURwVEC10shmFSgsF9aQw3XWMf6jBfnxL3gjdYj9dQguFMZ8hmqJP37hlkhuQnPBBswlS m3OxgL/7yQyNq0d7tm/myxDSbZNm0N5b0KqWjpIafVZFYZ/b17VSZErfWVOeGgJ+w3A8chxxKMM skvjmGn7XnlDPfDc/K2sSicwCYCsPo2BSw5O/12Y7OaM6JGCco3Q== X-Received: by 2002:a17:90b:3a8d:b0:365:a5f6:4a5c with SMTP id 98e67ed59e1d1-370ee5448f4mr2558375a91.1.1780648999316; Fri, 05 Jun 2026 01:43:19 -0700 (PDT) Received: from ubuntu22.mioffice.cn ([43.224.245.232]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3712fcb2607sm926632a91.0.2026.06.05.01.43.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2026 01:43:18 -0700 (PDT) From: Wenchao Hao To: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Minchan Kim , Sergey Senozhatsky Cc: Nhat Pham , Joshua Hahn , Barry Song , Wenchao Hao Subject: [PATCH v3 4/4] mm/zsmalloc: document free_zspage helper variants Date: Fri, 5 Jun 2026 16:42:42 +0800 Message-Id: <20260605084242.1549811-5-haowenchao22@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260605084242.1549811-1-haowenchao22@gmail.com> References: <20260605084242.1549811-1-haowenchao22@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wenchao Hao After splitting __free_zspage() into a lockless core and a wrapper that does the class-stat bookkeeping, three similarly-named helpers coexist: free_zspage / __free_zspage / __free_zspage_lockless. Add a comment block above them describing what each does and where it is used, so the names are not easy to confuse. No functional change. Suggested-by: Nhat Pham Signed-off-by: Wenchao Hao Reviewed-by: Nhat Pham --- mm/zsmalloc.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 2ecdf79cea03..9f588b63ec0d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -856,6 +856,21 @@ static int trylock_zspage(struct zspage *zspage) return 0; } =20 +/* + * Three free helpers, kept apart here: + * + * __free_zspage_lockless(): bare core; walks zpdescs and returns pages + * to the buddy allocator. Caller owns all zpdesc locks and has + * removed the zspage from its class list. Used by zs_free() outside + * class->lock so the buddy-side work does not stall the class. + * + * __free_zspage(): __free_zspage_lockless() + per-class accounting, + * under class->lock. Used by async_free_zspage(). + * + * free_zspage(): full wrapper - trylock zpdescs, remove from class + * list, call __free_zspage(); kicks deferred free on contention. + * Used by compaction. + */ static inline void __free_zspage_lockless(struct zs_pool *pool, struct zsp= age *zspage) { struct zpdesc *zpdesc, *next; --=20 2.34.1