From nobody Mon Feb 9 05:43:23 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 736DCEB64D7 for ; Fri, 23 Jun 2023 04:40:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231272AbjFWEkk (ORCPT ); Fri, 23 Jun 2023 00:40:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229994AbjFWEkg (ORCPT ); Fri, 23 Jun 2023 00:40:36 -0400 Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com [IPv6:2607:f8b0:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 493852126 for ; Thu, 22 Jun 2023 21:40:34 -0700 (PDT) Received: by mail-ot1-x334.google.com with SMTP id 46e09a7af769-6b71cdb47e1so280626a34.2 for ; Thu, 22 Jun 2023 21:40:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1687495233; x=1690087233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A39gdokmxlm4U38P/Zql/B/dp1bRrgyGGMBd2ncXVBo=; b=Cbp1e0QDlBoMFSQUoDU44b/Cu4zloKVkubwfpVG0voArrICkUtyxlHNDLJm7uQaPWs yMtq2C3UOdpl3PwFBZ6fbcJm75BISUhsaIgsCU5qiVlrVsBW+skGS0Karwa520O8DOTP ScA5bFfNO0Evz3c7RIm+D6tZD1CBxKd0YmmyQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687495233; x=1690087233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A39gdokmxlm4U38P/Zql/B/dp1bRrgyGGMBd2ncXVBo=; b=YOhcC/C4bL3dkjLwTN/HxBHP3iYC8TUJKMYNFNleZQZoE0ij+mvPk5TW9EsqauVjVN c25SnukxJ764oT4Dx3/BcVXn9i2CXnV/murWNnF35CvBNyVoMxaS8oi8LbL2XudEYBIf ieBsPBwSIRCpVN+G7qXZjxmtycNk4cTVMHcoKrZA8beZ7uU9xWIVV/wOSZM5k93/3RmY FqwGkMzc2ISpdm2JUA4HxIyePUsitKD4K+HOwo0+kXWXD7SDLbAFc9rv0dVZdVB8nmxk PVr0ekf9t8bYx29EyDpDd6IH1myZWzD9LvsnmlBI8QkbLMqZ1U00G4Af+rx5WQR8xCqZ sEYw== X-Gm-Message-State: AC+VfDwV3sS0tIz+rEnyWsZ1RikC4IAP6ylHjHtc7zmNYq3fTrhf5KKw kgtzW/0rO6BeGS/WodSykPqu6Q== X-Google-Smtp-Source: ACHHUZ40SAmzVB3zyhbJ6Kmva/XRtgJ/rNm6Czlb8EAz6WtSXnsypo+QCnvyyJ2uHG4IQuWlO/8oRA== X-Received: by 2002:a05:6358:edc:b0:12b:c378:b090 with SMTP id 28-20020a0563580edc00b0012bc378b090mr14548296rwh.26.1687495233523; Thu, 22 Jun 2023 21:40:33 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:3383:b451:fa2:1538]) by smtp.gmail.com with ESMTPSA id q25-20020a62ae19000000b00666e17004a4sm4237350pff.58.2023.06.22.21.40.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Jun 2023 21:40:33 -0700 (PDT) From: Sergey Senozhatsky To: Minchan Kim , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky , Alexey Romanov Subject: [PATCH 1/2] zsmalloc: do not scan for allocated objects in empty zspage Date: Fri, 23 Jun 2023 13:40:01 +0900 Message-ID: <20230623044016.366793-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog In-Reply-To: <20230623044016.366793-1-senozhatsky@chromium.org> References: <20230623044016.366793-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" zspage migration can terminate as soon as it moves the last allocated object from the source zspage. Add a simple helper zspage_empty() that tests zspage ->inuse on each migration iteration. Suggested-by: Alexey Romanov Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3f057970504e..5d60eaedc3b7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1147,6 +1147,11 @@ static bool zspage_full(struct size_class *class, st= ruct zspage *zspage) return get_zspage_inuse(zspage) =3D=3D class->objs_per_zspage; } =20 +static bool zspage_empty(struct zspage *zspage) +{ + return get_zspage_inuse(zspage) =3D=3D 0; +} + /** * zs_lookup_class_index() - Returns index of the zsmalloc &size_class * that hold objects of the provided size. @@ -1625,6 +1630,10 @@ static void migrate_zspage(struct zs_pool *pool, str= uct size_class *class, obj_idx++; record_obj(handle, free_obj); obj_free(class->size, used_obj); + + /* Stop if there are no more objects to migrate */ + if (zspage_empty(get_zspage(s_page))) + break; } =20 /* Remember last position in this iteration */ --=20 2.41.0.162.gfafddb0af9-goog From nobody Mon Feb 9 05:43:23 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9C32EB64D7 for ; Fri, 23 Jun 2023 04:40:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230022AbjFWEkq (ORCPT ); Fri, 23 Jun 2023 00:40:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230237AbjFWEkl (ORCPT ); Fri, 23 Jun 2023 00:40:41 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F8932136 for ; Thu, 22 Jun 2023 21:40:40 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-668711086f4so198524b3a.1 for ; Thu, 22 Jun 2023 21:40:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1687495240; x=1690087240; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JXozlQ7AGAAc+Nx/NUdVZ9+4KpO/wevqL//8Wl3bnA8=; b=JZkAmcGHq0XM4nsbfCm1f7kHv9MXYtpNPYSkjpMZEj94i95KEaw66wEhaDEXP14a0u v8UJKbJSz5QSo+cbz1k/Jn3kM0ZCI73rrZrfiPok1GnRO+rKCsXMtzz+f6LH0ISwHvNg PomoU6ex9zABeAcfQIPJ3ly1lyoWkasiGVFMY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687495240; x=1690087240; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JXozlQ7AGAAc+Nx/NUdVZ9+4KpO/wevqL//8Wl3bnA8=; b=LWGS31mHYN1GZsYJA0ZT4U3AJSAUrTOxJJRnazQJ0QKUktCK7wyYJey2aBIvgZZljR nWmrYbKPU6noFzESznC+umR78oQiVs7qfM/lSmdaMwleiIe9e9vGjQZIy0/yYV6UG3BK UKVz+tEEjq6UjWflEFWjQr0nqhIrdGaxSVq/g4UdIGVkFdT/sUROyoJ7Hm8uuOG8DHxV jlNqdb/FVc8rW/i40AfCV0y1ADI0l9ZF66F3A/CwQ1941KmIR8ithjbs2s+8frOXmT18 4GmMJ4Eag8CKxdPbTei9ubR+OP8g7O+Wi+7r6YVSfEQ89MfbvtN5X+oWubn9NfNUcxqN HlIQ== X-Gm-Message-State: AC+VfDw7nhR7ymSTYcKMsrI7dRpFWFcUcIn/ssLbZmKM0tu1Pl6UaAla rPG+IRfAV1czhtijRTN6zDhimQ== X-Google-Smtp-Source: ACHHUZ4n1mZuM7TTU/iqzcSryywffqB9jdQbsWlWm4H5TvuH/7J0J6/AUoSNwQC5ppUUQt0Nu5rzTA== X-Received: by 2002:a05:6a00:1309:b0:64d:42b9:6895 with SMTP id j9-20020a056a00130900b0064d42b96895mr39202386pfu.5.1687495240030; Thu, 22 Jun 2023 21:40:40 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:3383:b451:fa2:1538]) by smtp.gmail.com with ESMTPSA id q25-20020a62ae19000000b00666e17004a4sm4237350pff.58.2023.06.22.21.40.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Jun 2023 21:40:39 -0700 (PDT) From: Sergey Senozhatsky To: Minchan Kim , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH 2/2] zsmalloc: move migration destination zspage inuse check Date: Fri, 23 Jun 2023 13:40:02 +0900 Message-ID: <20230623044016.366793-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog In-Reply-To: <20230623044016.366793-1-senozhatsky@chromium.org> References: <20230623044016.366793-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Destination zspage fullness check need to be done after zs_object_copy() because that's where source and destination zspages fullness change. Checking destination zspage fullness before zs_object_copy() may cause migration to loop through source zspage sub-pages scanning for allocate objects just to find out at the end that the destination zspage is full. Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d60eaedc3b7..4a84f7877669 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1620,10 +1620,6 @@ static void migrate_zspage(struct zs_pool *pool, str= uct size_class *class, continue; } =20 - /* Stop if there is no more space */ - if (zspage_full(class, get_zspage(d_page))) - break; - used_obj =3D handle_to_obj(handle); free_obj =3D obj_malloc(pool, get_zspage(d_page), handle); zs_object_copy(class, free_obj, used_obj); @@ -1631,6 +1627,10 @@ static void migrate_zspage(struct zs_pool *pool, str= uct size_class *class, record_obj(handle, free_obj); obj_free(class->size, used_obj); =20 + /* Stop if there is no more space */ + if (zspage_full(class, get_zspage(d_page))) + break; + /* Stop if there are no more objects to migrate */ if (zspage_empty(get_zspage(s_page))) break; --=20 2.41.0.162.gfafddb0af9-goog