From nobody Mon Feb 9 00:42:00 2026 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AC2914A8E for ; Sat, 13 Apr 2024 02:24:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975054; cv=none; b=XaDXC5Mafx1eR2q74Tn5EAFEgR2ZBnyBO8kewotnUruZrfQU1ty9++rd8X7I1hY/HjMIYMsEXbuRfcGrkx2WBSSni0VmbUBaDqWnZzsWC5I5cmDqQMOoT2lG/fcY9dAg3HUttkXWAsQ3MJ2rr9QgAvM6imZOJ08iWoTBlINVF/I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975054; c=relaxed/simple; bh=uRXFHOd3GYWfX4hZygsGS6tgsg+JAiT7ierWKzgvHk8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sHKpJlhATcuWp3+5QU5M6n6Sx589H4dSurx/kG6fRkzaHtZ7WZCpQw0afnRZcMtRLloQdpLLHy+LhPO2NTm+K4/mg5gSo51BtyG7/AROTgyOceggor9l4mrd37mZtQjmXoHcZJCsZamAmjupYRj2vCZgXW3S29wW4O44dEiiyTc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qWs9ENIN; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qWs9ENIN" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6150e36ca0dso25066737b3.1 for ; Fri, 12 Apr 2024 19:24:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712975051; x=1713579851; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hOPR1bddD//aUuRH48FEXSlgeu+YRgqWmevGjhR2ITY=; b=qWs9ENINeF0m20/cY0Av3i+furL+uW4LBZfsUuzU5fTHeU8GBBcIMq6U0Pd0+hfX9T 5jz2fdahb8MgXbjGeIARouTT2/L7+QevdI+5hMhJIVeom1/tVbJ/8fJFLsYCJk3hEP7Y 2fVuD9uf5FB1uXYj57asstKkn2YBLzoRwzVDK2Ce3AmnfOWz+IuOz8S844mvFEwQ821L Lm/PLZ8ICelAYJF9uuD1aBGGUzIfA7YHGV2mxYP/GakDLT4jnhj1RnB9SzYvMoADOytj 7ScKKQeNmPdVXbYPgpj+Ie82LFEZbNbD5IFJcvgzdbgqnZrW/kmpLNPsbMWNHfQSvs+u bt0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712975051; x=1713579851; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hOPR1bddD//aUuRH48FEXSlgeu+YRgqWmevGjhR2ITY=; b=Jtu9qgLwWhVi38WzllzXA93R+BT2K13lVrgSQhzdKc3/DhQhtztO9IVJc+zChLQwwT e9s6G9lsVeoX1VWqaEdqm1phX8xx9LKQId2uDOCW9fEfCSTAW8OR5gIll9JnSdmflBhr KKWpvO1AWf2b+YJDvrGU8+hESdeVgznoNGIifFHm9gSjQm7YprGpDoFjd4/Zn/4KdQgU uU2H9QUX0L2zTeA8fezfs73c4V1/0DvDMpI6w7dseN9yGCNy04e+XGF3ekj8cLP3pdro B5azK+Ske/j04mPw6Wcub9GvvIEf0IEHDFu3yjopvFdAT7DR2Qk2DX5FTlK16vMeDlSt mGFg== X-Forwarded-Encrypted: i=1; AJvYcCXQdSixhY9lLEYJXzos5QK03Gq/3aRkvUR2mb7vcPJTe5QN9st3jJEoz33pXjG/tAfLO9Wkpkfjga3ZjdpC49v1Z/Z6O6qWHElH2hpB X-Gm-Message-State: AOJu0YzW0/PDdKZV14LFJurqAq+LzXzg1IhOEJNXpsUjUnVZzJ3GXVnh zP/d1BZDPeTvl8UfTNbWVSvSAAi73q4bBEgHjbjFyMyHTIJolBk+FmXB7xDo1X6403UACxL/xul UujzSS08sQyQ/WEGrpg== X-Google-Smtp-Source: AGHT+IEIyih7J9L34SbhPW1Fs19b+bJp1g0bk94AkYr3vPB7zcxk8Z47Mpof1dxsaJMt8h1S+jqO5WIy+IoUE+4+ X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:690c:dc3:b0:610:f5e6:375a with SMTP id db3-20020a05690c0dc300b00610f5e6375amr996613ywb.9.1712975051525; Fri, 12 Apr 2024 19:24:11 -0700 (PDT) Date: Sat, 13 Apr 2024 02:24:04 +0000 In-Reply-To: <20240413022407.785696-1-yosryahmed@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240413022407.785696-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240413022407.785696-2-yosryahmed@google.com> Subject: [PATCH v3 1/4] mm: zswap: always shrink in zswap_store() if zswap_pool_reached_full From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The cleanup code in zswap_store() is not pretty, particularly the 'shrink' label at the bottom that ends up jumping between cleanup labels. Instead of having a dedicated label to shrink the pool, just use zswap_pool_reached_full directly to figure out if the pool needs shrinking. zswap_pool_reached_full should be true if and only if the pool needs shrinking. The only caveat is that the value of zswap_pool_reached_full may be changed by concurrent zswap_store() calls between checking the limit and testing zswap_pool_reached_full in the cleanup code. This is fine because: - If zswap_pool_reached_full was true during limit checking then became false during the cleanup code, then someone else already took care of shrinking the pool and there is no need to queue the worker. That would be a good change. - If zswap_pool_reached_full was false during limit checking then became true during the cleanup code, then someone else hit the limit meanwhile. In this case, both threads will try to queue the worker, but it never gets queued more than once anyway. Also, calling queue_work() multiple times when the limit is hit could already happen today, so this isn't a significant change in any way. Signed-off-by: Yosry Ahmed Reviewed-by: Nhat Pham Reviewed-by: Chengming Zhou Acked-by: Johannes Weiner --- mm/zswap.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 741957f36f38b..77b6bb2099763 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1429,12 +1429,12 @@ bool zswap_store(struct folio *folio) if (cur_pages >=3D max_pages) { zswap_pool_limit_hit++; zswap_pool_reached_full =3D true; - goto shrink; + goto reject; } =20 if (zswap_pool_reached_full) { if (cur_pages > zswap_accept_thr_pages()) - goto shrink; + goto reject; else zswap_pool_reached_full =3D false; } @@ -1540,6 +1540,8 @@ bool zswap_store(struct folio *folio) zswap_entry_cache_free(entry); reject: obj_cgroup_put(objcg); + if (zswap_pool_reached_full) + queue_work(shrink_wq, &zswap_shrink_work); check_old: /* * If the zswap store fails or zswap is disabled, we must invalidate the @@ -1550,10 +1552,6 @@ bool zswap_store(struct folio *folio) if (entry) zswap_entry_free(entry); return false; - -shrink: - queue_work(shrink_wq, &zswap_shrink_work); - goto reject; } =20 bool zswap_load(struct folio *folio) --=20 2.44.0.683.g7961c838ac-goog From nobody Mon Feb 9 00:42:00 2026 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0152F17C6B for ; Sat, 13 Apr 2024 02:24:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975055; cv=none; b=iUB2B8j1qxqfRx8P8mjZaJRa2s5V4LWENBb9MbmwUt5nLY3RReo0H8WieCaERyA+ugcFGznUj0R0/gudlOIpVtojwM0VZUqGP+F2au2tMkagVzkBrOHioUb+vileyL7/yH2vL1K6o829wno7i9rBHmi6efEFcu6b0UyOJr5HnqY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975055; c=relaxed/simple; bh=3qZROuIu+1JIpszYBziqVDLibz3qxq/ZHL9aJhpHmNo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZOrLTpi+1DbbLAt551kNpVFV2OF/f9FOvbLICUJWhnN8h5I+H3YmTc5EZHcCCRur56T5dEQxtj3F33S5wD4aGCYDcolVjcFXKbOUJ9xprDALtlIhjOJzCW/1Oex+Xl3cN/DhrAOKjf8LHoMgJmgqZQkHuUO2iwlvnepPlUUXzG4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=t/t/P7aq; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="t/t/P7aq" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6150dcdf83fso30140967b3.2 for ; Fri, 12 Apr 2024 19:24:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712975053; x=1713579853; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pFcpj0TvebZcgya6byXyrm5KOadn+Q+ON/Bz5Cqkl5I=; b=t/t/P7aqZq5rTkxe8hYT5gfvR+tUrxppwvfZVbHHj6OxJlmJDzGUAWwvzlq8xa0zBU hk5gClJzfLawiwSsP5vuqrJ2YB9DlyPb57VuOTauEVV/RvXR7ewP4SinR0KpmRASDW4k LxGhatOAQ3XvWgjCHnb1AwWiEZlELdtRpMPg97LvvEqTPfOMNRGcr5JVtTiS7ss9DR4V HKTDk+zoq/wga7M2DMDwII1M+bNESN2B/vYYUGQ/cj+9PK/UkbzIjEnoeQ24g0fIqJIM cIf2cq0aQVCmmwBf9a0Gr4Ic5wGzTreVEPMXpqtptKu3NSnN6WgUSNuAK4AKyCTC9A7b l47A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712975053; x=1713579853; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pFcpj0TvebZcgya6byXyrm5KOadn+Q+ON/Bz5Cqkl5I=; b=U5vp7T949toYYFyZdroaRaeZLKKK/ant5bcHdqSETRkBdOUj5Az/+SRluHh1dUEgRQ 64lY4QLaIawWNC9QTK+js0ON23Zzms+UaTKP5kvuZM8w7bregfrIwkuOuqyBOGl8uuvW hVO7V19/79JiP7J2Eepih0ZPvOhaoDY5RSNPlqM75H991kale4qH5V/qqJ2F9II1TRKJ GvSEhlsuxvUIaW6QmsSHq10yMNKJs1YVDGT2XFww87AAuaWcPLhiz5jaZ/JjogAdBKzr iWDdIu2JfItXtJI0rH0OA47oftpd96SvE8aYZ8xdvv7BPTj7u19TfRGNlpR25GdU+IFl 5xRQ== X-Forwarded-Encrypted: i=1; AJvYcCVH8ZJGrTTT0CWus03MP2YZT+N2r+UGk4gKH4O0ZxjwONu8Zn6YSeqoZZ7MSyWYqpWD8q4OvxtmfuCivkSLR7qOLyVSP3Z4seJj/eQS X-Gm-Message-State: AOJu0YxMvgzSAPl/lf3TpLPjIL3UHhso4JJDcy0AdEdqy84RDwi5nAY6 8cKy+faK7F3QbPZLb2u3Q0mpxEZ/Los6SEeRCBEEYonH4N40Za812LHeMF7CNu5wJng+9/Maxjw uZG6tzuYNEBzowWQr2w== X-Google-Smtp-Source: AGHT+IHAlbeOydfvBUQ+AYR1qIuBuBzl/BOKahweaVpfLg8kz58GaVMgBySWi8xxEMGZJABW5nfoPZGjO8NmCCMB X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a0d:e209:0:b0:618:80c0:1a2e with SMTP id l9-20020a0de209000000b0061880c01a2emr834009ywe.6.1712975053165; Fri, 12 Apr 2024 19:24:13 -0700 (PDT) Date: Sat, 13 Apr 2024 02:24:05 +0000 In-Reply-To: <20240413022407.785696-1-yosryahmed@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240413022407.785696-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240413022407.785696-3-yosryahmed@google.com> Subject: [PATCH v3 2/4] mm: zswap: refactor limit checking from zswap_store() From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Refactor limit and acceptance threshold checking outside of zswap_store(). This code will be moved around in a following patch, so it would be cleaner to move a function call around. Signed-off-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 77b6bb2099763..7cddbca3ac62c 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -517,6 +517,21 @@ unsigned long zswap_total_pages(void) return total; } =20 +static bool zswap_check_limits(void) +{ + unsigned long cur_pages =3D zswap_total_pages(); + unsigned long max_pages =3D zswap_max_pages(); + + if (cur_pages >=3D max_pages) { + zswap_pool_limit_hit++; + zswap_pool_reached_full =3D true; + } else if (zswap_pool_reached_full && + cur_pages <=3D zswap_accept_thr_pages()) { + zswap_pool_reached_full =3D false; + } + return zswap_pool_reached_full; +} + /********************************* * param callbacks **********************************/ @@ -1399,7 +1414,6 @@ bool zswap_store(struct folio *folio) struct zswap_entry *entry, *old; struct obj_cgroup *objcg =3D NULL; struct mem_cgroup *memcg =3D NULL; - unsigned long max_pages, cur_pages; =20 VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1422,22 +1436,8 @@ bool zswap_store(struct folio *folio) mem_cgroup_put(memcg); } =20 - /* Check global limits */ - cur_pages =3D zswap_total_pages(); - max_pages =3D zswap_max_pages(); - - if (cur_pages >=3D max_pages) { - zswap_pool_limit_hit++; - zswap_pool_reached_full =3D true; + if (zswap_check_limits()) goto reject; - } - - if (zswap_pool_reached_full) { - if (cur_pages > zswap_accept_thr_pages()) - goto reject; - else - zswap_pool_reached_full =3D false; - } =20 /* allocate entry */ entry =3D zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(folio)); --=20 2.44.0.683.g7961c838ac-goog From nobody Mon Feb 9 00:42:00 2026 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEB1518EB1 for ; Sat, 13 Apr 2024 02:24:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975057; cv=none; b=qFFUSb8bYDpXMYc5I0ajIe70xCrVNGzF6LC8y5vm1yZOKCwk22eG8WR8Zg69ycvX6AXagq9+/3DJr2YDHnurx0s+bVQdYoj0VoIsofH7y7ZDFNyuBiPNb4sgt+pkDFPBiHmbPInSiICRwJLxxEPCd8jp+JOxgSxNjSklY/lU+a4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975057; c=relaxed/simple; bh=qDusTmxjn2kC5lBhg/CWFsj5+co4gZBN2F/Gz2WVfPs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XLP9nZViQmQHtBFuh3T4UMHqEq79uiaPfipRieQS+h3f7cr//aFNwYuUqZoj51J8fnRYpjIIdLaAkpHi8jEdYeOlT8/8rxvfshHTZ6ph79r43X+rsL9IHlJT0RWfKaox9tC1M6yzR3MJNyUeZe2UH5r9+sb1U/Uc3VG4I7GaUJQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lO+GtrUe; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lO+GtrUe" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-618946f8e7bso4078637b3.1 for ; Fri, 12 Apr 2024 19:24:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712975055; x=1713579855; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hZYUWNTiUhiZC6PVJ2X+TVQi5b4PqWXfNQ2Zo+Rl5yk=; b=lO+GtrUeBFRPHuUAJ8VCq1XAfh3kf1TzzlhC/2pa3kMDsIRCYd9jVOsZAPNMNea+Qi kIdijuN6ADfeGjpMRJmNhb/eoU+yA7OjiRZkePjW+l8pMsDlYtstjClXYuJ133LKM3XB fzlOvA/YkoIS9y6gvNXoSIeaesFHCC+MON45xYXA8lyW79UDnDYPbB+43Tggdzjzu8eX k8hgdhkVaj1e+q0CB32GChdXFS06hxLFcFI75SP7zytWcej5jtDAQHdp+K0EgTHrBQ1E 0toUSandXUodSZ6EhQX4SfkvMJP0xV21l0wxh4ofXClzmanpPAhVo0HJ4Ms8IapTDtzA 7W4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712975055; x=1713579855; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hZYUWNTiUhiZC6PVJ2X+TVQi5b4PqWXfNQ2Zo+Rl5yk=; b=lAM9X9YCtPY5mtDkrthgVic3WXTFlYUXV7GwhbTON90mHzc/zXIP2rDhzNFlAnB2Aj ZwjuMj1QsWyZ3GyzJFx8aKpUlwUq8kD7D9Lml3/EuZ+s4zLHZcfyk9MEIOVR3IIINgU0 k7WZ7sxfOeAcexRDv5o1VDnMNwEW5KnzfKFK/AS9Pav8zpuHty9aCMhG7S1QOPyNdPcH SDgkmsaYR72W3zygJii+xL2vML66pds7VHkVV1UOxm1bznoU8hlUbWGLBNWcPwmEXQIs plWABrs1w45/gRehvMVZ1Pl4tdj0u/EM9vv6RP1cJqwyH5LQj+SSsXhs17ZUWZFvgtvT KU0w== X-Forwarded-Encrypted: i=1; AJvYcCXIzDqPXhgmb8obWBYRfBG9korzgZlj9LWiuR5I1iKXMPfWgDSi4RAOdIudc5XGX7HVRF88tE5d/TIrlGvLJVZVq2XcaqXjg5LNuNEl X-Gm-Message-State: AOJu0Yy9TOk35o5ssNNMKfy3PpSkd1pyBnksD9KITNxJmU/LZ+/4d0vY WV/cBcKVfPqiaWh/gcTTTJNLQOkaFuEogrlZc2swvPJ9CMCWk4C/AtvHSqsP4i/5G/t+5BVfOfG 26ES9g84jk72I9RTEgw== X-Google-Smtp-Source: AGHT+IGTrx1GnvC7Seu3eWAOb4tBFHfkiqWBD9kd67Bwq3v4nuuqOaFgb15AT9eRdbOLIz4wHqs3/FUeUlx91LlG X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:1243:b0:dbd:ee44:8908 with SMTP id t3-20020a056902124300b00dbdee448908mr1442080ybu.0.1712975054816; Fri, 12 Apr 2024 19:24:14 -0700 (PDT) Date: Sat, 13 Apr 2024 02:24:06 +0000 In-Reply-To: <20240413022407.785696-1-yosryahmed@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240413022407.785696-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240413022407.785696-4-yosryahmed@google.com> Subject: [PATCH v3 3/4] mm: zswap: move more same-filled pages checks outside of zswap_store() From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, zswap_store() checks zswap_same_filled_pages_enabled, kmaps the folio, then calls zswap_is_page_same_filled() to check the folio contents. Move this logic into zswap_is_page_same_filled() as well (and rename it to use 'folio' while we are at it). This makes zswap_store() cleaner, and makes following changes to that logic contained within the helper. While we are at it: - Rename the insert_entry label to store_entry to match xa_store(). - Add comment headers for same-filled functions and the main API functions (load, store, invalidate, swapon, swapoff). No functional change intended. Signed-off-by: Yosry Ahmed Reviewed-by: Nhat Pham Reviewed-by: Chengming Zhou Acked-by: Johannes Weiner --- mm/zswap.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 7cddbca3ac62c..f1d3204c604bd 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1376,26 +1376,35 @@ static void shrink_worker(struct work_struct *w) } while (zswap_total_pages() > thr); } =20 -static int zswap_is_page_same_filled(void *ptr, unsigned long *value) +/********************************* +* same-filled functions +**********************************/ +static bool zswap_is_folio_same_filled(struct folio *folio, unsigned long = *value) { unsigned long *page; unsigned long val; unsigned int pos, last_pos =3D PAGE_SIZE / sizeof(*page) - 1; + bool ret =3D false; =20 - page =3D (unsigned long *)ptr; + if (!zswap_same_filled_pages_enabled) + return false; + + page =3D kmap_local_folio(folio, 0); val =3D page[0]; =20 if (val !=3D page[last_pos]) - return 0; + goto out; =20 for (pos =3D 1; pos < last_pos; pos++) { if (val !=3D page[pos]) - return 0; + goto out; } =20 *value =3D val; - - return 1; + ret =3D true; +out: + kunmap_local(page); + return ret; } =20 static void zswap_fill_page(void *ptr, unsigned long value) @@ -1406,6 +1415,9 @@ static void zswap_fill_page(void *ptr, unsigned long = value) memset_l(page, value, PAGE_SIZE / sizeof(unsigned long)); } =20 +/********************************* +* main API +**********************************/ bool zswap_store(struct folio *folio) { swp_entry_t swp =3D folio->swap; @@ -1414,6 +1426,7 @@ bool zswap_store(struct folio *folio) struct zswap_entry *entry, *old; struct obj_cgroup *objcg =3D NULL; struct mem_cgroup *memcg =3D NULL; + unsigned long value; =20 VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1446,19 +1459,11 @@ bool zswap_store(struct folio *folio) goto reject; } =20 - if (zswap_same_filled_pages_enabled) { - unsigned long value; - u8 *src; - - src =3D kmap_local_folio(folio, 0); - if (zswap_is_page_same_filled(src, &value)) { - kunmap_local(src); - entry->length =3D 0; - entry->value =3D value; - atomic_inc(&zswap_same_filled_pages); - goto insert_entry; - } - kunmap_local(src); + if (zswap_is_folio_same_filled(folio, &value)) { + entry->length =3D 0; + entry->value =3D value; + atomic_inc(&zswap_same_filled_pages); + goto store_entry; } =20 if (!zswap_non_same_filled_pages_enabled) @@ -1481,7 +1486,7 @@ bool zswap_store(struct folio *folio) if (!zswap_compress(folio, entry)) goto put_pool; =20 -insert_entry: +store_entry: entry->swpentry =3D swp; entry->objcg =3D objcg; =20 --=20 2.44.0.683.g7961c838ac-goog From nobody Mon Feb 9 00:42:00 2026 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 232221B978 for ; Sat, 13 Apr 2024 02:24:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975058; cv=none; b=VSplSDEyiL3l+prAYhXgIgxPHSp5EiisY/vslD12P6FtItCBczC55ECMqzHo/U+clhInhD6tyrC8J5pr3/ZzwPr+LgEzYzgkOkXFc1WixQSzTZBxFtSAAr34Mm+wV9BvwEF5oZdh4YCOPdN/owrOPww1X9i5T5paoxXe3nXYZYQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712975058; c=relaxed/simple; bh=zyDgBGE0uOh7yhHXUN8OqLzqFOje2WXWPFCwIO8VvvQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dO5MryBcCCe20v3S65qoaLAqK1/14x0Z9Ku0Y55I2SEXtI3JuYJa4cWfUqQ97R8cK/df4PLvYN0yJl1z0gtEZVyPhDQKLaxaYPncsnLJk6fcOwj6EALIjtC88RCsjr+w2aP361HG6n6qZgFM9e3lGWBxczkbSOfmuczbVIdsgzg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=d4pxaDSq; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="d4pxaDSq" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-617bd0cf61fso28196047b3.3 for ; Fri, 12 Apr 2024 19:24:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712975056; x=1713579856; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GGSekAACS+5O/41JM9vPAXBnj4QEps+gftm74rD23xg=; b=d4pxaDSqBGDGO9wHMrgP4boezbPfC5OKND7tT9B3V9YzKCl/mT+fRPvw7r9dkSIGGt ZOvvSgT/M9BvI9AAsbPm9uZCFGL90zcdHQWfvWVLFhTrVGZOz8JYCaUu6XtOsoZBaqOU 5tNJv6xJPXJoY2yeriYx9gg2rKBiIXdcIDxgWjavTjWENSg7dTl39yLkVqVLkQUDo9Lr qSBvAhstyV56k+4GMJeiFJ80Jjdmmi1vol80thOpC0VjQmUUfreala1wlsTiB0P8YHM8 7DUtTU5uEM9uKjWIGvUUj+q0ZXeC+cR3PQm0sqN9mffRIPNEQ3pDSnzfGinJsZ3jbu+D XsIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712975056; x=1713579856; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GGSekAACS+5O/41JM9vPAXBnj4QEps+gftm74rD23xg=; b=KycZpQ6owxkavxak6+Sd7/smwx+XAwB4+cxej9EZoIpTjM8L7+uDejGlo9MZwqxFDO P7M3l9qfe77jg13cz+ZC6ssxmo8NyN7m1NEROEeYfARoAwubK1jRb4v6L1Q+2KHoHwhf s5h/hM7/bxODFw+0OU9EHk3akkMJkknOgu58kJepuH6KWe2S8UKGoNUUaiLn72NN8wDB 1x+JLyFf3XTaJMqiS4qFvfmczdUHVS/5/Kuxth8GHlcJxzFsg1AmQfp1R9m8zLBvFZBw 2mwZFjnLqsDmRg/Rr/6CdeXyGxyJwkvX+wHhpUhSSAOxu5WLNvrN8pV4bpAEgNKhM8CG 7X0g== X-Forwarded-Encrypted: i=1; AJvYcCXkEmymhrPZeNB2HKBboQya38vc2QBSTRkhWg534SoGpKlKyeSkuLdACh1XALKyWfU8XUq8wvPs+1kn/Ws+Jfi1SKJ8HEsvBXJKs1uo X-Gm-Message-State: AOJu0YwOoU+l3Ub6FD0tqvZyhcUEm7Bq7qnsa3Wf57TtK6YDv2K/MYhk bchSsjPh0Y+diWJV7gg+sL4VvyT0XV8wtlEsKbwvOH73MY1DF71dx5a0P9iobMQUHO8IDkJMiWJ Ha+dEnGqbgbx9FEflhg== X-Google-Smtp-Source: AGHT+IFlkK/1syKwe6dvTycUcSxBT93UngoIpgQpJlcuebA8rvVZJNpQ+d8xJDsJnJOws2HVuhkwwg+6UgtMptz/ X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a81:4fcd:0:b0:618:a530:10aa with SMTP id d196-20020a814fcd000000b00618a53010aamr174476ywb.8.1712975056398; Fri, 12 Apr 2024 19:24:16 -0700 (PDT) Date: Sat, 13 Apr 2024 02:24:07 +0000 In-Reply-To: <20240413022407.785696-1-yosryahmed@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240413022407.785696-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240413022407.785696-5-yosryahmed@google.com> Subject: [PATCH v3 4/4] mm: zswap: remove same_filled module params From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed , "Maciej S. Szmigiero" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" These knobs offer more fine-grained control to userspace than needed and directly expose/influence kernel implementation; remove them. For disabling same_filled handling, there is no logical reason to refuse storing same-filled pages more efficiently and opt for compression. Scanning pages for patterns may be an argument, but the page contents will be read into the CPU cache anyway during compression. Also, removing the same_filled handling code does not move the needle significantly in terms of performance anyway [1]. For disabling non_same_filled handling, it was added when the compressed pages in zswap were not being properly charged to memcgs, as workloads could escape the accounting with compression [2]. This is no longer the case after commit f4840ccfca25 ("zswap: memcg accounting"), and using zswap without compression does not make much sense. [1]https://lore.kernel.org/lkml/CAJD7tkaySFP2hBQw4pnZHJJwe3bMdjJ1t9VC2VJd= =3Dkhn1_TXvA@mail.gmail.com/ [2]https://lore.kernel.org/lkml/19d5cdee-2868-41bd-83d5-6da75d72e940@maciej= .szmigiero.name/ Cc: "Maciej S. Szmigiero" Signed-off-by: Yosry Ahmed Acked-by: Johannes Weiner Reviewed-by: Nhat Pham Reviewed-by: Chengming Zhou --- mm/zswap.c | 19 ------------------- 1 file changed, 19 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index f1d3204c604bd..243a7c202c6c7 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -123,19 +123,6 @@ static unsigned int zswap_accept_thr_percent =3D 90; /= * of max pool size */ module_param_named(accept_threshold_percent, zswap_accept_thr_percent, uint, 0644); =20 -/* - * Enable/disable handling same-value filled pages (enabled by default). - * If disabled every page is considered non-same-value filled. - */ -static bool zswap_same_filled_pages_enabled =3D true; -module_param_named(same_filled_pages_enabled, zswap_same_filled_pages_enab= led, - bool, 0644); - -/* Enable/disable handling non-same-value filled pages (enabled by default= ) */ -static bool zswap_non_same_filled_pages_enabled =3D true; -module_param_named(non_same_filled_pages_enabled, zswap_non_same_filled_pa= ges_enabled, - bool, 0644); - /* Number of zpools in zswap_pool (empirically determined for scalability)= */ #define ZSWAP_NR_ZPOOLS 32 =20 @@ -1386,9 +1373,6 @@ static bool zswap_is_folio_same_filled(struct folio *= folio, unsigned long *value unsigned int pos, last_pos =3D PAGE_SIZE / sizeof(*page) - 1; bool ret =3D false; =20 - if (!zswap_same_filled_pages_enabled) - return false; - page =3D kmap_local_folio(folio, 0); val =3D page[0]; =20 @@ -1466,9 +1450,6 @@ bool zswap_store(struct folio *folio) goto store_entry; } =20 - if (!zswap_non_same_filled_pages_enabled) - goto freepage; - /* if entry is successfully added, it keeps the reference */ entry->pool =3D zswap_pool_current_get(); if (!entry->pool) --=20 2.44.0.683.g7961c838ac-goog