From nobody Wed Dec 24 19:58:51 2025 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1EE5C2D2 for ; Wed, 24 Jan 2024 04:51:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706071880; cv=none; b=Zp0MRqOLplEiOmWNhHR7zCCdWbYv0kpqXeBwmNsjH4p46fgbbwMftRVBVpJ+POSxToXcD+ZmFlkKlUH+tSmUGo+c3SywY0Lg5dXml13Kznf9gc3M8WPpZSc4Qfp3c1Ua2cTDkDaDqka2jSpXq8ryZUI9cfIC1PpHAqbxGqQc36g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706071880; c=relaxed/simple; bh=QQ4iFc7pBwn0Cc5xww2UkCQwKEjS5fJ6B1OQhRJ1HtA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Yb8Y2yIf4bVe3qXN6pd2WxIOI0tP7LPYGUYL9lqaaj3v338RcHOx2LEtzYItBm1TYyfQy8D625+RUEiKFHHIX6mI+TXrZM4zsJWC+cH5JiKaWzFAZBR5z8Ezdx0Z4ehSc+ZfeNaNCLIlY1OQUIlFXjl8g6gumb5Hi/azN8fji0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Obpp6kWR; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Obpp6kWR" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6029c85922dso3813667b3.3 for ; Tue, 23 Jan 2024 20:51:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706071877; x=1706676677; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tZ4zMgobYN5FtUkXrR87lWOCtHQq0Np8Yg1aQ6/7PDQ=; b=Obpp6kWRyAVsKVDvm5xY5j/VKQe0BA64N4XL7zXp79e2RKQ2rZbpLduMKW3EI2xsNZ l6ATDIjUsBsIr7GKfwB+NTrooc11NVeC4jXwXdW/3QTyAOLmNt7AEi5NZ41+6RIwIFIl g5trN/KTNKdH6fqC8LGh7KJDhgbPmTK72b8wPbocNGihdRfIAjeitHGq4YZPKB8g6aw6 NNKWCfT1yffC3Y5xmCkjsJ5xRJNMk703t8vqqsXeqlw+YBkL78Ru7B5QTwqWEmxPOAxH Lc9QnYH9a+GdCsmSQ5I0c5n08IMX2o1v0EdmchMXwU0xx5kqnY/1HAsgbWW1DyG/gVWy fmRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706071877; x=1706676677; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tZ4zMgobYN5FtUkXrR87lWOCtHQq0Np8Yg1aQ6/7PDQ=; b=wriGVbZzqF0asXl7MbN+wow451IiuHIdq4fLzE4CSFgUCLoyEwFOYa5JsU+2YxIQZJ 4HJv3Ob+AbVQtkUukTxk4qlDJQU4qh5zZ/Df5OKKvAxSgdcrRU59xQzQsNFsrD/bN2hF dkUxyrBb1eDTyyScVbFxelR5xUJNCo+VNoWCLHnUcM6gwUWRjMPkPWG4YE7C38h0BkTF c/6m8UVQz7fBu16Sxfq5PTmfSNjTCwo9Plr9ukBUxGzMyDVVjo3ZVCbIDKulyzrd3mMN SPr67wZKSm4vrZIqX3oeErDegI9ANt/CRPke4V1S5FSea4txXTDAHsyToc6RUKn5RUKR FgMg== X-Gm-Message-State: AOJu0YySlzvo5nlWbTg0RX0hyOAy4f5tW/I8ujNBEWjunanUJjBmZ2qY hO/624/f2pRkCt6NS7VJWYTMRBIORvYEo6a8N3t2XWIjpXgr8jLAau3amlp8alThLI16sx4ysB0 TYZInysyXXRsCKeUP5A== X-Google-Smtp-Source: AGHT+IH4dlOAA/aNVlbOqFDsv6qz2rL4n/3QH5h9ZkNH6atvYVWvPw72AEqTtaPY+eysFAbiENcG3HUSpBnk9xox X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a81:9806:0:b0:5fa:4896:7114 with SMTP id p6-20020a819806000000b005fa48967114mr148615ywg.5.1706071877573; Tue, 23 Jan 2024 20:51:17 -0800 (PST) Date: Wed, 24 Jan 2024 04:51:11 +0000 In-Reply-To: <20240124045113.415378-1-yosryahmed@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240124045113.415378-1-yosryahmed@google.com> X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240124045113.415378-2-yosryahmed@google.com> Subject: [PATCH v2 1/2] mm: swap: enforce updating inuse_pages at the end of swap_range_free() From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chris Li , Chengming Zhou , Huang Ying , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In swap_range_free(), we update inuse_pages then do some cleanups (arch invalidation, zswap invalidation, swap cache cleanups, etc). During swapoff, try_to_unuse() checks that inuse_pages is 0 to make sure all swap entries are freed. Make sure we only update inuse_pages after we are done with the cleanups in swap_range_free(), and use the proper memory barriers to enforce it. This makes sure that code following try_to_unuse() can safely assume that swap_range_free() ran for all entries in thr swapfile (e.g. swap cache cleanup, zswap_swapoff()). In practice, this currently isn't a problem because swap_range_free() is called with the swap info lock held, and the swapoff code happens to spin for that after try_to_unuse(). However, this seems fragile and unintentional, so make it more relable and future-proof. This also facilitates a following simplification of zswap_swapoff(). Signed-off-by: Yosry Ahmed Reviewed-by: "Huang, Ying" --- mm/swapfile.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index b11b6057d8b5f..0580bb3e34d77 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -737,8 +737,6 @@ static void swap_range_free(struct swap_info_struct *si= , unsigned long offset, if (was_full && (si->flags & SWP_WRITEOK)) add_to_avail_list(si); } - atomic_long_add(nr_entries, &nr_swap_pages); - WRITE_ONCE(si->inuse_pages, si->inuse_pages - nr_entries); if (si->flags & SWP_BLKDEV) swap_slot_free_notify =3D si->bdev->bd_disk->fops->swap_slot_free_notify; @@ -752,6 +750,14 @@ static void swap_range_free(struct swap_info_struct *s= i, unsigned long offset, offset++; } clear_shadow_from_swap_cache(si->type, begin, end); + + /* + * Make sure that try_to_unuse() observes si->inuse_pages reaching 0 + * only after the above cleanups are done. + */ + smp_wmb(); + atomic_long_add(nr_entries, &nr_swap_pages); + WRITE_ONCE(si->inuse_pages, si->inuse_pages - nr_entries); } =20 static void set_cluster_next(struct swap_info_struct *si, unsigned long ne= xt) @@ -2049,7 +2055,7 @@ static int try_to_unuse(unsigned int type) unsigned int i; =20 if (!READ_ONCE(si->inuse_pages)) - return 0; + goto success; =20 retry: retval =3D shmem_unuse(type); @@ -2130,6 +2136,12 @@ static int try_to_unuse(unsigned int type) return -EINTR; } =20 +success: + /* + * Make sure that further cleanups after try_to_unuse() returns happen + * after swap_range_free() reduces si->inuse_pages to 0. + */ + smp_mb(); return 0; } =20 --=20 2.43.0.429.g432eaa2c6b-goog From nobody Wed Dec 24 19:58:51 2025 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59690DDAF for ; Wed, 24 Jan 2024 04:51:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706071881; cv=none; b=qFWt/9gSvueG22nt8wBueMdQJEkcyAXPSk1Hmivv+fh0Im6X534sNZlCxzeoH+O9N3+fH5tds2bpc22zxbpr+Fy4CE/WcWBghFQ+0XYTVU+6iWk3cxbEmgLvkAWIdK6R9NIQX5CkaRK46ysP7/VRsz5w5lbx8rQaEtJ0ubicSVQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706071881; c=relaxed/simple; bh=oojZa1iMptE5sMnL+2TXHHLFHCC7HlS7H7JRK1lQHt4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ua/EWrNOI8mw5oiYgqTPdVSFyvf92B3za7FORITxwmmqVNuN8f1IPHn1qhH4nG2VhZd9AblpqjYDdzhOW2NIThhqTX8Cl3ImGvl0kVNeFLxA+cglKBPQtsySf9Hff0a6XfXrBsc9ruoKKSMbkQ9az4ukmoT+tNR202wp/WUTBoM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oMSZ9i9z; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oMSZ9i9z" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dc24eb13fecso6293098276.0 for ; Tue, 23 Jan 2024 20:51:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706071879; x=1706676679; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tkuiKBs8Z3LCxzx9sFvb10Ddko8C8l9RPXC0zaKu0yw=; b=oMSZ9i9zYwT1EZ4nJrQgaraSMGz5CuSrQxQDKodNHA/qfvRF4sXHw+SmCzdJY9Yzr3 FLTE3RuF038TPNsaVQUCkkt+Bm0+8g40EIu7EJha9PsXNbjFxtFL2+fNVKDh9WUAqlAM zShT5JQKCE/XWCwXIHW4Fk1KbDeynsbmFzmFFu5uueacIjAkfhQIxHbrcWfuAONpjGrP MkL3BDFtUszSURW9bAnezB8siXGCRrDBQxGDCKtXLzl7KGjm4BoUYCBWwGcLh4LSVD1X 2cvb2KtQmkGXUGVeUkTro4wRt+JArIuEwYM9IylXH9Oo35NZTXWmcCcgBH0eE17uarx2 eZKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706071879; x=1706676679; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tkuiKBs8Z3LCxzx9sFvb10Ddko8C8l9RPXC0zaKu0yw=; b=Uj2iK86aM5c+ISBQnO1NOc5urwVeNevfejS+eHRJIuzglzAiJ+ld1JMrydHiYU0NG6 hvtwaTMgqsd9GeQopP+lWrUpcl3ACsK4drvX5fTG1p4P41kHSF9iLp1zBaYW0TPfilmn tGha5pFyyalofst1aOzaIKaePEv3vr3xUXAM6Vy6FAHvVLtRtU9jymYmmlbQg03OBgP4 OLqqiuteX9dJAQRX0gxtappfzDoh1qR07pzpiY6jDAF9YnGNEba/PgdsctWXwPTZdr/v sjksW5h3BX0owr9Pw/EBEih+gVKpztu0zYyLrLhcAHkNn5nG/CIKbYSgRdnFAWLyDeVt 5DYg== X-Gm-Message-State: AOJu0YyWhbQe39AisBhLt1xa+KdT+AwMrTSBxqS41YZy/WMriFd7aZJd 7PJq5Sfbl6thhnyX4902JQHiIut36MlcBOYeZjRJ6kERAe3ib7xRlY6Zw2VhcBRk4kmYP3GkZ+O hf7xDjDvPW1ZvVRKuhQ== X-Google-Smtp-Source: AGHT+IHUBuTswzf59MXI254TJk24hUrp1Y6H9iQqapqBpe212J3nM0SfBnTsOF5kbITCa8mamWqtFQowkAG0He9g X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:a82:b0:dc3:6b67:9342 with SMTP id cd2-20020a0569020a8200b00dc36b679342mr126270ybb.4.1706071879410; Tue, 23 Jan 2024 20:51:19 -0800 (PST) Date: Wed, 24 Jan 2024 04:51:12 +0000 In-Reply-To: <20240124045113.415378-1-yosryahmed@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240124045113.415378-1-yosryahmed@google.com> X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240124045113.415378-3-yosryahmed@google.com> Subject: [PATCH v2 2/2] mm: zswap: remove unnecessary trees cleanups in zswap_swapoff() From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chris Li , Chengming Zhou , Huang Ying , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" During swapoff, try_to_unuse() makes sure that zswap_invalidate() is called for all swap entries before zswap_swapoff() is called. This means that all zswap entries should already be removed from the tree. Simplify zswap_swapoff() by removing the trees cleanup code, and leave an assertion in its place. Signed-off-by: Yosry Ahmed Reviewed-by: Chengming Zhou --- mm/zswap.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index dcdd5ecfedb09..78df16d307aa8 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1808,19 +1808,9 @@ void zswap_swapoff(int type) if (!trees) return; =20 - for (i =3D 0; i < nr_zswap_trees[type]; i++) { - struct zswap_tree *tree =3D trees + i; - struct zswap_entry *entry, *n; - - /* walk the tree and free everything */ - spin_lock(&tree->lock); - rbtree_postorder_for_each_entry_safe(entry, n, - &tree->rbroot, - rbnode) - zswap_free_entry(entry); - tree->rbroot =3D RB_ROOT; - spin_unlock(&tree->lock); - } + /* try_to_unuse() invalidated all the entries already */ + for (i =3D 0; i < nr_zswap_trees[type]; i++) + WARN_ON_ONCE(!RB_EMPTY_ROOT(&trees[i].rbroot)); =20 kvfree(trees); nr_zswap_trees[type] =3D 0; --=20 2.43.0.429.g432eaa2c6b-goog