From nobody Sun Feb 8 09:11:14 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF7711A0B0C for ; Thu, 19 Dec 2024 21:24:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734643483; cv=none; b=g3Z5++63JbWU5///r36sPHlwCNdkCrrPkajvqhmDrujqKHJvX2ccptEj6PN2FBYa0XyR/xwf0QfzU3Sce+5G6GMnpW3geHX2uEKMw66ywM+IATjm04oe4FjWDWuYM4tJhJNr7XvYVTL7A9o7fMERBhEMBFvthAABlrFRMxCwwys= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734643483; c=relaxed/simple; bh=cefdfB1yGhxUINcQUilu5D19OlaisDnr306U1LVHeRU=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=GgD3Px9cmdk9VKngDJm5/83lGlMm9IpN4FFTxNmabEJtjEAbaULSbTcVoBo586Dnf/dMV9rljbWTM9RFkgZq9Ym28njbesNVmbRLYCvnLeJs6QibbU+MCuWQE8y6Xdv4+d4EUBGKATg6lkBUgMZNcCL8x81gjlZXkpobujZAR0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WiLFYh2n; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yosryahmed.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WiLFYh2n" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7fd4dd6c506so657345a12.1 for ; Thu, 19 Dec 2024 13:24:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1734643480; x=1735248280; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=u7O8GUIoD1UZG/rG8KBOAiEwEk9YL8KfYsJ7/2RZcq0=; b=WiLFYh2nXT8NtrnJfC0+8sBbhttr//Ghmza4c0IRoN1PeHCnnplANERUX1iAG/JLOG TLqQX8/68goFwqACbKPlCKVsbr3a4ev3ZMe8pj8J7KgZhFc/UwnH8sWznGpSG8Vb1ufO 2ux/efkC2vaw1GBb5TOJn0+X0BMBazo0mNU1yv6+alt2Yk4mriFW0MFXHsqJZOhE3nut UmD3hQI87RVbMzN1AaFg+NZe1oqLO6WSKpgfyYpX6UeCCL4Op/2vBlO4r5XgY9f1o0Cd d4Gh2czp9gMYWIQriRhu71tVHeb21qwUp1hPCROIQX/k6WZBmMJZpdupo3nlW5JUpHqp sRfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734643480; x=1735248280; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=u7O8GUIoD1UZG/rG8KBOAiEwEk9YL8KfYsJ7/2RZcq0=; b=OWYmttzlqUuopQFA72RZpeMWNY8IPcC6nYz3OolXfg0P1t/CAPSlA51seRldk66A4P RymzuAJVQt7jZD8wOz5dqiSPpvRojrKl4ZCaBdXoi2KAZfns+RlQZmHZymLwgLjfeXR9 yZU9Ndh+ylGTuQOC4IPGK5ThFr+nJx6iBPk1GsoSXj6l07Qvm7mInzyXIY5eOjJQv4ld 0f6J0ckCU6QLj9DXpCkt8DtyLH0HqKGPZvlLrrnfarfcwki1l/tw6QQW6955J49oLTFY 44KX9fjYKfLkSIeFXEFdk88VsqRd8NFayB+d2pFBio+eAoYtXNZoQ6DJP6emoL+wHg+5 Iw3g== X-Forwarded-Encrypted: i=1; AJvYcCXjTRDpdnXRDQy9fcSFPO54kZU+9yDfZRif28PkWUDpC5PEEh29gvU50t9OgXBQJJgyvml4M5PB2wG2rc0=@vger.kernel.org X-Gm-Message-State: AOJu0YxsnHzUmP8fJIXyghudLDXYYwFEgLEZH6ji+I0sUqNnFJ4UUNA7 hdyJJcEQ8+yxE9dYX+Ov8Cp4kJm96DxhlacTvIghhv9UF+/TE6DlluRzzD0Eyve7L7Esz1I9XXi MPJDy06ADtxpQj8z4FQ== X-Google-Smtp-Source: AGHT+IFnH6UQrf3C3P3rhQlihyUOrX9sGtuPFMiRoQuLliycrZpnMFxS2E+N7rAXPa1MSWZDj7o1ChPfPBNiASrl X-Received: from pgbfu11.prod.google.com ([2002:a05:6a02:4a8b:b0:7fd:5569:7b79]) (user=yosryahmed job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:7487:b0:1e1:ab63:c5ed with SMTP id adf61e73a8af0-1e5e048b22emr887197637.23.1734643480234; Thu, 19 Dec 2024 13:24:40 -0800 (PST) Date: Thu, 19 Dec 2024 21:24:37 +0000 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20241219212437.2714151-1-yosryahmed@google.com> Subject: [PATCH] mm: zswap: fix race between [de]compression and CPU hotunplug From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chengming Zhou , Vitaly Wool , Barry Song , Sam Sun , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed , stable@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In zswap_compress() and zswap_decompress(), the per-CPU acomp_ctx of the current CPU at the beginning of the operation is retrieved and used throughout. However, since neither preemption nor migration are disabled, it is possible that the operation continues on a different CPU. If the original CPU is hotunplugged while the acomp_ctx is still in use, we run into a UAF bug as the resources attached to the acomp_ctx are freed during hotunplug in zswap_cpu_comp_dead(). The problem was introduced in commit 1ec3b5fe6eec ("mm/zswap: move to use crypto_acomp API for hardware acceleration") when the switch to the crypto_acomp API was made. Prior to that, the per-CPU crypto_comp was retrieved using get_cpu_ptr() which disables preemption and makes sure the CPU cannot go away from under us. Preemption cannot be disabled with the crypto_acomp API as a sleepable context is needed. Commit 8ba2f844f050 ("mm/zswap: change per-cpu mutex and buffer to per-acomp_ctx") increased the UAF surface area by making the per-CPU buffers dynamic, adding yet another resource that can be freed from under zswap compression/decompression by CPU hotunplug. There are a few ways to fix this: (a) Add a refcount for acomp_ctx. (b) Disable migration while using the per-CPU acomp_ctx. (c) Disable CPU hotunplug while using the per-CPU acomp_ctx by holding the CPUs read lock. Implement (c) since it's simpler than (a), and (b) involves using migrate_disable() which is apparently undesired (see huge comment in include/linux/preempt.h). Fixes: 1ec3b5fe6eec ("mm/zswap: move to use crypto_acomp API for hardware a= cceleration") Reported-by: Johannes Weiner Closes: https://lore.kernel.org/lkml/20241113213007.GB1564047@cmpxchg.org/ Reported-by: Sam Sun Closes: https://lore.kernel.org/lkml/CAEkJfYMtSdM5HceNsXUDf5haghD5+o2e7Qv4O= curuL4tPg6OaQ@mail.gmail.com/ Cc: Signed-off-by: Yosry Ahmed Acked-by: Barry Song Reviewed-by: Chengming Zhou Reviewed-by: Nhat Pham --- mm/zswap.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index f6316b66fb236..5a27af8d86ea9 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -880,6 +880,18 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struc= t hlist_node *node) return 0; } =20 +/* Prevent CPU hotplug from freeing up the per-CPU acomp_ctx resources */ +static struct crypto_acomp_ctx *acomp_ctx_get_cpu(struct crypto_acomp_ctx = __percpu *acomp_ctx) +{ + cpus_read_lock(); + return raw_cpu_ptr(acomp_ctx); +} + +static void acomp_ctx_put_cpu(void) +{ + cpus_read_unlock(); +} + static bool zswap_compress(struct page *page, struct zswap_entry *entry, struct zswap_pool *pool) { @@ -893,8 +905,7 @@ static bool zswap_compress(struct page *page, struct zs= wap_entry *entry, gfp_t gfp; u8 *dst; =20 - acomp_ctx =3D raw_cpu_ptr(pool->acomp_ctx); - + acomp_ctx =3D acomp_ctx_get_cpu(pool->acomp_ctx); mutex_lock(&acomp_ctx->mutex); =20 dst =3D acomp_ctx->buffer; @@ -950,6 +961,7 @@ static bool zswap_compress(struct page *page, struct zs= wap_entry *entry, zswap_reject_alloc_fail++; =20 mutex_unlock(&acomp_ctx->mutex); + acomp_ctx_put_cpu(); return comp_ret =3D=3D 0 && alloc_ret =3D=3D 0; } =20 @@ -960,7 +972,7 @@ static void zswap_decompress(struct zswap_entry *entry,= struct folio *folio) struct crypto_acomp_ctx *acomp_ctx; u8 *src; =20 - acomp_ctx =3D raw_cpu_ptr(entry->pool->acomp_ctx); + acomp_ctx =3D acomp_ctx_get_cpu(entry->pool->acomp_ctx); mutex_lock(&acomp_ctx->mutex); =20 src =3D zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); @@ -990,6 +1002,7 @@ static void zswap_decompress(struct zswap_entry *entry= , struct folio *folio) =20 if (src !=3D acomp_ctx->buffer) zpool_unmap_handle(zpool, entry->handle); + acomp_ctx_put_cpu(); } =20 /********************************* --=20 2.47.1.613.gc27f4b7a9f-goog