From nobody Sun Dec 14 13:58:36 2025 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 155C019A for ; Wed, 18 Dec 2024 06:35:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503725; cv=none; b=bgrEciNWxfhQR07hE/IXq38RjeDf325Z+FxLxtlf9mgIT/11yNinRs6c7V5XqCcqc751N5T4EsOowcNtA32DiyugpakBNjLcH7ZU9R/+06p5aha4PolZaPja0oJSm5jKWEmZzCxdKTmNCCkPnBSfbNdVjynXzEN0cyLMvalfuRY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503725; c=relaxed/simple; bh=r9j0bsoPlow28/GLcWExnZNZ5xalbO9yNQzFgmUdZRg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=elyMEzdVN6Vdyx5LXhSNrgSrzxLDwDg0A7tG2/tuzCxtqEW84fXYets6QSjdbPELJZJiPk+gMGCFWFbZCyvUTNQD1rkQznKYV85LMbnXqouVPh92l9HToNMwqRONR/2/j3CLcBA4EpJJl5x61Lykszsk/YRGCS82HpcFBaEB3yA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=mB1d9gHj; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="mB1d9gHj" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2166651f752so64466305ad.3 for ; Tue, 17 Dec 2024 22:35:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503723; x=1735108523; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YM5tbrHoFcnr7Yi5hBwjzOD0QAW+fJr+gF9sDLsbf7Q=; b=mB1d9gHj6a3uALzTY0i16gB6h25Tzaw6pH2HM2xWxsbY/uG6RJD1namQ5SfZCCxp+P HKtIDVQcwNoJvXWfHoh08G2hAYvYnv1faspbReMxbboYiZeNjAMTDf9h5xWR6ImKurwY zSkHuMM131r+awQ4H4grYNcL6fjlJb8LoCYU8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503723; x=1735108523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YM5tbrHoFcnr7Yi5hBwjzOD0QAW+fJr+gF9sDLsbf7Q=; b=hgkJta32HsSy0o1rbQk0uT0uMOiD0dtD1xUdWt8Ee1s22p5vjIfT1pZY4LRbFPJNZQ HKgsbDt5lF9UbSY78fsOSyE1vw5GIdSHlITsHWbTBeXLaFCHonC24zFrKaTH/JsnMsef g5zoEz5bwH4XJR8HONt4d4BPl13iFPdXlKeBHFGRSA4XK2ZtNru72GQJUZZjzDa2i75P 5Axd78rb1xFZhStIEChYbiwLLo/HIxc1aF3q5CoKy9ccq0gH3UPQiiEaOSVuJ/3fGjML 4vw/0VP+fh6mvMSX0YqlEKQ+9/6CDCJxsCp2jT8Yrf0eTq9LWISkyt8kux7EUsrDRUcA BGHQ== X-Forwarded-Encrypted: i=1; AJvYcCXJe7p89lLESm1AKU88r2yjSCS84UEJLlK15adnRqZwOsSaI0HFK97Bo3q+DRP0WDgv+VtBLrfLs+uX+Ic=@vger.kernel.org X-Gm-Message-State: AOJu0Yz1VrrYN0hXIskUG+s2sNrvFwCdswjYrX55+vxIrTJDutx/mpLS dV7wNhkfPh7cNIJXhOwcDSoEv7+LkLzxvVu0387xutpOffEMiShrfPK5s85OAg== X-Gm-Gg: ASbGncvD5+AWUpsp+BR4BNF2u+Ujd99EI1FPZXzkrG1Z4VIJ/Ib2bqcd2iKSaIv/5MT aGMIpIbOQihrkByYHfkD8OeiaDbRn5ltzIrAZeTgLqKNIC8HOXwHMqwYGZ7MdnBqDp2XgeOWI/I vunOnGE84i3CJtT1xMl2zCjT/AhVM0WFnDD48+rGDaWXh5ruWK0Vr0i+0P6frxVuktybG7qKFVn 989KeWNsc1UonOxg7LT0NsLmLLX5fSLs1BC6zsMNmHaLjd+oCX82ZHas2U= X-Google-Smtp-Source: AGHT+IEzRX0s0ipw+MkmY6rxb4hZkmahH0+HA2BPVeSSTzwnMahAGsE7gUMp3e22K+tr163jhvT0PQ== X-Received: by 2002:a17:902:dac6:b0:216:7cd8:e964 with SMTP id d9443c01a7336-218d710417emr25045545ad.22.1734503723387; Tue, 17 Dec 2024 22:35:23 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-218a1dcb8e9sm69394115ad.83.2024.12.17.22.35.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:23 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 1/7] zram: free slot memory early during write Date: Wed, 18 Dec 2024 15:34:18 +0900 Message-ID: <20241218063513.297475-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the current implementation entry's previously allocated memory is released in the very last moment, when we already have allocated a new memory for new data. This, basically, temporarily increases memory usage for no good reason. For example, consider the case when both old (stale) and new entry data are incompressible so such entry will temporarily use two physical pages - one for stale (old) data and one for new data. We can release old memory as soon as we get a write request for entry. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 45df5eeabc5e..cda0f3fd4058 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1648,6 +1648,11 @@ static int zram_write_page(struct zram *zram, struct= page *page, u32 index) unsigned long element =3D 0; enum zram_pageflags flags =3D 0; =20 + /* First, free memory allocated to this slot (if any) */ + zram_slot_lock(zram, index); + zram_free_page(zram, index); + zram_slot_unlock(zram, index); + mem =3D kmap_local_page(page); if (page_same_filled(mem, &element)) { kunmap_local(mem); @@ -1736,13 +1741,7 @@ static int zram_write_page(struct zram *zram, struct= page *page, u32 index) zs_unmap_object(zram->mem_pool, handle); atomic64_add(comp_len, &zram->stats.compr_data_size); out: - /* - * Free memory associated with this sector - * before overwriting unused sectors. - */ zram_slot_lock(zram, index); - zram_free_page(zram, index); - if (comp_len =3D=3D PAGE_SIZE) { zram_set_flag(zram, index, ZRAM_HUGE); atomic64_inc(&zram->stats.huge_pages); --=20 2.47.1.613.gc27f4b7a9f-goog From nobody Sun Dec 14 13:58:36 2025 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA90D19A for ; Wed, 18 Dec 2024 06:35:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503730; cv=none; b=Z5njFyCaWRqw/A2kF045Zmgt5wltOkilgyG7ambMDxfeFNSZLbu+ir1I08pnTavXAfZEMf+Xv57jx80VImt9O+y+Hcz6qjCDEux/2ZnX/MVy83mR6T0Ae6aOkkEEA7IBaEcFADYlkTqHFHpRgHCSJ3jsOkkGnmzXNMh+4TsCacI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503730; c=relaxed/simple; bh=77bMsso8gaWOdNVm5cZRZYObNUW9NcbFG8cE4uuZ2n4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lA9/CYgIf0rTyZNKtP5iUvEYe3/scbeBN2kP5C4iduRts6uUW8pLjZ79FCdEdwAYmXXimXzgOp8BHEHn6qZ0zbP1tIrY+qAGWPCW0D3GyTkZgXAswZlp8Vl/flSR7iYrLaKwdPzIi1t7HCacuQyI460AbUP4fR/Wv7mYRJk3p8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=B5l7OmED; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="B5l7OmED" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2166651f752so64466625ad.3 for ; Tue, 17 Dec 2024 22:35:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503727; x=1735108527; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1lsNLWZe1xx8LIy2DkJ6EwI7DqhKpPjMJ1gyizR1Gbo=; b=B5l7OmEDFNFbIwOUGzGJrzx0UEdwJi0sIqRJ27j8xa3Y0gGlD2454otIMOx4DzVWHf pdhFkHfjXYEq9GoeJsXc4Ul2TBvzlRpF/dp1PWPcXU7f+OjOt40asRaSkfAw2FPKTyZb dqDb2BjJRSy2HCgYIaq8KbWFsZWItt+zNa75o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503727; x=1735108527; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1lsNLWZe1xx8LIy2DkJ6EwI7DqhKpPjMJ1gyizR1Gbo=; b=cVjyoZD91lBKOxBh9kKtjEyxOrc5qJCTtDmLbE2TBRhAb1hEUVs2RAh2oXRS1Cet25 CUuB6v4+TO6p7I3KZfC++YkP+6CYcjNTbO9CggkCD+D3AUBTQGaEif6KnQR71Zni5DxX Vu6A27nWHqH3FrHDxI6ZUEOAM5eoH6Fepk4tYIFhwggtI7wN4SapMpNjw1cy6nyDxvCP 0R7mnYgz00z5GQL5kQMYgZjXard2cXmjq4eBRwFLjG6ZdrIXG94d2SajU16raFr28tJj FOH7O4SMcyhjVMu7QK0IRZsDKqkiPiGjnq6MbEZE3RvJAMeSP0WagRbq2YWVywUJsEMe v4JA== X-Forwarded-Encrypted: i=1; AJvYcCXUo2uggivWJcejPEF2lPeGrshl/HP6pSaZHQqw8VMFQk+CI81UPF7TpTEu5eZ7A8A2+bRQv/r5iFQtwK8=@vger.kernel.org X-Gm-Message-State: AOJu0YxnE8eR8kBww2EtTZXLk14Cg5m1krnzlKmPZcg4Zo0iDxqR/R1e BtGVOlV6WuIS16374nxvHHwhy+RVgy4CO/TaOScv2dUAKSf/KdeSwJ5AH5MUtcPRHsrktNqF7Io = X-Gm-Gg: ASbGncui88wOWhHjojmYtFtBuL7qjtodgmFlU7mIRCAqGM71ELn1bV32t9pHyiFNkuj NIAq6gjDz3pt9Cs5InlXSa3wwQ8E5iuwPuq/yN+UWB5bWOEo+Iw/Ezan06lSn2xPZ+viAped8BA l/C+AaUpk/ywynfaEUpdB+Loxc+NEgR1np6iKxTaBvBgudts8Sy/6LLQxwTKWtgxhMzwelmRAUV CmW1ib3h4Y6TaQZg+BqaQiL1zbkhX004CSKV3wmsLTBNKIjwSWhoyojwsA= X-Google-Smtp-Source: AGHT+IGtvXF84vSzKEt9nvcRn17lafPxKZcVbWSXwCigI3N+2LoaMAbFgPOkqbnj99UeaG1XnEyJIA== X-Received: by 2002:a17:903:2447:b0:216:725c:a12c with SMTP id d9443c01a7336-218d6fd7926mr23537805ad.9.1734503727216; Tue, 17 Dec 2024 22:35:27 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f2ed62cddbsm656772a91.15.2024.12.17.22.35.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:26 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 2/7] zram: remove entry element member Date: Wed, 18 Dec 2024 15:34:19 +0900 Message-ID: <20241218063513.297475-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Element is in the same anon union as handle and hence holds the same value, which makes code below sort of confusing handle =3D zram_get_handle() if (!handle) element =3D zram_get_element() Element doesn't really simplify the code, let's just remove it. We already re-purpose handle to store the block id a written back page. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 23 +++++------------------ drivers/block/zram/zram_drv.h | 5 +---- 2 files changed, 6 insertions(+), 22 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index cda0f3fd4058..8c71ddd17024 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -112,17 +112,6 @@ static void zram_clear_flag(struct zram *zram, u32 ind= ex, zram->table[index].flags &=3D ~BIT(flag); } =20 -static inline void zram_set_element(struct zram *zram, u32 index, - unsigned long element) -{ - zram->table[index].element =3D element; -} - -static unsigned long zram_get_element(struct zram *zram, u32 index) -{ - return zram->table[index].element; -} - static size_t zram_get_obj_size(struct zram *zram, u32 index) { return zram->table[index].flags & (BIT(ZRAM_FLAG_SHIFT) - 1); @@ -879,7 +868,7 @@ static ssize_t writeback_store(struct device *dev, =20 zram_free_page(zram, index); zram_set_flag(zram, index, ZRAM_WB); - zram_set_element(zram, index, blk_idx); + zram_set_handle(zram, index, blk_idx); blk_idx =3D 0; atomic64_inc(&zram->stats.pages_stored); spin_lock(&zram->wb_limit_lock); @@ -1504,7 +1493,7 @@ static void zram_free_page(struct zram *zram, size_t = index) =20 if (zram_test_flag(zram, index, ZRAM_WB)) { zram_clear_flag(zram, index, ZRAM_WB); - free_block_bdev(zram, zram_get_element(zram, index)); + free_block_bdev(zram, zram_get_handle(zram, index)); goto out; } =20 @@ -1548,12 +1537,10 @@ static int zram_read_from_zspool(struct zram *zram,= struct page *page, =20 handle =3D zram_get_handle(zram, index); if (!handle || zram_test_flag(zram, index, ZRAM_SAME)) { - unsigned long value; void *mem; =20 - value =3D handle ? zram_get_element(zram, index) : 0; mem =3D kmap_local_page(page); - zram_fill_page(mem, PAGE_SIZE, value); + zram_fill_page(mem, PAGE_SIZE, handle); kunmap_local(mem); return 0; } @@ -1599,7 +1586,7 @@ static int zram_read_page(struct zram *zram, struct p= age *page, u32 index, */ zram_slot_unlock(zram, index); =20 - ret =3D read_from_bdev(zram, page, zram_get_element(zram, index), + ret =3D read_from_bdev(zram, page, zram_get_handle(zram, index), parent); } =20 @@ -1750,7 +1737,7 @@ static int zram_write_page(struct zram *zram, struct = page *page, u32 index) =20 if (flags) { zram_set_flag(zram, index, flags); - zram_set_element(zram, index, element); + zram_set_handle(zram, index, element); } else { zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 134be414e210..db78d7c01b9a 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -62,10 +62,7 @@ enum zram_pageflags { =20 /* Allocated for each disk page */ struct zram_table_entry { - union { - unsigned long handle; - unsigned long element; - }; + unsigned long handle; unsigned int flags; spinlock_t lock; #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME --=20 2.47.1.613.gc27f4b7a9f-goog From nobody Sun Dec 14 13:58:36 2025 Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FB2A15853C for ; Wed, 18 Dec 2024 06:35:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503733; cv=none; b=mLbvXcdxXixRmBoGr2plJjATu7XZ2QwhOF3K6f5z9UKyPmZzagUlB87tV1KIc/AF4M49iGZT+OvaR9PNKYGalo/o1q0hGy28YM/+7Ijl+r5qSxDgVUwXFIWRbYV+I4znZaHXzUDeKjVhXGKsaW679hqtwN3It8P8T2SzHWGA7KY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503733; c=relaxed/simple; bh=HiHrcfRXYSUA14GWIIG3XGVwz9lXXXRXJ9mmzmSAFzs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KxqJyeVgrJ5WZFIEkfwzYi9lD+RHWINnBDZUVahn39BdLBRJJuw5ZTxR6ER/yNur+KlzaZhlbWms718rRbSRjAV1ahA0haBEgIcfMaCJriZRKbukT5EPUGe9wjU+bgWdd9mgHYYKxy5meHF2RRXVz31F9eNhQ631j9W729a/1jQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=IKwXlEQh; arc=none smtp.client-ip=209.85.215.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="IKwXlEQh" Received: by mail-pg1-f178.google.com with SMTP id 41be03b00d2f7-7fdc6e04411so3792992a12.2 for ; Tue, 17 Dec 2024 22:35:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503731; x=1735108531; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K01D9WjQ4IDKLASlrrrpT/PTwcowLsveSV75IjoNXbE=; b=IKwXlEQhA9hDhCCmVaZ9E1CD6pe57rfQTIThgKATZiwmoLlc5/7qC8uWIVqMg+wzhh T/uUlVkYGgCKmGhDD2JtELVUxf9X/cItFQCc/ucAIRbmMh2cPWQ3QPnrmT7QVBoJn7v0 xpbIBJAGyD7LnqEan6bKYTQage2QaiqOPgQ8g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503731; x=1735108531; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K01D9WjQ4IDKLASlrrrpT/PTwcowLsveSV75IjoNXbE=; b=BWT396jC9BAD00GOqoSXjgZYXFXE77SW8VifeDQczcrJFWGcL1xsxEOahJNwt7fzQg XqP7sE1RvHWZbDtHJZMHKz5Yzw8Wr499ucwKL6dOQGhmkk0J64sJIs/SdDqLWeiDzykm Sa3soJW2pZqkU7Yygp0iwoBemLuZCDDlohirP7ML3giYqSa1o6hywy6PLipbmpytBuuN Emu2eKVJV0BeNcbANVx1yJwK5ko5eMvz6z8hQizHj1p20xZOMaP7hfctkfksFVVrBSr1 4J6PXP6CIBMli9YMLjIQwk8s2uxva5m70HnqCErwmsjTZOcXxfQl75cleY+7PQdVKof5 zAig== X-Forwarded-Encrypted: i=1; AJvYcCXTx+QWJzchu9SD8Faot3ANaZWoQsYdjmHzAY0NPExbsvAxxkZSHro59uHKmF0US7s9KFEh4ZBREzMFmso=@vger.kernel.org X-Gm-Message-State: AOJu0YwSIugZp8sAw/I5hol0lDpTXaiYziIXQbX9lbEq0B7+Ev76M13P Ue/5UeCPkPQ4XAPK2yfQdambBbATI5vUrrzz/j5NoRxbNBUSCbX1NmwMd70f1A== X-Gm-Gg: ASbGncust2L+aPCwb9dyYP4o/15PKfA75RoAlEdewp+XAANU4QkCLaIMMzyddMUvOMr 0SGmUdIHnom2BMTYkwl3+7QPWIA+mVY3EaxGFFW0kNHZUXDTOixnF69nMYKnAGrIekxSUszY0/i 3MKEX6fbney9ffAo8ewXyt4SnikXAoxhLR/EHcV7LMaRyrL6BozTCaJXa1lxr4RwGi/iuuroFpW Z/ffDt0253W+Oz+4VMlsYogwcPzhGZiNLVHA0OSWm5+6dbfUVDkcOQcUkE= X-Google-Smtp-Source: AGHT+IFK5WlahiR5aGL0BTCtzr0kzih6dT4LaJYbmSnHINvLQ21PzUVu87JBr7NRGop1fXLHbUC5yg== X-Received: by 2002:a17:90b:2703:b0:2ee:ed1c:e451 with SMTP id 98e67ed59e1d1-2f2e91db281mr2864827a91.15.1734503731353; Tue, 17 Dec 2024 22:35:31 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f2ed52c3bdsm587770a91.10.2024.12.17.22.35.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:30 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 3/7] zram: factor out ZRAM_SAME write Date: Wed, 18 Dec 2024 15:34:20 +0900 Message-ID: <20241218063513.297475-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Handling of ZRAM_SAME now uses a goto to the final stages of zram_write_page() plus it introduces a branch and flags variable, which is not making the code any simpler. In reality, we can handle ZRAM_SAME immediately when we detect such pages and remove a goto and a branch. Factor out ZRAM_SAME handling into a separate routine to simplify zram_write_page(). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 37 ++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 8c71ddd17024..89f3aaa23329 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1624,6 +1624,20 @@ static int zram_bvec_read(struct zram *zram, struct = bio_vec *bvec, return zram_read_page(zram, bvec->bv_page, index, bio); } =20 +static int write_same_filled_page(struct zram *zram, unsigned long fill, + u32 index) +{ + zram_slot_lock(zram, index); + zram_set_flag(zram, index, ZRAM_SAME); + zram_set_handle(zram, index, fill); + zram_slot_unlock(zram, index); + + atomic64_inc(&zram->stats.same_pages); + atomic64_inc(&zram->stats.pages_stored); + + return 0; +} + static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret =3D 0; @@ -1633,7 +1647,7 @@ static int zram_write_page(struct zram *zram, struct = page *page, u32 index) void *src, *dst, *mem; struct zcomp_strm *zstrm; unsigned long element =3D 0; - enum zram_pageflags flags =3D 0; + bool same_filled; =20 /* First, free memory allocated to this slot (if any) */ zram_slot_lock(zram, index); @@ -1641,14 +1655,10 @@ static int zram_write_page(struct zram *zram, struc= t page *page, u32 index) zram_slot_unlock(zram, index); =20 mem =3D kmap_local_page(page); - if (page_same_filled(mem, &element)) { - kunmap_local(mem); - /* Free memory associated with this sector now. */ - flags =3D ZRAM_SAME; - atomic64_inc(&zram->stats.same_pages); - goto out; - } + same_filled =3D page_same_filled(mem, &element); kunmap_local(mem); + if (same_filled) + return write_same_filled_page(zram, element, index); =20 compress_again: zstrm =3D zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); @@ -1727,7 +1737,7 @@ static int zram_write_page(struct zram *zram, struct = page *page, u32 index) zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_unmap_object(zram->mem_pool, handle); atomic64_add(comp_len, &zram->stats.compr_data_size); -out: + zram_slot_lock(zram, index); if (comp_len =3D=3D PAGE_SIZE) { zram_set_flag(zram, index, ZRAM_HUGE); @@ -1735,13 +1745,8 @@ static int zram_write_page(struct zram *zram, struct= page *page, u32 index) atomic64_inc(&zram->stats.huge_pages_since); } =20 - if (flags) { - zram_set_flag(zram, index, flags); - zram_set_handle(zram, index, element); - } else { - zram_set_handle(zram, index, handle); - zram_set_obj_size(zram, index, comp_len); - } + zram_set_handle(zram, index, handle); + zram_set_obj_size(zram, index, comp_len); zram_slot_unlock(zram, index); =20 /* Update stats */ --=20 2.47.1.613.gc27f4b7a9f-goog From nobody Sun Dec 14 13:58:36 2025 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 470FF15A868 for ; Wed, 18 Dec 2024 06:35:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503737; cv=none; b=BSmh9OLjiSHZQtmqNVyONJxiRlnLXa/s1gcbLUOMTVZNVoJUlKSDAez5rTPoubBnIOm0hWjVwZTB6r6FNSSJ1ATH+C5zDpc3DVUGJX8nquBJ35eZvolaR01giQLspLqnasWsoehzRBznd9HQZDiw4AKBzOHvAVvgYqHF/Zx7reM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503737; c=relaxed/simple; bh=cQK9+yohWJfVY3p/FismgfK2Y4qEB4nHwdEOOmxzU7w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jMYwjxVxzQCxYe/83NVIMpQ2csTGyjwWCzabWQWAygZa4kvIjJukZ3Rqm/L6nMWYZA4DOq+S4D8NilDCr90GoWjSohyiNT5DCIBetX5yGbIPjXjxF1uNkAmNdsMh16SdEhMB0Z8dB+aAXJFfXqHSNsVKm3w5u5u6Dk0aIzWjpyA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=KP0yF2si; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="KP0yF2si" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-72739105e02so6659184b3a.0 for ; Tue, 17 Dec 2024 22:35:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503735; x=1735108535; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kcFgQDfUGNa4RpmE9ZvQmkln//OMWOItv9XoEUwGa60=; b=KP0yF2si3zZU3TctkJ9AoC1IQVzntBYcd2wp4qFi2jI8ojJPzqwgpaG8qNuvaS0gzu IlAxbxCihjnpa8s5ZBO1xQsGL9+sWpnp2Yhtbq5h3iu79+OhDf/Ttpt4sU8fspM+XN4S K/od3QpBjwlbRHo7JzffSA5Jj+r4UlQ1D3xZ8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503735; x=1735108535; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kcFgQDfUGNa4RpmE9ZvQmkln//OMWOItv9XoEUwGa60=; b=TVHgNCqAI/g8UTgpM/Y98H+szyfb/pb8NVB1FtEpFBUJIAhtpHiv+vUw44uPNc5gu5 /XuXC0xSVoBb9BJO7OjJQkw7Q4XBd1rMI8qOiKgEOz/OIzPDayz2Em9Db7vLAgLkSzVW EYAJ6dWwCSmZ9N86qWxZi7IC+ZEGULphllvs4KEU26XHqeYyJ9oZXsh5k7n5F6wGUkZE WnoB7LbgATHRg29WuNbk9Tz8o8o7WEnCpyRDilHYQtWklP+L6s9JRda+A4t7TogrtGSm 2d1RrPcDeudzTRmgkry2kU45lCrvRXRS6DyOUsxL7bRdZ/h9eQkSOTikD5xJsmpYWqni Wm1w== X-Forwarded-Encrypted: i=1; AJvYcCUKhI5XVNy7ki8A+CPiUEAeXepxihhiqa7m7fRuAhi0PCZfgPavq9HHZ5LHwbwX2q89t37oQ07L+zCA34E=@vger.kernel.org X-Gm-Message-State: AOJu0YylBEzDWRJOPtFHYD8YS09jyobuZTZhBcFmV90tBnCA8s0hl6Po VAJVuZnv5NfS1lnH5yCu4ELmBeMleDsLZbBzmsBZsVcPZWa9cP8eBYt7kRB/ng== X-Gm-Gg: ASbGncv8ZrtkndqsBAdWsEIrDmix1ouSki14f1VBTwu9vkqnKoN/2LBTYp4kJFMbeTB kEs7V+J3bs5fQh8woyTCR2KfPua7qsKWvEDXbXy3qt9KhgK7JSsTuR3QPg6y8+v7HO6DsBMFLcw a/4uMcmDNv40GA1m3mO9HiqjpN/kYpVlECRaxEBfH1unaQsXwk90nHaYaPpWCPlausY8QeQSSSF M2lXfMYBBoifKOqs2e8XGsMKFocY/jDI/uTlXJzd6hYCLWOS22Knb44Sn0= X-Google-Smtp-Source: AGHT+IH4+dvzwEYz6RMzf3+gGHyop3pBmEqsMorp12eUh0yPbMZK6wxgVt40cdSmpIiL6/jj76oo3w== X-Received: by 2002:a05:6a20:3d88:b0:1e1:b023:6c98 with SMTP id adf61e73a8af0-1e5b482147emr2917670637.26.1734503735547; Tue, 17 Dec 2024 22:35:35 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72918b77335sm7690904b3a.118.2024.12.17.22.35.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:35 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 4/7] zram: factor out ZRAM_HUGE write Date: Wed, 18 Dec 2024 15:34:21 +0900 Message-ID: <20241218063513.297475-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" zram_write_page() handles: ZRAM_SAME pages (which was already factored out) stores, regular page stores and ZRAM_HUGE pages stores. ZRAM_HUGE handling adds a significant amount of complexity. Instead, we can handle ZRAM_HUGE in a separate function. This allows us to simplify zs_handle allocations slow-path, as it now does not handle ZRAM_HUGE case. ZRAM_HUGE zs_handle allocation, on the other hand, can now drop __GFP_KSWAPD_RECLAIM because we handle ZRAM_HUGE in preemptible context (outside of local-lock scope). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 136 +++++++++++++++++++++------------- 1 file changed, 83 insertions(+), 53 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 89f3aaa23329..1339776bc6c5 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -132,6 +132,27 @@ static inline bool zram_allocated(struct zram *zram, u= 32 index) zram_test_flag(zram, index, ZRAM_WB); } =20 +static inline void update_used_max(struct zram *zram, const unsigned long = pages) +{ + unsigned long cur_max =3D atomic_long_read(&zram->stats.max_used_pages); + + do { + if (cur_max >=3D pages) + return; + } while (!atomic_long_try_cmpxchg(&zram->stats.max_used_pages, + &cur_max, pages)); +} + +static bool zram_can_store_page(struct zram *zram) +{ + unsigned long alloced_pages; + + alloced_pages =3D zs_get_total_pages(zram->mem_pool); + update_used_max(zram, alloced_pages); + + return !zram->limit_pages || alloced_pages <=3D zram->limit_pages; +} + #if PAGE_SIZE !=3D 4096 static inline bool is_partial_io(struct bio_vec *bvec) { @@ -266,18 +287,6 @@ static struct zram_pp_slot *select_pp_slot(struct zram= _pp_ctl *ctl) } #endif =20 -static inline void update_used_max(struct zram *zram, - const unsigned long pages) -{ - unsigned long cur_max =3D atomic_long_read(&zram->stats.max_used_pages); - - do { - if (cur_max >=3D pages) - return; - } while (!atomic_long_try_cmpxchg(&zram->stats.max_used_pages, - &cur_max, pages)); -} - static inline void zram_fill_page(void *ptr, unsigned long len, unsigned long value) { @@ -1638,13 +1647,54 @@ static int write_same_filled_page(struct zram *zram= , unsigned long fill, return 0; } =20 +static int write_incompressible_page(struct zram *zram, struct page *page, + u32 index) +{ + unsigned long handle; + void *src, *dst; + + /* + * This function is called from preemptible context so we don't need + * to do optimistic and fallback to pessimistic handle allocation, + * like we do for compressible pages. + */ + handle =3D zs_malloc(zram->mem_pool, PAGE_SIZE, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); + if (IS_ERR_VALUE(handle)) + return PTR_ERR((void *)handle); + + if (!zram_can_store_page(zram)) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zs_free(zram->mem_pool, handle); + return -ENOMEM; + } + + dst =3D zs_map_object(zram->mem_pool, handle, ZS_MM_WO); + src =3D kmap_local_page(page); + memcpy(dst, src, PAGE_SIZE); + kunmap_local(src); + zs_unmap_object(zram->mem_pool, handle); + + zram_slot_lock(zram, index); + zram_set_flag(zram, index, ZRAM_HUGE); + zram_set_handle(zram, index, handle); + zram_set_obj_size(zram, index, PAGE_SIZE); + zram_slot_unlock(zram, index); + + atomic64_add(PAGE_SIZE, &zram->stats.compr_data_size); + atomic64_inc(&zram->stats.huge_pages); + atomic64_inc(&zram->stats.huge_pages_since); + atomic64_inc(&zram->stats.pages_stored); + + return 0; +} + static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret =3D 0; - unsigned long alloced_pages; unsigned long handle =3D -ENOMEM; unsigned int comp_len =3D 0; - void *src, *dst, *mem; + void *dst, *mem; struct zcomp_strm *zstrm; unsigned long element =3D 0; bool same_filled; @@ -1662,10 +1712,10 @@ static int zram_write_page(struct zram *zram, struc= t page *page, u32 index) =20 compress_again: zstrm =3D zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); - src =3D kmap_local_page(page); + mem =3D kmap_local_page(page); ret =3D zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, - src, &comp_len); - kunmap_local(src); + mem, &comp_len); + kunmap_local(mem); =20 if (unlikely(ret)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); @@ -1674,8 +1724,11 @@ static int zram_write_page(struct zram *zram, struct= page *page, u32 index) return ret; } =20 - if (comp_len >=3D huge_class_size) - comp_len =3D PAGE_SIZE; + if (comp_len >=3D huge_class_size) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + return write_incompressible_page(zram, page, index); + } + /* * handle allocation has 2 paths: * a) fast path is executed with preemption disabled (for @@ -1691,35 +1744,23 @@ static int zram_write_page(struct zram *zram, struc= t page *page, u32 index) */ if (IS_ERR_VALUE(handle)) handle =3D zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + __GFP_KSWAPD_RECLAIM | + __GFP_NOWARN | + __GFP_HIGHMEM | + __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); atomic64_inc(&zram->stats.writestall); handle =3D zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_HIGHMEM | + __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) return PTR_ERR((void *)handle); =20 - if (comp_len !=3D PAGE_SIZE) - goto compress_again; - /* - * If the page is not compressible, you need to acquire the - * lock and execute the code below. The zcomp_stream_get() - * call is needed to disable the cpu hotplug and grab the - * zstrm buffer back. It is necessary that the dereferencing - * of the zstrm variable below occurs correctly. - */ - zstrm =3D zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); + goto compress_again; } =20 - alloced_pages =3D zs_get_total_pages(zram->mem_pool); - update_used_max(zram, alloced_pages); - - if (zram->limit_pages && alloced_pages > zram->limit_pages) { + if (!zram_can_store_page(zram)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_free(zram->mem_pool, handle); return -ENOMEM; @@ -1727,30 +1768,19 @@ static int zram_write_page(struct zram *zram, struc= t page *page, u32 index) =20 dst =3D zs_map_object(zram->mem_pool, handle, ZS_MM_WO); =20 - src =3D zstrm->buffer; - if (comp_len =3D=3D PAGE_SIZE) - src =3D kmap_local_page(page); - memcpy(dst, src, comp_len); - if (comp_len =3D=3D PAGE_SIZE) - kunmap_local(src); - + memcpy(dst, zstrm->buffer, comp_len); zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_unmap_object(zram->mem_pool, handle); - atomic64_add(comp_len, &zram->stats.compr_data_size); =20 zram_slot_lock(zram, index); - if (comp_len =3D=3D PAGE_SIZE) { - zram_set_flag(zram, index, ZRAM_HUGE); - atomic64_inc(&zram->stats.huge_pages); - atomic64_inc(&zram->stats.huge_pages_since); - } - zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); zram_slot_unlock(zram, index); =20 /* Update stats */ atomic64_inc(&zram->stats.pages_stored); + atomic64_add(comp_len, &zram->stats.compr_data_size); + return ret; } =20 --=20 2.47.1.613.gc27f4b7a9f-goog From nobody Sun Dec 14 13:58:36 2025 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7984615C13F for ; Wed, 18 Dec 2024 06:35:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503742; cv=none; b=bFh4Pd1bjug6uBqa7pjVbMLjHuad+kVaVv9BvdpUO8nvLcAtOKqn/HkAd6cv4SKSJ4Br32UhZdhq4Y6hHMoSSMzjohop5AHVqraz6mEKaggkItFrZKoPXJlIJ8dH08kZPdJ2Ik6RapcStusPygK9+nMDwvCCcCCrd5knwWUnYcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503742; c=relaxed/simple; bh=ueT58dOzzmPY8vm9egfT8Vk1XXLTgehj6PW0oAzS6OQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BWvtkeTIksxXXL+be9X5olE5ub3x5p8K6ysMj5u/1phTnytLFRW6rtL97w4frJo1E632SfOKrMEnNpeG3T438CROSaAM6fOVhOpu0QRMTD0k0WxlYCKtbjOgsCyjmAWUc5Dn5X+9EuNvgBbdFhbTnVae+ywWRntspYWtbhk7Ye8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=SaiEO/bv; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="SaiEO/bv" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-725d9f57d90so4574331b3a.1 for ; Tue, 17 Dec 2024 22:35:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503739; x=1735108539; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8UlIfnWhj8akIUQw35UgTdzrn/T1WzL/Vbski8Obleg=; b=SaiEO/bvSHHpeecwC+CoXgSkErOzmoK8Mh1oGRaWwf9ZoDH317aL01KhAvFaagNm+w pD4brEHcIk+1seP8HjnhGZN57XRdHdsRIrw6v+5tePV7NDLhiAprwy5vnFYDqvRjf77k +Ww2kT4YyjruACjYt32TUFl7JFemIYG7CnhjY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503739; x=1735108539; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8UlIfnWhj8akIUQw35UgTdzrn/T1WzL/Vbski8Obleg=; b=CMnkByFi/CTDg6AgzFwXgOmuOg2uCd5TR2V8vipxWaAk/DNE9EQWAtLCSDA0oPXpo3 9Ldw/qGKQb5KXQrwMwEz85et+6/xhWMh0CKXyutOfbsDzmDSUcxlRVJZVDxrx9xQfxay RCQcqk2fIN2tAuQSpXsmsL1P2/5z/SguMIji1WWQSe8vY7OMF0t7TfwhCHiiX9Z1k60S 83cQ9v8U9jNQL8Zq7bbn/ae/L8FjnpqBGn73NjgGCYUx+NdkHW2mLsIK1YreQJ0G7mmM WFx4lgHP+Mf4udYCGknQl7T5SfZe4dXAn/nsMrXp0y/rej6O4S538Xsf8Va5IpykiHQS jIMg== X-Forwarded-Encrypted: i=1; AJvYcCX06ISBXdOo38EHGKzAPBV97mDAKYrhjQal0xdGPamAWxgnr/jH1E8PI/wCdueC8qX7IYn0AFYA+FIjwfc=@vger.kernel.org X-Gm-Message-State: AOJu0Yyy1WYNdF+EOFVNjTufTzeHw1+aefzOICDw6jTQ+TdWx9Tfblac oAdSx/DkrY3aa9mOG3QsBCR1Vv6b6rD/nnbwq2DbBp+A9FeMWfLc2ojE01CAJlvKqjROWAcn9ac = X-Gm-Gg: ASbGncs2qjPH95ta6JfB6UTh0xIWQo0Fz8IGUqImtBjOqTXOQK4APok+73Hl3CJ6xzJ nL1Tc/Hf+uR3tmMjNhxg6AtjzUSvd6Yk3RuFHWQmXID8V0y4iLX7cAnGuH+FW0Dofg5P1qxq5t6 DANz+ylEXH+eZa9sVhZ9onoZbRe5N8B2OE7myCXECOAs7C5W8/RpRu4ktI/TdzZ0hqAf04gnkb3 6R0vu352WF04aCEEz8wucNf2lyOID7b5TE6WXKlq5X4Un07DqNgisunAeI= X-Google-Smtp-Source: AGHT+IGwQ0wQAXBUGI22Q3J+XJvZdIgvdCXo+/PUjQeIZFjzJ+5utt6loeABpMh/C7JssxnEtaeHvw== X-Received: by 2002:a05:6a20:3948:b0:1e1:aef4:9ce8 with SMTP id adf61e73a8af0-1e5b48a15a5mr2391517637.28.1734503739653; Tue, 17 Dec 2024 22:35:39 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72918bad760sm7760879b3a.152.2024.12.17.22.35.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:39 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 5/7] zram: factor out different page types read Date: Wed, 18 Dec 2024 15:34:22 +0900 Message-ID: <20241218063513.297475-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Similarly to write, split the page read code into ZRAM_HUGE read, ZRAM_SAME read and compressed page read to simplify the code. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 85 +++++++++++++++++++++-------------- 1 file changed, 52 insertions(+), 33 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 1339776bc6c5..465297d31bdf 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1530,54 +1530,73 @@ static void zram_free_page(struct zram *zram, size_= t index) zram_set_obj_size(zram, index, 0); } =20 -/* - * Reads (decompresses if needed) a page from zspool (zsmalloc). - * Corresponding ZRAM slot should be locked. - */ -static int zram_read_from_zspool(struct zram *zram, struct page *page, +static int read_same_filled_page(struct zram *zram, struct page *page, u32 index) { - struct zcomp_strm *zstrm; + void *mem; + + mem =3D kmap_local_page(page); + zram_fill_page(mem, PAGE_SIZE, zram_get_handle(zram, index)); + kunmap_local(mem); + return 0; +} + +static int read_incompressible_page(struct zram *zram, struct page *page, + u32 index) +{ unsigned long handle; - unsigned int size; void *src, *dst; - u32 prio; - int ret; =20 handle =3D zram_get_handle(zram, index); - if (!handle || zram_test_flag(zram, index, ZRAM_SAME)) { - void *mem; + src =3D zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + dst =3D kmap_local_page(page); + copy_page(dst, src); + kunmap_local(dst); + zs_unmap_object(zram->mem_pool, handle); =20 - mem =3D kmap_local_page(page); - zram_fill_page(mem, PAGE_SIZE, handle); - kunmap_local(mem); - return 0; - } + return 0; +} =20 - size =3D zram_get_obj_size(zram, index); +static int read_compressed_page(struct zram *zram, struct page *page, u32 = index) +{ + struct zcomp_strm *zstrm; + unsigned long handle; + unsigned int size; + void *src, *dst; + int ret, prio; =20 - if (size !=3D PAGE_SIZE) { - prio =3D zram_get_priority(zram, index); - zstrm =3D zcomp_stream_get(zram->comps[prio]); - } + handle =3D zram_get_handle(zram, index); + size =3D zram_get_obj_size(zram, index); + prio =3D zram_get_priority(zram, index); =20 + zstrm =3D zcomp_stream_get(zram->comps[prio]); src =3D zs_map_object(zram->mem_pool, handle, ZS_MM_RO); - if (size =3D=3D PAGE_SIZE) { - dst =3D kmap_local_page(page); - copy_page(dst, src); - kunmap_local(dst); - ret =3D 0; - } else { - dst =3D kmap_local_page(page); - ret =3D zcomp_decompress(zram->comps[prio], zstrm, - src, size, dst); - kunmap_local(dst); - zcomp_stream_put(zram->comps[prio]); - } + dst =3D kmap_local_page(page); + ret =3D zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); + kunmap_local(dst); zs_unmap_object(zram->mem_pool, handle); + zcomp_stream_put(zram->comps[prio]); + return ret; } =20 +/* + * Reads (decompresses if needed) a page from zspool (zsmalloc). + * Corresponding ZRAM slot should be locked. + */ +static int zram_read_from_zspool(struct zram *zram, struct page *page, + u32 index) +{ + if (zram_test_flag(zram, index, ZRAM_SAME) || + !zram_get_handle(zram, index)) + return read_same_filled_page(zram, page, index); + + if (!zram_test_flag(zram, index, ZRAM_HUGE)) + return read_compressed_page(zram, page, index); + else + return read_incompressible_page(zram, page, index); +} + static int zram_read_page(struct zram *zram, struct page *page, u32 index, struct bio *parent) { --=20 2.47.1.613.gc27f4b7a9f-goog From nobody Sun Dec 14 13:58:36 2025 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02BA21662E9 for ; Wed, 18 Dec 2024 06:35:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503745; cv=none; b=SgRLwB/VESIsnjiI5ip7THyaHQeFXtReFHqI31vNGIcsvXE3dBaGkRUQH0mZd7mtI9kbueDA72TbKjdyWuyDIaie4Ivne9Xf5vOkK4Qbxt+nkHaVK07pBvuIbwOLvhfgXTMJh9U5j2qqHM6N4gvpDbodUsoWdz1g/dc9NC7X7tg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503745; c=relaxed/simple; bh=qppdETxx2GmdR+QBG/OxKqjh419vfOkRJEgYdBmlDjo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hnT3kREdZ3zUoLd/kPKw9BkNugrusMw+Qk4uKLnqNqFomiHzhmbIKDm7JHk/O1ffobffdn679ApTN6nFXEtcuhv2JqNfCy7g7w3cibsbGC+yvm/HrRPLSLy65NBIpJ234gloT3zG5mTNl6Am69YHDAIA27t8135flA5EqjYq+Do= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=CsGEeh4B; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="CsGEeh4B" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-725ed193c9eso5249474b3a.1 for ; Tue, 17 Dec 2024 22:35:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503743; x=1735108543; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jx9bIgiEIqkEeB+/C7YzsZikx7IBwIxJmTKfYS6wvFk=; b=CsGEeh4BNxFPMU8YMakldux7ba0uhGa7criO89yFEW3QyU3SejN1dN8vHWwxJDyYg1 0pLYCOrPIJiaW3iidP7OVz64nrBW/G+QXlbFJjx6XxKxGMigMnLb5A+4S3DCKPLw3Ypj h0NQBfRHHFOtrMk1A+jkvuON+P71WBJES+1rc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503743; x=1735108543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jx9bIgiEIqkEeB+/C7YzsZikx7IBwIxJmTKfYS6wvFk=; b=ecOklnxNuwLVN210Rg2R/wsk0BTxh6g31Rom9ALULBntpveF0vTxDFHyo4Ti8iiNoT ZNpoFvC1C86GPC4K/ySNY1UF42KvGtC4DBEToV/+L6iQNUm94WYff8U8fAOgjF+zIxZx vJQc/3wsrF9DneDXNCXio2o2F+gom8575/pvxYccj0RgHL+no/azSoQyKIPGQa82GKa/ nDT9ttt4kdDMDjhWvv7cZwAQdaFbA7hZc1D5a8yf5MqiTfR/4dp3hfM3SmLvSfhYV7i2 X1L3BKNTBH8gStX52ZT48Lkh0aohNpRr8acN0maEu/hG9qo/Mjg2YHtbsHkkA+asVtfx XcCw== X-Forwarded-Encrypted: i=1; AJvYcCUyUPENBLxLzpBJNcic+YCnUcJJp67iNQ53tpsFXjX4zt9rdfSpvVFOOlGKbgrOj68tUaKqci2khq2FKxo=@vger.kernel.org X-Gm-Message-State: AOJu0Yzsqhf1lud+q83IasCxbb+rHk8Mx7V5nnrGnTfmhmVM5Hde4xA4 9x1sK80k2DkKYmp29iTLj4SC6/PBlDILQGGnpN0c8ybHFewT8jUH5OE3UWxkTw== X-Gm-Gg: ASbGncvRwiessalY4ybUgu6mOc3F10p9RUjSh7SY8UubLmUQiI8rZ9P+2ivCQifV6bo TgpO/uAMBzjDUFFVW58Wc3LD+Pa3E0D/Zk0Z+E6YAyDwjzhv905nfas5gTQUTA3/qdXEk/uFrju vax4uUwec0Y1nUIpxHMFHUyMwdDIAtLS62pQMzE4luN15IROppLqYN5061lax1/w/n4Mz8UNoKV yqnUdZ5c5h8UBMPSVSo9hVZg/Tx+NFGVg5FRB+wZgOc4AyhTebCvkhJvdI= X-Google-Smtp-Source: AGHT+IFnXzThR3kYgLdl6KAfkV3e2DAw4nVMXA+afaVN1un8QDO+bjEptYoZDse8hnX1mt8BUslL8Q== X-Received: by 2002:aa7:888c:0:b0:725:f097:ed21 with SMTP id d2e1a72fcca58-72a8d23e809mr2464921b3a.15.1734503743201; Tue, 17 Dec 2024 22:35:43 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7291bca1069sm7446008b3a.114.2024.12.17.22.35.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:42 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 6/7] zram: use zram_read_from_zspool() in writeback Date: Wed, 18 Dec 2024 15:34:23 +0900 Message-ID: <20241218063513.297475-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We only can read pages from zspool in writeback, zram_read_page() is not really right in that context not only because it's a more generic function that handles ZRAM_WB pages, but also because it requires us to unlock slot between slot flag check and actual page read. Use zram_read_from_zspool() instead and do slot flags check and page read under the same slot lock. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 465297d31bdf..7dd72b58e921 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -55,8 +55,8 @@ static size_t huge_class_size; static const struct block_device_operations zram_devops; =20 static void zram_free_page(struct zram *zram, size_t index); -static int zram_read_page(struct zram *zram, struct page *page, u32 index, - struct bio *parent); +static int zram_read_from_zspool(struct zram *zram, struct page *page, + u32 index); =20 static int zram_slot_trylock(struct zram *zram, u32 index) { @@ -831,13 +831,10 @@ static ssize_t writeback_store(struct device *dev, */ if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) goto next; + if (zram_read_from_zspool(zram, page, index)) + goto next; zram_slot_unlock(zram, index); =20 - if (zram_read_page(zram, page, index, NULL)) { - release_pp_slot(zram, pps); - continue; - } - bio_init(&bio, zram->bdev, &bio_vec, 1, REQ_OP_WRITE | REQ_SYNC); bio.bi_iter.bi_sector =3D blk_idx * (PAGE_SIZE >> 9); --=20 2.47.1.613.gc27f4b7a9f-goog From nobody Sun Dec 14 13:58:36 2025 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 836A916D9B8 for ; Wed, 18 Dec 2024 06:35:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503749; cv=none; b=EY5p26MhJ+Nb6/ZwypVupC6hlzeAb+N7t8YwXY5I4iNHpDlHSHd6rTGTDBNI5wtCsBVb8TfBF9rznIxLkISnCwfj4RfRS30apsbrC776kpYLBJ3jH6evd1FYUqzpF2QaP/mUJ1lz5yCoEoPxeYTgR/EzhlXV+JvNp5MZnOrHqHM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734503749; c=relaxed/simple; bh=Z7Q3IK3DKIWUy+W+tyBjH+edLwCfI7prnmKZg3nmKLI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lOhRrAaRT2got8LjB2kTThvfLiEJMzKc/F1Obt+XymHI4O9gjs3zRaA4qI/AQYlR5Iz3C2WqBDXGbldL0z/TckQ7zu96sZdeg9ld6Q/1ToeIJMpQuUvlv5YoMUycl8yPAsSvviRs5W7Sw9eBlf3/Rq0gckrk6L1nSSQBV0rtyF0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=itiC4McN; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="itiC4McN" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-725dc290c00so337942b3a.0 for ; Tue, 17 Dec 2024 22:35:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503747; x=1735108547; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=06dLgy7e/21/5Xh2W7aoiU59dLBqGpGpJ1upbma1LzU=; b=itiC4McNLozgYRFX0YlENXrPkn5xFk70b4yDp7AUJ700ajdDUODUy0e//sJgBNAP1T 77NdmAdaqP5OxpJJrfN2vAQF84y9MfPdnliwE3KXts8POVogmnaGy0QOTMPMCzy1lAxM 3uJgw9gdGVjZbRr7IhrzHD98YNvE+e/CFj1Q8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503747; x=1735108547; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=06dLgy7e/21/5Xh2W7aoiU59dLBqGpGpJ1upbma1LzU=; b=PSHy/CYVB1tkbnobeOKA3qxkqXBxfWkLkZ0CY9aqchxSbHkqTHs960Y/Ulwv3BzF6L NlcdZmkUe6Z/bplMVLwDtIZBww7gj8SQISRhN44KjmWXU0+OxL41KdZeCT8sY1dDhCqA 65EIlvzOeALOi6wYi0N+RcLlzv1Mukpdm5MdnNf1jf64AnWdn3tPTqZoN6x8RyAD9iW0 eyLhuhNSGbjmya0f5nVzc0KwLhb4wtewNNfSKAa9GWkCCKUM2yY7F/1hWWxuDLLJ2wmg 0Abk25pbEx0LfDmx7YXo/pWtWB8snCavOp6kdJubov9SRJDutoHhyzSTvIz5svCZXE7t TfVg== X-Forwarded-Encrypted: i=1; AJvYcCUvCVs4pURlH2QkQTpAsb/KBxDUcH3VCuRrButuS8hkIIJ8fBI1i+TcYwl3ApigW9Pg17eoakohWCoGP1w=@vger.kernel.org X-Gm-Message-State: AOJu0Yz0QToF0hPmbxjJ+Vix7Ey7B2RUY97mMh0B8HGdwSGCE0nFxfIs +lNWwO4yPlFCgw2tKZ3XtSYJ+q/hoyTSEiY5QlTRRtyao1Aqn/d8oxe0hGmtTM2dhzL0SL/7sgo = X-Gm-Gg: ASbGncsFNe5aOaJbrQ6jy8svUMPQacapQPllNV8Xo8F1JNZhGqvAW+fiZJED2Dm4boQ miJUSEEHS5OjYE1vcFTwP/uBccr3lKlOw3IqEd7qqfeEongQ9fG50a0k9e0rZHo/KgrhHdfui2+ sSlswCnzYWmu7RZuYhEnGWLOv6ES00jhYiBEKRmoe4QceIc26FrhOTVfgb15PTJrhZkfspEJtFI y0llz/wN9LjE6sA84FImhMZqYqDl9cJqdnp8pWu2PDtIrvZa9jXbwzxFBM= X-Google-Smtp-Source: AGHT+IGf8g96scNATdGQ6vdl6cixlVJ73v6GuTOkPIuKqnPqyqGhnKzXBehwZAxlCiqlPr2dEdXf8A== X-Received: by 2002:a05:6a20:431b:b0:1e1:aef4:9cd9 with SMTP id adf61e73a8af0-1e5b5a7da91mr2332683637.23.1734503746822; Tue, 17 Dec 2024 22:35:46 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72918ad5c07sm8023644b3a.60.2024.12.17.22.35.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:46 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 7/7] zram: cond_resched() in writeback loop Date: Wed, 18 Dec 2024 15:34:24 +0900 Message-ID: <20241218063513.297475-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" zram writeback is a costly operation, because every target slot (unless ZRAM_HUGE) is decompressed before it gets written to a backing device. The writeback to a backing device uses submit_bio_wait() which may look like a rescheduling point. However, if the backing device has BD_HAS_SUBMIT_BIO bit set __submit_bio() calls directly disk->fops->submit_bio(bio) on the backing device and so when submit_bio_wait() calls blk_wait_io() the I/O is already done. On such systems we effective end up in a loop for_each (target slot) { decompress(slot) __submit_bio() disk->fops->submit_bio(bio) } Which on PREEMPT_NONE systems triggers watchdogs (since there are no explicit rescheduling points). Add cond_resched() to the zram writeback loop. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 7dd72b58e921..5b8e4f4171ab 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -884,6 +884,8 @@ static ssize_t writeback_store(struct device *dev, next: zram_slot_unlock(zram, index); release_pp_slot(zram, pps); + + cond_resched(); } =20 if (blk_idx) --=20 2.47.1.613.gc27f4b7a9f-goog