From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78D6F250C1A for ; Wed, 5 Mar 2025 18:16:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198581; cv=none; b=qTxQIwM9qxBGO1bCeB4RgcCsbcchHSmoXtiuMMS7nksl2MQFs1Wsbt7yWbS7XfchCkQxAhg/pFxMMqJ6/NWK9Cz6SxuK8UNJDr0D+HGNOcss5NwwvpG45md6DaM2QcugVY4nRxfi4ptwGP6555nbr6Uz76X7lXRXCc3mMGUoSWY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198581; c=relaxed/simple; bh=sHWxz+0ShuFmutLGSt5hwCRMCNmucoCWmOorPvlaBq4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fHkWJJaGj0EMltqiX6SMf0pbVoSVmSag6Fr+DRG6uaogn652QPzsqNU2rpG65RdSmRIDiGbC7OBRIFmzAd5uqbKXi6H+7rwXxKD2kDRrrPxciuuzxk7FX49q/BBbSrgFmcpYVU8IPWMFWt6zfNoY2D5gsN9q/6XExdsAB2cab38= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sAXZYSeA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sAXZYSeA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1394DC4CEE2; Wed, 5 Mar 2025 18:16:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198581; bh=sHWxz+0ShuFmutLGSt5hwCRMCNmucoCWmOorPvlaBq4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sAXZYSeAbWhcz5LT0yvaAeEsnGtslYmnxLZZlGzmNk7/NTF1qAvm5yqfjEVeO9TdV Rif5lZv9l5yNiQTVhD7iR1rKRmVhxb6x0f7AtA5o+dDIi8dxA8k5+/hJNfj3M46CWo sitQN1QOJpXhDlw8MsqtGOU9rDauvpS566PKb/egzlDjOjw5HOOykW0p+6aaUy4+ZO MEFOD269bYvOMYz51UGc+NTi+U4dipo10wu5Cn47Yt7HxeoiYcEFt2g0Rr6CkLSXel R0AqI1OIzw8RiD5YCnRDB4PBzWL61je0d7EWQu7B8vxEk05bBfernmks5BaYa+d+GT f/9hPYOZmRYjA== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 01/16] mm/madvise: use is_memory_failure() from madvise_do_behavior() Date: Wed, 5 Mar 2025 10:15:56 -0800 Message-Id: <20250305181611.54484-2-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To reduce redundant open-coded checks of CONFIG_MEMORY_FAILURE and MADV_{HWPOISON,SOFT_OFFLINE} in madvise_[un]lock(), is_memory_failure() has introduced. madvise_do_behavior() is still doing the same open-coded check, though. Use is_memory_failure() instead. Signed-off-by: SeongJae Park --- mm/madvise.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 388dc289b5d1..dbc8fec05cc6 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1640,10 +1640,8 @@ static int madvise_do_behavior(struct mm_struct *mm, unsigned long end; int error; =20 -#ifdef CONFIG_MEMORY_FAILURE - if (behavior =3D=3D MADV_HWPOISON || behavior =3D=3D MADV_SOFT_OFFLINE) + if (is_memory_failure(behavior)) return madvise_inject_error(behavior, start, start + len_in); -#endif start =3D untagged_addr_remote(mm, start); end =3D start + len; =20 --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 984BD25291E for ; Wed, 5 Mar 2025 18:16:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198582; cv=none; b=m+VNRCYf3gFvj9n8uHfpArB4Ce3nZ2FpUCFekffOOxeE1lL+rDBLgKIgQhnLQInBTn4N5wLbbL48pjNuQw5EHPfIsFEGgyW0c5UESBn3tz69NFQKaRmzp2mseTCzpDTqMkXF1b9Q7/O5wV+EQSI4F+4BwVwL7b80o28aClrNKnI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198582; c=relaxed/simple; bh=k3u/LxEMkFkS7mrfeSjLL0Nnr2JwC5q9oxYyj2TJeAI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qt68Ix/Jb1zX5IsPeAWOCEgDOiO9eoI4dUD9lOVzmC5QJomX0InUfaXXTZhuMGYrM3mtD5eGgdsXN3V8ulFBKsJVlvivhemkPAi7LsHd1FoiUmPF19rQvpPcBPKGzSu7hMbZ1lzO7wf3935K56PsHO5ovHhzNDwQ0eT4TT5Po9Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=d+BXnAC1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="d+BXnAC1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39959C4CEF2; Wed, 5 Mar 2025 18:16:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198582; bh=k3u/LxEMkFkS7mrfeSjLL0Nnr2JwC5q9oxYyj2TJeAI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d+BXnAC1TkoainkqZ1u199F1FvvZiZ34Ndk/rNCx/M/1t+E94AGj5Quk5hqfuBfOR M1YClyfRGCUP1WoVAujO8KEYZPzAM5nyE+V9w6X04oqWzXMx1S63w8kjugzVIQPUqT vyzywq6x4ps9bDMoX5nQ0n4xouj/atCwPePYKOQDZ9OKJUcyAhG1rAbQvIxrVUDWuc fUV1pP5jvKW6nsjwv7a+fJk5BM7bKW+1NdufBwSEcLDO2AH5PRhrJ8rKbQOMJzAtor VkesKLWj63wstq8GYdHeUtU7YoUmatCsQXyNU7EPPvlIriyjFwml6cKma9iQLIPNg/ DVjdb/F+G4fYA== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 02/16] mm/madvise: split out populate behavior check logic Date: Wed, 5 Mar 2025 10:15:57 -0800 Message-Id: <20250305181611.54484-3-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_do_behavior() has a long open-coded 'behavior' check for MADV_POPULATE_{READ,WRITE}. It adds multiple layers[1] and make the code arguably take longer time to read. Like is_memory_failure(), split out the check to a separate function. This is not technically removing the additional layer but discourage further extending the switch-case. Also it makes madvise_do_behavior() code shorter and therefore easier to read. [1] https://lore.kernel.org/bd6d0bf1-c79e-46bd-a810-9791efb9ad73@lucifer.lo= cal Signed-off-by: SeongJae Park --- mm/madvise.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index dbc8fec05cc6..4a91590656dc 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1633,6 +1633,17 @@ static bool is_valid_madvise(unsigned long start, si= ze_t len_in, int behavior) return true; } =20 +static bool is_memory_populate(int behavior) +{ + switch (behavior) { + case MADV_POPULATE_READ: + case MADV_POPULATE_WRITE: + return true; + default: + return false; + } +} + static int madvise_do_behavior(struct mm_struct *mm, unsigned long start, size_t len_in, size_t len, int behavior) { @@ -1646,16 +1657,11 @@ static int madvise_do_behavior(struct mm_struct *mm, end =3D start + len; =20 blk_start_plug(&plug); - switch (behavior) { - case MADV_POPULATE_READ: - case MADV_POPULATE_WRITE: + if (is_memory_populate(behavior)) error =3D madvise_populate(mm, start, end, behavior); - break; - default: + else error =3D madvise_walk_vmas(mm, start, end, behavior, madvise_vma_behavior); - break; - } blk_finish_plug(&plug); return error; } --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D38EB1C5D4E for ; Wed, 5 Mar 2025 18:16:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198583; cv=none; b=k2ht3Nijnof4XNTF2iV5HYuA+I6r5TJlK/0n+toAaJIW10j64u3UfOhBv4IRi2Gx3fgzaiOX2WVxLfx9ZKK/tNUB/Tl66dNZZpX7a6FFmLuklpoWP2sZ+/4GzKseh/a79GVwAUasWYAVkUlo2RZQJSMDkn5hejS0vO38i4fcsLw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198583; c=relaxed/simple; bh=Hf/IsCKuh49/H9FxeO83Nv8vZIFMcDB1quEptKUrTMY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UxFtrMUjxPSa+gtMmiXvvEUvGBlTcgFUEmKPG2PEs9RfdCV3FovKyuSuoeLvI6Dxrr6nm4IMz+mVEyqLaZYFAFlrg2QqDmiO+YCs5KR+NlJZdj1zVS9cnmkYiR5hh2w6w6b0leVNAETaBHWs0t7j6dds5AI0416AblZqBr3cBIg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ia7q5BFn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ia7q5BFn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6F099C4CEE2; Wed, 5 Mar 2025 18:16:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198583; bh=Hf/IsCKuh49/H9FxeO83Nv8vZIFMcDB1quEptKUrTMY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ia7q5BFnvBQEdZiYG/J9924zrl9GGOQcILKyQtO2540WBcXGDi74PQbLvgXupSOUU uRq2YOJyjf13O6CJrtwZp7ybsYd/NWISFt06+mKck3lNr1V6HS+rsBPAtfhYtxAV/m 9mGzaCAZQ1VmtWZETYtdNHp93km18Jza5GUnZc8azzucVpDXHQay5QjAfaQ9Rxa6tQ ZEBt05yTnUAhWgCFjAhWlkb4Al10XBa5hpCLWeZ9fUMHELUlrz/Wn7MbpZTN5XSDOz rqOnzF+7m8gCiV6mCEjnfk01x6pPUjmbHD2tvZReVbxf58LtttlkbasvRkWlCQLwkV 8w93AORgrrNRQ== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 03/16] mm/madvise: deduplicate madvise_do_behavior() skip case handlings Date: Wed, 5 Mar 2025 10:15:58 -0800 Message-Id: <20250305181611.54484-4-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The logic for checking if a given madvise() request for a single memory range can skip real work, namely madvise_do_behavior(), is duplicated in do_madvise() and vector_madvise(). Split out the logic to a function and resue it. Signed-off-by: SeongJae Park --- mm/madvise.c | 53 +++++++++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 4a91590656dc..265b325d8829 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1633,6 +1633,27 @@ static bool is_valid_madvise(unsigned long start, si= ze_t len_in, int behavior) return true; } =20 +/* + * madvise_should_skip() - Return if an madivse request can skip real work= s. + * @start: Start address of madvise-requested address range. + * @len_in: Length of madvise-requested address range. + * @behavior: Requested madvise behavor. + * @err: Pointer to store an error code from the check. + */ +static bool madvise_should_skip(unsigned long start, size_t len_in, + int behavior, int *err) +{ + if (!is_valid_madvise(start, len_in, behavior)) { + *err =3D -EINVAL; + return true; + } + if (start + PAGE_ALIGN(len_in) =3D=3D start) { + *err =3D 0; + return true; + } + return false; +} + static bool is_memory_populate(int behavior) { switch (behavior) { @@ -1740,23 +1761,15 @@ static int madvise_do_behavior(struct mm_struct *mm, */ int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, i= nt behavior) { - unsigned long end; int error; - size_t len; - - if (!is_valid_madvise(start, len_in, behavior)) - return -EINVAL; - - len =3D PAGE_ALIGN(len_in); - end =3D start + len; - - if (end =3D=3D start) - return 0; =20 + if (madvise_should_skip(start, len_in, behavior, &error)) + return error; error =3D madvise_lock(mm, behavior); if (error) return error; - error =3D madvise_do_behavior(mm, start, len_in, len, behavior); + error =3D madvise_do_behavior(mm, start, len_in, PAGE_ALIGN(len_in), + behavior); madvise_unlock(mm, behavior); =20 return error; @@ -1783,19 +1796,13 @@ static ssize_t vector_madvise(struct mm_struct *mm,= struct iov_iter *iter, while (iov_iter_count(iter)) { unsigned long start =3D (unsigned long)iter_iov_addr(iter); size_t len_in =3D iter_iov_len(iter); - size_t len; - - if (!is_valid_madvise(start, len_in, behavior)) { - ret =3D -EINVAL; - break; - } + int error; =20 - len =3D PAGE_ALIGN(len_in); - if (start + len =3D=3D start) - ret =3D 0; + if (madvise_should_skip(start, len_in, behavior, &error)) + ret =3D error; else - ret =3D madvise_do_behavior(mm, start, len_in, len, - behavior); + ret =3D madvise_do_behavior(mm, start, len_in, + PAGE_ALIGN(len_in), behavior); /* * An madvise operation is attempting to restart the syscall, * but we cannot proceed as it would not be correct to repeat --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC1E9253B67 for ; Wed, 5 Mar 2025 18:16:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198584; cv=none; b=F2hD6ILZKLXSi5UXrEyekUAWoqQ3KGZDoCXi2nAeZsjdA5setm+IwMdJFrcTIB2kyQL0NYBtgLbyyCI4mVH8fodwN9HJwTHwqm3puagzPPNhVHWtyqZuoAczd0dLfhIjwq8YskRfUChL1Y+ujRomDoJMuyWtUH2HtD92brNREkI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198584; c=relaxed/simple; bh=4+ggVfIvFgjcdAiiBX8MuJhj0bQWEcWPio982NmoaDE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rR2Jr1uH5M67WSX/hcRfl4Ptxlze2K7XoWJ7H9JtRxY0b43WSdUvkrzGqps9VtWDdPXzMclWKALfYFGpMOKyOkdttb5c0xinUeAup9lRHIW6GbvcQlYvcDoIcWFdLY9zlbOSMEdg7E9AoarH7Y+qvCdZMBdQT79OXkVP5AJaUqs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XifHhBB9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XifHhBB9" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93788C4CEE9; Wed, 5 Mar 2025 18:16:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198584; bh=4+ggVfIvFgjcdAiiBX8MuJhj0bQWEcWPio982NmoaDE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XifHhBB9EGx7WTdY5Gq0ixSMzm4RkQK0ZqOZ9h5RteCQnJpOLL7PXW1hXZgCieJA9 bwRB8sDJVdiADPtwYVrSaV16QQDvsSjJXn+pyfHS0EDC707FUaUPhiSjwVtM3Oh0VI 9TwicXQjUUfeNlDPCuG4B+7B6aoA7+GLOw+mIhbRTl1E3iXz9ucdlFnYQXLTfXOSkM VSAmDDuARX2Rkz42f/sh9oUZ4giZ4AJ7MkwEZC95GbWgJq0KLCQP+qH0x17J9/cn0M pq1OiyZJ1eNsUiztLw9lmF4XyWY7x7T6q8DSyrjAxc9DQmXHpz62zgUp8YpDW+2c5/ XDl0CABH8E5Uw== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 04/16] mm/madvise: remove len parameter of madvise_do_behavior() Date: Wed, 5 Mar 2025 10:15:59 -0800 Message-Id: <20250305181611.54484-5-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Because madise_should_skip() logic is factored out, making madvise_do_behavior() calculates 'len' on its own rather then receiving it as a parameter makes code simpler. Remove the parameter. Signed-off-by: SeongJae Park --- mm/madvise.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 265b325d8829..c5e1a4d1df72 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1666,7 +1666,7 @@ static bool is_memory_populate(int behavior) } =20 static int madvise_do_behavior(struct mm_struct *mm, - unsigned long start, size_t len_in, size_t len, int behavior) + unsigned long start, size_t len_in, int behavior) { struct blk_plug plug; unsigned long end; @@ -1675,7 +1675,7 @@ static int madvise_do_behavior(struct mm_struct *mm, if (is_memory_failure(behavior)) return madvise_inject_error(behavior, start, start + len_in); start =3D untagged_addr_remote(mm, start); - end =3D start + len; + end =3D start + PAGE_ALIGN(len_in); =20 blk_start_plug(&plug); if (is_memory_populate(behavior)) @@ -1768,8 +1768,7 @@ int do_madvise(struct mm_struct *mm, unsigned long st= art, size_t len_in, int beh error =3D madvise_lock(mm, behavior); if (error) return error; - error =3D madvise_do_behavior(mm, start, len_in, PAGE_ALIGN(len_in), - behavior); + error =3D madvise_do_behavior(mm, start, len_in, behavior); madvise_unlock(mm, behavior); =20 return error; @@ -1801,8 +1800,7 @@ static ssize_t vector_madvise(struct mm_struct *mm, s= truct iov_iter *iter, if (madvise_should_skip(start, len_in, behavior, &error)) ret =3D error; else - ret =3D madvise_do_behavior(mm, start, len_in, - PAGE_ALIGN(len_in), behavior); + ret =3D madvise_do_behavior(mm, start, len_in, behavior); /* * An madvise operation is attempting to restart the syscall, * but we cannot proceed as it would not be correct to repeat --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D4CD253F09 for ; Wed, 5 Mar 2025 18:16:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198586; cv=none; b=XttIthAKQ/hWpgzraz0Ozt2NTAGB3+fCuTVkEp4R3xfKTzH/ckJLTTS47WbEC7Ck8QAf/N2eIoLDGVigg915QiYC+sS0ky5XyTQatN3O7wjuVrmLZCxdGpOqmKToaSo+0J5XJOBHbkyi1tS3F3Oz0HFgWGtasRlNMc8aBE+iiY8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198586; c=relaxed/simple; bh=zA6hArayCI2Rl1M1lVh5I5ZUimJAVrPkFcOBSQrdmgA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mq49T+4lXD+aDq4nsrqA6Mzz7Ej53AyHWsym3+7LBmMUT3fj0vudVEKpL/Z0+lzaEh5KCHE4sqGgW/OZ8OrjveZm7SO1Y+8mKDeI4IHzK8f1Iidge5eQamLYb47tyCD1xUIaIszspGrcznQaGLMVcYLk5kObz64aDi2tSFIr9VY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tuw7cR1M; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tuw7cR1M" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B9685C4CED1; Wed, 5 Mar 2025 18:16:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198585; bh=zA6hArayCI2Rl1M1lVh5I5ZUimJAVrPkFcOBSQrdmgA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tuw7cR1MHr5aXndjIuHntuhVG2we1MfPpIbDEiWcfzdWAVCjwriV5BA9ak4kJ43J5 FqOuVr5sM11QSKbSAQgOlmUL/KObNY95VB9FOfZr/6/PE2woIGfHoPq/nHYSGSzkjv 14y/WzbzFk3iklvMwASgDW5TnelJWbMGFd2jponx0g3PkCxDEwXsXbr4Y0c+/OkowP b4J1RfkKX3qpb6iPvKLdJpQYNDNJ8xYZJzm/xv/l24+SW8A5s8VU3AkpxnrVVLoxca Ge4I3F8AKetaxegjk6tf6YcIx1EdcyWgY6x2uyisSlmZ5wpedBfyqHKuLaw8dd2jcL tQw7ln6j3BFgQ== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 05/16] mm/madvise: define and use madvise_behavior struct for madvise_do_behavior() Date: Wed, 5 Mar 2025 10:16:00 -0800 Message-Id: <20250305181611.54484-6-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To implement batched tlb flushes for MADV_DONTNEED[_LOCKED] and MADV_FREE, an mmu_gather object in addition to the behavior integer need to be passed to the internal logics. Define a struct for passing such information together with the behavior value without increasing number of parameters of all code paths towards the internal logic. Signed-off-by: SeongJae Park --- mm/madvise.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index c5e1a4d1df72..3346e593e07d 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1665,9 +1665,15 @@ static bool is_memory_populate(int behavior) } } =20 +struct madvise_behavior { + int behavior; +}; + static int madvise_do_behavior(struct mm_struct *mm, - unsigned long start, size_t len_in, int behavior) + unsigned long start, size_t len_in, + struct madvise_behavior *madv_behavior) { + int behavior =3D madv_behavior->behavior; struct blk_plug plug; unsigned long end; int error; @@ -1762,13 +1768,14 @@ static int madvise_do_behavior(struct mm_struct *mm, int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, i= nt behavior) { int error; + struct madvise_behavior madv_behavior =3D {.behavior =3D behavior}; =20 if (madvise_should_skip(start, len_in, behavior, &error)) return error; error =3D madvise_lock(mm, behavior); if (error) return error; - error =3D madvise_do_behavior(mm, start, len_in, behavior); + error =3D madvise_do_behavior(mm, start, len_in, &madv_behavior); madvise_unlock(mm, behavior); =20 return error; @@ -1785,6 +1792,7 @@ static ssize_t vector_madvise(struct mm_struct *mm, s= truct iov_iter *iter, { ssize_t ret =3D 0; size_t total_len; + struct madvise_behavior madv_behavior =3D {.behavior =3D behavior}; =20 total_len =3D iov_iter_count(iter); =20 @@ -1800,7 +1808,8 @@ static ssize_t vector_madvise(struct mm_struct *mm, s= truct iov_iter *iter, if (madvise_should_skip(start, len_in, behavior, &error)) ret =3D error; else - ret =3D madvise_do_behavior(mm, start, len_in, behavior); + ret =3D madvise_do_behavior(mm, start, len_in, + &madv_behavior); /* * An madvise operation is attempting to restart the syscall, * but we cannot proceed as it would not be correct to repeat --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8761E253F3C for ; Wed, 5 Mar 2025 18:16:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198587; cv=none; b=cfcLxbvlWm3ZnKroQl4T/9RXnqlU/Byq6+p3p6huwVnDlSCqTFO8yx2BGPzWqCX7/mFkc6xqitVX8cLdfHg/OaqajLsIzkHpzSmdbr7+xZbiQQYmakcd1e9ZMclorv8gdbD9DZ+buV5vKHwAOxxcyShGIV+70JU6WBzUGviDy7Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198587; c=relaxed/simple; bh=OqLt2zId8LUFKARGxIVzcXznGs9Ytza2liDWujZpc8Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gOYapv8y9pHJ7rR7Y6WK/3aiAGMVYKhjsUpq8OnEhJC9bpQ5u2XkGom4BD6mNhVap/gklZtQyB/620lDvDJhw68njWd3bHKp0u4Fb6F5rb9YMDslUqXF5sujnHlJ90gmU2lKX/xpf26vtjGtZcU0AOg+USW1qo68Fx6T5V+6JeQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=E5tb1b18; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="E5tb1b18" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7FBFC4CED1; Wed, 5 Mar 2025 18:16:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198587; bh=OqLt2zId8LUFKARGxIVzcXznGs9Ytza2liDWujZpc8Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E5tb1b18FoCnv4378bvS3h3FozmbWmysT2y3RwuHzECCmJPFN9mtaSFgXIRlhhSQP NyjxjtXqPftmaLx3OTJhNzZNB0lE/f9exNaPLlRczJhYQ6ln/Xq8ZwyeZ6Aj/xk1NJ HBuMny/ZCKifdmvK3jAb65tdp0uY5mnirM6RyPEaonb9YW7JmIg5W24imxDPppoKJw hC5hGKAVqNNVLiH1G3FPg/mRTHxy+fpG/IvWx5jvkF5zrGVDT0R1f8vW4LJiEZBf5T fQmHOeNyU5pRA/ixwasJ+S2kiqveFW4zNS2NpGcWVuuYUbTlC6zcHerl3AXN0y+BOJ VPl9SODsmawoA== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 06/16] mm/madvise: pass madvise_behavior struct to madvise_vma_behavior() Date: Wed, 5 Mar 2025 10:16:01 -0800 Message-Id: <20250305181611.54484-7-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Most internal madvise logics are executed by madvise_walk_vmas() visit function, namely madvise_vma_behavior(). The function receives only a single behavior value, so difficult to extend for tlb flushes batching and potential future optimizations that may require information more than single behavior value. Modify it to receive 'struct madvise_behavior' object instead, for upcoming tlb flushes batching change and possible future optimizations. Signed-off-by: SeongJae Park --- mm/madvise.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 3346e593e07d..8c4c128eaeb7 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1241,6 +1241,10 @@ static long madvise_guard_remove(struct vm_area_stru= ct *vma, &guard_remove_walk_ops, NULL); } =20 +struct madvise_behavior { + int behavior; +}; + /* * Apply an madvise behavior to a region of a vma. madvise_update_vma * will handle splitting a vm area into separate areas, each area with its= own @@ -1249,8 +1253,10 @@ static long madvise_guard_remove(struct vm_area_stru= ct *vma, static int madvise_vma_behavior(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, unsigned long end, - unsigned long behavior) + unsigned long behavior_arg) { + struct madvise_behavior *arg =3D (struct madvise_behavior *)behavior_arg; + int behavior =3D arg->behavior; int error; struct anon_vma_name *anon_name; unsigned long new_flags =3D vma->vm_flags; @@ -1665,10 +1671,6 @@ static bool is_memory_populate(int behavior) } } =20 -struct madvise_behavior { - int behavior; -}; - static int madvise_do_behavior(struct mm_struct *mm, unsigned long start, size_t len_in, struct madvise_behavior *madv_behavior) @@ -1687,7 +1689,8 @@ static int madvise_do_behavior(struct mm_struct *mm, if (is_memory_populate(behavior)) error =3D madvise_populate(mm, start, end, behavior); else - error =3D madvise_walk_vmas(mm, start, end, behavior, + error =3D madvise_walk_vmas(mm, start, end, + (unsigned long)madv_behavior, madvise_vma_behavior); blk_finish_plug(&plug); return error; --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E39D254860 for ; Wed, 5 Mar 2025 18:16:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198588; cv=none; b=U+JQ+BYq4vuSWTnmhdG8oZi6CYHOdgGJVCCx4qYbzrxfGDTqa1UWRE+cpzIob6RrE/HLXyqN/Pwa1jaWmMXlErDdkgtgU5qQwjZLVTdYeYrHEKV0Lw/PFtZUhCMas/4aZtulCpUemL/UrizznKr0XUgTeVnaj8/I7VlA2VKE7VI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198588; c=relaxed/simple; bh=3LG3O8N+F8TPPzR8Mb+NGODzxzconMM3ZSxJjGwNDUk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=n28+wb0yq0740JVdFeAnQBMBrqqxOEKq7ypTiZqp6iaUBwEQEZRAtqO+8TS8caSJlVluaI9ZhZ5qqz0XV9WP+Yay3/xbU6mfSFgktdbufO0DPyT8V2CMWYAEpyyUXHaokpXHVwFaGZwGo/EjpSNUL2z7xYU8PtZD9uys4W1dGrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NJNXUPRx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NJNXUPRx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E25D9C4CED1; Wed, 5 Mar 2025 18:16:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198588; bh=3LG3O8N+F8TPPzR8Mb+NGODzxzconMM3ZSxJjGwNDUk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NJNXUPRxqjS1gIwtMlxNbHOdcW9b79WtuUL9PyyAEjlIn+Vmdx0iNdtdhmifJPV/R GCVbmonJu8xRM/kk1GlHomD/e5U5XgM1Mb4hpN+hhnvzr0gKLqLDQ6Kgx4D530bHXB Gme7DF42kCSOhJa6YAaCSpl/2/uZLtNh3bZ4RcoB8CGYExP2VclEe8CSPjg7C7oPfq 0CWC2UoS3J+VTBFxclXBq8041eB+Tr3mNqQTsnpMPctCSCNU8Caa8ZF3vNxC4wlPuT Xf7drlusvq8z2sRG9d+2OvVKAG+6QfWdADuPevd6T8pyRWvg80ned9ISgfgActsxyB dJGUsCap7Ys/g== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 07/16] mm/madvise: make madvise_walk_vmas() visit function receives a void pointer Date: Wed, 5 Mar 2025 10:16:02 -0800 Message-Id: <20250305181611.54484-8-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_walk_vmas() is used for two visit functions, namely madvise_vma_anon_name() and madvise_walk_vmas(). The visit function type is defined to receive an 'unsigned long' type argument. But, both visit functions need a pointer argument, so casting the arguments between 'unsinged long' and the real pointer types. It is more idiomatic and clean to use a void pointer type for such cases. Update the visit function type to receive a void pointer as the argument and cleanup the type-casting code. Signed-off-by: SeongJae Park --- mm/madvise.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 8c4c128eaeb7..6fa7dabe5bad 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1253,9 +1253,9 @@ struct madvise_behavior { static int madvise_vma_behavior(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, unsigned long end, - unsigned long behavior_arg) + void *behavior_arg) { - struct madvise_behavior *arg =3D (struct madvise_behavior *)behavior_arg; + struct madvise_behavior *arg =3D behavior_arg; int behavior =3D arg->behavior; int error; struct anon_vma_name *anon_name; @@ -1468,10 +1468,10 @@ static bool process_madvise_remote_valid(int behavi= or) */ static int madvise_walk_vmas(struct mm_struct *mm, unsigned long start, - unsigned long end, unsigned long arg, + unsigned long end, void *arg, int (*visit)(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, - unsigned long end, unsigned long arg)) + unsigned long end, void *arg)) { struct vm_area_struct *vma; struct vm_area_struct *prev; @@ -1529,7 +1529,7 @@ int madvise_walk_vmas(struct mm_struct *mm, unsigned = long start, static int madvise_vma_anon_name(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, unsigned long end, - unsigned long anon_name) + void *anon_name) { int error; =20 @@ -1538,7 +1538,7 @@ static int madvise_vma_anon_name(struct vm_area_struc= t *vma, return -EBADF; =20 error =3D madvise_update_vma(vma, prev, start, end, vma->vm_flags, - (struct anon_vma_name *)anon_name); + anon_name); =20 /* * madvise() returns EAGAIN if kernel resources, such as @@ -1570,7 +1570,7 @@ int madvise_set_anon_name(struct mm_struct *mm, unsig= ned long start, if (end =3D=3D start) return 0; =20 - return madvise_walk_vmas(mm, start, end, (unsigned long)anon_name, + return madvise_walk_vmas(mm, start, end, anon_name, madvise_vma_anon_name); } #endif /* CONFIG_ANON_VMA_NAME */ @@ -1689,8 +1689,7 @@ static int madvise_do_behavior(struct mm_struct *mm, if (is_memory_populate(behavior)) error =3D madvise_populate(mm, start, end, behavior); else - error =3D madvise_walk_vmas(mm, start, end, - (unsigned long)madv_behavior, + error =3D madvise_walk_vmas(mm, start, end, madv_behavior, madvise_vma_behavior); blk_finish_plug(&plug); return error; --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A7F925484D for ; Wed, 5 Mar 2025 18:16:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198589; cv=none; b=WUAZAKIrXYoZHt3SlUM3cFrvzVfyxAPrPhHLo0aZqnl5JVXsfnrXY/vp+bYQ56u/3cEFXyRI2c6ZIes4HPuI6yMt9dm2o2egkr8bAcUQEbTlFIDe3vlL/IOdkc+L1huZoQy/bv4pFY/WgEjrkR5hOqKyhQc60hzGBOqWXr/VzHA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198589; c=relaxed/simple; bh=5ks7fx/Q1sCl0dg2a2snRfWoft6X+IgkXl2rMy0gCGQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qVXi5gBNsgLDub0zqgIVS0wX39FIB5OQlI6FlualoABdBEFV6CNFNbwX6UvaCCfuZydTEMLVNaGPiCAcsMM5g6lmGrAXaj/vnBx695ee4UNHdgJZabkp5T5bgLz6ABJ1pkzjsNYQqN2fOssIepyaqkcSizDqORCpNtdUBddbe+E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sN32lAfO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sN32lAfO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 16D77C4CEE2; Wed, 5 Mar 2025 18:16:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198589; bh=5ks7fx/Q1sCl0dg2a2snRfWoft6X+IgkXl2rMy0gCGQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sN32lAfOUBUATYQrwl4BKcRsrxm4mSQ3kaYsz2Z0kSriFn1jgHlzV2U6C0xBm4hsD 7coxL6aUxhuXz8dmZDGAsaUAqHV7ZEehscytuEa1vzbq5q8IyAtkZv7VEsjmKuJud/ nTE3QtKiSWsnOMNuDEtJc+rEEuERdsBPbstKTMn31F18ss5rnYWoEV1kGzrwhf9So+ Oh78y+Kz0NuzwnB6DhGgCOqIyb7sWaPAX/f+xGUuDF9HqCWId0tYJK2QPmxz+eb4WU oQOBVsBu52cCCIl3YKWmT+1GhR+dupStWnrMDZwC2FnOPaLGckhmNZGnmQWJRAYYG1 kn3rTbJOOUrVA== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 08/16] mm/madvise: pass madvise_behavior struct to madvise_dontneed_free() Date: Wed, 5 Mar 2025 10:16:03 -0800 Message-Id: <20250305181611.54484-9-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_dontneed_free() can be optimized to batch tlb flushes by receiving mmu_gather object that initialized and finished outside, and doing only gathering of tlb entries that need to be flushed into the received mmu_gather object. But the function is receiving only one single behavior integer value. Update it to receive 'struct madvise_behavior' object so that upcoming tlb flushes batching optimization can pass the mmu_gather object. Signed-off-by: SeongJae Park --- mm/madvise.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 6fa7dabe5bad..73a4323197e2 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -890,11 +890,16 @@ static bool madvise_dontneed_free_valid_vma(struct vm= _area_struct *vma, return true; } =20 +struct madvise_behavior { + int behavior; +}; + static long madvise_dontneed_free(struct vm_area_struct *vma, struct vm_area_struct **prev, unsigned long start, unsigned long end, - int behavior) + struct madvise_behavior *madv_behavior) { + int behavior =3D madv_behavior->behavior; struct mm_struct *mm =3D vma->vm_mm; =20 *prev =3D vma; @@ -1241,10 +1246,6 @@ static long madvise_guard_remove(struct vm_area_stru= ct *vma, &guard_remove_walk_ops, NULL); } =20 -struct madvise_behavior { - int behavior; -}; - /* * Apply an madvise behavior to a region of a vma. madvise_update_vma * will handle splitting a vm area into separate areas, each area with its= own @@ -1276,7 +1277,7 @@ static int madvise_vma_behavior(struct vm_area_struct= *vma, case MADV_FREE: case MADV_DONTNEED: case MADV_DONTNEED_LOCKED: - return madvise_dontneed_free(vma, prev, start, end, behavior); + return madvise_dontneed_free(vma, prev, start, end, arg); case MADV_NORMAL: new_flags =3D new_flags & ~VM_RAND_READ & ~VM_SEQ_READ; break; --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CF1F255238 for ; Wed, 5 Mar 2025 18:16:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198590; cv=none; b=f32vVZvQR9nUUXY3+qxvQmps6IZ5SYHhk7oI3YbM7vnTd51YcaSBrGJFzGlb7VHcwmUEM4vR96YdmOfyY6jJJLeYyWW7wOkHgCX1MHqKNObAQEZyEHmU7hWfrc0dCfuQ02ikz8nLg5UFpzBhNpUGgFJmJFgBcgCevgCPJE1ec6Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198590; c=relaxed/simple; bh=2xphIriEKA8i5rp9aYhxLxFxGr/HuEVyUJQbKTIJ9W0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YNZIk18IpYZzvRacujPoT6BziVboX4ZsTWBuRozEE2CytdcPwtLFlTdWct96FopA4HkB2fy9N6LHNw/1/xB3BKnOwGOkVzdGtbS5Kc1eBPnN+dE6OcI5iqA/6e5MPdrhKHy+O/29QYikRwEa0Wz/3bpipwpI+dqrtF2dnkSMcxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sIdVI7Ez; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sIdVI7Ez" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3D2FFC4CEEC; Wed, 5 Mar 2025 18:16:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198590; bh=2xphIriEKA8i5rp9aYhxLxFxGr/HuEVyUJQbKTIJ9W0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sIdVI7Ezc8/ZF2n0e2HlbbzQczYnvzGI5HusymbHblTUb8ZINUYydPwmj3Ni8dGNU QylfDQt4OmqKTwR0TDaBNDmpHfEvCzdEbqbG+yzY8YFAbZ2JpoKjQYMf//LdaQIXjX YrN7k4OweALQuFfzViR1+pHQmbV+YkvlqVzz2NEVMRkCRUvWciCj/FXV5PmE23qEaJ 3a2AlWAM7CkaK/nFKTlG154acKna2tJovPZeBSrDc/OMc4qnOIR5detGdZsRiJ7cCa DrCp0ukPkgd8e/ouCQ2KiOQXvIFe8tTgEil7tRhrPP7G9gIvHyQSPx54gaSXTp7HJo orNrTtIB5HhqQ== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 09/16] mm/memory: split non-tlb flushing part from zap_page_range_single() Date: Wed, 5 Mar 2025 10:16:04 -0800 Message-Id: <20250305181611.54484-10-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some of zap_page_range_single() callers such as [process_]madvise() with MADV_DONEED[_LOCKED] cannot batch tlb flushes because zap_page_range_single() does tlb flushing for each invocation. Split out core part of zap_page_range_single() except mmu_gather object initialization and gathered tlb entries flushing part for such batched tlb flushing usage. Signed-off-by: SeongJae Park --- mm/memory.c | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a838c8c44bfd..aadb2844c701 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2011,38 +2011,46 @@ void unmap_vmas(struct mmu_gather *tlb, struct ma_s= tate *mas, mmu_notifier_invalidate_range_end(&range); } =20 -/** - * zap_page_range_single - remove user pages in a given range - * @vma: vm_area_struct holding the applicable pages - * @address: starting address of pages to zap - * @size: number of bytes to zap - * @details: details of shared cache invalidation - * - * The range must fit into one VMA. - */ -void zap_page_range_single(struct vm_area_struct *vma, unsigned long addre= ss, +static void unmap_vma_single(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long address, unsigned long size, struct zap_details *details) { const unsigned long end =3D address + size; struct mmu_notifier_range range; - struct mmu_gather tlb; =20 mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma->vm_mm, address, end); hugetlb_zap_begin(vma, &range.start, &range.end); - tlb_gather_mmu(&tlb, vma->vm_mm); update_hiwater_rss(vma->vm_mm); mmu_notifier_invalidate_range_start(&range); /* * unmap 'address-end' not 'range.start-range.end' as range * could have been expanded for hugetlb pmd sharing. */ - unmap_single_vma(&tlb, vma, address, end, details, false); + unmap_single_vma(tlb, vma, address, end, details, false); mmu_notifier_invalidate_range_end(&range); - tlb_finish_mmu(&tlb); hugetlb_zap_end(vma, details); } =20 +/** + * zap_page_range_single - remove user pages in a given range + * @vma: vm_area_struct holding the applicable pages + * @address: starting address of pages to zap + * @size: number of bytes to zap + * @details: details of shared cache invalidation + * + * The range must fit into one VMA. + */ +void zap_page_range_single(struct vm_area_struct *vma, unsigned long addre= ss, + unsigned long size, struct zap_details *details) +{ + struct mmu_gather tlb; + + tlb_gather_mmu(&tlb, vma->vm_mm); + unmap_vma_single(&tlb, vma, address, size, details); + tlb_finish_mmu(&tlb); +} + /** * zap_vma_ptes - remove ptes mapping the vma * @vma: vm_area_struct holding ptes to be zapped --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95B91255E3D for ; Wed, 5 Mar 2025 18:16:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198591; cv=none; b=E+3m/m5ZaU2KbOXTE+XprAxXIX4eNJym232XUMQzp5mj2Vg1E3kwwQ0graIIhEPnMtygLTRidLqUDJHOMDdSwTVqY5ezye/cMCKQkZRbDaLniEBFuI6a6qcsq0q16kUE9a5YDtU8lI1p6VALGAoApQv1bBcHOqfELhkVNOVH34g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198591; c=relaxed/simple; bh=qTz2EADhHkd12Pdb2rgzQeU5lnQ9KkIcBg1xzIMxfqw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KWN6oir9GFiNktCoNiF+7pnSUhMVSGAbrty6id2ZEbDeEdCmVsD2L21EbVNKotEzhXhB6vboHOm5cv7URAfpjU1IJGIpx5eH0lgj+cqPjoFDqOQxDnW4+XB/M2MsQ7xlgpXWAwk6B50FZzQqlM+3NgKBJNITKKGce4r0pAEsBKo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CgRqfs2N; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CgRqfs2N" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B554C4CED1; Wed, 5 Mar 2025 18:16:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198591; bh=qTz2EADhHkd12Pdb2rgzQeU5lnQ9KkIcBg1xzIMxfqw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CgRqfs2NseUkix5vrYEws4rsZfUtLGUzUiaQKz9cL3cUczbHcnhtImfhyuuyEter5 jReIK8APFWJgSQCwIor9x6PTZBSTjB3bEwYOXEQoO82GoaOItFlnPpmJ55nrb4pQEH Icz+IKD6OzUoGVchOBRUXmwZXhu8OTsKyeufqiejQsO9/JLFngmaTLNNy8cgBvvS6m eGZW+6snKE73VeZUE3xG4Q1vGmh0GAvWhjtQYXwCTzYQGH2pSqKvHU2zNH1KHyKUsi 9XZXa7koL3kvZxHcyftsU8YgULJP9WVJIazYRyRFmjTe4aa1mo4c2FD1hkRvwREDeX TV52sNWDWuA3w== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 10/16] mm/madvise: let madvise_dontneed_single_vma() caller batches tlb flushes Date: Wed, 5 Mar 2025 10:16:05 -0800 Message-Id: <20250305181611.54484-11-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Update madvise_dontneed_single_vma() function so that the caller can pass an mmu_gather object that should be initialized and will be finished outside, for batched tlb flushes. Also modify madvise_dontneed_single_vma() internal code to support such usage by skipping the initialization and finishing of self-allocated mmu_gather object if it received a valid mmu_gather object. Signed-off-by: SeongJae Park --- mm/internal.h | 3 +++ mm/madvise.c | 10 +++++++--- mm/memory.c | 6 +++--- 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 0413822ee34b..45b8d08cd176 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -438,6 +438,9 @@ void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long addr, unsigned long end, struct zap_details *details); +void unmap_vma_single(struct mmu_gather *tlb, struct vm_area_struct *vma, + unsigned long addr, unsigned long size, + struct zap_details *details); int folio_unmap_invalidate(struct address_space *mapping, struct folio *fo= lio, gfp_t gfp); =20 diff --git a/mm/madvise.c b/mm/madvise.c index 73a4323197e2..22da6699613c 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -848,7 +848,8 @@ static int madvise_free_single_vma(struct vm_area_struc= t *vma, * An interface that causes the system to free clean pages and flush * dirty pages is already available as msync(MS_INVALIDATE). */ -static long madvise_dontneed_single_vma(struct vm_area_struct *vma, +static long madvise_dontneed_single_vma(struct mmu_gather *tlb, + struct vm_area_struct *vma, unsigned long start, unsigned long end) { struct zap_details details =3D { @@ -856,7 +857,10 @@ static long madvise_dontneed_single_vma(struct vm_area= _struct *vma, .even_cows =3D true, }; =20 - zap_page_range_single(vma, start, end - start, &details); + if (!tlb) + zap_page_range_single(vma, start, end - start, &details); + else + unmap_vma_single(tlb, vma, start, end - start, &details); return 0; } =20 @@ -951,7 +955,7 @@ static long madvise_dontneed_free(struct vm_area_struct= *vma, } =20 if (behavior =3D=3D MADV_DONTNEED || behavior =3D=3D MADV_DONTNEED_LOCKED) - return madvise_dontneed_single_vma(vma, start, end); + return madvise_dontneed_single_vma(NULL, vma, start, end); else if (behavior =3D=3D MADV_FREE) return madvise_free_single_vma(vma, start, end); else diff --git a/mm/memory.c b/mm/memory.c index aadb2844c701..ba011c064936 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2011,9 +2011,9 @@ void unmap_vmas(struct mmu_gather *tlb, struct ma_sta= te *mas, mmu_notifier_invalidate_range_end(&range); } =20 -static void unmap_vma_single(struct mmu_gather *tlb, - struct vm_area_struct *vma, unsigned long address, - unsigned long size, struct zap_details *details) +void unmap_vma_single(struct mmu_gather *tlb, struct vm_area_struct *vma, + unsigned long address, unsigned long size, + struct zap_details *details) { const unsigned long end =3D address + size; struct mmu_notifier_range range; --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADBED2561B0 for ; Wed, 5 Mar 2025 18:16:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198592; cv=none; b=q6/c37NOtv843TryhoiPmr/f+8qCG4AUb1y/8r4szzsxRvXr6Zsj3rJOe/i6CSgcSXcqRReEChdxb3tBdmb/OlkyU8AB7dCVcrK9LZWdyapPeneCtCQ4BckNxbuFHV3Eqmk8WafJl+fIyTwxGKO1NAY1XqAp8yK2BlddYiHEomk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198592; c=relaxed/simple; bh=VbOLwUfPS96J+Dn4D6VO7VbIh87bUFOgKgh9tH90XtA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nKcWq83w8gkyoGYgPjU74ZpjCN+OPRFIcW0ce6p8cGjB4bxFp/Lvq6YlOTRBMJW2CYzOsudoZ9AAmKAUkLyjJ/EEa0lvVxtKSBWql8mh7feiDup/iao0FNV0paKIDAdd2eEp0f7Kh7oXIrXOoRGjR3e0nwqqfnBhhzffanQFV6E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=r8L/NUdf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="r8L/NUdf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6302EC4CED1; Wed, 5 Mar 2025 18:16:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198592; bh=VbOLwUfPS96J+Dn4D6VO7VbIh87bUFOgKgh9tH90XtA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r8L/NUdfSFARyAX4ToIaUViXGLmoyIt6P5pSeStwqeYkAQ5qsal4uIh28/nnnWeWx BCNP3KLDIJahIo1NSGAj+CfpF7d8ZolvmgY0DtPbCg2fS8a5Jr4bjfaHQLGTS06Bq5 K8SKd17LjNI6OuV8m+WGpk3/duWYX2ANVJpKET0autEmll2LvI72K+80s7Mqi+3zR4 JsjMvJHCTEHS3ViA/E1bI3BUEz9eoCp1fO7kFQxq+HD581fSbgb9kQFooJJabZRMqH mgGf2F2R3P624GAUf9a+ZalbXy3HjYOkzBrTDBfiIOHw1woUG9LbBrJygjfOzltkJX Xd305rSU3wOQQ== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 11/16] mm/madvise: let madvise_free_single_vma() caller batches tlb flushes Date: Wed, 5 Mar 2025 10:16:06 -0800 Message-Id: <20250305181611.54484-12-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Update madvise_free_single_vma() function so that the caller can pass an mmu_gather object that should be initialized and will be finished outside, for batched tlb flushes. Also modify madvise_free_single_vma() internal code to support such usage while keeping support of olde usage that the mmu_gather object is not passed. Signed-off-by: SeongJae Park --- mm/madvise.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 22da6699613c..767c5d21ee75 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -794,12 +794,19 @@ static const struct mm_walk_ops madvise_free_walk_ops= =3D { .walk_lock =3D PGWALK_RDLOCK, }; =20 -static int madvise_free_single_vma(struct vm_area_struct *vma, - unsigned long start_addr, unsigned long end_addr) +static int madvise_free_single_vma( + struct mmu_gather *caller_tlb, struct vm_area_struct *vma, + unsigned long start_addr, unsigned long end_addr) { struct mm_struct *mm =3D vma->vm_mm; struct mmu_notifier_range range; - struct mmu_gather tlb; + struct mmu_gather self_tlb; + struct mmu_gather *tlb; + + if (caller_tlb) + tlb =3D caller_tlb; + else + tlb =3D &self_tlb; =20 /* MADV_FREE works for only anon vma at the moment */ if (!vma_is_anonymous(vma)) @@ -815,16 +822,18 @@ static int madvise_free_single_vma(struct vm_area_str= uct *vma, range.start, range.end); =20 lru_add_drain(); - tlb_gather_mmu(&tlb, mm); + if (!caller_tlb) + tlb_gather_mmu(tlb, mm); update_hiwater_rss(mm); =20 mmu_notifier_invalidate_range_start(&range); - tlb_start_vma(&tlb, vma); + tlb_start_vma(tlb, vma); walk_page_range(vma->vm_mm, range.start, range.end, - &madvise_free_walk_ops, &tlb); - tlb_end_vma(&tlb, vma); + &madvise_free_walk_ops, tlb); + tlb_end_vma(tlb, vma); mmu_notifier_invalidate_range_end(&range); - tlb_finish_mmu(&tlb); + if (!caller_tlb) + tlb_finish_mmu(tlb); =20 return 0; } @@ -957,7 +966,7 @@ static long madvise_dontneed_free(struct vm_area_struct= *vma, if (behavior =3D=3D MADV_DONTNEED || behavior =3D=3D MADV_DONTNEED_LOCKED) return madvise_dontneed_single_vma(NULL, vma, start, end); else if (behavior =3D=3D MADV_FREE) - return madvise_free_single_vma(vma, start, end); + return madvise_free_single_vma(NULL, vma, start, end); else return -EINVAL; } --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C8582566D7 for ; Wed, 5 Mar 2025 18:16:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198594; cv=none; b=dLSFYZ1dEvKzq6944FMrdND8f+A/cMqDfi+C9cOcnscswcCWqQqLT++1NIpJqARhV40288ia+hC2IUnxNZMsyvnCQjh/Jm8/FkilvZKu+dDqWzJD+BoXqvIuciBoPfAPdQrV/WL5s3wJLCFl7fvHFaPvkIm+T+d9WawehhjWFMI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198594; c=relaxed/simple; bh=jzL4THh0ZOpM9TD9jtctj6vlHRlDDc1QOi0WL+ouKWc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Yh6yzzOehdy+sOG2E+jnPNZXaoKmoSPIjvoSF3MX0/0xH0xc/T9OeYG3w2AJEXrWVmeR1XJhsc+zMFKyonptlIc391HuopPo0SH2t1o2kYXH3zEqfA4wMPaWX7MYqxZACF/VL4c271FL8/Ua02o6jMQpRqPb8HrnP0OfipKzcks= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dfAZgM5N; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dfAZgM5N" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DEBAC4CED1; Wed, 5 Mar 2025 18:16:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198593; bh=jzL4THh0ZOpM9TD9jtctj6vlHRlDDc1QOi0WL+ouKWc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dfAZgM5NRq4d1FSeYirYc97Aa37ub9mtL5+bYLp6su191iwtezpNxNXtZ9QKnYlfG ZbU4WvCz8p8tYxGgqYlbUlcF7P0PADFKnlM/EF+o3gGKkkEDqFvW6xeVSkOCtyYwx3 8m2pMPtJp+qwemce3LMOXfT2qVT5VDOjvkpFLvUpflQ0olmQBk+jZngEtXZcUxs4iA oMGEqY6WB+JVziG+jlXgXL6tNNF+rh3EucCJmQC/exsYPZNpJ3IGf1YyfvdjOY3m1K 1cP0WLchBvEthG2w3kt4Kx3pa369OEKOQ0igOLvv2ef71WBsy4u/HRBVQwnfXD0QEf fDAJimD9a+P1A== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 12/16] mm/madvise: batch tlb flushes for process_madvise(MADV_DONTNEED[_LOCKED]) Date: Wed, 5 Mar 2025 10:16:07 -0800 Message-Id: <20250305181611.54484-13-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" MADV_DONTNEED[_LOCKED] internal logic can be invoked with batched tlb flushes. Update vector_madvise(), which is called for process_madvise(), to use that in the way, by passing an mmu_gather object that it initializes before starting the internal works, and flushing the gathered tlb entries at once after all the internal works are done. Signed-off-by: SeongJae Park --- mm/madvise.c | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 767c5d21ee75..efa4184d6cf5 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -905,6 +905,7 @@ static bool madvise_dontneed_free_valid_vma(struct vm_a= rea_struct *vma, =20 struct madvise_behavior { int behavior; + struct mmu_gather *tlb; }; =20 static long madvise_dontneed_free(struct vm_area_struct *vma, @@ -964,7 +965,8 @@ static long madvise_dontneed_free(struct vm_area_struct= *vma, } =20 if (behavior =3D=3D MADV_DONTNEED || behavior =3D=3D MADV_DONTNEED_LOCKED) - return madvise_dontneed_single_vma(NULL, vma, start, end); + return madvise_dontneed_single_vma( + madv_behavior->tlb, vma, start, end); else if (behavior =3D=3D MADV_FREE) return madvise_free_single_vma(NULL, vma, start, end); else @@ -1802,19 +1804,50 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size= _t, len_in, int, behavior) return do_madvise(current->mm, start, len_in, behavior); } =20 +static bool vector_madvise_batch_tlb_flush(int behavior) +{ + switch (behavior) { + case MADV_DONTNEED: + case MADV_DONTNEED_LOCKED: + return true; + default: + return false; + } +} + +static void vector_madvise_init_tlb(struct madvise_behavior *madv_behavior, + struct mm_struct *mm) +{ + if (!vector_madvise_batch_tlb_flush(madv_behavior->behavior)) + return; + tlb_gather_mmu(madv_behavior->tlb, mm); +} + +static void vector_madvise_finish_tlb(struct madvise_behavior *madv_behavi= or) +{ + if (!vector_madvise_batch_tlb_flush(madv_behavior->behavior)) + return; + tlb_finish_mmu(madv_behavior->tlb); +} + /* Perform an madvise operation over a vector of addresses and lengths. */ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, int behavior) { ssize_t ret =3D 0; size_t total_len; - struct madvise_behavior madv_behavior =3D {.behavior =3D behavior}; + struct mmu_gather tlb; + struct madvise_behavior madv_behavior =3D { + .behavior =3D behavior, + .tlb =3D &tlb, + }; =20 total_len =3D iov_iter_count(iter); =20 ret =3D madvise_lock(mm, behavior); if (ret) return ret; + vector_madvise_init_tlb(&madv_behavior, mm); =20 while (iov_iter_count(iter)) { unsigned long start =3D (unsigned long)iter_iov_addr(iter); @@ -1843,14 +1876,17 @@ static ssize_t vector_madvise(struct mm_struct *mm,= struct iov_iter *iter, } =20 /* Drop and reacquire lock to unwind race. */ + vector_madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); madvise_lock(mm, behavior); + vector_madvise_init_tlb(&madv_behavior, mm); continue; } if (ret < 0) break; iov_iter_advance(iter, iter_iov_len(iter)); } + vector_madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); =20 ret =3D (total_len - iov_iter_count(iter)) ? : ret; --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E39B256C61 for ; Wed, 5 Mar 2025 18:16:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198595; cv=none; b=JpZwCXMSV/d+8sTS/ZC3gX8rak5zx5dO5qoUb/lhchsFNxZe/2Abk/UPto48taFz27zYGOcehNYrwTqy+wIsnDS3FTt8Bmykb6/lIMu+yh2J+4+dUoyDQ7LMQqn93Rsv1GhNiAGIzxgpsYWvIAUvPm4aDnzKCFnHhpX/nGXQhto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198595; c=relaxed/simple; bh=8avla5b+KxmfOJySvGj+DodzRtTHcPd16UXdPAbl5pc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jC3SSZptJ7helnKHZQmJvUP2iTc1pH+NUafbRAj30GlfVmWziAEWREDQdcrCCTb/4iQNJ1hbIXMlQm9UcrJbvoJ5pkN/l3FrevZM0jBRvmnouNqbPrv0x3MX3qXY2uHpPyOw6yyze4NQG6PJCZmc3sgcRNubvTB+1hhNtoFjNmQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=J7zx6935; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J7zx6935" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C3FF4C4CED1; Wed, 5 Mar 2025 18:16:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198595; bh=8avla5b+KxmfOJySvGj+DodzRtTHcPd16UXdPAbl5pc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J7zx6935Lw/ggyahtkH9JuxQGHB7OziMFhKqYpIdYXK1AwF99oacmfuhm/HCW2zjq nMEFh3WjAxFhwaYN+/wR5UnDYbecUDIesx1CWpuYZRcbSNb7C9lZn5+OSoSL+WSxcX emekqO9bfmC/noXO7JmQTP8vEA5gwJWhac81xKU5Q6fnF+k1zykuU24bqfpptTmDJR tYUlA/oEk909vsO7NSImsB/xMvg7XIFT7l+1oQ3sLz9m/YnTeUHYYnHoX6zzrdC9kW SpBemJEY0tvPtLU55sGAYCm3ZMs8jVNAA4aZunXcGqurlEh8EElDLseZVuGQQ2WhnX /pM5FE9fLjbbA== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 13/16] mm/madvise: batch tlb flushes for process_madvise(MADV_FREE) Date: Wed, 5 Mar 2025 10:16:08 -0800 Message-Id: <20250305181611.54484-14-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" MADV_FREE internal logic can be invoked with batching tlb flushes. Update vector_madvise(), which is called for process_madvise(), to use that in the efficient way, by passing an mmu_gather object that it initializes before starting the work, and flushing the tlb entries at once after all the internal works are done. Signed-off-by: SeongJae Park --- mm/madvise.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/madvise.c b/mm/madvise.c index efa4184d6cf5..f1beadb6176a 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -968,7 +968,8 @@ static long madvise_dontneed_free(struct vm_area_struct= *vma, return madvise_dontneed_single_vma( madv_behavior->tlb, vma, start, end); else if (behavior =3D=3D MADV_FREE) - return madvise_free_single_vma(NULL, vma, start, end); + return madvise_free_single_vma( + madv_behavior->tlb, vma, start, end); else return -EINVAL; } --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5261E2571B8 for ; Wed, 5 Mar 2025 18:16:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198596; cv=none; b=lRB1HIqMeKWzyKCa2FjgOlJn5X6In7xlyP2Vajea0KU7f8FGWkt66FrwVyjpGW/3UaQ3pbgedqXDodJs4Yk8Hu6mYEt91LipZBnhgONxi1Pq2Zcocw1zg+ka7EwhYf5tTfiGbYt90uURu6YP9SuF1DwBsmsBc3BtaV7tUcBoY9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198596; c=relaxed/simple; bh=1LPOXs5PBkvrUa9S0qnB8qMN3Uc/uSVSQOTjrIyqAEE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZwQg0zoAX+Yd83VrLP3ld4VqPmwHUD/S49Nx082yF77fvOK6iNe4cNv13Z1+PFVdqsUxpK3E85C/kNU/iMSlTSI4BYexAikGdi6motdXjrTDlpvzK4QIaLEuEpEc00K/wVVEXRx0Bz6PnUDS7cexA85/GEiycDrcuDm8G7hzp5c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EHqFAg4A; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EHqFAg4A" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DDADAC4CED1; Wed, 5 Mar 2025 18:16:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198596; bh=1LPOXs5PBkvrUa9S0qnB8qMN3Uc/uSVSQOTjrIyqAEE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EHqFAg4AOa7S72mrOT/H3kNLtZLpur42QTtIRfYbMO+8bjFaIMS1gEdG7yG47ywRy 4rqL7AOqSLUArKtqalvcDEx3ixdn8dlfbcGDTM1pqiALKHLYqhtQxCLfVmbiUAhrTX 2RcH9u9I5jIJsy6XNsl18EuyAwWPEEvd/joVHWmWx0elQQUeWx3GfIyDmIE/G9M6Re XFyFhxghJiYElbYzzl1lLS52XUGPvwNvv+54MiCrwgqJI55xsVpJFhza5Ax/bx2POQ n1qO5tDPVQXxLPtGBrry7hNgEENXcWxlm8HRSwn0cqoTuKx/tfYYqSPbb7fu1+nRVF LQwENlJY4X9sw== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 14/16] mm/madvise: batch tlb flushes for madvise(MADV_{DONTNEED[_LOCKED],FREE} Date: Wed, 5 Mar 2025 10:16:09 -0800 Message-Id: <20250305181611.54484-15-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" MADV_DONTNEED[_LOCKED] and MADV_FREE internal logics can be invoked with batched tlb flushes. madvise() is called for single address range, but if there are multiple vmas for the range, tlb flush will happen multiple times. Update do_madvise(), which is called for madvise(), to use that in the efficient way, by passing an mmu_gather object that it initializes before starting the work, and flushing the tlb entries at once after all the internal works are done. Signed-off-by: SeongJae Park --- mm/madvise.c | 68 ++++++++++++++++++++++++++++------------------------ 1 file changed, 37 insertions(+), 31 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index f1beadb6176a..0d292b8e1a0e 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1635,6 +1635,32 @@ static void madvise_unlock(struct mm_struct *mm, int= behavior) mmap_read_unlock(mm); } =20 +static bool madvise_batch_tlb_flush(int behavior) +{ + switch (behavior) { + case MADV_DONTNEED: + case MADV_DONTNEED_LOCKED: + return true; + default: + return false; + } +} + +static void madvise_init_tlb(struct madvise_behavior *madv_behavior, + struct mm_struct *mm) +{ + if (!madvise_batch_tlb_flush(madv_behavior->behavior)) + return; + tlb_gather_mmu(madv_behavior->tlb, mm); +} + +static void madvise_finish_tlb(struct madvise_behavior *madv_behavior) +{ + if (!madvise_batch_tlb_flush(madv_behavior->behavior)) + return; + tlb_finish_mmu(madv_behavior->tlb); +} + static bool is_valid_madvise(unsigned long start, size_t len_in, int behav= ior) { size_t len; @@ -1787,14 +1813,20 @@ static int madvise_do_behavior(struct mm_struct *mm, int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, i= nt behavior) { int error; - struct madvise_behavior madv_behavior =3D {.behavior =3D behavior}; + struct mmu_gather tlb; + struct madvise_behavior madv_behavior =3D { + .behavior =3D behavior, + .tlb =3D &tlb, + }; =20 if (madvise_should_skip(start, len_in, behavior, &error)) return error; error =3D madvise_lock(mm, behavior); if (error) return error; + madvise_init_tlb(&madv_behavior, mm); error =3D madvise_do_behavior(mm, start, len_in, &madv_behavior); + madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); =20 return error; @@ -1805,32 +1837,6 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_= t, len_in, int, behavior) return do_madvise(current->mm, start, len_in, behavior); } =20 -static bool vector_madvise_batch_tlb_flush(int behavior) -{ - switch (behavior) { - case MADV_DONTNEED: - case MADV_DONTNEED_LOCKED: - return true; - default: - return false; - } -} - -static void vector_madvise_init_tlb(struct madvise_behavior *madv_behavior, - struct mm_struct *mm) -{ - if (!vector_madvise_batch_tlb_flush(madv_behavior->behavior)) - return; - tlb_gather_mmu(madv_behavior->tlb, mm); -} - -static void vector_madvise_finish_tlb(struct madvise_behavior *madv_behavi= or) -{ - if (!vector_madvise_batch_tlb_flush(madv_behavior->behavior)) - return; - tlb_finish_mmu(madv_behavior->tlb); -} - /* Perform an madvise operation over a vector of addresses and lengths. */ static ssize_t vector_madvise(struct mm_struct *mm, struct iov_iter *iter, int behavior) @@ -1848,7 +1854,7 @@ static ssize_t vector_madvise(struct mm_struct *mm, s= truct iov_iter *iter, ret =3D madvise_lock(mm, behavior); if (ret) return ret; - vector_madvise_init_tlb(&madv_behavior, mm); + madvise_init_tlb(&madv_behavior, mm); =20 while (iov_iter_count(iter)) { unsigned long start =3D (unsigned long)iter_iov_addr(iter); @@ -1877,17 +1883,17 @@ static ssize_t vector_madvise(struct mm_struct *mm,= struct iov_iter *iter, } =20 /* Drop and reacquire lock to unwind race. */ - vector_madvise_finish_tlb(&madv_behavior); + madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); madvise_lock(mm, behavior); - vector_madvise_init_tlb(&madv_behavior, mm); + madvise_init_tlb(&madv_behavior, mm); continue; } if (ret < 0) break; iov_iter_advance(iter, iter_iov_len(iter)); } - vector_madvise_finish_tlb(&madv_behavior); + madvise_finish_tlb(&madv_behavior); madvise_unlock(mm, behavior); =20 ret =3D (total_len - iov_iter_count(iter)) ? : ret; --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72DA2257431 for ; Wed, 5 Mar 2025 18:16:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198597; cv=none; b=ZH8avHdDS0OhTOdZEa1t7SXRg5snUrNEOJ2qKi1qlmAk2Q64k0+UJqmkFnbsV8ZS/UBrDlgj3+kWRbmTP+GMHBq0d1nYNMsS/b6q8OIgF1qjNBtYzL/AjSEu8cp0FKACbpIaTlDOtsKHC2U9MqNf3R7SU1HUrcXiIMYX23bArPU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198597; c=relaxed/simple; bh=61D1iQafSCEWPTeNlMInbGqRcctkDb5o3v4OcYZ+P3Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WM2AEaST/Ao+zhAYyUW8QG4paK8yCtUvkNWpXtML/nDXJ009jDZid0ZFHLXa2l9kinOvGQA5Uq5SWlq8n5BuWrcpt79sLuPzKsbWebj+sSp9GJzkzk4/rxBPX6IsanL4HYcDliFFGqLZ+D93iXKq+LfAwPjrXJnoxCxeGlS9C54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ODq+EsY3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ODq+EsY3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06F51C4CED1; Wed, 5 Mar 2025 18:16:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198597; bh=61D1iQafSCEWPTeNlMInbGqRcctkDb5o3v4OcYZ+P3Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ODq+EsY3/sWBCSNpxUzI9FMLsZgJlE7Dy483xYT6Kt4Mg9OSOluTVjXkexjRq06BP lMjO/Tvnxa+w8n00c6Ym/vYh9UQPMiZhyyeLvhnrMMamKOSuQlpWlcx0PMFjH/MX6F ytwabLKAQ+ZDeElAuiuMgHvEVF5zBtNbM5TzCIPf1hHdk4ishq53IQaBGF13pVfoYd VQaSMorn8QNtKh1+SKJh+KsAN8iL5J0q9YcWzqrSqxEYLL71uQrH5lDesIYn1ZCwQ4 JqI988Cd1g4LAGOkzGZOhhM7/4LgMTyYtyliqk4Pp9IshM7YYxA91Hcnc8DcngECjW ApD3cjLnF7fdw== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 15/16] mm/madvise: remove !tlb support from madvise_dontneed_single_vma() Date: Wed, 5 Mar 2025 10:16:10 -0800 Message-Id: <20250305181611.54484-16-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_dontneed_single_vma() supports both batched tlb flushes and unbatched tlb flushes use cases depending on received tlb parameter's value. Both were supported for safe and fine transition of the usages from the unbatched flushed to the batched ones. Now the transition is done, and therefore there is no real unbatched tlb flushes use case. Remove the code for supporting the no more being used input case. Signed-off-by: SeongJae Park --- mm/madvise.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 0d292b8e1a0e..1dd2c25c83d8 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -866,10 +866,7 @@ static long madvise_dontneed_single_vma(struct mmu_gat= her *tlb, .even_cows =3D true, }; =20 - if (!tlb) - zap_page_range_single(vma, start, end - start, &details); - else - unmap_vma_single(tlb, vma, start, end - start, &details); + unmap_vma_single(tlb, vma, start, end - start, &details); return 0; } =20 --=20 2.39.5 From nobody Fri Dec 19 20:33:09 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65FDD257ADE for ; Wed, 5 Mar 2025 18:16:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198598; cv=none; b=psz9PQIskrK2UlrNYX3EcCNaqLoH0Z+WbHLphiEY8a17dnK54ZAQO6oe7Knils3HRsMPtJZrfTvj4M8OyExC8v85AAMhF2wWiy/LMoX/ElDjotkue3cnfo43u2BRhjf/mk87deui/CbfT5fYiK9hX9pEH1SPEFcxwhvK1Yyc3Kk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741198598; c=relaxed/simple; bh=iyzH21UhzYqd7Tqlxj4XxNbXqpQgjQqkRv0b1OdLlb0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VYMSGZB2dBbmVlKwyQ0LkZTTHM/0L2agAJzGbFSw3Bpj861xHeM3noK/Y2Vxcie+DNYMsZJNmeE1vrMlEIdSMm5t9TJq/l7dPgtTg1lPIDd604hzvdxAiVlyEftEh8XVmSjmON2JLXCEqaZ4EZ8jpy2/LTQvPdxtT2Cp9ylHO70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WjwJEMeK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WjwJEMeK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F21FC4CED1; Wed, 5 Mar 2025 18:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198598; bh=iyzH21UhzYqd7Tqlxj4XxNbXqpQgjQqkRv0b1OdLlb0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WjwJEMeKovsett4/AR6x8oSEwzZnyBpsSPqAC5hvWjTXEPikGH4plKyr4a5j+irB2 kMkvzQV4rLf8MeNCGskmt4fnZSi0E/+ukl9CWU9VR7MNLtGGRPtIp4pC8DV7O99qUs ckSaJybTfvS50qC8CluLpozQzSUdtxe7VwJns/gpyd/WSidVLXrZ2je9ZuPfHzDdPY vQnD/QTFu2Wk+G5Oa01nPcfR3tl1uZt4Da93DPNVG4+5cmfGgrpgDV18EGv9E5xKzF ctBPkLmdrMp/kdrWE5sIUtOPiLvuAGSUpjwMFy9BZNiWihXtc+nEUWp2v71GSjP3Zj C+ErJUXLmDFcA== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 16/16] mm/madvise: remove !caller_tlb case of madvise_free_single_vma() Date: Wed, 5 Mar 2025 10:16:11 -0800 Message-Id: <20250305181611.54484-17-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" madvise_free_single_vma() supports both batched tlb flushes and unbatched tlb flushes use cases depending on received tlb parameter's value. Both were supported for safe and fine transition of the usages from the unbatched flushed to the batched ones. Now the transition is done, and therefore there is no real unbatched tlb flushes use case. Remove the code for supporting the no more being used input case. Signed-off-by: SeongJae Park --- mm/madvise.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 1dd2c25c83d8..03ba5ff0cf9b 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -795,18 +795,11 @@ static const struct mm_walk_ops madvise_free_walk_ops= =3D { }; =20 static int madvise_free_single_vma( - struct mmu_gather *caller_tlb, struct vm_area_struct *vma, + struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long start_addr, unsigned long end_addr) { struct mm_struct *mm =3D vma->vm_mm; struct mmu_notifier_range range; - struct mmu_gather self_tlb; - struct mmu_gather *tlb; - - if (caller_tlb) - tlb =3D caller_tlb; - else - tlb =3D &self_tlb; =20 /* MADV_FREE works for only anon vma at the moment */ if (!vma_is_anonymous(vma)) @@ -822,8 +815,6 @@ static int madvise_free_single_vma( range.start, range.end); =20 lru_add_drain(); - if (!caller_tlb) - tlb_gather_mmu(tlb, mm); update_hiwater_rss(mm); =20 mmu_notifier_invalidate_range_start(&range); @@ -832,9 +823,6 @@ static int madvise_free_single_vma( &madvise_free_walk_ops, tlb); tlb_end_vma(tlb, vma); mmu_notifier_invalidate_range_end(&range); - if (!caller_tlb) - tlb_finish_mmu(tlb); - return 0; } =20 --=20 2.39.5