Archs may need to do special things when flushing thp tlb,
so use the more applicable flush_pud_tlb_range() instead of
flush_tlb_range().
Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
mm/mremap.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/mremap.c b/mm/mremap.c
index 11e06e4ab33b..1883205fa22b 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr,
VM_BUG_ON(!pmd_none(*new_pmd));
pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
- flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
+ flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
if (new_ptl != old_ptl)
spin_unlock(new_ptl);
spin_unlock(old_ptl);
--
2.41.0Hi Kefeng, kernel test robot noticed the following build errors: [auto build test ERROR on arm64/for-next/core] [also build test ERROR on arm-perf/for-next/perf linus/master v6.5-rc4] [cannot apply to akpm-mm/mm-everything] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Kefeng-Wang/mm-hugetlb-use-flush_hugetlb_tlb_range-in-move_hugetlb_page_tables/20230731-154016 base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core patch link: https://lore.kernel.org/r/20230731074829.79309-3-wangkefeng.wang%40huawei.com patch subject: [PATCH 2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd() config: x86_64-randconfig-x003-20230731 (https://download.01.org/0day-ci/archive/20230801/202308010553.KxefZFdO-lkp@intel.com/config) compiler: clang version 16.0.4 (https://github.com/llvm/llvm-project.git ae42196bc493ffe877a7e3dff8be32035dea4d07) reproduce: (https://download.01.org/0day-ci/archive/20230801/202308010553.KxefZFdO-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202308010553.KxefZFdO-lkp@intel.com/ All errors (new ones prefixed by >>): >> ld.lld: error: call to __compiletime_assert_860 marked "dontcall-error": BUILD_BUG failed -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Hi Kefeng,
kernel test robot noticed the following build errors:
[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on arm-perf/for-next/perf linus/master v6.5-rc4 next-20230731]
[cannot apply to akpm-mm/mm-everything]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Kefeng-Wang/mm-hugetlb-use-flush_hugetlb_tlb_range-in-move_hugetlb_page_tables/20230731-154016
base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link: https://lore.kernel.org/r/20230731074829.79309-3-wangkefeng.wang%40huawei.com
patch subject: [PATCH 2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd()
config: x86_64-defconfig (https://download.01.org/0day-ci/archive/20230731/202307312137.ormxuS5g-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230731/202307312137.ormxuS5g-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202307312137.ormxuS5g-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from <command-line>:
In function 'move_normal_pmd',
inlined from 'move_pgt_entry' at mm/mremap.c:463:11,
inlined from 'move_page_tables' at mm/mremap.c:565:8:
>> include/linux/compiler_types.h:397:45: error: call to '__compiletime_assert_338' declared with attribute error: BUILD_BUG failed
397 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:378:25: note: in definition of macro '__compiletime_assert'
378 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:397:9: note: in expansion of macro '_compiletime_assert'
397 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
| ^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:59:21: note: in expansion of macro 'BUILD_BUG_ON_MSG'
59 | #define BUILD_BUG() BUILD_BUG_ON_MSG(1, "BUILD_BUG failed")
| ^~~~~~~~~~~~~~~~
include/linux/pgtable.h:1415:49: note: in expansion of macro 'BUILD_BUG'
1415 | #define flush_pmd_tlb_range(vma, addr, end) BUILD_BUG()
| ^~~~~~~~~
mm/mremap.c:287:9: note: in expansion of macro 'flush_pmd_tlb_range'
287 | flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
| ^~~~~~~~~~~~~~~~~~~
vim +/__compiletime_assert_338 +397 include/linux/compiler_types.h
eb5c2d4b45e3d2 Will Deacon 2020-07-21 383
eb5c2d4b45e3d2 Will Deacon 2020-07-21 384 #define _compiletime_assert(condition, msg, prefix, suffix) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21 385 __compiletime_assert(condition, msg, prefix, suffix)
eb5c2d4b45e3d2 Will Deacon 2020-07-21 386
eb5c2d4b45e3d2 Will Deacon 2020-07-21 387 /**
eb5c2d4b45e3d2 Will Deacon 2020-07-21 388 * compiletime_assert - break build and emit msg if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21 389 * @condition: a compile-time constant condition to check
eb5c2d4b45e3d2 Will Deacon 2020-07-21 390 * @msg: a message to emit if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21 391 *
eb5c2d4b45e3d2 Will Deacon 2020-07-21 392 * In tradition of POSIX assert, this macro will break the build if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21 393 * supplied condition is *false*, emitting the supplied error message if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21 394 * compiler has support to do so.
eb5c2d4b45e3d2 Will Deacon 2020-07-21 395 */
eb5c2d4b45e3d2 Will Deacon 2020-07-21 396 #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21 @397 _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
eb5c2d4b45e3d2 Will Deacon 2020-07-21 398
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
On Mon, Jul 31, 2023 at 03:48:27PM +0800, Kefeng Wang wrote:
> Archs may need to do special things when flushing thp tlb,
> so use the more applicable flush_pud_tlb_range() instead of
> flush_tlb_range().
>
> Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> ---
> mm/mremap.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/mremap.c b/mm/mremap.c
> index 11e06e4ab33b..1883205fa22b 100644
> --- a/mm/mremap.c
> +++ b/mm/mremap.c
> @@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr,
> VM_BUG_ON(!pmd_none(*new_pmd));
>
> pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
> - flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
> + flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
I don't think that's correct for arm64. The assumption in the
flush_p*d_tlb_range() was that they are called only for block mappings
at that p*d level (and we use FEAT_TTL on arm64 indicating that the leaf
level is level 2 for pmd, 1 for pud). IIUC move_normal_pmd() is only
called for table pmds which would have a leaf level of 3 (the pte).
Same for the next patch doing the equivalent for the pud.
--
Catalin
On 2023/7/31 19:05, Catalin Marinas wrote:
> On Mon, Jul 31, 2023 at 03:48:27PM +0800, Kefeng Wang wrote:
>> Archs may need to do special things when flushing thp tlb,
>> so use the more applicable flush_pud_tlb_range() instead of
>> flush_tlb_range().
>>
>> Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>> ---
>> mm/mremap.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/mremap.c b/mm/mremap.c
>> index 11e06e4ab33b..1883205fa22b 100644
>> --- a/mm/mremap.c
>> +++ b/mm/mremap.c
>> @@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr,
>> VM_BUG_ON(!pmd_none(*new_pmd));
>>
>> pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
>> - flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
>> + flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
>
> I don't think that's correct for arm64. The assumption in the
> flush_p*d_tlb_range() was that they are called only for block mappings
> at that p*d level (and we use FEAT_TTL on arm64 indicating that the leaf
> level is level 2 for pmd, 1 for pud). IIUC move_normal_pmd() is only
> called for table pmds which would have a leaf level of 3 (the pte).
oops, yes, this is for NORMAL_PMD case, not HPAGE_PMD, please ignore
patch 2/3.
>
> Same for the next patch doing the equivalent for the pud.
>
© 2016 - 2026 Red Hat, Inc.