Commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for
platforms with no DMA memory zones"), this lets the kernel benifit
due to BLOCK_MAPPINGS, we could do more if ZONE_DMA and ZONE_DMA32
enabled.

1) Don't defer reserve_crashkernel() if only ZONE_DMA32
2) Don't defer reserve_crashkernel() if ZONE_DMA with dma_force_32bit
   kernel parameter(newly added)

Here is another case to show the benefit of the block mapping.

Unixbench benchmark result shows between the block mapping and page mapping.
----------------+------------------+-------------------
        	| block mapping    |   page mapping    
----------------+------------------+-------------------
Process Creation|  5,030.7         |    4,711.8       
(in unixbench)  |                  |                   
----------------+------------------+-------------------

note: RODATA_FULL_DEFAULT_ENABLED is not enabled

v3:
- renaming crashkernel_could_early_reserve() to crashkernel_early_reserve() 
- drop dma32_phys_limit, directly use max_zone_phys(32)
- fix no previous prototype issue
- add RB of Vijay to patch2/3
v2 resend:
- fix build error reported-by lkp
v2:
- update patch1 according to Vijay and Florian, and RB of Vijay
- add new patch2

Kefeng Wang (3):
  arm64: mm: Do not defer reserve_crashkernel() if only ZONE_DMA32
  arm64: mm: Don't defer reserve_crashkernel() with dma_force_32bit
  arm64: mm: Cleanup useless parameters in zone_sizes_init()

 arch/arm64/include/asm/kexec.h |  1 +
 arch/arm64/mm/init.c           | 60 ++++++++++++++++++++++++----------
 arch/arm64/mm/mmu.c            |  6 ++--
 3 files changed, 46 insertions(+), 21 deletions(-)

-- 
2.26.2

On Mon, Apr 11, 2022 at 05:24:52PM +0800, Kefeng Wang wrote:
> Commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for
> platforms with no DMA memory zones"), this lets the kernel benifit
> due to BLOCK_MAPPINGS, we could do more if ZONE_DMA and ZONE_DMA32
> enabled.
> 
> 1) Don't defer reserve_crashkernel() if only ZONE_DMA32
> 2) Don't defer reserve_crashkernel() if ZONE_DMA with dma_force_32bit
>    kernel parameter(newly added)

I'm not really keen on a new kernel parameter for this. But even with
such parameter, there is another series that allows crashkernel
reservations above ZONE_DMA32, so that would also need
NO_BLOCK_MAPPINGS, at least initially. I think there was a proposal to
do the high reservation first and only defer the low one in ZONE_DMA but
suggested we get the reservations sorted first and look at optimisations
later.

If hardware is so bad with page mappings, I think we need to look at
different ways to enable the block mappings, e.g. some safe break
before make change of the mappings or maybe switching to another TTBR1
during boot.

Does FEAT_BBM level 2 allow us to change the block size without a break
before make? I think that can still trigger a TLB conflict abort, maybe
we can trap it and invalidate the TLBs (the conflict should be on the
linear map not where the kernel image is mapped).

-- 
Catalin

On 2022/5/4 2:20, Catalin Marinas wrote:
> On Mon, Apr 11, 2022 at 05:24:52PM +0800, Kefeng Wang wrote:
>> Commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for
>> platforms with no DMA memory zones"), this lets the kernel benifit
>> due to BLOCK_MAPPINGS, we could do more if ZONE_DMA and ZONE_DMA32
>> enabled.
>>
>> 1) Don't defer reserve_crashkernel() if only ZONE_DMA32
>> 2) Don't defer reserve_crashkernel() if ZONE_DMA with dma_force_32bit
>>     kernel parameter(newly added)
> I'm not really keen on a new kernel parameter for this. But even with
> such parameter, there is another series that allows crashkernel
> reservations above ZONE_DMA32, so that would also need
> NO_BLOCK_MAPPINGS, at least initially. I think there was a proposal to
> do the high reservation first and only defer the low one in ZONE_DMA but
> suggested we get the reservations sorted first and look at optimisations
> later.
OK, we could look it again after patch "support reserving crashkernel
above 4G on arm64 kdump".

The patch3 is a small cleanup, could you pick it up?

> If hardware is so bad with page mappings, I think we need to look at
> different ways to enable the block mappings, e.g. some safe break
> before make change of the mappings or maybe switching to another TTBR1
> during boot.
>
> Does FEAT_BBM level 2 allow us to change the block size without a break
> before make? I think that can still trigger a TLB conflict abort, maybe
> we can trap it and invalidate the TLBs (the conflict should be on the
> linear map not where the kernel image is mapped).

Block mapping is better than page mapping in some testcase(unixbench,
booting time, and mysql, maybe more). KFENCE will make the liner
mapping to page mapping too. If there is a new way to let's enable
the block mapping, that's great.

Hi Catalin and Will, any comments, thanks.

On 2022/4/11 17:24, Kefeng Wang wrote:
> Commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for
> platforms with no DMA memory zones"), this lets the kernel benifit
> due to BLOCK_MAPPINGS, we could do more if ZONE_DMA and ZONE_DMA32
> enabled.
>
> 1) Don't defer reserve_crashkernel() if only ZONE_DMA32
> 2) Don't defer reserve_crashkernel() if ZONE_DMA with dma_force_32bit
>     kernel parameter(newly added)
>
> Here is another case to show the benefit of the block mapping.
>
> Unixbench benchmark result shows between the block mapping and page mapping.
> ----------------+------------------+-------------------
>          	| block mapping    |   page mapping
> ----------------+------------------+-------------------
> Process Creation|  5,030.7         |    4,711.8
> (in unixbench)  |                  |
> ----------------+------------------+-------------------
>
> note: RODATA_FULL_DEFAULT_ENABLED is not enabled
>
> v3:
> - renaming crashkernel_could_early_reserve() to crashkernel_early_reserve()
> - drop dma32_phys_limit, directly use max_zone_phys(32)
> - fix no previous prototype issue
> - add RB of Vijay to patch2/3
> v2 resend:
> - fix build error reported-by lkp
> v2:
> - update patch1 according to Vijay and Florian, and RB of Vijay
> - add new patch2
>
> Kefeng Wang (3):
>    arm64: mm: Do not defer reserve_crashkernel() if only ZONE_DMA32
>    arm64: mm: Don't defer reserve_crashkernel() with dma_force_32bit
>    arm64: mm: Cleanup useless parameters in zone_sizes_init()
>
>   arch/arm64/include/asm/kexec.h |  1 +
>   arch/arm64/mm/init.c           | 60 ++++++++++++++++++++++++----------
>   arch/arm64/mm/mmu.c            |  6 ++--
>   3 files changed, 46 insertions(+), 21 deletions(-)
>

On Mon, 11 Apr 2022 17:24:52 +0800, Kefeng Wang wrote:
> Commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for
> platforms with no DMA memory zones"), this lets the kernel benifit
> due to BLOCK_MAPPINGS, we could do more if ZONE_DMA and ZONE_DMA32
> enabled.
> 
> 1) Don't defer reserve_crashkernel() if only ZONE_DMA32
> 2) Don't defer reserve_crashkernel() if ZONE_DMA with dma_force_32bit
>    kernel parameter(newly added)
> 
> [...]

Applied to arm64 (for-next/misc), thanks!

[3/3] arm64: mm: Cleanup useless parameters in zone_sizes_init()
      https://git.kernel.org/arm64/c/f41ef4c2ee99

-- 
Catalin