[RFC v2 00/12] xen/arm: Don't switch TTBR while the MMU is on

Julien Grall posted 12 patches 1 year, 6 months ago
Test gitlab-ci failed
Patches applied successfully (tree, apply log)
git fetch https://gitlab.com/xen-project/patchew/xen tags/patchew/20221022150422.17707-1-julien@xen.org
There is a newer version of this series
xen/arch/arm/arm32/head.S           | 253 ++++++++++++++++++----------
xen/arch/arm/arm32/smpboot.c        |   4 +
xen/arch/arm/arm64/Makefile         |   1 +
xen/arch/arm/arm64/head.S           |  86 +++++-----
xen/arch/arm/arm64/mm.c             | 160 ++++++++++++++++++
xen/arch/arm/arm64/smpboot.c        |  15 +-
xen/arch/arm/domain_page.c          |   9 +
xen/arch/arm/include/asm/arm32/mm.h |   4 +
xen/arch/arm/include/asm/arm64/mm.h |  12 ++
xen/arch/arm/include/asm/config.h   |  63 +++++--
xen/arch/arm/include/asm/mm.h       |   2 +
xen/arch/arm/include/asm/setup.h    |  11 ++
xen/arch/arm/include/asm/smp.h      |   1 +
xen/arch/arm/mm.c                   | 105 +++++++-----
xen/arch/arm/smpboot.c              |   1 +
15 files changed, 536 insertions(+), 191 deletions(-)
create mode 100644 xen/arch/arm/arm64/mm.c
[RFC v2 00/12] xen/arm: Don't switch TTBR while the MMU is on
Posted by Julien Grall 1 year, 6 months ago
From: Julien Grall <jgrall@amazon.com>

Hi all,

Currently, Xen on Arm will switch TTBR whilst the MMU is on. This is
similar to replacing existing mappings with new ones. So we need to
follow a break-before-make sequence.

When switching the TTBR, we need to temporary disable the MMU
before updating the TTBR. This means the page-tables must contain an
identity mapping.

The current memory layout is not very flexible and has an higher chance
to clash with the identity mapping.

On Arm64, we have plenty of unused virtual address space Therefore, we can
simply reshuffle the layout to leave the first part of the virtual
address space empty.

On Arm32, the virtual address space is already quite full. Even if we
find space, it would be necessary to have a dynamic layout. So a
different approach will be necessary. The chosen one is to have
a temporary mapping that will be used to jumped from the ID mapping
to the runtime mapping (or vice versa). The temporary mapping will
be overlapping with the domheap area as it should not be used when
switching on/off the MMU.

The Arm32 part is not yet addressed in this version. The series is
sent as an early RFC to gather some feedback on the approach.

After this series, most of Xen page-table code should be compliant
with the Arm Arm. The last two issues I am aware of are:
 - domheap: Mappings are replaced without using the Break-Before-Make
   approach.
 - The cache is not cleaned/invalidated when updating the page-tables
   with Data cache off (like during early boot).

The long term plan is to get rid of boot_* page tables and then
directly use the runtime pages. This means for coloring, we will
need to build the pages in the relocated Xen rather than the current
Xen.

For convience, I pushed a branch with everything applied:

https://xenbits.xen.org/git-http/people/julieng/xen-unstable.git
branch boot-pt-rework-v2

Cheers,

Julien Grall (12):
  xen/arm: Clean-up the memory layout
  xen/arm32: head: Jump to the runtime mapping in enable_mmu()
  xen/arm32: head: Introduce an helper to flush the TLBs
  xen/arm32: head: Remove restriction where to load Xen
  xen/arm32: head: Widen the use of the temporary mapping
  xen/arm: Enable use of dump_pt_walk() early during boot
  xen/arm64: Rework the memory layout
  xen/arm: mm: Allow xen_pt_update() to work with the current root table
  xen/arm: mm: Allow dump_hyp_walk() to work on the current root table
  xen/arm64: mm: Introduce helpers to prepare/enable/disable the
    identity mapping
  xen/arm64: mm: Rework switch_ttbr()
  xen/arm64: smpboot: Directly switch to the runtime page-tables

 xen/arch/arm/arm32/head.S           | 253 ++++++++++++++++++----------
 xen/arch/arm/arm32/smpboot.c        |   4 +
 xen/arch/arm/arm64/Makefile         |   1 +
 xen/arch/arm/arm64/head.S           |  86 +++++-----
 xen/arch/arm/arm64/mm.c             | 160 ++++++++++++++++++
 xen/arch/arm/arm64/smpboot.c        |  15 +-
 xen/arch/arm/domain_page.c          |   9 +
 xen/arch/arm/include/asm/arm32/mm.h |   4 +
 xen/arch/arm/include/asm/arm64/mm.h |  12 ++
 xen/arch/arm/include/asm/config.h   |  63 +++++--
 xen/arch/arm/include/asm/mm.h       |   2 +
 xen/arch/arm/include/asm/setup.h    |  11 ++
 xen/arch/arm/include/asm/smp.h      |   1 +
 xen/arch/arm/mm.c                   | 105 +++++++-----
 xen/arch/arm/smpboot.c              |   1 +
 15 files changed, 536 insertions(+), 191 deletions(-)
 create mode 100644 xen/arch/arm/arm64/mm.c

-- 
2.37.1
Re: [RFC v2 00/12] xen/arm: Don't switch TTBR while the MMU is on
Posted by Xenia Ragiadakou 1 year, 6 months ago
Hi Julien

On 10/22/22 18:04, Julien Grall wrote:
> From: Julien Grall <jgrall@amazon.com>
> 
> Hi all,
> 
> Currently, Xen on Arm will switch TTBR whilst the MMU is on. This is
> similar to replacing existing mappings with new ones. So we need to
> follow a break-before-make sequence.
> 
> When switching the TTBR, we need to temporary disable the MMU
> before updating the TTBR. This means the page-tables must contain an
> identity mapping.
> 
> The current memory layout is not very flexible and has an higher chance
> to clash with the identity mapping.
> 
> On Arm64, we have plenty of unused virtual address space Therefore, we can
> simply reshuffle the layout to leave the first part of the virtual
> address space empty.
> 
> On Arm32, the virtual address space is already quite full. Even if we
> find space, it would be necessary to have a dynamic layout. So a
> different approach will be necessary. The chosen one is to have
> a temporary mapping that will be used to jumped from the ID mapping
> to the runtime mapping (or vice versa). The temporary mapping will
> be overlapping with the domheap area as it should not be used when
> switching on/off the MMU.
> 
> The Arm32 part is not yet addressed in this version. The series is
> sent as an early RFC to gather some feedback on the approach.
> 
> After this series, most of Xen page-table code should be compliant
> with the Arm Arm. The last two issues I am aware of are:
>   - domheap: Mappings are replaced without using the Break-Before-Make
>     approach.
>   - The cache is not cleaned/invalidated when updating the page-tables
>     with Data cache off (like during early boot).
> 
> The long term plan is to get rid of boot_* page tables and then
> directly use the runtime pages. This means for coloring, we will
> need to build the pages in the relocated Xen rather than the current
> Xen.
> 
> For convience, I pushed a branch with everything applied:
> 
> https://xenbits.xen.org/git-http/people/julieng/xen-unstable.git
> branch boot-pt-rework-v2

Are you sure that the branch has been pushed remotely? If yes, ignore.

> 
> Cheers,
> 
> Julien Grall (12):
>    xen/arm: Clean-up the memory layout
>    xen/arm32: head: Jump to the runtime mapping in enable_mmu()
>    xen/arm32: head: Introduce an helper to flush the TLBs
>    xen/arm32: head: Remove restriction where to load Xen
>    xen/arm32: head: Widen the use of the temporary mapping
>    xen/arm: Enable use of dump_pt_walk() early during boot
>    xen/arm64: Rework the memory layout
>    xen/arm: mm: Allow xen_pt_update() to work with the current root table
>    xen/arm: mm: Allow dump_hyp_walk() to work on the current root table
>    xen/arm64: mm: Introduce helpers to prepare/enable/disable the
>      identity mapping
>    xen/arm64: mm: Rework switch_ttbr()
>    xen/arm64: smpboot: Directly switch to the runtime page-tables
> 
>   xen/arch/arm/arm32/head.S           | 253 ++++++++++++++++++----------
>   xen/arch/arm/arm32/smpboot.c        |   4 +
>   xen/arch/arm/arm64/Makefile         |   1 +
>   xen/arch/arm/arm64/head.S           |  86 +++++-----
>   xen/arch/arm/arm64/mm.c             | 160 ++++++++++++++++++
>   xen/arch/arm/arm64/smpboot.c        |  15 +-
>   xen/arch/arm/domain_page.c          |   9 +
>   xen/arch/arm/include/asm/arm32/mm.h |   4 +
>   xen/arch/arm/include/asm/arm64/mm.h |  12 ++
>   xen/arch/arm/include/asm/config.h   |  63 +++++--
>   xen/arch/arm/include/asm/mm.h       |   2 +
>   xen/arch/arm/include/asm/setup.h    |  11 ++
>   xen/arch/arm/include/asm/smp.h      |   1 +
>   xen/arch/arm/mm.c                   | 105 +++++++-----
>   xen/arch/arm/smpboot.c              |   1 +
>   15 files changed, 536 insertions(+), 191 deletions(-)
>   create mode 100644 xen/arch/arm/arm64/mm.c
> 

-- 
Xenia
Re: [RFC v2 00/12] xen/arm: Don't switch TTBR while the MMU is on
Posted by Julien Grall 1 year, 6 months ago

On 24/10/2022 16:39, Xenia Ragiadakou wrote:
> Hi Julien

Hi Xenia,

> 
> On 10/22/22 18:04, Julien Grall wrote:
>> From: Julien Grall <jgrall@amazon.com>
>>
>> Hi all,
>>
>> Currently, Xen on Arm will switch TTBR whilst the MMU is on. This is
>> similar to replacing existing mappings with new ones. So we need to
>> follow a break-before-make sequence.
>>
>> When switching the TTBR, we need to temporary disable the MMU
>> before updating the TTBR. This means the page-tables must contain an
>> identity mapping.
>>
>> The current memory layout is not very flexible and has an higher chance
>> to clash with the identity mapping.
>>
>> On Arm64, we have plenty of unused virtual address space Therefore, we 
>> can
>> simply reshuffle the layout to leave the first part of the virtual
>> address space empty.
>>
>> On Arm32, the virtual address space is already quite full. Even if we
>> find space, it would be necessary to have a dynamic layout. So a
>> different approach will be necessary. The chosen one is to have
>> a temporary mapping that will be used to jumped from the ID mapping
>> to the runtime mapping (or vice versa). The temporary mapping will
>> be overlapping with the domheap area as it should not be used when
>> switching on/off the MMU.
>>
>> The Arm32 part is not yet addressed in this version. The series is
>> sent as an early RFC to gather some feedback on the approach.
>>
>> After this series, most of Xen page-table code should be compliant
>> with the Arm Arm. The last two issues I am aware of are:
>>   - domheap: Mappings are replaced without using the Break-Before-Make
>>     approach.
>>   - The cache is not cleaned/invalidated when updating the page-tables
>>     with Data cache off (like during early boot).
>>
>> The long term plan is to get rid of boot_* page tables and then
>> directly use the runtime pages. This means for coloring, we will
>> need to build the pages in the relocated Xen rather than the current
>> Xen.
>>
>> For convience, I pushed a branch with everything applied:
>>
>> https://xenbits.xen.org/git-http/people/julieng/xen-unstable.git
>> branch boot-pt-rework-v2
> 
> Are you sure that the branch has been pushed remotely? If yes, ignore.

I forgot to push it. It should now be pushed.

Cheers,

-- 
Julien Grall