[v2] KVM: arm64: Optimize protected mode and FWB

[PATCH v2] KVM: arm64: Optimize protected mode and FWB

Posted by Mostafa Saleh 3 days, 15 hours ago

KVM opportunistically enables FWB if supported by the system for guest
VMs, which allows it to elude cache maintenance for data as they are
forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return.
However in protected mode, before calling __clean_dcache_guest_page()
it loops over the range and fix_map/unmap it, issuing TLB
invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
functions issue dsb() for the unmap path.

Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
index 4e329e39a695..049a09be2bbf 100644
--- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
+++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/*
+	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
+	 * In contrast, the host stage-2 never has FWB enabled.
+	 */
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return;
+
 	__apply_guest_page(va, size, __clean_dcache_guest_page);
 }
 
-- 
2.55.0.229.g6434b31f56-goog

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

Posted by Marc Zyngier 3 days, 14 hours ago

On Tue, 21 Jul 2026 09:56:45 +0100,
Mostafa Saleh <smostafa@google.com> wrote:
> 
> KVM opportunistically enables FWB if supported by the system for guest
> VMs, which allows it to elude cache maintenance for data as they are
> forced to be cacheable from stage-2.
> In that case, __clean_dcache_guest_page() will immediately return.
> However in protected mode, before calling __clean_dcache_guest_page()
> it loops over the range and fix_map/unmap it, issuing TLB
> invalidations, dsb() and isb() unnecessarily.
> 
> This can be optimized by returning early if FWB is supported,
> kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> functions issue dsb() for the unmap path.
> 
> Signed-off-by: Mostafa Saleh <smostafa@google.com>
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> index 4e329e39a695..049a09be2bbf 100644
> --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
>  
>  static void clean_dcache_guest_page(void *va, size_t size)
>  {
> +	/*
> +	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> +	 * In contrast, the host stage-2 never has FWB enabled.

Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
available.

Not sure that changes anything, but the comment is factually wrong.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

Posted by Mostafa Saleh 3 days, 14 hours ago

On Tue, Jul 21, 2026 at 11:10:07AM +0100, Marc Zyngier wrote:
> On Tue, 21 Jul 2026 09:56:45 +0100,
> Mostafa Saleh <smostafa@google.com> wrote:
> > 
> > KVM opportunistically enables FWB if supported by the system for guest
> > VMs, which allows it to elude cache maintenance for data as they are
> > forced to be cacheable from stage-2.
> > In that case, __clean_dcache_guest_page() will immediately return.
> > However in protected mode, before calling __clean_dcache_guest_page()
> > it loops over the range and fix_map/unmap it, issuing TLB
> > invalidations, dsb() and isb() unnecessarily.
> > 
> > This can be optimized by returning early if FWB is supported,
> > kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> > functions issue dsb() for the unmap path.
> > 
> > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> > ---
> >  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > index 4e329e39a695..049a09be2bbf 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
> >  
> >  static void clean_dcache_guest_page(void *va, size_t size)
> >  {
> > +	/*
> > +	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> > +	 * In contrast, the host stage-2 never has FWB enabled.
> 
> Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
> over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
> available.
> 
> Not sure that changes anything, but the comment is factually wrong.

Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not
change the patch, I will need to update the comment.

Thanks,
Mostafa

> 
> Thanks,
> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

Posted by Fuad Tabba 2 days, 12 hours ago

On Tue, 21 Jul 2026 at 11:19, Mostafa Saleh <smostafa@google.com> wrote:
>
> On Tue, Jul 21, 2026 at 11:10:07AM +0100, Marc Zyngier wrote:
> > On Tue, 21 Jul 2026 09:56:45 +0100,
> > Mostafa Saleh <smostafa@google.com> wrote:
> > >
> > > KVM opportunistically enables FWB if supported by the system for guest
> > > VMs, which allows it to elude cache maintenance for data as they are
> > > forced to be cacheable from stage-2.
> > > In that case, __clean_dcache_guest_page() will immediately return.
> > > However in protected mode, before calling __clean_dcache_guest_page()
> > > it loops over the range and fix_map/unmap it, issuing TLB
> > > invalidations, dsb() and isb() unnecessarily.
> > >
> > > This can be optimized by returning early if FWB is supported,
> > > kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> > > functions issue dsb() for the unmap path.
> > >
> > > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > > Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> > > ---
> > >  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
> > >  1 file changed, 7 insertions(+)
> > >
> > > diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > index 4e329e39a695..049a09be2bbf 100644
> > > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
> > >
> > >  static void clean_dcache_guest_page(void *va, size_t size)
> > >  {
> > > +   /*
> > > +    * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> > > +    * In contrast, the host stage-2 never has FWB enabled.
> >
> > Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
> > over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
> > available.
> >
> > Not sure that changes anything, but the comment is factually wrong.
>
> Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not
> change the patch, I will need to update the comment.

Sorry, this confusing is partially my fault. The original comment in v1

>  /* See __clean_dcache_guest_page() */

wasn't very clear to me, since I wasn't sure what I was looking for in
__clean_dcache_guest_page(). Talking to Mostafa in person, he's
referring to the comment _inside_ __clean_dcache_guest_page. I think
the best fix is to rephase as

+  /* See comment in __clean_dcache_guest_page() */

This would save duplication and having to repeat the comment.

Cheers,
/fuad

> Thanks,
> Mostafa
>
> >
> > Thanks,
> >
> >       M.
> >
> > --
> > Without deviation from the norm, progress is not possible.

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

Posted by Mostafa Saleh 1 day, 15 hours ago

On Wed, Jul 22, 2026 at 12:51:35PM +0100, Fuad Tabba wrote:
> On Tue, 21 Jul 2026 at 11:19, Mostafa Saleh <smostafa@google.com> wrote:
> >
> > On Tue, Jul 21, 2026 at 11:10:07AM +0100, Marc Zyngier wrote:
> > > On Tue, 21 Jul 2026 09:56:45 +0100,
> > > Mostafa Saleh <smostafa@google.com> wrote:
> > > >
> > > > KVM opportunistically enables FWB if supported by the system for guest
> > > > VMs, which allows it to elude cache maintenance for data as they are
> > > > forced to be cacheable from stage-2.
> > > > In that case, __clean_dcache_guest_page() will immediately return.
> > > > However in protected mode, before calling __clean_dcache_guest_page()
> > > > it loops over the range and fix_map/unmap it, issuing TLB
> > > > invalidations, dsb() and isb() unnecessarily.
> > > >
> > > > This can be optimized by returning early if FWB is supported,
> > > > kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> > > > functions issue dsb() for the unmap path.
> > > >
> > > > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > > > Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> > > > ---
> > > >  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
> > > >  1 file changed, 7 insertions(+)
> > > >
> > > > diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > > index 4e329e39a695..049a09be2bbf 100644
> > > > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > > @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
> > > >
> > > >  static void clean_dcache_guest_page(void *va, size_t size)
> > > >  {
> > > > +   /*
> > > > +    * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> > > > +    * In contrast, the host stage-2 never has FWB enabled.
> > >
> > > Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
> > > over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
> > > available.
> > >
> > > Not sure that changes anything, but the comment is factually wrong.
> >
> > Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not
> > change the patch, I will need to update the comment.
> 
> Sorry, this confusing is partially my fault. The original comment in v1
> 
> >  /* See __clean_dcache_guest_page() */
> 
> wasn't very clear to me, since I wasn't sure what I was looking for in
> __clean_dcache_guest_page(). Talking to Mostafa in person, he's
> referring to the comment _inside_ __clean_dcache_guest_page. I think
> the best fix is to rephase as
> 
> +  /* See comment in __clean_dcache_guest_page() */
> 
> This would save duplication and having to repeat the comment.
> 

Makes sense, also thinking about this, I think we should do the
same for invalidate_icache_guest_page() and ARM64_HAS_CACHE_DIC.

I can include that in the same patch.

Thanks,
Mostafa