VMCLEAR/VMREAD/VMWRITE/VMPTRLD access the internal VMCS cache, which
is not visible to the compiler; without a memory clobber, the compiler
can reorder them in troublesome ways because "asm volatile" and "asm goto"
only protect against removal of the asm.  For example, placing a VMWRITE
before the corresponding VMCS pointer is loaded can lead to corruption.
While none of this has been observed, it is better to prevent than cure.

Likewise, INVEPT and INVVPID access the TLB and, even though in their
case the effect is only visible to the next VMLAUNCH/VMRESUME, it is
technically correct to add the clobber there too.  So avoid any urge to
special case them, and simply hardcode "memory" into the clobber list
of vmx_asm1() and vmx_asm2().  __vmcs_readl() open-codes its own asm,
so add the clobber there as well.

Link: https://lore.kernel.org/kvm/CABgObfbL3t21yVeSwiLSjjOUER+rTYDPHYAH9YU4TWGRjx6XHg@mail.gmail.com/
Cc: Sean Christopherson <seanjc@google.com>
Cc: stable@vger.kernel.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/kvm/vmx/vmx_ops.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/vmx/vmx_ops.h b/arch/x86/kvm/vmx/vmx_ops.h
index 81784befaaf4..9ffe03a0c223 100644
--- a/arch/x86/kvm/vmx/vmx_ops.h
+++ b/arch/x86/kvm/vmx/vmx_ops.h
@@ -101,7 +101,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field)
 
 			  : [output] "=r" (value)
 			  : [field] "r" (field)
-			  : "cc"
+			  : "cc", "memory"
 			  : do_fail, do_exception);
 
 	return value;
@@ -145,7 +145,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field)
 
 		     : ASM_CALL_CONSTRAINT, [output] "=&r" (value)
 		     : [field] "r" (field)
-		     : "cc");
+		     : "cc", "memory");
 	return value;
 
 #endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */
@@ -192,7 +192,7 @@ do {									\
 	asm goto("1: " __stringify(insn) " %0\n\t"			\
 			  "jna %l[error]\n\t"				\
 			  _ASM_EXTABLE(1b, %l[fault])			\
-			  : : op1 : "cc" : error, fault);		\
+			  : : op1 : "cc", "memory" : error, fault);	\
 	return;								\
 error:									\
 	instrumentation_begin();					\
@@ -208,7 +208,7 @@ do {									\
 	asm goto("1: "  __stringify(insn) " %1, %0\n\t"			\
 			  "jna %l[error]\n\t"				\
 			  _ASM_EXTABLE(1b, %l[fault])			\
-			  : : op1, op2 : "cc" : error, fault);		\
+			  : : op1, op2 : "cc", "memory" : error, fault);\
 	return;								\
 error:									\
 	instrumentation_begin();					\
-- 
2.55.0

On Tue, Jul 21, 2026, Paolo Bonzini wrote:
> VMCLEAR/VMREAD/VMWRITE/VMPTRLD access the internal VMCS cache, which
> is not visible to the compiler; without a memory clobber, the compiler
> can reorder them in troublesome ways because "asm volatile" and "asm goto"
> only protect against removal of the asm.  For example, placing a VMWRITE
> before the corresponding VMCS pointer is loaded can lead to corruption.
> While none of this has been observed, it is better to prevent than cure.
> 
> Likewise, INVEPT and INVVPID access the TLB and, even though in their
> case the effect is only visible to the next VMLAUNCH/VMRESUME, it is
> technically correct to add the clobber there too.  So avoid any urge to
> special case them, and simply hardcode "memory" into the clobber list
> of vmx_asm1() and vmx_asm2().  __vmcs_readl() open-codes its own asm,
> so add the clobber there as well.
> 
> Link: https://lore.kernel.org/kvm/CABgObfbL3t21yVeSwiLSjjOUER+rTYDPHYAH9YU4TWGRjx6XHg@mail.gmail.com/
> Cc: Sean Christopherson <seanjc@google.com>
> Cc: stable@vger.kernel.org
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---

Reviewed-by: Sean Christopherson <seanjc@google.com>