From: Ankit Agrawal <ankita@nvidia.com>

Generalizing S2 setting from DEVICE_nGnRE to NormalNc for non PCI
devices may be problematic. E.g. GICv2 vCPU interface, which is
effectively a shared peripheral, can allow a guest to affect another
guest's interrupt distribution. The issue may be solved by limiting
the relaxation to mappings that have a user VMA. Still there is
insufficient information and uncertainity in the behavior of
non PCI drivers.

Add a new flag VM_VFIO_ALLOW_WC to indicate KVM that the device is
WC capable and these S2 changes can be extended to it. KVM can use
this flag to activate the code.

Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Jason Gunthorpe <jgg@nvidia.com>
---
 include/linux/mm.h | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index f5a97dec5169..884c068a79eb 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -391,6 +391,20 @@ extern unsigned int kobjsize(const void *objp);
 # define VM_UFFD_MINOR		VM_NONE
 #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_MINOR */
 
+/*
+ * This flag is used to connect VFIO to arch specific KVM code. It
+ * indicates that the memory under this VMA is safe for use with any
+ * non-cachable memory type inside KVM. Some VFIO devices, on some
+ * platforms, are thought to be unsafe and can cause machine crashes if
+ * KVM does not lock down the memory type.
+ */
+#ifdef CONFIG_64BIT
+#define VM_VFIO_ALLOW_WC_BIT	39
+#define VM_VFIO_ALLOW_WC	BIT(VM_VFIO_ALLOW_WC_BIT)
+#else
+#define VM_VFIO_ALLOW_WC	VM_NONE
+#endif
+
 /* Bits set in the VMA until the stack is in its final location */
 #define VM_STACK_INCOMPLETE_SETUP (VM_RAND_READ | VM_SEQ_READ | VM_STACK_EARLY)
 
-- 
2.34.1

On Thu, Feb 08, 2024 at 02:16:50AM +0530, ankita@nvidia.com wrote:
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index f5a97dec5169..884c068a79eb 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -391,6 +391,20 @@ extern unsigned int kobjsize(const void *objp);
>  # define VM_UFFD_MINOR		VM_NONE
>  #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_MINOR */
>  
> +/*
> + * This flag is used to connect VFIO to arch specific KVM code. It
> + * indicates that the memory under this VMA is safe for use with any
> + * non-cachable memory type inside KVM. Some VFIO devices, on some
> + * platforms, are thought to be unsafe and can cause machine crashes if
> + * KVM does not lock down the memory type.
> + */
> +#ifdef CONFIG_64BIT
> +#define VM_VFIO_ALLOW_WC_BIT	39
> +#define VM_VFIO_ALLOW_WC	BIT(VM_VFIO_ALLOW_WC_BIT)
> +#else
> +#define VM_VFIO_ALLOW_WC	VM_NONE
> +#endif

Adding David Hildenbrand to this thread as well since we briefly
discussed potential alternatives (not sure we came to any conclusion).

-- 
Catalin

On Thu, Feb 08, 2024 at 01:03:27PM +0000, Catalin Marinas wrote:
> On Thu, Feb 08, 2024 at 02:16:50AM +0530, ankita@nvidia.com wrote:
> > diff --git a/include/linux/mm.h b/include/linux/mm.h
> > index f5a97dec5169..884c068a79eb 100644
> > --- a/include/linux/mm.h
> > +++ b/include/linux/mm.h
> > @@ -391,6 +391,20 @@ extern unsigned int kobjsize(const void *objp);
> >  # define VM_UFFD_MINOR		VM_NONE
> >  #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_MINOR */
> >  
> > +/*
> > + * This flag is used to connect VFIO to arch specific KVM code. It
> > + * indicates that the memory under this VMA is safe for use with any
> > + * non-cachable memory type inside KVM. Some VFIO devices, on some
> > + * platforms, are thought to be unsafe and can cause machine crashes if
> > + * KVM does not lock down the memory type.
> > + */
> > +#ifdef CONFIG_64BIT
> > +#define VM_VFIO_ALLOW_WC_BIT	39
> > +#define VM_VFIO_ALLOW_WC	BIT(VM_VFIO_ALLOW_WC_BIT)
> > +#else
> > +#define VM_VFIO_ALLOW_WC	VM_NONE
> > +#endif
> 
> Adding David Hildenbrand to this thread as well since we briefly
> discussed potential alternatives (not sure we came to any conclusion).

FWIW, with my mm hat on:

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

But I'm interested if David has an alternative. We don't have a
shortage of bits here so I'm not sure it is worth much fuss.

Jason