Series comparison

-[PATCH v2 0/2] ring-buffer: Allow persistent memory to be user space mmapped
+[PATCH v3 0/5] ring-buffer: Allow persistent memory to be user space mmapped
-Allow the persistent memory mapped memory to be memory mapped to user
+Now that I learned that the memory passed back from reserve_mem is part
-space as well. Currently, the user space memory mapping requires the
+of the memory allocator and just "reserved" and the memory is already
-buffers to have been allocated via page_alloc() and converted to virtual
+virtually mapped, it can simply use phys_to_virt() on the physical memory
-address via page_address(). But the persistent memory is memory mapped
+that is returned to get the virtual mapping for that memory!
-via vmap() and a simple virt_to_page() can not be used. Move the control
+  (Thanks Mike!)
 of the physical mapping via vmap() to the ring buffer code so that it
 can then use the saved physical and virtual mapping to find the pages
 needed for memory mapping user space.
-The first patch moves the memory mapping of the physical memory returned
+That makes things much easier, especially since it means that the memory
-by reserve_mem from the tracing code to the ring buffer code. This makes
+returned by reserve_mem is no different than the memory retrieved by
-sense as this gives more control over to the ring buffer in knowing exactly
+page_alloc(). This allows that memory to be memory mapped to user space
-how the pages were created. It keeps track of where the physical memory
+no differently than it is mapped by the normal buffer.
 that was mapped and also handles the freeing of this memory (removing the
 burden from the tracing code from having to do this). It also handles
 knowing if the buffer may be memory mapped or not. The check is removed
 from the tracing code, but if the tracing code tries to memory map the
 persistent ring buffer, the call to the ring buffer code will fail with
 the same error as before.
-The second patch implements the user space memory mapping of the persistent
+This new series does the following:
 ring buffer. It does so by adding several helper functions to annotate
 what the code is doing. Note, there's two meta pages here. One is mapped
 between the kernel and user space and is used to inform user space of updates
 to the ring buffer. The other is inside the persistent memory that is used to
 pass information across boots. The persistent memory meta data is never exposed
 to user space. The meta data for user space mapping is always allocated via the
 normal memory allocation.
-The helper functions are:
+- Enforce the memory mapping is page aligned (both the address and the
   size). If not, it errors out.
- rb_struct_page() - This is the rb_get_page() from our discussions, but
+- Use phys_to_virt() to get to the virtual memory from the reserve_mem
-                    I renamed it because "get" implies "put".
+  returned addresses. Also use free_reserved_area() to give it
-                    This function will return the struct page for a given
+  back to the buddy allocator when it is freed.
                     buffer page by either virt_to_page() if the page was
                     allocated via the normal memory allocator, or it
                     is found via pfn_to_page() by using the saved physical
                     and virtual address of the mapped location. It uses
                     that to calculate the physical address from the virtual
                     address of the page and then pfn_to_page() can be used
                     from that.
-  rb_fush_buffer_page() - this calls the above rb_struct_page() and then
+- Treat the buffer allocated via memmap differently. It still needs to
-                    calls flush_dcache_folio() to make sure the kernel
+  be virtually mapped (cannot use phys_to_virt) and it must not be
-                    and user space is coherent.
+  freed nor memory mapped to user space. A new flag is added when a buffer
   is created this way to prevent it from ever being memory mapped to user
   space and the ref count is upped so that it can never be freed.
-  rb_flush_meta() - This just uses virt_to_page() and calls flush_dcache_folio()
+- Use vmap_page_range() instead of using kmalloc_array() to create an array
-                    as it is always allocated by the normal memory allocator.
+  of struct pages for vmap().
                     I created it just to be consistent.
-  rb_page_id() - The mappings require knowing where they are mapped.
+- Use flush_kernel_vmap_range() instead of flush_dcache_folio()
                  As the normal allocated pages are done in a way that they
                  may exist anywhere from the kernel's point of view, they
                  need to be labelled to know where they are mapped in user
                  space. The bpage->id is used for this. But for the persistent
                  memory, that bpage->id is already used for knowing the order
                  of the pages that are still active in the write part of
                  the buffer. This means that they are not consecutive. For
                  the user space mapping, the index of where the pages exist
                  in the physical memory is used for the placement in user
                  space. In order to manage this difference between how the
                  ids are used, this helper function handles that.
-These helper functions make the code obvious to what is being mapped
+- Allow the reserve_mem persistent ring buffer to be memory mapped.
-and how they are mapped.
+  There is no difference now with how the memory is mapped to user space,
   only the accounting of what pages are mapped where is updated as
   the meta data is different between the two.
-Changes since v1: https://lore.kernel.org/all/20250328220836.812222422@goodmis.org/
+Note, the first 4 patches makes the code a bit more correct. Especially
 since the vunmap() does not give the buffer back to the buddy allocator.
 I will be looking to get the first 4 patches into this merge window.
-- Changed map_pages counters page_count and i to unsigned long
+The last patch which enables he persistent memory mapping to user space can
 wait till the 6.16.
-  In case someone uses over 4 billion pages on a 64 bit machine to map
+Changes since v2: https://lore.kernel.org/all/20250331143426.947281958@goodmis.org/
   the memory, have the counters be unsigned long and unsigned int.
   The way machine memory is growing, this may just happen in the near future!
+- Basically a full rewrite once I found out that you can get the virtual
+  address of the memory returned by reserve_mem via phys_to_virt()!
-Steven Rostedt (2):
+Steven Rostedt (5):
-      tracing: ring-buffer: Have the ring buffer code do the vmap of physical memory
+      tracing: Enforce the persistent ring buffer to be page aligned
-      ring-buffer: Allow persistent ring buffers to be mmapped
+      tracing: Have reserve_mem use phys_to_virt() and separate from memmap buffer
       tracing: Use vmap_page_range() to map memmap ring buffer
       ring-buffer: Use flush_kernel_vmap_range() over flush_dcache_folio()
       ring-buffer: Allow reserve_mem persistent ring buffers to be mmapped
 ----
- include/linux/ring_buffer.h |  19 ++---
+ Documentation/admin-guide/kernel-parameters.txt |  2 +
- kernel/trace/ring_buffer.c  | 180 +++++++++++++++++++++++++++++++++++++++-----
+ Documentation/trace/debugging.rst               |  2 +
- kernel/trace/trace.c        |  65 ++++------------
+ kernel/trace/ring_buffer.c                      | 54 ++++++++++++++++--
-files changed, 186 insertions(+), 78 deletions(-)
+ kernel/trace/trace.c                            | 75 ++++++++++++++++---------
  kernel/trace/trace.h                            |  1 +
 files changed, 102 insertions(+), 32 deletions(-)

-New patch
+[PATCH v3 1/5] tracing: Enforce the persistent ring buffer to be page aligned
+From: Steven Rostedt <rostedt@goodmis.org>
+Enforce that the address and the size of the memory used by the persistent
+ring buffer is page aligned. Also update the documentation to reflect this
+requirement.
+Link: https://lore.kernel.org/all/CAHk-=whUOfVucfJRt7E0AH+GV41ELmS4wJqxHDnui6Giddfkzw@mail.gmail.com/
+Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
+Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
+---
+ Documentation/admin-guide/kernel-parameters.txt |  2 ++
+ Documentation/trace/debugging.rst               |  2 ++
+ kernel/trace/trace.c                            | 12 ++++++++++++
+files changed, 16 insertions(+)
+diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
+index XXXXXXX..XXXXXXX 100644
+--- a/Documentation/admin-guide/kernel-parameters.txt
++++ b/Documentation/admin-guide/kernel-parameters.txt
+@@ -XXX,XX +XXX,XX @@
+             This is just one of many ways that can clear memory. Make sure your system
+             keeps the content of memory across reboots before relying on this option.
++            NB: Both the mapped address and size must be page aligned for the architecture.
++
+             See also Documentation/trace/debugging.rst
+diff --git a/Documentation/trace/debugging.rst b/Documentation/trace/debugging.rst
+index XXXXXXX..XXXXXXX 100644
+--- a/Documentation/trace/debugging.rst
++++ b/Documentation/trace/debugging.rst
+@@ -XXX,XX +XXX,XX @@ kernel, so only the same kernel is guaranteed to work if the mapping is
+ preserved. Switching to a different kernel version may find a different
+ layout and mark the buffer as invalid.
++NB: Both the mapped address and size must be page aligned for the architecture.
++
+ Using trace_printk() in the boot instance
+ -----------------------------------------
+ By default, the content of trace_printk() goes into the top level tracing
+diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
+index XXXXXXX..XXXXXXX 100644
+--- a/kernel/trace/trace.c
++++ b/kernel/trace/trace.c
+@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
+         }
+         if (start) {
++            /* Start and size must be page aligned */
++            if (start & ~PAGE_MASK) {
++                pr_warn("Tracing: mapping start addr %lx is not page aligned\n",
++                    (unsigned long)start);
++                continue;
++            }
++            if (size & ~PAGE_MASK) {
++                pr_warn("Tracing: mapping size %lx is not page aligned\n",
++                    (unsigned long)size);
++                continue;
++            }
++
+             addr = map_pages(start, size);
+             if (addr) {
+                 pr_info("Tracing: mapped boot instance %s at physical memory %pa of size 0x%lx\n",
+--
+.47.2

-New patch
+[PATCH v3 2/5] tracing: Have reserve_mem use phys_to_virt() and separate from memmap buffer
+From: Steven Rostedt <rostedt@goodmis.org>
+The reserve_mem kernel command line option may pass back a physical
+address, but the memory is still part of the normal memory just like
+using memblock_reserve() would be. This means that the physical memory
+returned by the reserve_mem command line option can be converted directly
+to virtual memory by simply using phys_to_virt().
+When freeing the buffer allocated by reserve_mem, use free_reserved_area().
+Because the persistent ring buffer can also be allocated via the memmap
+option, which *is* different than normal memory as it cannot be added back
+to the buddy system, it must be treated differently. It still needs to be
+virtually mapped to have access to it. It also can not be freed nor can it
+ever be memory mapped to user space.
+Create a new trace_array flag called TRACE_ARRAY_FL_MEMMAP which gets set
+if the buffer is created by the memmap option, and this will prevent the
+buffer from being memory mapped by user space.
+Also increment the ref count for memmap'ed buffers so that they can never
+be freed.
+Link: https://lore.kernel.org/all/Z-wFszhJ_9o4dc8O@kernel.org/
+Suggested-by: Mike Rapoport <rppt@kernel.org>
+Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
+---
+ kernel/trace/trace.c | 28 ++++++++++++++++++++++------
+ kernel/trace/trace.h |  1 +
+files changed, 23 insertions(+), 6 deletions(-)
+diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
+index XXXXXXX..XXXXXXX 100644
+--- a/kernel/trace/trace.c
++++ b/kernel/trace/trace.c
+@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
+     struct trace_iterator *iter = &info->iter;
+     int ret = 0;
++    /* A memmap'ed buffer is not supported for user space mmap */
++    if (iter->tr->flags & TRACE_ARRAY_FL_MEMMAP)
++        return -ENODEV;
++
+     /* Currently the boot mapped buffer is not supported for mmap */
+     if (iter->tr->flags & TRACE_ARRAY_FL_BOOT)
+         return -ENODEV;
+@@ -XXX,XX +XXX,XX @@ static void free_trace_buffers(struct trace_array *tr)
+     free_trace_buffer(&tr->max_buffer);
+ #endif
+-    if (tr->range_addr_start)
+-        vunmap((void *)tr->range_addr_start);
++    if (tr->range_addr_start) {
++        void *start = (void *)tr->range_addr_start;
++        void *end = start + tr->range_addr_size;
++
++        free_reserved_area(start, end, 0, tr->range_name);
++    }
+ }
+ static void init_trace_flags_index(struct trace_array *tr)
+@@ -XXX,XX +XXX,XX @@ static inline void do_allocate_snapshot(const char *name) { }
+ __init static void enable_instances(void)
+ {
+     struct trace_array *tr;
++    bool memmap_area = false;
+     char *curr_str;
+     char *name;
+     char *str;
+@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
+                     name);
+                 continue;
+             }
++            memmap_area = true;
+         } else if (tok) {
+             if (!reserve_mem_find_by_name(tok, &start, &size)) {
+                 start = 0;
+@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
+                 continue;
+             }
+-            addr = map_pages(start, size);
++            if (memmap_area)
++                addr = map_pages(start, size);
++            else
++                addr = (unsigned long)phys_to_virt(start);
+             if (addr) {
+                 pr_info("Tracing: mapped boot instance %s at physical memory %pa of size 0x%lx\n",
+                     name, &start, (unsigned long)size);
+@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
+             update_printk_trace(tr);
+         /*
+-         * If start is set, then this is a mapped buffer, and
+-         * cannot be deleted by user space, so keep the reference
+-         * to it.
++         * memmap'd buffers can not be freed.
+          */
++        if (memmap_area) {
++            tr->flags |= TRACE_ARRAY_FL_MEMMAP;
++            tr->ref++;
++        }
++
+         if (start) {
+             tr->flags |= TRACE_ARRAY_FL_BOOT | TRACE_ARRAY_FL_LAST_BOOT;
+             tr->range_name = no_free_ptr(rname);
+diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
+index XXXXXXX..XXXXXXX 100644
+--- a/kernel/trace/trace.h
++++ b/kernel/trace/trace.h
+@@ -XXX,XX +XXX,XX @@ enum {
+     TRACE_ARRAY_FL_BOOT        = BIT(1),
+     TRACE_ARRAY_FL_LAST_BOOT    = BIT(2),
+     TRACE_ARRAY_FL_MOD_INIT        = BIT(3),
++    TRACE_ARRAY_FL_MEMMAP        = BIT(4),
+ };
+ #ifdef CONFIG_MODULES
+--
+.47.2

-[PATCH v2 1/2] tracing: ring-buffer: Have the ring buffer code do the vmap of physical memory
+[PATCH v3 3/5] tracing: Use vmap_page_range() to map memmap ring buffer
 From: Steven Rostedt <rostedt@goodmis.org>
-Currently, when reserve_mem is used on the kernel command line to reserve
+The code to map the physical memory retrieved by memmap currently
-"persistent" memory to map the ring buffer on. The tracing code will do
+allocates an array of pages to cover the physical memory and then calls
-the vmap() on the physical memory provided by reserve_mem and pass that to
+vmap() to map it to a virtual address. Instead of using this temporary
-ring_buffer_alloc_range() where it will map the ring buffer on top of the
+array of struct page descriptors, simply use vmap_page_range() that can
-given memory. It will also look at the current content of the memory there
+directly map the contiguous physical memory to a virtual address.
 and if the memory already contains valid content, it will use that content
 for the ring buffer.
-But this method makes the ring buffer code not know where that memory came
+Link: https://lore.kernel.org/all/CAHk-=whUOfVucfJRt7E0AH+GV41ELmS4wJqxHDnui6Giddfkzw@mail.gmail.com/
 from. Here, the tracing code used vmap() but it could have also used
 vmalloc(), or whatever. And many of these methods may not be supported by
 the ring buffer code.
-Instead, rename ring_buffer_alloc_range() to ring_buffer_alloc_physical()
+Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
 where contiguous physical memory is passed to the ring buffer code, and it
 will be responsible for mapping it as well as freeing it. This simplifies
 the callers from having to keep track of whether the code is mapped or
 not.
 The ring buffer can also take control of whether it can memory map the
 buffer to user space or not. Currently it does not allow this physical
 memory to be mapped to user space, but now that it has control over the
 struct pages of this memory, it can easily do so in the future.
 As the ring buffer mapping of a physical memory mapped buffer will now
 fail, the tracing code no longer keeps track if the buffer is the "last
 boot" buffer or not and will try to map it anyway. It will still fail, but
 when the ring buffer code is modified to allow it, it will then succeed.
 Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
 ---
- include/linux/ring_buffer.h | 19 ++++----
+ kernel/trace/trace.c | 35 +++++++++++++++++------------------
- kernel/trace/ring_buffer.c  | 86 ++++++++++++++++++++++++++++++++-----
+file changed, 17 insertions(+), 18 deletions(-)
  kernel/trace/trace.c        | 65 +++++++---------------------
 files changed, 101 insertions(+), 69 deletions(-)
-diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/linux/ring_buffer.h
-+++ b/include/linux/ring_buffer.h
-@@ -XXX,XX +XXX,XX @@ void ring_buffer_discard_commit(struct trace_buffer *buffer,
- struct trace_buffer *
- __ring_buffer_alloc(unsigned long size, unsigned flags, struct lock_class_key *key);
--struct trace_buffer *__ring_buffer_alloc_range(unsigned long size, unsigned flags,
--                           int order, unsigned long start,
--                           unsigned long range_size,
--                           unsigned long scratch_size,
--                           struct lock_class_key *key);
-+struct trace_buffer *__ring_buffer_alloc_physical(unsigned long size, unsigned flags,
-+                          int order, unsigned long start,
-+                          unsigned long range_size,
-+                          unsigned long scratch_size,
-+                          struct lock_class_key *key);
- void *ring_buffer_meta_scratch(struct trace_buffer *buffer, unsigned int *size);
-@@ -XXX,XX +XXX,XX @@ void *ring_buffer_meta_scratch(struct trace_buffer *buffer, unsigned int *size);
-  * traced by ftrace, it can produce lockdep warnings. We need to keep each
-  * ring buffer's lock class separate.
-  */
--#define ring_buffer_alloc_range(size, flags, order, start, range_size, s_size)    \
-+#define ring_buffer_alloc_physical(size, flags, order, start, range_size, s_size) \
- ({                                    \
-     static struct lock_class_key __key;                \
--    __ring_buffer_alloc_range((size), (flags), (order), (start),    \
--                  (range_size), (s_size), &__key);    \
-+    __ring_buffer_alloc_physical((size), (flags), (order), (start),    \
-+                     (range_size), (s_size), &__key);    \
- })
- typedef bool (*ring_buffer_cond_fn)(void *data);
-@@ -XXX,XX +XXX,XX @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order);
- int ring_buffer_subbuf_size_get(struct trace_buffer *buffer);
- enum ring_buffer_flags {
--    RB_FL_OVERWRITE        = 1 << 0,
-+    RB_FL_OVERWRITE        = BIT(0),
-+    RB_FL_PHYSICAL        = BIT(1),
- };
- #ifdef CONFIG_RING_BUFFER
-diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
-index XXXXXXX..XXXXXXX 100644
---- a/kernel/trace/ring_buffer.c
-+++ b/kernel/trace/ring_buffer.c
-@@ -XXX,XX +XXX,XX @@
- #include <linux/uaccess.h>
- #include <linux/hardirq.h>
- #include <linux/kthread.h>    /* for self test */
-+#include <linux/vmalloc.h>    /* for vmap */
- #include <linux/module.h>
- #include <linux/percpu.h>
- #include <linux/mutex.h>
-@@ -XXX,XX +XXX,XX @@ struct trace_buffer {
-     struct ring_buffer_meta        *meta;
-+    unsigned long            phys_start;
-+
-     unsigned int            subbuf_size;
-     unsigned int            subbuf_order;
-     unsigned int            max_data_size;
-@@ -XXX,XX +XXX,XX @@ static void rb_free_cpu_buffer(struct ring_buffer_per_cpu *cpu_buffer)
-     kfree(cpu_buffer);
- }
-+static unsigned long map_pages(unsigned long *start, unsigned long *end)
-+{
-+    struct page **pages;
-+    phys_addr_t page_start;
-+    unsigned long size;
-+    unsigned long page_count;
-+    unsigned long i;
-+    void *vaddr;
-+
-+    /* Make sure the mappings are page aligned */
-+    *start = ALIGN(*start, PAGE_SIZE);
-+
-+    size = *end - *start;
-+
-+    /* The size must fit full pages */
-+    page_count = size >> PAGE_SHIFT;
-+
-+    if (!page_count)
-+        return 0;
-+
-+    page_start = *start;
-+    pages = kmalloc_array(page_count, sizeof(struct page *), GFP_KERNEL);
-+    if (!pages)
-+        return 0;
-+
-+    for (i = 0; i < page_count; i++) {
-+        phys_addr_t addr = page_start + i * PAGE_SIZE;
-+        pages[i] = pfn_to_page(addr >> PAGE_SHIFT);
-+    }
-+    vaddr = vmap(pages, page_count, VM_MAP, PAGE_KERNEL);
-+    kfree(pages);
-+
-+    *end = *start + page_count * PAGE_SIZE;
-+
-+    return (unsigned long)vaddr;
-+}
-+
- static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
-                      int order, unsigned long start,
-                      unsigned long end,
-@@ -XXX,XX +XXX,XX @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
-     if (!buffer->buffers)
-         goto fail_free_cpumask;
--    /* If start/end are specified, then that overrides size */
-+    /* If start/end are specified, then this is a physical mapping */
-     if (start && end) {
-         unsigned long buffers_start;
-+        unsigned long addr;
-         unsigned long ptr;
-+        u64 size;
-         int n;
--        /* Make sure that start is word aligned */
--        start = ALIGN(start, sizeof(long));
-+        addr = map_pages(&start, &end);
-+        if (!addr)
-+            goto fail_free_cpumask;
-+
-+        /* end and start have have been updated for alignment */
-+        size = end - start;
-+
-+        buffer->phys_start = start;
-+        buffer->flags |= RB_FL_PHYSICAL;
-+
-+        start = addr;
-+        end = start + size;
-         /* scratch_size needs to be aligned too */
-         scratch_size = ALIGN(scratch_size, sizeof(long));
-@@ -XXX,XX +XXX,XX @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
-     }
-     kfree(buffer->buffers);
-+    if (buffer->phys_start)
-+        vunmap((void *)buffer->phys_start);
-+
-  fail_free_cpumask:
-     free_cpumask_var(buffer->cpumask);
-@@ -XXX,XX +XXX,XX @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags,
- EXPORT_SYMBOL_GPL(__ring_buffer_alloc);
- /**
-- * __ring_buffer_alloc_range - allocate a new ring_buffer from existing memory
-+ * __ring_buffer_alloc_physical - allocate a new ring_buffer from physical memory
-  * @size: the size in bytes per cpu that is needed.
-  * @flags: attributes to set for the ring buffer.
-  * @order: sub-buffer order
-- * @start: start of allocated range
-+ * @start: start of the physical memory range
-  * @range_size: size of allocated range
-  * @scratch_size: size of scratch area (for preallocated memory buffers)
-  * @key: ring buffer reader_lock_key.
-@@ -XXX,XX +XXX,XX @@ EXPORT_SYMBOL_GPL(__ring_buffer_alloc);
-  * when the buffer wraps. If this flag is not set, the buffer will
-  * drop data when the tail hits the head.
-  */
--struct trace_buffer *__ring_buffer_alloc_range(unsigned long size, unsigned flags,
--                           int order, unsigned long start,
--                           unsigned long range_size,
--                           unsigned long scratch_size,
--                           struct lock_class_key *key)
-+struct trace_buffer *__ring_buffer_alloc_physical(unsigned long size, unsigned flags,
-+                          int order, unsigned long start,
-+                          unsigned long range_size,
-+                          unsigned long scratch_size,
-+                          struct lock_class_key *key)
- {
-     return alloc_buffer(size, flags, order, start, start + range_size,
-                 scratch_size, key);
-@@ -XXX,XX +XXX,XX @@ ring_buffer_free(struct trace_buffer *buffer)
-     kfree(buffer->buffers);
-     free_cpumask_var(buffer->cpumask);
-+    if (buffer->flags & RB_FL_PHYSICAL)
-+        vunmap((void *)buffer->phys_start);
-+
-     kfree(buffer);
- }
- EXPORT_SYMBOL_GPL(ring_buffer_free);
-@@ -XXX,XX +XXX,XX @@ int ring_buffer_map(struct trace_buffer *buffer, int cpu,
-     unsigned long flags, *subbuf_ids;
-     int err = 0;
-+    /*
-+     * Currently, this does not support vmap()'d buffers.
-+     * Return -ENODEV as that is what is returned when a file
-+     * does not support memory mapping.
-+     */
-+    if (buffer->flags & RB_FL_PHYSICAL)
-+        return -ENODEV;
-+
-     if (!cpumask_test_cpu(cpu, buffer->cpumask))
-         return -EINVAL;
 diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
 index XXXXXXX..XXXXXXX 100644
 --- a/kernel/trace/trace.c
 +++ b/kernel/trace/trace.c
-@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
+@@ -XXX,XX +XXX,XX @@
-     struct trace_iterator *iter = &info->iter;
+ #include <linux/irq_work.h>
-     int ret = 0;
+ #include <linux/workqueue.h>
+ #include <linux/sort.h>
--    /* Currently the boot mapped buffer is not supported for mmap */
++#include <linux/io.h> /* vmap_page_range() */
--    if (iter->tr->flags & TRACE_ARRAY_FL_BOOT)
--        return -ENODEV;
+ #include <asm/setup.h> /* COMMAND_LINE_SIZE */
--
      ret = get_snapshot_map(iter->tr);
      if (ret)
          return ret;
@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
      ret = ring_buffer_map(iter->array_buffer->buffer, iter->cpu_file, vma);
      if (ret)
          put_snapshot_map(iter->tr);
 -
 -    vma->vm_ops = &tracing_buffers_vmops;
 +    else
 +        vma->vm_ops = &tracing_buffers_vmops;
      return ret;
  }
@@ -XXX,XX +XXX,XX @@ allocate_trace_buffer(struct trace_array *tr, struct array_buffer *buf, int size
      if (tr->range_addr_start && tr->range_addr_size) {
          /* Add scratch buffer to handle 128 modules */
 -        buf->buffer = ring_buffer_alloc_range(size, rb_flags, 0,
 -                              tr->range_addr_start,
 -                              tr->range_addr_size,
 -                              struct_size(tscratch, entries, 128));
 +        buf->buffer = ring_buffer_alloc_physical(size, rb_flags, 0,
 +                             tr->range_addr_start,
 +                             tr->range_addr_size,
 +                             struct_size(tscratch, entries, 128));
 +        if (!buf->buffer) {
 +            pr_warn("Tracing: Failed to map boot instance %s\n", tr->name);
 +            return -1;
 +        }
 +
 +        pr_info("Tracing: mapped boot instance %s at physical memory %lx of size 0x%lx\n",
 +            tr->name, tr->range_addr_start, tr->range_addr_size);
          tscratch = ring_buffer_meta_scratch(buf->buffer, &scratch_size);
          setup_trace_scratch(tr, tscratch, scratch_size);
@@ -XXX,XX +XXX,XX @@ static void free_trace_buffers(struct trace_array *tr)
  #ifdef CONFIG_TRACER_MAX_TRACE
      free_trace_buffer(&tr->max_buffer);
  #endif
 -
 -    if (tr->range_addr_start)
 -        vunmap((void *)tr->range_addr_start);
  }
  static void init_trace_flags_index(struct trace_array *tr)
 @@ -XXX,XX +XXX,XX @@ static int instance_mkdir(const char *name)
      return ret;
  }
 -static u64 map_pages(u64 start, u64 size)
--{
++static u64 map_pages(unsigned long start, unsigned long size)
  {
 -    struct page **pages;
 -    phys_addr_t page_start;
 -    unsigned int page_count;
 -    unsigned int i;
 -    void *vaddr;
--
++        unsigned long vmap_start, vmap_end;
 +    struct vm_struct *area;
 +    int ret;
 -    page_count = DIV_ROUND_UP(size, PAGE_SIZE);
--
++        area = get_vm_area(size, VM_IOREMAP);
 +        if (!area)
 +                return 0;
 -    page_start = start;
 -    pages = kmalloc_array(page_count, sizeof(struct page *), GFP_KERNEL);
 -    if (!pages)
 -        return 0;
--
++        vmap_start = (unsigned long) area->addr;
 +        vmap_end = vmap_start + size;
 -    for (i = 0; i < page_count; i++) {
 -        phys_addr_t addr = page_start + i * PAGE_SIZE;
 -        pages[i] = pfn_to_page(addr >> PAGE_SHIFT);
 -    }
 -    vaddr = vmap(pages, page_count, VM_MAP, PAGE_KERNEL);
 -    kfree(pages);
--
++        ret = vmap_page_range(vmap_start, vmap_end,
 +                  start, pgprot_nx(PAGE_KERNEL));
 +        if (ret < 0) {
 +                free_vm_area(area);
 +                return 0;
 +        }
 -    return (u64)(unsigned long)vaddr;
--}
++    return (u64)vmap_start;
--
+ }
  /**
-  * trace_array_get_by_name - Create/Lookup a trace array, given its name.
-  * @name: The name of the trace array to be looked up/created.
-@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
-     while ((curr_str = strsep(&str, "\t"))) {
-         phys_addr_t start = 0;
-         phys_addr_t size = 0;
--        unsigned long addr = 0;
-         bool traceprintk = false;
-         bool traceoff = false;
-         char *flag_delim;
-@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
-             rname = kstrdup(tok, GFP_KERNEL);
-         }
--        if (start) {
--            addr = map_pages(start, size);
--            if (addr) {
--                pr_info("Tracing: mapped boot instance %s at physical memory %pa of size 0x%lx\n",
--                    name, &start, (unsigned long)size);
--            } else {
--                pr_warn("Tracing: Failed to map boot instance %s\n", name);
--                continue;
--            }
--        } else {
-+        if (!start) {
-             /* Only non mapped buffers have snapshot buffers */
-             if (IS_ENABLED(CONFIG_TRACER_MAX_TRACE))
-                 do_allocate_snapshot(name);
-         }
--        tr = trace_array_create_systems(name, NULL, addr, size);
-+        tr = trace_array_create_systems(name, NULL, (unsigned long)start, size);
-         if (IS_ERR(tr)) {
-             pr_warn("Tracing: Failed to create instance buffer %s\n", curr_str);
-             continue;
 --
 .47.2

-New patch
+[PATCH v3 4/5] ring-buffer: Use flush_kernel_vmap_range() over flush_dcache_folio()
+From: Steven Rostedt <rostedt@goodmis.org>
+Some architectures do not have data cache coherency between user and
+kernel space. For these architectures, the cache needs to be flushed on
+both the kernel and user addresses so that user space can see the updates
+the kernel has made.
+Instead of using flush_dcache_folio() and playing with virt_to_folio()
+within the call to that function, use flush_kernel_vmap_range() which
+takes the virtual address and does the work for those architectures that
+need it.
+Link: https://lore.kernel.org/all/CAG48ez3w0my4Rwttbc5tEbNsme6tc0mrSN95thjXUFaJ3aQ6SA@mail.gmail.com/
+Suggested-by: Jann Horn <jannh@google.com>
+Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
+---
+ kernel/trace/ring_buffer.c | 5 +++--
+file changed, 3 insertions(+), 2 deletions(-)
+diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
+index XXXXXXX..XXXXXXX 100644
+--- a/kernel/trace/ring_buffer.c
++++ b/kernel/trace/ring_buffer.c
+@@ -XXX,XX +XXX,XX @@ static void rb_update_meta_page(struct ring_buffer_per_cpu *cpu_buffer)
+     meta->read = cpu_buffer->read;
+     /* Some archs do not have data cache coherency between kernel and user-space */
+-    flush_dcache_folio(virt_to_folio(cpu_buffer->meta_page));
++    flush_kernel_vmap_range(cpu_buffer->meta_page, PAGE_SIZE);
+ }
+ static void
+@@ -XXX,XX +XXX,XX @@ int ring_buffer_map_get_reader(struct trace_buffer *buffer, int cpu)
+ out:
+     /* Some archs do not have data cache coherency between kernel and user-space */
+-    flush_dcache_folio(virt_to_folio(cpu_buffer->reader_page->page));
++    flush_kernel_vmap_range(cpu_buffer->reader_page->page,
++                buffer->subbuf_size + BUF_PAGE_HDR_SIZE);
+     rb_update_meta_page(cpu_buffer);
+--
+.47.2

-[PATCH v2 2/2] ring-buffer: Allow persistent ring buffers to be mmapped
+[PATCH v3 5/5] ring-buffer: Allow reserve_mem persistent ring buffers to be mmapped
 From: Steven Rostedt <rostedt@goodmis.org>
-The persistent ring buffer uses vmap()'d memory to map the reserved memory
+When the persistent ring buffer is created from the memory returned by
-from boot. But the user space mmap() to the ring buffer requires
+reserve_mem there is nothing prohibiting it to be memory mapped to user
-virt_to_page() to return a valid page. But that only works for core kernel
+space. The memory is the same as the pages allocated by alloc_page().
 addresses and not for vmap() addresses.
-To address this, save the physical and virtual address of where the
+The way the memory is managed by the ring buffer code is slightly
-persistent memory is mapped. Create a rb_struct_page() helper function
+different though and needs to be addressed.
 that returns the page via virt_to_page() for normal buffer pages that were
 allocated with page_alloc() but for the physical memory vmap()'d pages, it
 uses the saved physical and virtual addresses of where the memory was
 located to calculate the physical address of the virtual page that needs
 the struct page. Then it uses pfn_to_page() to get the page for that
 physical address.
-New helper functions are created for flushing the cache for architectures
+The persistent memory uses the page->id for its own purpose where as the
-that need it between user and kernel space.
+user mmap buffer currently uses that for the subbuf array mapped to user
+space. If the buffer is a persistent buffer, use the page index into that
-Also, the persistent memory uses the page->id for its own purpose where as
+buffer as the identifier instead of the page->id.
 the user mmap buffer currently uses that for the subbuf array mapped to
 user space. If the buffer is a persistent buffer, use the page index into
 that buffer as the identifier instead of the page->id.
 That is, the page->id for a persistent buffer, represents the order of the
 buffer is in the link list. ->id == 0 means it is the reader page.
 When a reader page is swapped, the new reader page's ->id gets zero, and
 the old reader page gets the ->id of the page that it swapped with.
 ...
 A new rb_page_id() helper function is used to get and set the id depending
 on if the page is a normal memory allocated buffer or a physical memory
 mapped buffer.
-Link: https://lore.kernel.org/all/CAHk-=wgD9MQOoMAGtT=fXZsWY39exRVyZgxuBXix4u=1bdHA=g@mail.gmail.com/
 Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
 ---
- kernel/trace/ring_buffer.c | 110 +++++++++++++++++++++++++++++++------
+ kernel/trace/ring_buffer.c | 49 ++++++++++++++++++++++++++++++++++----
-file changed, 93 insertions(+), 17 deletions(-)
+ kernel/trace/trace.c       |  4 ----
 files changed, 45 insertions(+), 8 deletions(-)
 diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
 index XXXXXXX..XXXXXXX 100644
 --- a/kernel/trace/ring_buffer.c
 +++ b/kernel/trace/ring_buffer.c
-@@ -XXX,XX +XXX,XX @@ struct trace_buffer {
-     struct ring_buffer_meta        *meta;
-     unsigned long            phys_start;
-+    unsigned long            virt_start;
-     unsigned int            subbuf_size;
-     unsigned int            subbuf_order;
-@@ -XXX,XX +XXX,XX @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
-         start = addr;
-         end = start + size;
-+        buffer->virt_start = start;
-         /* scratch_size needs to be aligned too */
-         scratch_size = ALIGN(scratch_size, sizeof(long));
 @@ -XXX,XX +XXX,XX @@ static void rb_clear_buffer_page(struct buffer_page *page)
      page->read = 0;
  }
-+/*
-+ * Get the struct page for the given buffer page.
-+ *
-+ * For normal ring buffer pages that are allocated via page_alloc()
-+ * the struct page can simply be retrieved via virt_to_page().
-+ *
-+ * But if the buffer was created via a physical mapping and vmap()
-+ * was used to get to the virtual addresses, use the stored virtual
-+ * and physical of the start address to calculate the original
-+ * physical address of the given page and use pfn_to_page() to return
-+ * the struct page.
-+ */
-+static struct page *rb_struct_page(struct trace_buffer *buffer, void *vaddr)
-+{
-+    if (buffer->flags & RB_FL_PHYSICAL) {
-+        unsigned long addr = (unsigned long)vaddr;
-+
-+        addr -= buffer->virt_start;
-+        addr += buffer->phys_start;
-+        return pfn_to_page(addr >> PAGE_SHIFT);
-+    }
-+    return virt_to_page(vaddr);
-+}
-+
-+/* Some archs do not have data cache coherency between kernel and user-space */
-+static void rb_flush_buffer_page(struct trace_buffer *buffer,
-+                 struct buffer_page *bpage)
-+{
-+    struct page *page = rb_struct_page(buffer, bpage->page);
-+
-+    flush_dcache_folio(page_folio(page));
-+}
-+
-+/* The user mapped meta page is always allocated via page_alloc() */
-+static void rb_flush_meta(void *meta)
-+{
-+    struct page *page = virt_to_page(meta);
-+
-+    flush_dcache_folio(page_folio(page));
-+}
-+
 +/*
 + * When the buffer is memory mapped to user space, each sub buffer
 + * has a unique id that is used by the meta data to tell the user
 + * where the current reader page is.
 + *
 + * For a normal allocated ring buffer, the id is saved in the buffer page
 + * id field, and updated via this function.
 + *
-+ * But for a physical memory mapped buffer, the id is already assigned for
++ * But for a fixed memory mapped buffer, the id is already assigned for
-+ * memory ording in the physical memory layout and can not be used. Instead
++ * fixed memory ording in the memory layout and can not be used. Instead
 + * the index of where the page lies in the memory layout is used.
 + *
 + * For the normal pages, set the buffer page id with the passed in @id
 + * value and return that.
 + *
-+ * For memory mapped pages, get the page index in the physical memory layout
++ * For fixed memory mapped pages, get the page index in the memory layout
 + * and return that as the id.
 + */
 +static int rb_page_id(struct ring_buffer_per_cpu *cpu_buffer,
 +              struct buffer_page *bpage, int id)
 +{
 ...
 +                     cpu_buffer->reader_page->id);
 +
      meta->reader.lost_events = cpu_buffer->lost_events;
      meta->entries = local_read(&cpu_buffer->entries);
-     meta->overrun = local_read(&cpu_buffer->overrun);
-     meta->read = cpu_buffer->read;
--    /* Some archs do not have data cache coherency between kernel and user-space */
--    flush_dcache_folio(virt_to_folio(cpu_buffer->meta_page));
-+    rb_flush_meta(meta);
- }
- static void
 @@ -XXX,XX +XXX,XX @@ static void rb_setup_ids_meta_page(struct ring_buffer_per_cpu *cpu_buffer,
      struct trace_buffer_meta *meta = cpu_buffer->meta_page;
      unsigned int nr_subbufs = cpu_buffer->nr_pages + 1;
      struct buffer_page *first_subbuf, *subbuf;
 +    int cnt = 0;
 ...
 +    WARN_ON(cnt != nr_subbufs);
 +
      /* install subbuf ID to kern VA translation */
      cpu_buffer->subbuf_ids = subbuf_ids;
-@@ -XXX,XX +XXX,XX @@ static int __rb_map_vma(struct ring_buffer_per_cpu *cpu_buffer,
+diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
-     if (!pgoff) {
+index XXXXXXX..XXXXXXX 100644
-         unsigned long meta_page_padding;
+--- a/kernel/trace/trace.c
++++ b/kernel/trace/trace.c
-+        /* The meta page is always allocated via alloc_page() */
+@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
-         pages[p++] = virt_to_page(cpu_buffer->meta_page);
+     if (iter->tr->flags & TRACE_ARRAY_FL_MEMMAP)
+         return -ENODEV;
-         /*
-@@ -XXX,XX +XXX,XX @@ static int __rb_map_vma(struct ring_buffer_per_cpu *cpu_buffer,
+-    /* Currently the boot mapped buffer is not supported for mmap */
-             goto out;
+-    if (iter->tr->flags & TRACE_ARRAY_FL_BOOT)
          }
 -        page = virt_to_page((void *)cpu_buffer->subbuf_ids[s]);
 +        page = rb_struct_page(cpu_buffer->buffer,
 +                      (void *)cpu_buffer->subbuf_ids[s]);
          for (; off < (1 << (subbuf_order)); off++, page++) {
              if (p >= nr_pages)
@@ -XXX,XX +XXX,XX @@ int ring_buffer_map(struct trace_buffer *buffer, int cpu,
      unsigned long flags, *subbuf_ids;
      int err = 0;
 -    /*
 -     * Currently, this does not support vmap()'d buffers.
 -     * Return -ENODEV as that is what is returned when a file
 -     * does not support memory mapping.
 -     */
 -    if (buffer->flags & RB_FL_PHYSICAL)
 -        return -ENODEV;
 -
-     if (!cpumask_test_cpu(cpu, buffer->cpumask))
+     ret = get_snapshot_map(iter->tr);
-         return -EINVAL;
+     if (ret)
+         return ret;
@@ -XXX,XX +XXX,XX @@ int ring_buffer_map_get_reader(struct trace_buffer *buffer, int cpu)
      goto consume;
  out:
 -    /* Some archs do not have data cache coherency between kernel and user-space */
 -    flush_dcache_folio(virt_to_folio(cpu_buffer->reader_page->page));
 +    rb_flush_buffer_page(buffer, cpu_buffer->reader_page);
      rb_update_meta_page(cpu_buffer);
 --
 .47.2

Allow the persistent memory mapped memory to be memory mapped to user
space as well. Currently, the user space memory mapping requires the
buffers to have been allocated via page_alloc() and converted to virtual
address via page_address(). But the persistent memory is memory mapped
via vmap() and a simple virt_to_page() can not be used. Move the control
of the physical mapping via vmap() to the ring buffer code so that it
can then use the saved physical and virtual mapping to find the pages
needed for memory mapping user space.

The first patch moves the memory mapping of the physical memory returned
by reserve_mem from the tracing code to the ring buffer code. This makes
sense as this gives more control over to the ring buffer in knowing exactly
how the pages were created. It keeps track of where the physical memory
that was mapped and also handles the freeing of this memory (removing the
burden from the tracing code from having to do this). It also handles
knowing if the buffer may be memory mapped or not. The check is removed
from the tracing code, but if the tracing code tries to memory map the
persistent ring buffer, the call to the ring buffer code will fail with
the same error as before.

The second patch implements the user space memory mapping of the persistent
ring buffer. It does so by adding several helper functions to annotate
what the code is doing. Note, there's two meta pages here. One is mapped
between the kernel and user space and is used to inform user space of updates
to the ring buffer. The other is inside the persistent memory that is used to
pass information across boots. The persistent memory meta data is never exposed
to user space. The meta data for user space mapping is always allocated via the
normal memory allocation.

The helper functions are:

rb_struct_page() - This is the rb_get_page() from our discussions, but
                    I renamed it because "get" implies "put".
                    This function will return the struct page for a given
                    buffer page by either virt_to_page() if the page was
                    allocated via the normal memory allocator, or it
                    is found via pfn_to_page() by using the saved physical
                    and virtual address of the mapped location. It uses
                    that to calculate the physical address from the virtual
                    address of the page and then pfn_to_page() can be used
                    from that.

rb_fush_buffer_page() - this calls the above rb_struct_page() and then
                    calls flush_dcache_folio() to make sure the kernel
                    and user space is coherent.

rb_flush_meta() - This just uses virt_to_page() and calls flush_dcache_folio()
                    as it is always allocated by the normal memory allocator.
                    I created it just to be consistent.

rb_page_id() - The mappings require knowing where they are mapped.
                 As the normal allocated pages are done in a way that they
                 may exist anywhere from the kernel's point of view, they
                 need to be labelled to know where they are mapped in user
                 space. The bpage->id is used for this. But for the persistent
                 memory, that bpage->id is already used for knowing the order
                 of the pages that are still active in the write part of
                 the buffer. This means that they are not consecutive. For
                 the user space mapping, the index of where the pages exist
                 in the physical memory is used for the placement in user
                 space. In order to manage this difference between how the
                 ids are used, this helper function handles that.

These helper functions make the code obvious to what is being mapped
and how they are mapped.

Changes since v1: https://lore.kernel.org/all/20250328220836.812222422@goodmis.org/

- Changed map_pages counters page_count and i to unsigned long

In case someone uses over 4 billion pages on a 64 bit machine to map
  the memory, have the counters be unsigned long and unsigned int.
  The way machine memory is growing, this may just happen in the near future!

Steven Rostedt (2):
      tracing: ring-buffer: Have the ring buffer code do the vmap of physical memory
      ring-buffer: Allow persistent ring buffers to be mmapped

----
 include/linux/ring_buffer.h |  19 ++---
 kernel/trace/ring_buffer.c  | 180 +++++++++++++++++++++++++++++++++++++++-----
 kernel/trace/trace.c        |  65 ++++------------
 3 files changed, 186 insertions(+), 78 deletions(-)

From: Steven Rostedt <rostedt@goodmis.org>

Currently, when reserve_mem is used on the kernel command line to reserve
"persistent" memory to map the ring buffer on. The tracing code will do
the vmap() on the physical memory provided by reserve_mem and pass that to
ring_buffer_alloc_range() where it will map the ring buffer on top of the
given memory. It will also look at the current content of the memory there
and if the memory already contains valid content, it will use that content
for the ring buffer.

But this method makes the ring buffer code not know where that memory came
from. Here, the tracing code used vmap() but it could have also used
vmalloc(), or whatever. And many of these methods may not be supported by
the ring buffer code.

Instead, rename ring_buffer_alloc_range() to ring_buffer_alloc_physical()
where contiguous physical memory is passed to the ring buffer code, and it
will be responsible for mapping it as well as freeing it. This simplifies
the callers from having to keep track of whether the code is mapped or
not.

The ring buffer can also take control of whether it can memory map the
buffer to user space or not. Currently it does not allow this physical
memory to be mapped to user space, but now that it has control over the
struct pages of this memory, it can easily do so in the future.

As the ring buffer mapping of a physical memory mapped buffer will now
fail, the tracing code no longer keeps track if the buffer is the "last
boot" buffer or not and will try to map it anyway. It will still fail, but
when the ring buffer code is modified to allow it, it will then succeed.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 include/linux/ring_buffer.h | 19 ++++----
 kernel/trace/ring_buffer.c  | 86 ++++++++++++++++++++++++++++++++-----
 kernel/trace/trace.c        | 65 +++++++---------------------
 3 files changed, 101 insertions(+), 69 deletions(-)

diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
index XXXXXXX..XXXXXXX 100644
--- a/include/linux/ring_buffer.h
+++ b/include/linux/ring_buffer.h
@@ -XXX,XX +XXX,XX @@ void ring_buffer_discard_commit(struct trace_buffer *buffer,
 struct trace_buffer *
 __ring_buffer_alloc(unsigned long size, unsigned flags, struct lock_class_key *key);
 
-struct trace_buffer *__ring_buffer_alloc_range(unsigned long size, unsigned flags,
-					       int order, unsigned long start,
-					       unsigned long range_size,
-					       unsigned long scratch_size,
-					       struct lock_class_key *key);
+struct trace_buffer *__ring_buffer_alloc_physical(unsigned long size, unsigned flags,
+						  int order, unsigned long start,
+						  unsigned long range_size,
+						  unsigned long scratch_size,
+						  struct lock_class_key *key);
 
 void *ring_buffer_meta_scratch(struct trace_buffer *buffer, unsigned int *size);
 
@@ -XXX,XX +XXX,XX @@ void *ring_buffer_meta_scratch(struct trace_buffer *buffer, unsigned int *size);
  * traced by ftrace, it can produce lockdep warnings. We need to keep each
  * ring buffer's lock class separate.
  */
-#define ring_buffer_alloc_range(size, flags, order, start, range_size, s_size)	\
+#define ring_buffer_alloc_physical(size, flags, order, start, range_size, s_size) \
 ({									\
 	static struct lock_class_key __key;				\
-	__ring_buffer_alloc_range((size), (flags), (order), (start),	\
-				  (range_size), (s_size), &__key);	\
+	__ring_buffer_alloc_physical((size), (flags), (order), (start),	\
+				     (range_size), (s_size), &__key);	\
 })
 
 typedef bool (*ring_buffer_cond_fn)(void *data);
@@ -XXX,XX +XXX,XX @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order);
 int ring_buffer_subbuf_size_get(struct trace_buffer *buffer);
 
 enum ring_buffer_flags {
-	RB_FL_OVERWRITE		= 1 << 0,
+	RB_FL_OVERWRITE		= BIT(0),
+	RB_FL_PHYSICAL		= BIT(1),
 };
 
 #ifdef CONFIG_RING_BUFFER
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -XXX,XX +XXX,XX @@
 #include <linux/uaccess.h>
 #include <linux/hardirq.h>
 #include <linux/kthread.h>	/* for self test */
+#include <linux/vmalloc.h>	/* for vmap */
 #include <linux/module.h>
 #include <linux/percpu.h>
 #include <linux/mutex.h>
@@ -XXX,XX +XXX,XX @@ struct trace_buffer {
 
 	struct ring_buffer_meta		*meta;
 
+	unsigned long			phys_start;
+
 	unsigned int			subbuf_size;
 	unsigned int			subbuf_order;
 	unsigned int			max_data_size;
@@ -XXX,XX +XXX,XX @@ static void rb_free_cpu_buffer(struct ring_buffer_per_cpu *cpu_buffer)
 	kfree(cpu_buffer);
 }
 
+static unsigned long map_pages(unsigned long *start, unsigned long *end)
+{
+	struct page **pages;
+	phys_addr_t page_start;
+	unsigned long size;
+	unsigned long page_count;
+	unsigned long i;
+	void *vaddr;
+
+	/* Make sure the mappings are page aligned */
+	*start = ALIGN(*start, PAGE_SIZE);
+
+	size = *end - *start;
+
+	/* The size must fit full pages */
+	page_count = size >> PAGE_SHIFT;
+
+	if (!page_count)
+		return 0;
+
+	page_start = *start;
+	pages = kmalloc_array(page_count, sizeof(struct page *), GFP_KERNEL);
+	if (!pages)
+		return 0;
+
+	for (i = 0; i < page_count; i++) {
+		phys_addr_t addr = page_start + i * PAGE_SIZE;
+		pages[i] = pfn_to_page(addr >> PAGE_SHIFT);
+	}
+	vaddr = vmap(pages, page_count, VM_MAP, PAGE_KERNEL);
+	kfree(pages);
+
+	*end = *start + page_count * PAGE_SIZE;
+
+	return (unsigned long)vaddr;
+}
+
 static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
 					 int order, unsigned long start,
 					 unsigned long end,
@@ -XXX,XX +XXX,XX @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
 	if (!buffer->buffers)
 		goto fail_free_cpumask;
 
-	/* If start/end are specified, then that overrides size */
+	/* If start/end are specified, then this is a physical mapping */
 	if (start && end) {
 		unsigned long buffers_start;
+		unsigned long addr;
 		unsigned long ptr;
+		u64 size;
 		int n;
 
-		/* Make sure that start is word aligned */
-		start = ALIGN(start, sizeof(long));
+		addr = map_pages(&start, &end);
+		if (!addr)
+			goto fail_free_cpumask;
+
+		/* end and start have have been updated for alignment */
+		size = end - start;
+
+		buffer->phys_start = start;
+		buffer->flags |= RB_FL_PHYSICAL;
+
+		start = addr;
+		end = start + size;
 
 		/* scratch_size needs to be aligned too */
 		scratch_size = ALIGN(scratch_size, sizeof(long));
@@ -XXX,XX +XXX,XX @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
 	}
 	kfree(buffer->buffers);
 
+	if (buffer->phys_start)
+		vunmap((void *)buffer->phys_start);
+
  fail_free_cpumask:
 	free_cpumask_var(buffer->cpumask);
 
@@ -XXX,XX +XXX,XX @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags,
 EXPORT_SYMBOL_GPL(__ring_buffer_alloc);
 
 /**
- * __ring_buffer_alloc_range - allocate a new ring_buffer from existing memory
+ * __ring_buffer_alloc_physical - allocate a new ring_buffer from physical memory
  * @size: the size in bytes per cpu that is needed.
  * @flags: attributes to set for the ring buffer.
  * @order: sub-buffer order
- * @start: start of allocated range
+ * @start: start of the physical memory range
  * @range_size: size of allocated range
  * @scratch_size: size of scratch area (for preallocated memory buffers)
  * @key: ring buffer reader_lock_key.
@@ -XXX,XX +XXX,XX @@ EXPORT_SYMBOL_GPL(__ring_buffer_alloc);
  * when the buffer wraps. If this flag is not set, the buffer will
  * drop data when the tail hits the head.
  */
-struct trace_buffer *__ring_buffer_alloc_range(unsigned long size, unsigned flags,
-					       int order, unsigned long start,
-					       unsigned long range_size,
-					       unsigned long scratch_size,
-					       struct lock_class_key *key)
+struct trace_buffer *__ring_buffer_alloc_physical(unsigned long size, unsigned flags,
+						  int order, unsigned long start,
+						  unsigned long range_size,
+						  unsigned long scratch_size,
+						  struct lock_class_key *key)
 {
 	return alloc_buffer(size, flags, order, start, start + range_size,
 			    scratch_size, key);
@@ -XXX,XX +XXX,XX @@ ring_buffer_free(struct trace_buffer *buffer)
 	kfree(buffer->buffers);
 	free_cpumask_var(buffer->cpumask);
 
+	if (buffer->flags & RB_FL_PHYSICAL)
+		vunmap((void *)buffer->phys_start);
+
 	kfree(buffer);
 }
 EXPORT_SYMBOL_GPL(ring_buffer_free);
@@ -XXX,XX +XXX,XX @@ int ring_buffer_map(struct trace_buffer *buffer, int cpu,
 	unsigned long flags, *subbuf_ids;
 	int err = 0;
 
+	/*
+	 * Currently, this does not support vmap()'d buffers.
+	 * Return -ENODEV as that is what is returned when a file
+	 * does not support memory mapping.
+	 */
+	if (buffer->flags & RB_FL_PHYSICAL)
+		return -ENODEV;
+
 	if (!cpumask_test_cpu(cpu, buffer->cpumask))
 		return -EINVAL;
 
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
 	struct trace_iterator *iter = &info->iter;
 	int ret = 0;
 
-	/* Currently the boot mapped buffer is not supported for mmap */
-	if (iter->tr->flags & TRACE_ARRAY_FL_BOOT)
-		return -ENODEV;
-
 	ret = get_snapshot_map(iter->tr);
 	if (ret)
 		return ret;
@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
 	ret = ring_buffer_map(iter->array_buffer->buffer, iter->cpu_file, vma);
 	if (ret)
 		put_snapshot_map(iter->tr);
-
-	vma->vm_ops = &tracing_buffers_vmops;
+	else
+		vma->vm_ops = &tracing_buffers_vmops;
 
 	return ret;
 }
@@ -XXX,XX +XXX,XX @@ allocate_trace_buffer(struct trace_array *tr, struct array_buffer *buf, int size
 
 	if (tr->range_addr_start && tr->range_addr_size) {
 		/* Add scratch buffer to handle 128 modules */
-		buf->buffer = ring_buffer_alloc_range(size, rb_flags, 0,
-						      tr->range_addr_start,
-						      tr->range_addr_size,
-						      struct_size(tscratch, entries, 128));
+		buf->buffer = ring_buffer_alloc_physical(size, rb_flags, 0,
+							 tr->range_addr_start,
+							 tr->range_addr_size,
+							 struct_size(tscratch, entries, 128));
+		if (!buf->buffer) {
+			pr_warn("Tracing: Failed to map boot instance %s\n", tr->name);
+			return -1;
+		}
+
+		pr_info("Tracing: mapped boot instance %s at physical memory %lx of size 0x%lx\n",
+			tr->name, tr->range_addr_start, tr->range_addr_size);
 
 		tscratch = ring_buffer_meta_scratch(buf->buffer, &scratch_size);
 		setup_trace_scratch(tr, tscratch, scratch_size);
@@ -XXX,XX +XXX,XX @@ static void free_trace_buffers(struct trace_array *tr)
 #ifdef CONFIG_TRACER_MAX_TRACE
 	free_trace_buffer(&tr->max_buffer);
 #endif
-
-	if (tr->range_addr_start)
-		vunmap((void *)tr->range_addr_start);
 }
 
 static void init_trace_flags_index(struct trace_array *tr)
@@ -XXX,XX +XXX,XX @@ static int instance_mkdir(const char *name)
 	return ret;
 }
 
-static u64 map_pages(u64 start, u64 size)
-{
-	struct page **pages;
-	phys_addr_t page_start;
-	unsigned int page_count;
-	unsigned int i;
-	void *vaddr;
-
-	page_count = DIV_ROUND_UP(size, PAGE_SIZE);
-
-	page_start = start;
-	pages = kmalloc_array(page_count, sizeof(struct page *), GFP_KERNEL);
-	if (!pages)
-		return 0;
-
-	for (i = 0; i < page_count; i++) {
-		phys_addr_t addr = page_start + i * PAGE_SIZE;
-		pages[i] = pfn_to_page(addr >> PAGE_SHIFT);
-	}
-	vaddr = vmap(pages, page_count, VM_MAP, PAGE_KERNEL);
-	kfree(pages);
-
-	return (u64)(unsigned long)vaddr;
-}
-
 /**
  * trace_array_get_by_name - Create/Lookup a trace array, given its name.
  * @name: The name of the trace array to be looked up/created.
@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
 	while ((curr_str = strsep(&str, "\t"))) {
 		phys_addr_t start = 0;
 		phys_addr_t size = 0;
-		unsigned long addr = 0;
 		bool traceprintk = false;
 		bool traceoff = false;
 		char *flag_delim;
@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
 			rname = kstrdup(tok, GFP_KERNEL);
 		}
 
-		if (start) {
-			addr = map_pages(start, size);
-			if (addr) {
-				pr_info("Tracing: mapped boot instance %s at physical memory %pa of size 0x%lx\n",
-					name, &start, (unsigned long)size);
-			} else {
-				pr_warn("Tracing: Failed to map boot instance %s\n", name);
-				continue;
-			}
-		} else {
+		if (!start) {
 			/* Only non mapped buffers have snapshot buffers */
 			if (IS_ENABLED(CONFIG_TRACER_MAX_TRACE))
 				do_allocate_snapshot(name);
 		}
 
-		tr = trace_array_create_systems(name, NULL, addr, size);
+		tr = trace_array_create_systems(name, NULL, (unsigned long)start, size);
 		if (IS_ERR(tr)) {
 			pr_warn("Tracing: Failed to create instance buffer %s\n", curr_str);
 			continue;
-- 
2.47.2

From: Steven Rostedt <rostedt@goodmis.org>

The persistent ring buffer uses vmap()'d memory to map the reserved memory
from boot. But the user space mmap() to the ring buffer requires
virt_to_page() to return a valid page. But that only works for core kernel
addresses and not for vmap() addresses.

To address this, save the physical and virtual address of where the
persistent memory is mapped. Create a rb_struct_page() helper function
that returns the page via virt_to_page() for normal buffer pages that were
allocated with page_alloc() but for the physical memory vmap()'d pages, it
uses the saved physical and virtual addresses of where the memory was
located to calculate the physical address of the virtual page that needs
the struct page. Then it uses pfn_to_page() to get the page for that
physical address.

New helper functions are created for flushing the cache for architectures
that need it between user and kernel space.

Also, the persistent memory uses the page->id for its own purpose where as
the user mmap buffer currently uses that for the subbuf array mapped to
user space. If the buffer is a persistent buffer, use the page index into
that buffer as the identifier instead of the page->id.

That is, the page->id for a persistent buffer, represents the order of the
buffer is in the link list. ->id == 0 means it is the reader page.
When a reader page is swapped, the new reader page's ->id gets zero, and
the old reader page gets the ->id of the page that it swapped with.

The user space mapping has the ->id is the index of where it was mapped in
user space and does not change while it is mapped.

Since the persistent buffer is fixed in its location, the index of where
a page is in the memory range can be used as the "id" to put in the meta
page array, and it can be mapped in the same order to user space as it is
in the persistent memory.

A new rb_page_id() helper function is used to get and set the id depending
on if the page is a normal memory allocated buffer or a physical memory
mapped buffer.

Link: https://lore.kernel.org/all/CAHk-=wgD9MQOoMAGtT=fXZsWY39exRVyZgxuBXix4u=1bdHA=g@mail.gmail.com/

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 kernel/trace/ring_buffer.c | 110 +++++++++++++++++++++++++++++++------
 1 file changed, 93 insertions(+), 17 deletions(-)

diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -XXX,XX +XXX,XX @@ struct trace_buffer {
 	struct ring_buffer_meta		*meta;
 
 	unsigned long			phys_start;
+	unsigned long			virt_start;
 
 	unsigned int			subbuf_size;
 	unsigned int			subbuf_order;
@@ -XXX,XX +XXX,XX @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
 
 		start = addr;
 		end = start + size;
+		buffer->virt_start = start;
 
 		/* scratch_size needs to be aligned too */
 		scratch_size = ALIGN(scratch_size, sizeof(long));
@@ -XXX,XX +XXX,XX @@ static void rb_clear_buffer_page(struct buffer_page *page)
 	page->read = 0;
 }
 
+/*
+ * Get the struct page for the given buffer page.
+ *
+ * For normal ring buffer pages that are allocated via page_alloc()
+ * the struct page can simply be retrieved via virt_to_page().
+ *
+ * But if the buffer was created via a physical mapping and vmap()
+ * was used to get to the virtual addresses, use the stored virtual
+ * and physical of the start address to calculate the original
+ * physical address of the given page and use pfn_to_page() to return
+ * the struct page.
+ */
+static struct page *rb_struct_page(struct trace_buffer *buffer, void *vaddr)
+{
+	if (buffer->flags & RB_FL_PHYSICAL) {
+		unsigned long addr = (unsigned long)vaddr;
+
+		addr -= buffer->virt_start;
+		addr += buffer->phys_start;
+		return pfn_to_page(addr >> PAGE_SHIFT);
+	}
+	return virt_to_page(vaddr);
+}
+
+/* Some archs do not have data cache coherency between kernel and user-space */
+static void rb_flush_buffer_page(struct trace_buffer *buffer,
+				 struct buffer_page *bpage)
+{
+	struct page *page = rb_struct_page(buffer, bpage->page);
+
+	flush_dcache_folio(page_folio(page));
+}
+
+/* The user mapped meta page is always allocated via page_alloc() */
+static void rb_flush_meta(void *meta)
+{
+	struct page *page = virt_to_page(meta);
+
+	flush_dcache_folio(page_folio(page));
+}
+
+/*
+ * When the buffer is memory mapped to user space, each sub buffer
+ * has a unique id that is used by the meta data to tell the user
+ * where the current reader page is.
+ *
+ * For a normal allocated ring buffer, the id is saved in the buffer page
+ * id field, and updated via this function.
+ *
+ * But for a physical memory mapped buffer, the id is already assigned for
+ * memory ording in the physical memory layout and can not be used. Instead
+ * the index of where the page lies in the memory layout is used.
+ *
+ * For the normal pages, set the buffer page id with the passed in @id
+ * value and return that.
+ *
+ * For memory mapped pages, get the page index in the physical memory layout
+ * and return that as the id.
+ */
+static int rb_page_id(struct ring_buffer_per_cpu *cpu_buffer,
+		      struct buffer_page *bpage, int id)
+{
+	/*
+	 * For boot buffers, the id is the index,
+	 * otherwise, set the buffer page with this id
+	 */
+	if (cpu_buffer->ring_meta)
+		id = rb_meta_subbuf_idx(cpu_buffer->ring_meta, bpage->page);
+	else
+		bpage->id = id;
+
+	return id;
+}
+
 static void rb_update_meta_page(struct ring_buffer_per_cpu *cpu_buffer)
 {
 	struct trace_buffer_meta *meta = cpu_buffer->meta_page;
@@ -XXX,XX +XXX,XX @@ static void rb_update_meta_page(struct ring_buffer_per_cpu *cpu_buffer)
 		return;
 
 	meta->reader.read = cpu_buffer->reader_page->read;
-	meta->reader.id = cpu_buffer->reader_page->id;
+	meta->reader.id = rb_page_id(cpu_buffer, cpu_buffer->reader_page,
+				     cpu_buffer->reader_page->id);
+
 	meta->reader.lost_events = cpu_buffer->lost_events;
 
 	meta->entries = local_read(&cpu_buffer->entries);
 	meta->overrun = local_read(&cpu_buffer->overrun);
 	meta->read = cpu_buffer->read;
 
-	/* Some archs do not have data cache coherency between kernel and user-space */
-	flush_dcache_folio(virt_to_folio(cpu_buffer->meta_page));
+	rb_flush_meta(meta);
 }
 
 static void
@@ -XXX,XX +XXX,XX @@ static void rb_setup_ids_meta_page(struct ring_buffer_per_cpu *cpu_buffer,
 	struct trace_buffer_meta *meta = cpu_buffer->meta_page;
 	unsigned int nr_subbufs = cpu_buffer->nr_pages + 1;
 	struct buffer_page *first_subbuf, *subbuf;
+	int cnt = 0;
 	int id = 0;
 
-	subbuf_ids[id] = (unsigned long)cpu_buffer->reader_page->page;
-	cpu_buffer->reader_page->id = id++;
+	id = rb_page_id(cpu_buffer, cpu_buffer->reader_page, id);
+	subbuf_ids[id++] = (unsigned long)cpu_buffer->reader_page->page;
+	cnt++;
 
 	first_subbuf = subbuf = rb_set_head_page(cpu_buffer);
 	do {
+		id = rb_page_id(cpu_buffer, subbuf, id);
+
 		if (WARN_ON(id >= nr_subbufs))
 			break;
 
 		subbuf_ids[id] = (unsigned long)subbuf->page;
-		subbuf->id = id;
 
 		rb_inc_page(&subbuf);
 		id++;
+		cnt++;
 	} while (subbuf != first_subbuf);
 
+	WARN_ON(cnt != nr_subbufs);
+
 	/* install subbuf ID to kern VA translation */
 	cpu_buffer->subbuf_ids = subbuf_ids;
 
@@ -XXX,XX +XXX,XX @@ static int __rb_map_vma(struct ring_buffer_per_cpu *cpu_buffer,
 	if (!pgoff) {
 		unsigned long meta_page_padding;
 
+		/* The meta page is always allocated via alloc_page() */
 		pages[p++] = virt_to_page(cpu_buffer->meta_page);
 
 		/*
@@ -XXX,XX +XXX,XX @@ static int __rb_map_vma(struct ring_buffer_per_cpu *cpu_buffer,
 			goto out;
 		}
 
-		page = virt_to_page((void *)cpu_buffer->subbuf_ids[s]);
+		page = rb_struct_page(cpu_buffer->buffer,
+				      (void *)cpu_buffer->subbuf_ids[s]);
 
 		for (; off < (1 << (subbuf_order)); off++, page++) {
 			if (p >= nr_pages)
@@ -XXX,XX +XXX,XX @@ int ring_buffer_map(struct trace_buffer *buffer, int cpu,
 	unsigned long flags, *subbuf_ids;
 	int err = 0;
 
-	/*
-	 * Currently, this does not support vmap()'d buffers.
-	 * Return -ENODEV as that is what is returned when a file
-	 * does not support memory mapping.
-	 */
-	if (buffer->flags & RB_FL_PHYSICAL)
-		return -ENODEV;
-
 	if (!cpumask_test_cpu(cpu, buffer->cpumask))
 		return -EINVAL;
 
@@ -XXX,XX +XXX,XX @@ int ring_buffer_map_get_reader(struct trace_buffer *buffer, int cpu)
 	goto consume;
 
 out:
-	/* Some archs do not have data cache coherency between kernel and user-space */
-	flush_dcache_folio(virt_to_folio(cpu_buffer->reader_page->page));
+	rb_flush_buffer_page(buffer, cpu_buffer->reader_page);
 
 	rb_update_meta_page(cpu_buffer);
 
-- 
2.47.2

Now that I learned that the memory passed back from reserve_mem is part
of the memory allocator and just "reserved" and the memory is already
virtually mapped, it can simply use phys_to_virt() on the physical memory
that is returned to get the virtual mapping for that memory!
  (Thanks Mike!)

That makes things much easier, especially since it means that the memory
returned by reserve_mem is no different than the memory retrieved by
page_alloc(). This allows that memory to be memory mapped to user space
no differently than it is mapped by the normal buffer.

This new series does the following:

- Enforce the memory mapping is page aligned (both the address and the
  size). If not, it errors out.

- Use phys_to_virt() to get to the virtual memory from the reserve_mem
  returned addresses. Also use free_reserved_area() to give it
  back to the buddy allocator when it is freed.

- Treat the buffer allocated via memmap differently. It still needs to
  be virtually mapped (cannot use phys_to_virt) and it must not be
  freed nor memory mapped to user space. A new flag is added when a buffer
  is created this way to prevent it from ever being memory mapped to user
  space and the ref count is upped so that it can never be freed.

- Use vmap_page_range() instead of using kmalloc_array() to create an array
  of struct pages for vmap().

- Use flush_kernel_vmap_range() instead of flush_dcache_folio()

- Allow the reserve_mem persistent ring buffer to be memory mapped.
  There is no difference now with how the memory is mapped to user space,
  only the accounting of what pages are mapped where is updated as
  the meta data is different between the two.

Note, the first 4 patches makes the code a bit more correct. Especially
since the vunmap() does not give the buffer back to the buddy allocator.
I will be looking to get the first 4 patches into this merge window.

The last patch which enables he persistent memory mapping to user space can
wait till the 6.16.

Changes since v2: https://lore.kernel.org/all/20250331143426.947281958@goodmis.org/

- Basically a full rewrite once I found out that you can get the virtual
  address of the memory returned by reserve_mem via phys_to_virt()!

Steven Rostedt (5):
      tracing: Enforce the persistent ring buffer to be page aligned
      tracing: Have reserve_mem use phys_to_virt() and separate from memmap buffer
      tracing: Use vmap_page_range() to map memmap ring buffer
      ring-buffer: Use flush_kernel_vmap_range() over flush_dcache_folio()
      ring-buffer: Allow reserve_mem persistent ring buffers to be mmapped

----
 Documentation/admin-guide/kernel-parameters.txt |  2 +
 Documentation/trace/debugging.rst               |  2 +
 kernel/trace/ring_buffer.c                      | 54 ++++++++++++++++--
 kernel/trace/trace.c                            | 75 ++++++++++++++++---------
 kernel/trace/trace.h                            |  1 +
 5 files changed, 102 insertions(+), 32 deletions(-)

From: Steven Rostedt <rostedt@goodmis.org>

Enforce that the address and the size of the memory used by the persistent
ring buffer is page aligned. Also update the documentation to reflect this
requirement.

Link: https://lore.kernel.org/all/CAHk-=whUOfVucfJRt7E0AH+GV41ELmS4wJqxHDnui6Giddfkzw@mail.gmail.com/

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 Documentation/admin-guide/kernel-parameters.txt |  2 ++
 Documentation/trace/debugging.rst               |  2 ++
 kernel/trace/trace.c                            | 12 ++++++++++++
 3 files changed, 16 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index XXXXXXX..XXXXXXX 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -XXX,XX +XXX,XX @@
 			This is just one of many ways that can clear memory. Make sure your system
 			keeps the content of memory across reboots before relying on this option.
 
+			NB: Both the mapped address and size must be page aligned for the architecture.
+
 			See also Documentation/trace/debugging.rst
 
 
diff --git a/Documentation/trace/debugging.rst b/Documentation/trace/debugging.rst
index XXXXXXX..XXXXXXX 100644
--- a/Documentation/trace/debugging.rst
+++ b/Documentation/trace/debugging.rst
@@ -XXX,XX +XXX,XX @@ kernel, so only the same kernel is guaranteed to work if the mapping is
 preserved. Switching to a different kernel version may find a different
 layout and mark the buffer as invalid.
 
+NB: Both the mapped address and size must be page aligned for the architecture.
+
 Using trace_printk() in the boot instance
 -----------------------------------------
 By default, the content of trace_printk() goes into the top level tracing
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
 		}
 
 		if (start) {
+			/* Start and size must be page aligned */
+			if (start & ~PAGE_MASK) {
+				pr_warn("Tracing: mapping start addr %lx is not page aligned\n",
+					(unsigned long)start);
+				continue;
+			}
+			if (size & ~PAGE_MASK) {
+				pr_warn("Tracing: mapping size %lx is not page aligned\n",
+					(unsigned long)size);
+				continue;
+			}
+
 			addr = map_pages(start, size);
 			if (addr) {
 				pr_info("Tracing: mapped boot instance %s at physical memory %pa of size 0x%lx\n",
-- 
2.47.2

From: Steven Rostedt <rostedt@goodmis.org>

The reserve_mem kernel command line option may pass back a physical
address, but the memory is still part of the normal memory just like
using memblock_reserve() would be. This means that the physical memory
returned by the reserve_mem command line option can be converted directly
to virtual memory by simply using phys_to_virt().

When freeing the buffer allocated by reserve_mem, use free_reserved_area().

Because the persistent ring buffer can also be allocated via the memmap
option, which *is* different than normal memory as it cannot be added back
to the buddy system, it must be treated differently. It still needs to be
virtually mapped to have access to it. It also can not be freed nor can it
ever be memory mapped to user space.

Create a new trace_array flag called TRACE_ARRAY_FL_MEMMAP which gets set
if the buffer is created by the memmap option, and this will prevent the
buffer from being memory mapped by user space.

Also increment the ref count for memmap'ed buffers so that they can never
be freed.

Link: https://lore.kernel.org/all/Z-wFszhJ_9o4dc8O@kernel.org/

Suggested-by: Mike Rapoport <rppt@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 kernel/trace/trace.c | 28 ++++++++++++++++++++++------
 kernel/trace/trace.h |  1 +
 2 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
 	struct trace_iterator *iter = &info->iter;
 	int ret = 0;
 
+	/* A memmap'ed buffer is not supported for user space mmap */
+	if (iter->tr->flags & TRACE_ARRAY_FL_MEMMAP)
+		return -ENODEV;
+
 	/* Currently the boot mapped buffer is not supported for mmap */
 	if (iter->tr->flags & TRACE_ARRAY_FL_BOOT)
 		return -ENODEV;
@@ -XXX,XX +XXX,XX @@ static void free_trace_buffers(struct trace_array *tr)
 	free_trace_buffer(&tr->max_buffer);
 #endif
 
-	if (tr->range_addr_start)
-		vunmap((void *)tr->range_addr_start);
+	if (tr->range_addr_start) {
+		void *start = (void *)tr->range_addr_start;
+		void *end = start + tr->range_addr_size;
+
+		free_reserved_area(start, end, 0, tr->range_name);
+	}
 }
 
 static void init_trace_flags_index(struct trace_array *tr)
@@ -XXX,XX +XXX,XX @@ static inline void do_allocate_snapshot(const char *name) { }
 __init static void enable_instances(void)
 {
 	struct trace_array *tr;
+	bool memmap_area = false;
 	char *curr_str;
 	char *name;
 	char *str;
@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
 					name);
 				continue;
 			}
+			memmap_area = true;
 		} else if (tok) {
 			if (!reserve_mem_find_by_name(tok, &start, &size)) {
 				start = 0;
@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
 				continue;
 			}
 
-			addr = map_pages(start, size);
+			if (memmap_area)
+				addr = map_pages(start, size);
+			else
+				addr = (unsigned long)phys_to_virt(start);
 			if (addr) {
 				pr_info("Tracing: mapped boot instance %s at physical memory %pa of size 0x%lx\n",
 					name, &start, (unsigned long)size);
@@ -XXX,XX +XXX,XX @@ __init static void enable_instances(void)
 			update_printk_trace(tr);
 
 		/*
-		 * If start is set, then this is a mapped buffer, and
-		 * cannot be deleted by user space, so keep the reference
-		 * to it.
+		 * memmap'd buffers can not be freed.
 		 */
+		if (memmap_area) {
+			tr->flags |= TRACE_ARRAY_FL_MEMMAP;
+			tr->ref++;
+		}
+
 		if (start) {
 			tr->flags |= TRACE_ARRAY_FL_BOOT | TRACE_ARRAY_FL_LAST_BOOT;
 			tr->range_name = no_free_ptr(rname);
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -XXX,XX +XXX,XX @@ enum {
 	TRACE_ARRAY_FL_BOOT		= BIT(1),
 	TRACE_ARRAY_FL_LAST_BOOT	= BIT(2),
 	TRACE_ARRAY_FL_MOD_INIT		= BIT(3),
+	TRACE_ARRAY_FL_MEMMAP		= BIT(4),
 };
 
 #ifdef CONFIG_MODULES
-- 
2.47.2

From: Steven Rostedt <rostedt@goodmis.org>

The code to map the physical memory retrieved by memmap currently
allocates an array of pages to cover the physical memory and then calls
vmap() to map it to a virtual address. Instead of using this temporary
array of struct page descriptors, simply use vmap_page_range() that can
directly map the contiguous physical memory to a virtual address.

Link: https://lore.kernel.org/all/CAHk-=whUOfVucfJRt7E0AH+GV41ELmS4wJqxHDnui6Giddfkzw@mail.gmail.com/

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 kernel/trace/trace.c | 35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -XXX,XX +XXX,XX @@
 #include <linux/irq_work.h>
 #include <linux/workqueue.h>
 #include <linux/sort.h>
+#include <linux/io.h> /* vmap_page_range() */
 
 #include <asm/setup.h> /* COMMAND_LINE_SIZE */
 
@@ -XXX,XX +XXX,XX @@ static int instance_mkdir(const char *name)
 	return ret;
 }
 
-static u64 map_pages(u64 start, u64 size)
+static u64 map_pages(unsigned long start, unsigned long size)
 {
-	struct page **pages;
-	phys_addr_t page_start;
-	unsigned int page_count;
-	unsigned int i;
-	void *vaddr;
+        unsigned long vmap_start, vmap_end;
+	struct vm_struct *area;
+	int ret;
 
-	page_count = DIV_ROUND_UP(size, PAGE_SIZE);
+        area = get_vm_area(size, VM_IOREMAP);
+        if (!area)
+                return 0;
 
-	page_start = start;
-	pages = kmalloc_array(page_count, sizeof(struct page *), GFP_KERNEL);
-	if (!pages)
-		return 0;
+        vmap_start = (unsigned long) area->addr;
+        vmap_end = vmap_start + size;
 
-	for (i = 0; i < page_count; i++) {
-		phys_addr_t addr = page_start + i * PAGE_SIZE;
-		pages[i] = pfn_to_page(addr >> PAGE_SHIFT);
-	}
-	vaddr = vmap(pages, page_count, VM_MAP, PAGE_KERNEL);
-	kfree(pages);
+        ret = vmap_page_range(vmap_start, vmap_end,
+			      start, pgprot_nx(PAGE_KERNEL));
+        if (ret < 0) {
+                free_vm_area(area);
+                return 0;
+        }
 
-	return (u64)(unsigned long)vaddr;
+	return (u64)vmap_start;
 }
 
 /**
-- 
2.47.2

From: Steven Rostedt <rostedt@goodmis.org>

Some architectures do not have data cache coherency between user and
kernel space. For these architectures, the cache needs to be flushed on
both the kernel and user addresses so that user space can see the updates
the kernel has made.

Instead of using flush_dcache_folio() and playing with virt_to_folio()
within the call to that function, use flush_kernel_vmap_range() which
takes the virtual address and does the work for those architectures that
need it.

Link: https://lore.kernel.org/all/CAG48ez3w0my4Rwttbc5tEbNsme6tc0mrSN95thjXUFaJ3aQ6SA@mail.gmail.com/

Suggested-by: Jann Horn <jannh@google.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 kernel/trace/ring_buffer.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -XXX,XX +XXX,XX @@ static void rb_update_meta_page(struct ring_buffer_per_cpu *cpu_buffer)
 	meta->read = cpu_buffer->read;
 
 	/* Some archs do not have data cache coherency between kernel and user-space */
-	flush_dcache_folio(virt_to_folio(cpu_buffer->meta_page));
+	flush_kernel_vmap_range(cpu_buffer->meta_page, PAGE_SIZE);
 }
 
 static void
@@ -XXX,XX +XXX,XX @@ int ring_buffer_map_get_reader(struct trace_buffer *buffer, int cpu)
 
 out:
 	/* Some archs do not have data cache coherency between kernel and user-space */
-	flush_dcache_folio(virt_to_folio(cpu_buffer->reader_page->page));
+	flush_kernel_vmap_range(cpu_buffer->reader_page->page,
+				buffer->subbuf_size + BUF_PAGE_HDR_SIZE);
 
 	rb_update_meta_page(cpu_buffer);
 
-- 
2.47.2

From: Steven Rostedt <rostedt@goodmis.org>

When the persistent ring buffer is created from the memory returned by
reserve_mem there is nothing prohibiting it to be memory mapped to user
space. The memory is the same as the pages allocated by alloc_page().

The way the memory is managed by the ring buffer code is slightly
different though and needs to be addressed.

The persistent memory uses the page->id for its own purpose where as the
user mmap buffer currently uses that for the subbuf array mapped to user
space. If the buffer is a persistent buffer, use the page index into that
buffer as the identifier instead of the page->id.

The user space mapping has the ->id is the index of where it was mapped in
user space and does not change while it is mapped.

A new rb_page_id() helper function is used to get and set the id depending
on if the page is a normal memory allocated buffer or a physical memory
mapped buffer.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 kernel/trace/ring_buffer.c | 49 ++++++++++++++++++++++++++++++++++----
 kernel/trace/trace.c       |  4 ----
 2 files changed, 45 insertions(+), 8 deletions(-)

diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -XXX,XX +XXX,XX @@ static void rb_clear_buffer_page(struct buffer_page *page)
 	page->read = 0;
 }
 
+/*
+ * When the buffer is memory mapped to user space, each sub buffer
+ * has a unique id that is used by the meta data to tell the user
+ * where the current reader page is.
+ *
+ * For a normal allocated ring buffer, the id is saved in the buffer page
+ * id field, and updated via this function.
+ *
+ * But for a fixed memory mapped buffer, the id is already assigned for
+ * fixed memory ording in the memory layout and can not be used. Instead
+ * the index of where the page lies in the memory layout is used.
+ *
+ * For the normal pages, set the buffer page id with the passed in @id
+ * value and return that.
+ *
+ * For fixed memory mapped pages, get the page index in the memory layout
+ * and return that as the id.
+ */
+static int rb_page_id(struct ring_buffer_per_cpu *cpu_buffer,
+		      struct buffer_page *bpage, int id)
+{
+	/*
+	 * For boot buffers, the id is the index,
+	 * otherwise, set the buffer page with this id
+	 */
+	if (cpu_buffer->ring_meta)
+		id = rb_meta_subbuf_idx(cpu_buffer->ring_meta, bpage->page);
+	else
+		bpage->id = id;
+
+	return id;
+}
+
 static void rb_update_meta_page(struct ring_buffer_per_cpu *cpu_buffer)
 {
 	struct trace_buffer_meta *meta = cpu_buffer->meta_page;
@@ -XXX,XX +XXX,XX @@ static void rb_update_meta_page(struct ring_buffer_per_cpu *cpu_buffer)
 		return;
 
 	meta->reader.read = cpu_buffer->reader_page->read;
-	meta->reader.id = cpu_buffer->reader_page->id;
+	meta->reader.id = rb_page_id(cpu_buffer, cpu_buffer->reader_page,
+				     cpu_buffer->reader_page->id);
+
 	meta->reader.lost_events = cpu_buffer->lost_events;
 
 	meta->entries = local_read(&cpu_buffer->entries);
@@ -XXX,XX +XXX,XX @@ static void rb_setup_ids_meta_page(struct ring_buffer_per_cpu *cpu_buffer,
 	struct trace_buffer_meta *meta = cpu_buffer->meta_page;
 	unsigned int nr_subbufs = cpu_buffer->nr_pages + 1;
 	struct buffer_page *first_subbuf, *subbuf;
+	int cnt = 0;
 	int id = 0;
 
-	subbuf_ids[id] = (unsigned long)cpu_buffer->reader_page->page;
-	cpu_buffer->reader_page->id = id++;
+	id = rb_page_id(cpu_buffer, cpu_buffer->reader_page, id);
+	subbuf_ids[id++] = (unsigned long)cpu_buffer->reader_page->page;
+	cnt++;
 
 	first_subbuf = subbuf = rb_set_head_page(cpu_buffer);
 	do {
+		id = rb_page_id(cpu_buffer, subbuf, id);
+
 		if (WARN_ON(id >= nr_subbufs))
 			break;
 
 		subbuf_ids[id] = (unsigned long)subbuf->page;
-		subbuf->id = id;
 
 		rb_inc_page(&subbuf);
 		id++;
+		cnt++;
 	} while (subbuf != first_subbuf);
 
+	WARN_ON(cnt != nr_subbufs);
+
 	/* install subbuf ID to kern VA translation */
 	cpu_buffer->subbuf_ids = subbuf_ids;
 
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index XXXXXXX..XXXXXXX 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -XXX,XX +XXX,XX @@ static int tracing_buffers_mmap(struct file *filp, struct vm_area_struct *vma)
 	if (iter->tr->flags & TRACE_ARRAY_FL_MEMMAP)
 		return -ENODEV;
 
-	/* Currently the boot mapped buffer is not supported for mmap */
-	if (iter->tr->flags & TRACE_ARRAY_FL_BOOT)
-		return -ENODEV;
-
 	ret = get_snapshot_map(iter->tr);
 	if (ret)
 		return ret;
-- 
2.47.2