drivers/media/common/videobuf2/Makefile | 1 + drivers/media/common/videobuf2/v4l2-allocator.c | 186 +++++++++++++++++++++ .../media/common/videobuf2/videobuf2-dma-contig.c | 27 ++- .../media/platform/rockchip/rkvdec/rkvdec-h264.c | 14 +- .../media/platform/rockchip/rkvdec/rkvdec-hevc.c | 14 +- .../media/platform/rockchip/rkvdec/rkvdec-rcb.c | 21 ++- .../platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c | 14 +- .../platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c | 14 +- .../platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c | 14 +- .../platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c | 14 +- .../media/platform/rockchip/rkvdec/rkvdec-vp9.c | 33 ++-- drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 + drivers/media/platform/verisilicon/hantro.h | 1 + drivers/media/platform/verisilicon/hantro_drv.c | 4 + drivers/media/platform/verisilicon/hantro_h264.c | 7 +- drivers/media/platform/verisilicon/hantro_hevc.c | 100 ++++++----- drivers/media/platform/verisilicon/hantro_mpeg2.c | 16 +- .../media/platform/verisilicon/hantro_postproc.c | 14 +- drivers/media/platform/verisilicon/hantro_vp8.c | 25 +-- drivers/media/platform/verisilicon/hantro_vp9.c | 34 +++- .../verisilicon/rockchip_vpu981_hw_av1_dec.c | 160 +++++++++++------- drivers/media/v4l2-core/v4l2-device.c | 4 +- include/media/v4l2-allocator.h | 26 +++ include/media/v4l2-device.h | 2 + include/media/videobuf2-core.h | 3 + 25 files changed, 555 insertions(+), 197 deletions(-)
Hello,
Currently, the only way to track buffers allocated by v4l2 from usespace
is to use the subsystem available debug information (e.g.:
/sys/kernel/debug/dma_buf/bufinfo).
But that information is generic and cannot be matched to a v4l2 driver or
to a userspace application: It is merely information about the allocation.
Other types of allocations require the developper to find where they are
exposed and how to link them to their test.
This can become hard to track when mutliple drivers are working at the
same time.
To improve that, add a small wrapper around buffer allocations to keep
track of them at the video device level so that we can add debug
information to them like a name, userspace pid/fd that did the
allocation,... and expose them to userspace via a debugfs entry.
It currently only supports DMA buffer allocations and adds support for
VB2 allocations too.
Other kind of memory tracking can be added later.
The verisilicon and rkvdec drivers have been ported to use the tracked
dma alloactions.
Note that this depends on the ftrace support patch series[1] that
provides fd/pid info in the v4l2_fh struct.
That series is a bit old, so this is based on an older linux version, but
that only changes things for the last 2 commits.
A v4l2top utility[2] has been made, to be used in parallel with the
fdinfo patch series[3], to show the list of active streams with their HW
and memory usage.
With this, debugfs looks like this when decoding a HEVC 1080p stream with
rkvdec on rk3588:
root # cat /sys/kernel/debug/v4l2/fdc38100.video-codec/mem
created-by fd pid size label
-------------------------------------------------------------------------------------
gst-launch-1.0 7 635 39184 vdpu381-hevc-priv-tbl
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-7
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-6
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-5
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-4
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-3
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-2
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-1
gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-0
gst-launch-1.0 7 635 3133440 out-000000005c0230f6-1
gst-launch-1.0 7 635 3133440 out-000000005c0230f6-0
=====================================================================================
Total size: 39729424
[1]: https://lore.kernel.org/all/20260610-v4l2-add-ftrace-v2-0-9756edf72ac1@collabora.com/
[2]: https://github.com/cazou/v4l2top/tree/upstream
[3]: https://lore.kernel.org/all/20260706-v4l2-add-fdinfo-v3-0-d556568cf38e@collabora.com/
Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
---
Detlev Casanova (4):
media: Add a v4l2 memory allocations tracker
media: Store v4l2_fh in the vb_queue
media: verisilicon: Switch to tracked dma allocations
media: rkvdec: Switch to tracked dma allocations
drivers/media/common/videobuf2/Makefile | 1 +
drivers/media/common/videobuf2/v4l2-allocator.c | 186 +++++++++++++++++++++
.../media/common/videobuf2/videobuf2-dma-contig.c | 27 ++-
.../media/platform/rockchip/rkvdec/rkvdec-h264.c | 14 +-
.../media/platform/rockchip/rkvdec/rkvdec-hevc.c | 14 +-
.../media/platform/rockchip/rkvdec/rkvdec-rcb.c | 21 ++-
.../platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c | 14 +-
.../platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c | 14 +-
.../platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c | 14 +-
.../platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c | 14 +-
.../media/platform/rockchip/rkvdec/rkvdec-vp9.c | 33 ++--
drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 +
drivers/media/platform/verisilicon/hantro.h | 1 +
drivers/media/platform/verisilicon/hantro_drv.c | 4 +
drivers/media/platform/verisilicon/hantro_h264.c | 7 +-
drivers/media/platform/verisilicon/hantro_hevc.c | 100 ++++++-----
drivers/media/platform/verisilicon/hantro_mpeg2.c | 16 +-
.../media/platform/verisilicon/hantro_postproc.c | 14 +-
drivers/media/platform/verisilicon/hantro_vp8.c | 25 +--
drivers/media/platform/verisilicon/hantro_vp9.c | 34 +++-
.../verisilicon/rockchip_vpu981_hw_av1_dec.c | 160 +++++++++++-------
drivers/media/v4l2-core/v4l2-device.c | 4 +-
include/media/v4l2-allocator.h | 26 +++
include/media/v4l2-device.h | 2 +
include/media/videobuf2-core.h | 3 +
25 files changed, 555 insertions(+), 197 deletions(-)
---
base-commit: 66affa37cfac0aec061cc4bcf4a065b0c52f7e19
change-id: 20260612-v4l2-add-mem-tracker-da0088c74a64
prerequisite-change-id: 20260608-v4l2-add-ftrace-aec6e7f60a6c:v2
prerequisite-patch-id: bd45b4df66799a7f9f22b49973a78d1bd3590a4a
prerequisite-patch-id: 6a5ed5615f08257cf854d441785066b5ff4d5d47
prerequisite-patch-id: 02594c869498b9e91e416d70242afbaa6b3a8d6a
prerequisite-patch-id: 9f25ec78f3b99ac1c5d42eb732f5bd9eae73fadd
prerequisite-patch-id: 4ed2c2e8c3abfa7f6328d0ea4f488ca5a3eec8fb
prerequisite-patch-id: 8787d50eb80e1a63a57a123268cade23c9e64618
prerequisite-patch-id: bfe85822378fa771f9bbc3e1e8c8ff6bce0eb7fd
prerequisite-patch-id: 784d0801da320f7b41194f4632ee422c32f1b5b8
prerequisite-patch-id: 362c8366e2efbfb5e6f1a3f71cfddbd46e8d1506
Best regards,
--
Detlev Casanova <detlev.casanova@collabora.com>
Hi Detlev, On Wed Sep 16, 2026 at 4:25 PM CEST, Detlev Casanova wrote: > Hello, > > Currently, the only way to track buffers allocated by v4l2 from usespace > is to use the subsystem available debug information (e.g.: > /sys/kernel/debug/dma_buf/bufinfo). > But that information is generic and cannot be matched to a v4l2 driver or > to a userspace application: It is merely information about the allocation. > > Other types of allocations require the developper to find where they are > exposed and how to link them to their test. > > This can become hard to track when mutliple drivers are working at the > same time. > > To improve that, add a small wrapper around buffer allocations to keep > track of them at the video device level so that we can add debug > information to them like a name, userspace pid/fd that did the > allocation,... and expose them to userspace via a debugfs entry. > > It currently only supports DMA buffer allocations and adds support for > VB2 allocations too. > Other kind of memory tracking can be added later. > The verisilicon and rkvdec drivers have been ported to use the tracked > dma alloactions. > > Note that this depends on the ftrace support patch series[1] that > provides fd/pid info in the v4l2_fh struct. > That series is a bit old, so this is based on an older linux version, but > that only changes things for the last 2 commits. > > A v4l2top utility[2] has been made, to be used in parallel with the > fdinfo patch series[3], to show the list of active streams with their HW > and memory usage. > > With this, debugfs looks like this when decoding a HEVC 1080p stream with > rkvdec on rk3588: > > root # cat /sys/kernel/debug/v4l2/fdc38100.video-codec/mem > created-by fd pid size label > ------------------------------------------------------------------------------------- > gst-launch-1.0 7 635 39184 vdpu381-hevc-priv-tbl > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-7 > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-6 > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-5 > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-4 > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-3 > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-2 > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-1 > gst-launch-1.0 7 635 4177920 cap-00000000ac34392e-0 > gst-launch-1.0 7 635 3133440 out-000000005c0230f6-1 > gst-launch-1.0 7 635 3133440 out-000000005c0230f6-0 > ===================================================================================== > Total size: 39729424 I build a kernel with your 3 patch series and I also build the v4l2top utility. Cool tool :-D Especially after I found out I should build the 'upstream' branch ;-) There were a couple of things I noticed: 1) With 'plain' kernel 7.3-rc3 but also with all my patches included, but excluding the patches from this series, I see this: ``` root@nanopc-t6-plus:~# ls -lh /sys/kernel/debug/v4l2/ total 0 drwxr-xr-x 3 root root 0 Sep 20 15:10 fdee0000.hdmi_receiver ``` Which is presumably the reason that with a kernel which also includes this patch series, I get the following kernel error: ``[ 10.849405] debugfs: 'v4l2' already exists in '/'`` I don't think this should generate an error/warning/info message at all, but just silently dealt with. Checking v4l2 debug dir again results in this: ``` root@nanopc-t6-plus:~# ls -lh /sys/kernel/debug/v4l2/ total 0 drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb60000.rga drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb80000.rga drwxr-xr-x 2 root root 0 Sep 20 15:13 fdc38000.video-codec drwxr-xr-x 2 root root 0 Sep 20 15:13 fdee0000.hdmi_receiver ``` I noticed my video-codec address was slightly different from yours. Playing a 1080p BBB x264 video with a patched ffmpeg+mpv, results in this: ``` root@nanopc-t6-plus:~# cat /sys/kernel/debug/v4l2/fdc38000.video-codec/mem created-by fd pid size label ------------------------------------------------------------------------------------- mpv 26 2027 4177920 cap-00000000205a3b3a-10 mpv 26 2027 4177920 cap-00000000205a3b3a-9 mpv 26 2027 4177920 cap-00000000205a3b3a-8 mpv 26 2027 16736 vdpu381-h264-priv-tbl mpv 26 2027 3133440 out-0000000079002cb1-3 mpv 26 2027 3133440 out-0000000079002cb1-2 mpv 26 2027 3133440 out-0000000079002cb1-1 mpv 26 2027 3133440 out-0000000079002cb1-0 mpv 26 2027 4177920 cap-00000000205a3b3a-7 mpv 26 2027 4177920 cap-00000000205a3b3a-6 mpv 26 2027 4177920 cap-00000000205a3b3a-5 mpv 26 2027 4177920 cap-00000000205a3b3a-4 mpv 26 2027 4177920 cap-00000000205a3b3a-3 mpv 26 2027 4177920 cap-00000000205a3b3a-2 mpv 26 2027 4177920 cap-00000000205a3b3a-1 mpv 26 2027 4177920 cap-00000000205a3b3a-0 ===================================================================================== Total size: 58507616 ``` And looking at the v4l2top utility, I noticed the following: 1. I see differing indications of the load while it's playing :-D 2. The ``Clock rate`` is always 750.0MHz (or 786431991Hz) (expected I think?) 3. The ``Total Memory`` is always 0B and the ``Memory usage details`` column is always empty. That's also the case when I press one of the arrow keys which seems to select the entry. Am I missing sth or doing sth wrong? Cheers, Diederik > [1]: https://lore.kernel.org/all/20260610-v4l2-add-ftrace-v2-0-9756edf72ac1@collabora.com/ > [2]: https://github.com/cazou/v4l2top/tree/upstream > [3]: https://lore.kernel.org/all/20260706-v4l2-add-fdinfo-v3-0-d556568cf38e@collabora.com/ > > Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com> > --- > Detlev Casanova (4): > media: Add a v4l2 memory allocations tracker > media: Store v4l2_fh in the vb_queue > media: verisilicon: Switch to tracked dma allocations > media: rkvdec: Switch to tracked dma allocations > > drivers/media/common/videobuf2/Makefile | 1 + > drivers/media/common/videobuf2/v4l2-allocator.c | 186 +++++++++++++++++++++ > .../media/common/videobuf2/videobuf2-dma-contig.c | 27 ++- > .../media/platform/rockchip/rkvdec/rkvdec-h264.c | 14 +- > .../media/platform/rockchip/rkvdec/rkvdec-hevc.c | 14 +- > .../media/platform/rockchip/rkvdec/rkvdec-rcb.c | 21 ++- > .../platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c | 14 +- > .../platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c | 14 +- > .../platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c | 14 +- > .../platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c | 14 +- > .../media/platform/rockchip/rkvdec/rkvdec-vp9.c | 33 ++-- > drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 + > drivers/media/platform/verisilicon/hantro.h | 1 + > drivers/media/platform/verisilicon/hantro_drv.c | 4 + > drivers/media/platform/verisilicon/hantro_h264.c | 7 +- > drivers/media/platform/verisilicon/hantro_hevc.c | 100 ++++++----- > drivers/media/platform/verisilicon/hantro_mpeg2.c | 16 +- > .../media/platform/verisilicon/hantro_postproc.c | 14 +- > drivers/media/platform/verisilicon/hantro_vp8.c | 25 +-- > drivers/media/platform/verisilicon/hantro_vp9.c | 34 +++- > .../verisilicon/rockchip_vpu981_hw_av1_dec.c | 160 +++++++++++------- > drivers/media/v4l2-core/v4l2-device.c | 4 +- > include/media/v4l2-allocator.h | 26 +++ > include/media/v4l2-device.h | 2 + > include/media/videobuf2-core.h | 3 + > 25 files changed, 555 insertions(+), 197 deletions(-) > --- > base-commit: 66affa37cfac0aec061cc4bcf4a065b0c52f7e19 > change-id: 20260612-v4l2-add-mem-tracker-da0088c74a64 > prerequisite-change-id: 20260608-v4l2-add-ftrace-aec6e7f60a6c:v2 > prerequisite-patch-id: bd45b4df66799a7f9f22b49973a78d1bd3590a4a > prerequisite-patch-id: 6a5ed5615f08257cf854d441785066b5ff4d5d47 > prerequisite-patch-id: 02594c869498b9e91e416d70242afbaa6b3a8d6a > prerequisite-patch-id: 9f25ec78f3b99ac1c5d42eb732f5bd9eae73fadd > prerequisite-patch-id: 4ed2c2e8c3abfa7f6328d0ea4f488ca5a3eec8fb > prerequisite-patch-id: 8787d50eb80e1a63a57a123268cade23c9e64618 > prerequisite-patch-id: bfe85822378fa771f9bbc3e1e8c8ff6bce0eb7fd > prerequisite-patch-id: 784d0801da320f7b41194f4632ee422c32f1b5b8 > prerequisite-patch-id: 362c8366e2efbfb5e6f1a3f71cfddbd46e8d1506 > > Best regards, > -- > Detlev Casanova <detlev.casanova@collabora.com> > > > _______________________________________________ > Linux-rockchip mailing list > Linux-rockchip@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-rockchip
Hi Diederik, Thank you for testing ! On Sunday, September 20, 2026 9:50:24 a.m. Eastern Daylight Time Diederik de Haas wrote: > Hi Detlev, > > On Wed Sep 16, 2026 at 4:25 PM CEST, Detlev Casanova wrote: > > Hello, > > > > Currently, the only way to track buffers allocated by v4l2 from usespace > > is to use the subsystem available debug information (e.g.: > > /sys/kernel/debug/dma_buf/bufinfo). > > But that information is generic and cannot be matched to a v4l2 driver or > > to a userspace application: It is merely information about the allocation. > > > > Other types of allocations require the developper to find where they are > > exposed and how to link them to their test. > > > > This can become hard to track when mutliple drivers are working at the > > same time. > > > > To improve that, add a small wrapper around buffer allocations to keep > > track of them at the video device level so that we can add debug > > information to them like a name, userspace pid/fd that did the > > allocation,... and expose them to userspace via a debugfs entry. > > > > It currently only supports DMA buffer allocations and adds support for > > VB2 allocations too. > > Other kind of memory tracking can be added later. > > The verisilicon and rkvdec drivers have been ported to use the tracked > > dma alloactions. > > > > Note that this depends on the ftrace support patch series[1] that > > provides fd/pid info in the v4l2_fh struct. > > That series is a bit old, so this is based on an older linux version, but > > that only changes things for the last 2 commits. > > > > A v4l2top utility[2] has been made, to be used in parallel with the > > fdinfo patch series[3], to show the list of active streams with their HW > > and memory usage. > > > > With this, debugfs looks like this when decoding a HEVC 1080p stream with > > > > rkvdec on rk3588: > > root # cat /sys/kernel/debug/v4l2/fdc38100.video-codec/mem > > created-by fd pid size > > label > > ----------------------------------------------------------------------- > > -------------- gst-launch-1.0 7 635 > > 39184 vdpu381-hevc-priv-tbl gst-launch-1.0 > > 7 635 4177920 > > cap-00000000ac34392e-7 gst-launch-1.0 7 > > 635 4177920 cap-00000000ac34392e-6 gst-launch-1.0 > > 7 635 4177920 > > cap-00000000ac34392e-5 gst-launch-1.0 7 > > 635 4177920 cap-00000000ac34392e-4 gst-launch-1.0 > > 7 635 4177920 > > cap-00000000ac34392e-3 gst-launch-1.0 7 > > 635 4177920 cap-00000000ac34392e-2 gst-launch-1.0 > > 7 635 4177920 > > cap-00000000ac34392e-1 gst-launch-1.0 7 > > 635 4177920 cap-00000000ac34392e-0 gst-launch-1.0 > > 7 635 3133440 > > out-000000005c0230f6-1 gst-launch-1.0 7 > > 635 3133440 out-000000005c0230f6-0 > > ======================================================================= > > ============== Total size: 39729424 > > I build a kernel with your 3 patch series and I also build the v4l2top > utility. Cool tool :-D > Especially after I found out I should build the 'upstream' branch ;-) > > There were a couple of things I noticed: > 1) With 'plain' kernel 7.3-rc3 but also with all my patches included, but > excluding the patches from this series, I see this: > > ``` > root@nanopc-t6-plus:~# ls -lh /sys/kernel/debug/v4l2/ > total 0 > drwxr-xr-x 3 root root 0 Sep 20 15:10 fdee0000.hdmi_receiver > ``` > > Which is presumably the reason that with a kernel which also includes this > patch series, I get the following kernel error: > ``[ 10.849405] debugfs: 'v4l2' already exists in '/'`` > > I don't think this should generate an error/warning/info message at all, > but just silently dealt with. Indeed, that is something I will fix in the next version. > > Checking v4l2 debug dir again results in this: > ``` > root@nanopc-t6-plus:~# ls -lh /sys/kernel/debug/v4l2/ > total 0 > drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb60000.rga > drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb80000.rga > drwxr-xr-x 2 root root 0 Sep 20 15:13 fdc38000.video-codec > drwxr-xr-x 2 root root 0 Sep 20 15:13 fdee0000.hdmi_receiver > ``` > > I noticed my video-codec address was slightly different from yours. Yes, mine is on rk3588. A different SoC will have a different device register address. > Playing a 1080p BBB x264 video with a patched ffmpeg+mpv, results in this: > ``` > root@nanopc-t6-plus:~# cat /sys/kernel/debug/v4l2/fdc38000.video-codec/mem > created-by fd pid size > label > --------------------------------------------------------------------------- > ---------- mpv 26 2027 > 4177920 cap-00000000205a3b3a-10 mpv 26 > 2027 4177920 cap-00000000205a3b3a-9 mpv > 26 2027 4177920 > cap-00000000205a3b3a-8 mpv 26 2027 > 16736 vdpu381-h264-priv-tbl mpv > 26 2027 3133440 out-0000000079002cb1-3 > mpv 26 2027 3133440 > out-0000000079002cb1-2 mpv 26 > 2027 3133440 out-0000000079002cb1-1 mpv > 26 2027 3133440 > out-0000000079002cb1-0 mpv 26 2027 > 4177920 cap-00000000205a3b3a-7 mpv > 26 2027 4177920 > cap-00000000205a3b3a-6 mpv 26 2027 > 4177920 cap-00000000205a3b3a-5 mpv > 26 2027 4177920 > cap-00000000205a3b3a-4 mpv 26 2027 > 4177920 cap-00000000205a3b3a-3 mpv > 26 2027 4177920 > cap-00000000205a3b3a-2 mpv 26 2027 > 4177920 cap-00000000205a3b3a-1 mpv > 26 2027 4177920 > cap-00000000205a3b3a-0 > =========================================================================== > ========== Total size: 58507616 > ``` > > And looking at the v4l2top utility, I noticed the following: > 1. I see differing indications of the load while it's playing :-D \o/ > 2. The ``Clock rate`` is always 750.0MHz (or 786431991Hz) (expected I > think?) Yes, the clock won't change, for now at least. Some decoders could adapt their clocks depending on the work. The frequency value will also be used to determine exact core usage for those that expose a used cycle count after each job (I did not implement that yet in v4l2top) > 3. The ``Total Memory`` is always 0B and the ``Memory usage > details`` column is always empty. That's also the case when I press one of > the arrow keys which seems to select the entry. Am I missing sth or doing > sth wrong? My guess would be that you didn't run it as root (this is usually needed to access the debugfs*). But if you did, we will have to dig a little. *: Note that later, we'd like to report global memory usage through fdinfo, so that the metric is accessible as a user, with buffer details still from the debugfs (so that'd need root access) Detlev. > > > [1]: > > https://lore.kernel.org/all/20260610-v4l2-add-ftrace-v2-0-9756edf72ac1@co > > llabora.com/ [2]: https://github.com/cazou/v4l2top/tree/upstream > > [3]: > > https://lore.kernel.org/all/20260706-v4l2-add-fdinfo-v3-0-d556568cf38e@co > > llabora.com/ > > > > Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com> > > --- > > > > Detlev Casanova (4): > > media: Add a v4l2 memory allocations tracker > > media: Store v4l2_fh in the vb_queue > > media: verisilicon: Switch to tracked dma allocations > > media: rkvdec: Switch to tracked dma allocations > > > > drivers/media/common/videobuf2/Makefile | 1 + > > drivers/media/common/videobuf2/v4l2-allocator.c | 186 > > +++++++++++++++++++++ .../media/common/videobuf2/videobuf2-dma-contig.c > > | 27 ++- > > .../media/platform/rockchip/rkvdec/rkvdec-h264.c | 14 +- > > .../media/platform/rockchip/rkvdec/rkvdec-hevc.c | 14 +- > > .../media/platform/rockchip/rkvdec/rkvdec-rcb.c | 21 ++- > > .../platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c | 14 +- > > .../platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c | 14 +- > > .../platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c | 14 +- > > .../platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c | 14 +- > > .../media/platform/rockchip/rkvdec/rkvdec-vp9.c | 33 ++-- > > drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 + > > drivers/media/platform/verisilicon/hantro.h | 1 + > > drivers/media/platform/verisilicon/hantro_drv.c | 4 + > > drivers/media/platform/verisilicon/hantro_h264.c | 7 +- > > drivers/media/platform/verisilicon/hantro_hevc.c | 100 ++++++----- > > drivers/media/platform/verisilicon/hantro_mpeg2.c | 16 +- > > .../media/platform/verisilicon/hantro_postproc.c | 14 +- > > drivers/media/platform/verisilicon/hantro_vp8.c | 25 +-- > > drivers/media/platform/verisilicon/hantro_vp9.c | 34 +++- > > .../verisilicon/rockchip_vpu981_hw_av1_dec.c | 160 > > +++++++++++------- > > drivers/media/v4l2-core/v4l2-device.c | 4 +- > > include/media/v4l2-allocator.h | 26 +++ > > include/media/v4l2-device.h | 2 + > > include/media/videobuf2-core.h | 3 + > > 25 files changed, 555 insertions(+), 197 deletions(-) > > > > --- > > base-commit: 66affa37cfac0aec061cc4bcf4a065b0c52f7e19 > > change-id: 20260612-v4l2-add-mem-tracker-da0088c74a64 > > prerequisite-change-id: 20260608-v4l2-add-ftrace-aec6e7f60a6c:v2 > > prerequisite-patch-id: bd45b4df66799a7f9f22b49973a78d1bd3590a4a > > prerequisite-patch-id: 6a5ed5615f08257cf854d441785066b5ff4d5d47 > > prerequisite-patch-id: 02594c869498b9e91e416d70242afbaa6b3a8d6a > > prerequisite-patch-id: 9f25ec78f3b99ac1c5d42eb732f5bd9eae73fadd > > prerequisite-patch-id: 4ed2c2e8c3abfa7f6328d0ea4f488ca5a3eec8fb > > prerequisite-patch-id: 8787d50eb80e1a63a57a123268cade23c9e64618 > > prerequisite-patch-id: bfe85822378fa771f9bbc3e1e8c8ff6bce0eb7fd > > prerequisite-patch-id: 784d0801da320f7b41194f4632ee422c32f1b5b8 > > prerequisite-patch-id: 362c8366e2efbfb5e6f1a3f71cfddbd46e8d1506 > > > > Best regards, > > -- > > Detlev Casanova <detlev.casanova@collabora.com> > > > > > > _______________________________________________ > > Linux-rockchip mailing list > > Linux-rockchip@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-rockchip
Hi,
On Thu Sep 24, 2026 at 2:36 PM CEST, Detlev Casanova wrote:
> On Sunday, September 20, 2026 9:50:24 a.m. Eastern Daylight Time Diederik de
> Haas wrote:
>> On Wed Sep 16, 2026 at 4:25 PM CEST, Detlev Casanova wrote:
>> > Currently, the only way to track buffers allocated by v4l2 from usespace
>> > is to use the subsystem available debug information (e.g.:
>> > /sys/kernel/debug/dma_buf/bufinfo).
>> > But that information is generic and cannot be matched to a v4l2 driver or
>> > to a userspace application: It is merely information about the allocation.
>> >
>> > ...
>> >
>> > Note that this depends on the ftrace support patch series[1] that
>> > provides fd/pid info in the v4l2_fh struct.
>> > That series is a bit old, so this is based on an older linux version, but
>> > that only changes things for the last 2 commits.
>> >
>> > A v4l2top utility[2] has been made, to be used in parallel with the
>> > fdinfo patch series[3], to show the list of active streams with their HW
>> > and memory usage.
>> >
>> > With this, debugfs looks like this when decoding a HEVC 1080p stream with
>> >
>> > rkvdec on rk3588:
>> > root # cat /sys/kernel/debug/v4l2/fdc38100.video-codec/mem
>> > created-by fd pid size
>> > label
>> > -----------------------------------------------------------------------
>> > ...
>>
>> I build a kernel with your 3 patch series and I also build the v4l2top
>> utility. Cool tool :-D
>> Especially after I found out I should build the 'upstream' branch ;-)
>>
>> There were a couple of things I noticed:
>> 1) With 'plain' kernel 7.3-rc3 but also with all my patches included, but
>> excluding the patches from this series, I see this:
>> ...
>>
>> Checking v4l2 debug dir again results in this:
>> ```
>> root@nanopc-t6-plus:~# ls -lh /sys/kernel/debug/v4l2/
>> total 0
>> drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb60000.rga
>> drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb80000.rga
>> drwxr-xr-x 2 root root 0 Sep 20 15:13 fdc38000.video-codec
>> drwxr-xr-x 2 root root 0 Sep 20 15:13 fdee0000.hdmi_receiver
>> ```
>>
>> I noticed my video-codec address was slightly different from yours.
>
> Yes, mine is on rk3588. A different SoC will have a different device register
> address.
That's why I noticed/mentioned it as mine is rk3588 too. Maybe this commit?
b481c11cd20a ("arm64: dts: rockchip: Update vdec register blocks order on RK3588")
>> Playing a 1080p BBB x264 video with a patched ffmpeg+mpv, results in this:
>> ...
>>
>> And looking at the v4l2top utility, I noticed the following:
>> 1. I see differing indications of the load while it's playing :-D
>
> \o/
Just did another quick test (for 3.) and noticed sometimes green 'lines' and
sometimes red 'lines'. Any significance to the color?
There also seems to be some kind of 'delay' when stopping a video and playing
another one (can be the same video). It looks like it's 'flatlining' until the
line of the previous video is scrolled off ... and then it seems to 'repaint'
itself as the 'flatline' comes back alive :-P
But this is REALLY minor and not bothersome at all.
>> 2. The ``Clock rate`` is always 750.0MHz (or 786431991Hz) (expected I
>> think?)
>
> Yes, the clock won't change, for now at least. Some decoders could adapt their
> clocks depending on the work. The frequency value will also be used to
> determine exact core usage for those that expose a used cycle count after each
> job (I did not implement that yet in v4l2top)
>
>> 3. The ``Total Memory`` is always 0B and the ``Memory usage
>> details`` column is always empty. That's also the case when I press one of
>> the arrow keys which seems to select the entry. Am I missing sth or doing
>> sth wrong?
>
> My guess would be that you didn't run it as root (this is usually needed to
> access the debugfs*). But if you did, we will have to dig a little.
I did indeed not run it as root the first time around. But just did another
test and with root the memory data is displayed :-)
Cheers,
Diederik
> *: Note that later, we'd like to report global memory usage through fdinfo, so
> that the metric is accessible as a user, with buffer details still from the
> debugfs (so that'd need root access)
>
> Detlev.
>
>>
>> > [1]:
>> > https://lore.kernel.org/all/20260610-v4l2-add-ftrace-v2-0-9756edf72ac1@co
>> > llabora.com/ [2]: https://github.com/cazou/v4l2top/tree/upstream
>> > [3]:
>> > https://lore.kernel.org/all/20260706-v4l2-add-fdinfo-v3-0-d556568cf38e@co
>> > llabora.com/
>> >
>> > Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
>> > ---
>> >
>> > Detlev Casanova (4):
>> > media: Add a v4l2 memory allocations tracker
>> > media: Store v4l2_fh in the vb_queue
>> > media: verisilicon: Switch to tracked dma allocations
>> > media: rkvdec: Switch to tracked dma allocations
>> >
>> > drivers/media/common/videobuf2/Makefile | 1 +
>> > drivers/media/common/videobuf2/v4l2-allocator.c | 186
>> > +++++++++++++++++++++ .../media/common/videobuf2/videobuf2-dma-contig.c
>> > | 27 ++-
>> > .../media/platform/rockchip/rkvdec/rkvdec-h264.c | 14 +-
>> > .../media/platform/rockchip/rkvdec/rkvdec-hevc.c | 14 +-
>> > .../media/platform/rockchip/rkvdec/rkvdec-rcb.c | 21 ++-
>> > .../platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c | 14 +-
>> > .../platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c | 14 +-
>> > .../platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c | 14 +-
>> > .../platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c | 14 +-
>> > .../media/platform/rockchip/rkvdec/rkvdec-vp9.c | 33 ++--
>> > drivers/media/platform/rockchip/rkvdec/rkvdec.c | 4 +
>> > drivers/media/platform/verisilicon/hantro.h | 1 +
>> > drivers/media/platform/verisilicon/hantro_drv.c | 4 +
>> > drivers/media/platform/verisilicon/hantro_h264.c | 7 +-
>> > drivers/media/platform/verisilicon/hantro_hevc.c | 100 ++++++-----
>> > drivers/media/platform/verisilicon/hantro_mpeg2.c | 16 +-
>> > .../media/platform/verisilicon/hantro_postproc.c | 14 +-
>> > drivers/media/platform/verisilicon/hantro_vp8.c | 25 +--
>> > drivers/media/platform/verisilicon/hantro_vp9.c | 34 +++-
>> > .../verisilicon/rockchip_vpu981_hw_av1_dec.c | 160
>> > +++++++++++-------
>> > drivers/media/v4l2-core/v4l2-device.c | 4 +-
>> > include/media/v4l2-allocator.h | 26 +++
>> > include/media/v4l2-device.h | 2 +
>> > include/media/videobuf2-core.h | 3 +
>> > 25 files changed, 555 insertions(+), 197 deletions(-)
>> >
>> > ---
>> > base-commit: 66affa37cfac0aec061cc4bcf4a065b0c52f7e19
>> > change-id: 20260612-v4l2-add-mem-tracker-da0088c74a64
>> > prerequisite-change-id: 20260608-v4l2-add-ftrace-aec6e7f60a6c:v2
>> > prerequisite-patch-id: bd45b4df66799a7f9f22b49973a78d1bd3590a4a
>> > prerequisite-patch-id: 6a5ed5615f08257cf854d441785066b5ff4d5d47
>> > prerequisite-patch-id: 02594c869498b9e91e416d70242afbaa6b3a8d6a
>> > prerequisite-patch-id: 9f25ec78f3b99ac1c5d42eb732f5bd9eae73fadd
>> > prerequisite-patch-id: 4ed2c2e8c3abfa7f6328d0ea4f488ca5a3eec8fb
>> > prerequisite-patch-id: 8787d50eb80e1a63a57a123268cade23c9e64618
>> > prerequisite-patch-id: bfe85822378fa771f9bbc3e1e8c8ff6bce0eb7fd
>> > prerequisite-patch-id: 784d0801da320f7b41194f4632ee422c32f1b5b8
>> > prerequisite-patch-id: 362c8366e2efbfb5e6f1a3f71cfddbd46e8d1506
>> >
>> > Best regards,
>> > --
>> > Detlev Casanova <detlev.casanova@collabora.com>
>> >
>> >
>> > _______________________________________________
>> > Linux-rockchip mailing list
>> > Linux-rockchip@lists.infradead.org
>> > http://lists.infradead.org/mailman/listinfo/linux-rockchip
© 2016 - 2026 Red Hat, Inc.