On Fri, May 30, 2025 at 8:50 PM Jagan Teki <jagan@edgeble.ai> wrote:
>
> On Mon, 19 May 2025 at 19:14, Tomeu Vizoso <tomeu@tomeuvizoso.net> wrote:
> >
> > This series adds a new driver for the NPU that Rockchip includes in its
> > newer SoCs, developed by them on the NVDLA base.
> >
> > In its current form, it supports the specific NPU in the RK3588 SoC.
> >
> > The userspace driver is part of Mesa and an initial draft can be found at:
> >
> > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29698
> >
> > Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
> > ---
> > Changes in v4:
> > - Several fixes to DT bindings.
> > - Link to v3: https://lore.kernel.org/r/20250516-6-10-rocket-v3-0-7051ac9225db@tomeuvizoso.net
> >
> > Changes in v3:
> > - Reference in the device tree only the register blocks that are
> > actually used.
> > - Several style and robustness fixes suggested in the mailing list.
> > - Added patches from Nicolas Frattaroli that add support to the NPU for
> > the Rock 5B board.
> > - Link to v2: https://lore.kernel.org/r/20250225-6-10-rocket-v2-0-d4dbcfafc141@tomeuvizoso.net
> >
> > Changes in v2:
> > - Drop patch adding the rk3588 compatible to rockchip-iommu (Sebastian Reichel)
> > - Drop patch adding support for multiple power domains to rockchip-iommu (Sebastian Reichel)
> > - Link to v1: https://lore.kernel.org/r/20240612-6-10-rocket-v1-0-060e48eea250@tomeuvizoso.net
> >
> > ---
> > Nicolas Frattaroli (2):
> > arm64: dts: rockchip: add pd_npu label for RK3588 power domains
> > arm64: dts: rockchip: enable NPU on ROCK 5B
> >
> > Tomeu Vizoso (8):
> > dt-bindings: npu: rockchip,rknn: Add bindings
> > arm64: dts: rockchip: Add nodes for NPU and its MMU to rk3588s
> > arm64: dts: rockchip: Enable the NPU on quartzpro64
> > accel/rocket: Add registers header
> > accel/rocket: Add a new driver for Rockchip's NPU
> > accel/rocket: Add IOCTL for BO creation
> > accel/rocket: Add job submission IOCTL
> > accel/rocket: Add IOCTLs for synchronizing memory accesses
>
> Can this be possible to infer yolov8/10? Do we need to convert PT/ONNX
> to any other common format's unlike rknn?
Both considerations are entirely dependent on the userspace driver.
This kernel driver should be able to support a userspace driver that
accelerates any YOLO version. Should also be able to support without
changes a userspace driver that implements execution of ONNX, PyTorch
models, etc. With or without conversion to an intermediate model
format.
Regarding the particular userspace driver that has been submitted for
review to Mesa, you can put questions and comments at:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29698
Thanks,
Tomeu