This series moves nova-core from the r570 GSP (GPU System Processor)
firmware onto the ABI that the r615 and later firmware releases share.
The driver calls that ABI "r000", because the firmware files carry no
version number in their names.
This should boot up on firmware extracted from NVIDIA's very recent
publicly released r615 branch, although my testing has been so far on a
slightly modified version of that, which Timur Tabi is working on making
publicly available for anyone who is hacking on nova-core. The changes
are small things for ABI stability for vGPU and don't affect this
series.
The GSP runs only one set (version) of firmware at a time, so patch 27
makes the one-commit change over to use r000 firmware. Every patch
before patch 27 still boots r570.
In other words, it is safe to merge this patchset, once reviewers are
happy with it, because publicly available firmware now exists for it.
I've tested on Turing, Ampere, and Blackwell on a single x86_64 test
machine, so far: nova-core dmesg excerpt:
0000:c1:00.0: NVIDIA T400 4GB
0000:c2:00.0: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
0000:01:00.0: NVIDIA RTX A4000
Changes in v3:
* Grouped the series into consecuitive sets of patches, each doing
something related, per Alex and Eliot.
* The three radix3 patches have been removed and sent out as a separate
prerequisite own series.
* The four debug-logging patches have been dropped (I have them ready
to post as a follow-up series, though).
* The big switch-to-r000 patch is smaller and much simpler and therefore
much easier to review.
* Every commit message and code comment has been rewritten (more than
once, in some cases), with a much stronger focus on making it easily
readable.
* A refactoring pass over the whole series, with each defect fixed in
the patch that introduced it. Most of the fixes either merge
duplicated code into shared helpers or move code to the type that owns
it.
* Improved how the switch to msgq v2 is done.
The 33 patches come in groups:
* Patches 1 through 8: prerequisites. They add a PCI domain accessor,
the MCTP header version, the r000 bindings and the msgq v2 registers,
and they prepare the queue code for msgq v2 and for a second kind of
element header.
* Patches 9 through 12: the GMC API types and their send and receive
paths.
* Patches 13 through 18: the two load-and-execute boot events and their
dispatcher.
* Patches 19 through 26: the static configuration returned from boot,
the GSP_INIT request and reply, the LIBOS log and state buffers, the
ucodes image, the GSP HAL method that loads the generic bootloader
for the load-and-execute handler, and the GSP_SUSPEND request.
* Patches 27 through 29: the switch to r000, the cleanup that makes the
decoded GSP_INIT reply the static configuration type, and the
deletion of the r570 bindings. The switch patch also adds the larger
framebuffer reservations that r000 requires on GB100 and GB20x.
* Patches 30 through 33: the receive path hardening. The receive path
matches an RPC reply by sequence number, decodes each element by its
NVDM type, and matches a GMC response by flag, id and sequence.
The series applies on top of drm-rust-next plus four dependencies:
* the interrupt tree series for GIN, the GPU Interrupt and Notification
unit [1]
* Eliot Courtney's NVKV codec series [2]
* the three-patch radix3 cleanup [3]
* two lint fixes for the generated bindings, both already in mainline
A branch with everything applied:
https://github.com/johnhubbard/linux/tree/nova-core-run-on-r615-or-later-v3/
[1] https://lore.kernel.org/all/20260912044400.677097-1-jhubbard@nvidia.com/
[2] https://lore.kernel.org/all/20260827-b4-nvkv-v2-0-0de9d5c8658c@nvidia.com/
[3] https://lore.kernel.org/all/20260913195413.742143-1-jhubbard@nvidia.com/
John Hubbard (32):
rust: pci: add domain_nr() accessor
gpu: nova-core: set MCTP transport header version to 1
gpu: nova-core: gsp: give the command queue its own BAR0 mapping
gpu: nova-core: firmware: add r000 bindings
gpu: nova-core: regs: add msgq v2 BAR0 register declarations
gpu: nova-core: gsp: ring the GSP doorbell from the queue memory
gpu: nova-core: gsp: make command allocation generic over the header
gpu: nova-core: gsp: compute the queue regions from a count and a slot
gpu: nova-core: add GMC API message types
gpu: nova-core: add GMC send path
gpu: nova-core: add GMC transport receive path
gpu: nova-core: gsp: add GMC dispatch on receive
gpu: nova-core: separate the generic falcon bootloader from FWSEC
gpu: nova-core: add the falcon DMA and suspend helpers for r000 boot
gpu: nova-core: add the r000 load-and-execute HS binary handler
gpu: nova-core: move the bootloader DMEM descriptor out of FWSEC
gpu: nova-core: add the r000 load-and-execute bootloader handler
gpu: nova-core: gsp: add the GMC boot event dispatcher
gpu: nova-core: gsp: rename the static configuration type
gpu: nova-core: gsp: return the static GPU configuration from boot
gpu: nova-core: gsp: add the GSP_INIT request builder
gpu: nova-core: gsp: send GSP_INIT and decode its reply
gpu: nova-core: add LIBOS3 log buffers and state monitor buffer
gpu: nova-core: add the ucodes firmware loader
gpu: nova-core: gsp: let the GSP HAL load the generic bootloader
gpu: nova-core: gsp: add the GSP_SUSPEND request
gpu: nova-core: switch to the r000 GSP firmware
gpu: nova-core: gsp: make the GSP_INIT reply the static configuration
gpu: nova-core: firmware: delete the r570 bindings
gpu: nova-core: match GSP RPC replies by sequence, not just function
gpu: nova-core: gsp: split the reply match out of the RPC receive path
gpu: nova-core: gsp: decode queue elements by their NVDM type
Zhi Wang (1):
gpu: nova-core: gsp: match a GMC response by flag, id and sequence
Documentation/gpu/nova/core/fsp.rst | 2 +
Documentation/gpu/nova/core/interrupts.rst | 41 +-
drivers/gpu/nova-core/falcon.rs | 203 +++-
drivers/gpu/nova-core/falcon/gsp.rs | 10 +-
drivers/gpu/nova-core/fb/hal/gb100.rs | 24 +-
drivers/gpu/nova-core/fb/hal/gb202.rs | 6 +-
drivers/gpu/nova-core/firmware.rs | 8 +-
drivers/gpu/nova-core/firmware/bindata.rs | 58 +
.../nova-core/firmware/fwsec/bootloader.rs | 160 +--
.../gpu/nova-core/firmware/gen_bootloader.rs | 216 ++++
drivers/gpu/nova-core/firmware/gsp.rs | 15 +-
drivers/gpu/nova-core/firmware/tlv.rs | 40 +-
drivers/gpu/nova-core/fsp.rs | 3 +-
drivers/gpu/nova-core/gpu.rs | 52 +-
drivers/gpu/nova-core/gsp.rs | 156 ++-
drivers/gpu/nova-core/gsp/boot.rs | 483 +++++++-
drivers/gpu/nova-core/gsp/cmdq.rs | 1036 +++++++++++------
drivers/gpu/nova-core/gsp/commands.rs | 368 ++----
drivers/gpu/nova-core/gsp/fw.rs | 832 ++++++-------
drivers/gpu/nova-core/gsp/fw/commands.rs | 405 ++++---
.../gsp/fw/{r570_144.rs => r000_00.rs} | 10 +-
.../gsp/fw/{r570_144 => r000_00}/bindings.rs | 691 ++++-------
drivers/gpu/nova-core/gsp/hal.rs | 32 +-
drivers/gpu/nova-core/gsp/hal/tu102.rs | 24 +-
drivers/gpu/nova-core/gsp/regs.rs | 19 +
drivers/gpu/nova-core/gsp/sequencer.rs | 379 ------
drivers/gpu/nova-core/mctp.rs | 33 +-
drivers/gpu/nova-core/mm.rs | 4 +-
drivers/gpu/nova-core/regs.rs | 9 +-
drivers/gpu/nova-core/sbuffer.rs | 2 +-
rust/helpers/pci.c | 5 +
rust/kernel/pci.rs | 12 +
32 files changed, 2973 insertions(+), 2365 deletions(-)
create mode 100644 drivers/gpu/nova-core/firmware/bindata.rs
create mode 100644 drivers/gpu/nova-core/firmware/gen_bootloader.rs
rename drivers/gpu/nova-core/gsp/fw/{r570_144.rs => r000_00.rs} (71%)
rename drivers/gpu/nova-core/gsp/fw/{r570_144 => r000_00}/bindings.rs (71%)
delete mode 100644 drivers/gpu/nova-core/gsp/sequencer.rs
--
2.55.0