Fix issues preventing rmmod of ti-am65-cpsw-nuss from working properly.

v3:

  - more patch submission minutiae

v2: https://lore.kernel.org/netdev/20241003172105.2712027-2-nico@fluxnic.net/T/

  - conform to netdev patch submission customs
  - address patch review trivias

v1: https://lore.kernel.org/netdev/20240927025301.1312590-2-nico@fluxnic.net/T/