From nobody Fri Sep 25 11:10:09 2026 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8795B350D7D for ; Sun, 13 Sep 2026 18:37:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789324678; cv=none; b=GdKDE5rK6nQOyqZVDLZlLE3nf1WsJRVafFIRzIH82JSqW9hbZAZRSmH+RXdKekSBEBIT/yicsgj7x2EY0ndhQcakPKkwdctwwtd6ROGVX/GVv2HRU64ViS2/Y1dogo5NUWCi3OwtS9GKGkYx/02Wr+iYRR0Ny3KDeHkse+gW2do= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789324678; c=relaxed/simple; bh=5MkNv5HhmnRadnPiFZ7Kpq6QLGkjqqFh7YYd35NzyS8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kroLA4BHzxZ2nwcjsa6yaC2XznWq4bVUmLQDouFwN+4LkxvprBmHxH9IoULYpnED2KMbDygh64fWZrQp7T1zZDrsLSk7gouH2pOFq1qQlL/BAU/wPYb9nMmw4a0HgXSq+P2iLi9TS5OqUaQbbbjwqodKuUTHgEsKKwWDNjKBnwY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TCkRJ1bx; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TCkRJ1bx" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-38e42560ebcso2481157a91.1 for ; Sun, 13 Sep 2026 11:37:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789324676; x=1789929476; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=hLpXy9tNi5ms1dbYyVi6I1as/81y8YWcT2cY9iWq8NU=; b=TCkRJ1bx2IFTofsKflqvb3SRACSiTu8znYe+y1i7ZQ2/uMlJ/0sWm4IqOe4pIfK0Tq 1meGfoI9jCRdpBO9/DISfAYGkpOv8lqs3tOPo5nnE9Ya9RjFGpb/65KsN4r6QGrOoDL5 /t1/B575G6iwlKHn6J2WTHwjW/bUYKHMUNoQPIKcIxGE+fHALyLVhBNaY1c6GLMr67BT w3v1TupA6m61ZW6CYb+QdqWVp4BmpXsANymn/dqQLhk/OVy69wV0RdeXWMbj7gXxNZ0N Vx2YlvUgokZ2zsoNaOM0zyuRH07o2iL2E7pXBALHF3Tmtc8cpBDsQZoLdxKD9HnwhP3M KViw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789324676; x=1789929476; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=hLpXy9tNi5ms1dbYyVi6I1as/81y8YWcT2cY9iWq8NU=; b=CdvaCMZd5vVyFUyXmxmw7h7sOTRMrmlXALodPDvwG0o7Nloxh0cWlgzQ+Y7gAJkhbh GxzlW5HOn55wdFwE63BDwhC55XZXHHEsMh11QFbpv4zceWRO+5LY4v/nAweeHW2w6oGU GZjnnfeRztDmBuoU57B+F3omaWjMgFWFzatnX6YOKpE8Y6WY+hv6AQmsnxOALXD+gXLL b8CfpmSn+/3gVXK+d48glSk+M3c3XQ3E9aQpATrNVpI71oiy6j2mZ0SnMESgKuAzfqGO kjBG+Qb/xofrgsaGsraUZas0QBiJXzruBPfKjqGy8RGxhRO4l0/+DS+QQg5B+tKAhe3u jr3w== X-Forwarded-Encrypted: i=1; AKwUvBwC+VAdW7LunYm+i6snvIVV3ACMCgCh1ekvLqRZR5Rt2qA/M+FBy1JA28/8NpQEnAO22sT7HLjwrlKrKLM=@vger.kernel.org X-Gm-Message-State: AFuF++k+3aYlPIAN+qcndM42YAbGYtcxHRaOM3C5Sb+cOA5Qj2C0M5ZK 5Fz+BEGwA5pgb79y4fRcU4RfzGxnGkjIjTlNAvyYycXzTxSpnj4jTgnv X-Gm-Gg: AYBFou38VsatYRYTAQC1eJ2R03i/9Pr+uroVDsnPDJIfVP/3ZrVzQEWfEDnNzjgQxU2 Le49iAHXiDAsv7pbxq/jjRpKwzsaXkFhcYjT77ppJ3RbdFvJMesi2pY1u0WufDxjyO8tebuOgO5 yLYj9GFwMTWYfRsnd1LmbFuo4SaTMzjW+sdGt4gBb5ZTF3s/LrrpOqAtEo/IWYU9O3EfcGIaRGe In1yOnbMnIwCQ7hGg9vQNWyP++n1NbHzYK4Swm4N8C+ejSZpKEi7PV3QXmppaj4cStUtS090FNG hBmNadLmo5VGEZ7MsjlEXmVEISUAO/Sm1kAG8SaarPVSVSu7/K+JT1ldmXMRqcf5nUYE1zjJedZ 48W2KbcUKUv8chPfe30mqCDUpmv/9xvvb8h7VjQbYyvaVepVexkAqJFAb9IouEJAgIrY9BHIGvq km00fxyqT9+wlA0Zf2z+CiDlbE56qeDmD+R6cPhB3TReG73ThpmsHJ/Oo4NK91R0JAULhEfA4uG KNL9g== X-Received: by 2002:a17:90b:3812:b0:398:990e:1587 with SMTP id 98e67ed59e1d1-39d9beb8e80mr23558089a91.9.1789324675485; Sun, 13 Sep 2026 11:37:55 -0700 (PDT) Received: from localhost ([95.190.112.239]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39d9b08165fsm4118447a91.1.2026.09.13.11.37.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 13 Sep 2026 11:37:55 -0700 (PDT) From: Vladislav Zaharov To: dakr@kernel.org, jhubbard@nvidia.com Cc: acourbot@nvidia.com, aliceryhl@google.com, ttabi@nvidia.com, gary@garyguo.net, nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Vladislav Zaharov Subject: [PATCH v4 1/3] gpu: nova-core: move the debugfs root into the module data Date: Mon, 14 Sep 2026 01:37:32 +0700 Message-ID: <20260913183734.134307-2-vladazaharova2018@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260913183734.134307-1-vladazaharova2018@gmail.com> References: <20260913183734.134307-1-vladazaharova2018@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The debugfs root lives in a static that init() fills in and a guard field of the module data clears again. That costs a `static mut`, an unsafe write on each side and a guard type whose only job is to undo the write. It also leaks. try_pin_init! drops only the fields it has already built, and the guard is written after the Registration, so a registration that fails leaves the guard unbuilt and the static set. Statics are never dropped, and the module is unloaded right away, so the directory outlives everything that could remove it. The next load then finds the name taken: debugfs_create_dir() returns -EEXIST, which Entry keeps as it would any other pointer, and the driver comes up with no debugfs at all until the machine is rebooted. Have the module data own a DebugfsData instead, built before the registration and dropped after it, and keep only a pointer to it in the static, for devices that have no other way to reach the data of their module. What is left is one unsafe read for the users and one write on each side, with no guard type. A registration that fails now drops the data that was built before it, and the directory goes with it. Assisted-by: Claude:claude-opus-5 Signed-off-by: Vladislav Zaharov --- drivers/gpu/nova-core/gsp.rs | 15 +++--- drivers/gpu/nova-core/nova_core.rs | 82 +++++++++++++++++++++++------- 2 files changed, 69 insertions(+), 28 deletions(-) diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs index 25ea43f1cbe9..f29e601e6753 100644 --- a/drivers/gpu/nova-core/gsp.rs +++ b/drivers/gpu/nova-core/gsp.rs @@ -196,15 +196,12 @@ pub(crate) fn new(pdev: &'gsp pci::Device) -> impl PinInit::NAME; =20 -// TODO: Move this into per-module data once that exists. -static mut DEBUGFS_ROOT: Option =3D None; +/// Pointer to the [`DebugfsData`] the module owns. +/// +/// A device has no way to reach the data of its module, so probe() goes t= hrough here instead. +// TODO: Drop this once devices can reach the data of their module. +static mut DEBUGFS_DATA: *const DebugfsData =3D core::ptr::null(); =20 -/// Guard that clears `DEBUGFS_ROOT` when dropped. -struct DebugfsRootGuard; +/// Data the module shares with every GPU it drives. +/// +/// # Invariants +/// +/// A non-null `DEBUGFS_DATA` points at a live, pinned instance of this ty= pe that outlives the +/// driver registration. +#[pin_data(PinnedDrop)] +pub(crate) struct DebugfsData { + /// Root directory of the driver in debugfs. + root: debugfs::Dir, +} + +impl DebugfsData { + /// Creates the shared data and publishes it, so that [`debugfs_data()= `] can hand it out. + fn new() -> impl PinInit { + pin_init!(&this in Self { + root: debugfs::Dir::new(c"nova-core"), + _: { + // SAFETY: Module initialization runs once and before the = driver is registered, so + // nothing can be reading `DEBUGFS_DATA` while it is writt= en here. `this` is where + // the data is being built, and it stays there: the module= data never moves. + unsafe { DEBUGFS_DATA =3D this.as_ptr() }; + }, + }) + } + + /// Returns the root directory of the driver in debugfs. + pub(crate) fn root(&self) -> &debugfs::Dir { + &self.root + } +} =20 -impl Drop for DebugfsRootGuard { - fn drop(&mut self) { - // SAFETY: This guard is dropped after `_driver` (due to field ord= er), - // so the driver is unregistered and no probe() can be running. - unsafe { DEBUGFS_ROOT =3D None }; +#[pinned_drop] +impl PinnedDrop for DebugfsData { + fn drop(self: Pin<&mut Self>) { + // SAFETY: This runs after the registration is dropped, as the fie= lds of `NovaCoreModule` + // are dropped in declaration order, so the driver is unregistered= and neither a probe() + // nor the teardown of a device can be reading `DEBUGFS_DATA`. + unsafe { DEBUGFS_DATA =3D core::ptr::null() }; } } =20 +/// Returns the data the module shares with its devices, or [`None`] if th= ere is none yet. +/// +/// Only ever call this while the driver is registered, which is to say fr= om probe() or from the +/// teardown of a device that is bound: the data is built before the regis= tration and dropped +/// after it, and nothing else keeps what is returned here alive. +pub(crate) fn debugfs_data() -> Option<&'static DebugfsData> { + // SAFETY: `DEBUGFS_DATA` is written while the module data is initiali= zed, before the driver + // is registered, and again when that data is dropped, after the drive= r is unregistered. Both + // happen with no device bound, so a caller in probe() or in the teard= own of a device cannot + // race with either, and by the type invariant what it gets points at = live data that outlives + // the device it is used from. + unsafe { DEBUGFS_DATA.as_ref() } +} + #[pin_data] struct NovaCoreModule { - // Fields are dropped in declaration order, so `_driver` is dropped fi= rst, - // then `_debugfs_guard` clears `DEBUGFS_ROOT`. + // Fields are dropped in declaration order, so the registration goes f= irst and no probe() can + // still be running once the shared data is torn down. `init()` builds= them the other way + // round, as the data has to be there before the first probe() reaches= for it. #[pin] _driver: Registration>, - _debugfs_guard: DebugfsRootGuard, + #[pin] + _debugfs: DebugfsData, } =20 impl InPlaceModule for NovaCoreModule { fn init(module: &'static kernel::ThisModule) -> impl PinInit { - let dir =3D debugfs::Dir::new(c"nova-core"); - - // SAFETY: We are the only driver code running during init, so the= re - // cannot be any concurrent access to `DEBUGFS_ROOT`. - unsafe { DEBUGFS_ROOT =3D Some(dir) }; - try_pin_init!(Self { + _debugfs <- DebugfsData::new(), _driver <- Registration::new(MODULE_NAME, module), - _debugfs_guard: DebugfsRootGuard, }) } } --=20 2.55.0 From nobody Fri Sep 25 11:10:09 2026 Received: from mail-pj2-f13.google.com (mail-pj2-f13.google.com [74.125.227.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FAA035F197 for ; Sun, 13 Sep 2026 18:38:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789324686; cv=none; b=UHiJIpDuEhOsAlb6qzTL6lnWSSJLJPJ9wyoBcxvNvTu2twkeOe/hVMqmLSkwv3y7u+rknr/UuQXMDn9dbHM9/lPTgOHJ5ZaWL6mTMJfi/mlyXQ72Pu8f5MpBXk/6sG3IC29ch7b2UsGbCu6TU3BMGHDvdSgXwKVZfa5M9YvG4iY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789324686; c=relaxed/simple; bh=zj8+wk8OOX0/4FWfYcxMOJZLw6PDM7FKGkEAW8Mz/tk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PjhemKUjBNQVzE320w0w3eQ+60qz24KpRXKpH9s2iTiJe6pK8G8cSREt5ShHjVExKSSCThGniLYK92kanXOi+x1xdlPjVlhjEmjaXoCrx7Loi3wzPNFLdrYRP0GHw2gkZVlNNuK0FqBbT1UfsBGfgB1a8TRy+olfjM/sj0+n9v8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JxdyRSdf; arc=none smtp.client-ip=74.125.227.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JxdyRSdf" Received: by mail-pj2-f13.google.com with SMTP id 98e67ed59e1d1-39dbdfaef3cso1260174a91.1 for ; Sun, 13 Sep 2026 11:38:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789324684; x=1789929484; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=F1ucxwCu9RQD4D679j8zHYPWIOfpF32/a1fvobfc1os=; b=JxdyRSdfZvoddWHr3PdBqrkjCcE0JeNLjIF82ijrsVhQ/ucxXiKyc+JgS/5ktu7iCW PTCJDfpA4OaC0COZyRtiYfHwbzgoM+es4b+uxkP2N8xcArM9hlvXmPA1brqKeVMHTLzS GRor+ugGcMgkMPnZSf1x40QbosJY9PfFGJIappxhsBxGjBMxTxdQsEFiDFgMNXWmZr8e L+Sg+KO8/HOwSQX4NqsV9ioq7+9YDEGGzvM9Nk5K/XBgfNMvlO/vMWL5uJnidNrhab9V UNlRBYcymX6f1zRr6ZUfRj7/5wkEdLMWjxIDht+fg/x4sLfFqnYrbmmtKwaJco03bc2s xDmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789324684; x=1789929484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=F1ucxwCu9RQD4D679j8zHYPWIOfpF32/a1fvobfc1os=; b=Bd3yj09czlGkqgwKFdXmlZaiQMfhSX1P2jg7e28gmP0QrKPyiQ91Gwk3UZrz01Tn7/ o18gCUfq75URJkbWXgndMNmqWpDx1KZCt+Bfr718qY3t4CBrpSjD66/RLITW9BWaYtMM c1BjKZyWT1W8IFiABJuPLIi75Ge280H6rUt6JFEqVDZYx76uq2gAt7e9Nniz9n06YQvp fjGJnM37ccCM/kr388w3Z/X4Ykpn+Igw+eizapDZ5okmWSDSYl1pqsZvgzWuvY8yuMq6 cw7VxfkXIx14MrfWQpwlfpabQbKmep9wNHtHOKVSCnwKMiN7/h5L2JpOAaIBMx2m3R3E yTwQ== X-Forwarded-Encrypted: i=1; AKwUvBzk9zzKJXXF0gAEsVOd9Zjt5HN4jYZzY+vy3SXJNK7ueqBxm6hzoHEX8d56+tS/M+2KHU332J6OKzLSytM=@vger.kernel.org X-Gm-Message-State: AFuF++lvgbDjLjbRpQRG0vT0uBzJWUGCvsN8gXZfN2saPaDd2BMxtDC2 F32qWeoEiPx3vi7EENyupB7+IWEWH9JxL8QWKpsuAk7eEqJrRRm+EA5+ X-Gm-Gg: AYBFou27Vfid2C8x61Xg1nuKoViEsG61389ygqmH9UiaiCGXogxg8mfHqLwWcAr8kd4 loOvCq8dgOjkLqN4jmJWxYZt8XdVJZbCmiVSBmaxKw9U9ipLWtrGw2YLgdKjE8H998+q+JeDqjd HmkeEyq2qAu0Kzyy94KyqgyAGqF/wi0Byr761/28WOLq3uE0eG7vUf9oFarbhYpEyRjfRsnx4nO ncU28kjqZl9DGnRe86Lshf6HdnafKykXGr0k8dmOs1y3cmPN3oZt2S3dZesgj9BVYGoc+g3enYl +d7sajgsgssnEKxmpnOm1IYKOkRTKhyHu7ha72MoLeTRMUnkfw6zbn3WCr1deZNW3lEUXBM9i/q P+KQorE6le6fxWPhkfpXuYCo5K/YCbHa9LATVfxQZ8E/Zegs08Z6Yso0XcymYCy/rmfI7QEJ963 aSBdAibo7iFXyvQn9d5LxmrHf3azNlgydt19H4msk51BmWlZoQ4IyEWBRkL2s7uaV0bR+WSSCYu UR0Nw== X-Received: by 2002:a17:90b:3a0c:b0:38e:9eb2:9d43 with SMTP id 98e67ed59e1d1-39d9c1dbb33mr27898773a91.16.1789324683339; Sun, 13 Sep 2026 11:38:03 -0700 (PDT) Received: from localhost ([95.190.112.239]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39d9b08165fsm4118447a91.1.2026.09.13.11.37.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 13 Sep 2026 11:38:02 -0700 (PDT) From: Vladislav Zaharov To: dakr@kernel.org, jhubbard@nvidia.com Cc: acourbot@nvidia.com, aliceryhl@google.com, ttabi@nvidia.com, gary@garyguo.net, nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Vladislav Zaharov Subject: [PATCH v4 2/3] gpu: nova-core: gsp: retain the GSP-RM log buffers after unbind Date: Mon, 14 Sep 2026 01:37:33 +0700 Message-ID: <20260913183734.134307-3-vladazaharova2018@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260913183734.134307-1-vladazaharova2018@gmail.com> References: <20260913183734.134307-1-vladazaharova2018@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The GSP-RM log buffers are exposed through debugfs, but the Scope that owns them lives in Gsp, inside GspResources, inside the Gpu built by probe(). They are DMA allocations of the device and cannot outlive it, so the entries go away as soon as the GPU is unbound - and, more to the point, as soon as probe() fails, which is exactly when the log of a GSP that did not come up is the thing one wants to read. Add a gsp_keep_logs module parameter. When it is set, dropping the log buffers copies whatever the GSP wrote into memory owned by the module and exposes the copies until the module is unloaded. A buffer whose "put" pointer is still zero was never written to and is skipped. The GSP has normally been stopped by the time the buffers are dropped, but a boot that timed out can leave it still appending, so a DMA read barrier orders the read of the "put" pointer before the copy. The copies belong to the module data next to the debugfs root, and live in a "retained" directory created during module init rather than on first use, which keeps the teardown path of a device from having to create anything. Keeping them out of the directory used by bound GPUs also means a device coming back does not find its debugfs name taken by its own history; nouveau, which recreates the entries under the name of the GPU that just went away, has that problem. While at it, move the log buffer code out of gsp.rs into gsp/logbuffer.rs. Assisted-by: Claude:claude-opus-5 Signed-off-by: Vladislav Zaharov --- drivers/gpu/nova-core/gsp.rs | 97 ++-------- drivers/gpu/nova-core/gsp/logbuffer.rs | 253 +++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 29 ++- 3 files changed, 300 insertions(+), 79 deletions(-) create mode 100644 drivers/gpu/nova-core/gsp/logbuffer.rs diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs index f29e601e6753..1a1eb7f37075 100644 --- a/drivers/gpu/nova-core/gsp.rs +++ b/drivers/gpu/nova-core/gsp.rs @@ -12,11 +12,7 @@ CoherentView, DmaAddress, // }, - io::{ - io_project, - io_write, - Io, // - }, + io::io_write, pci, prelude::*, // }; @@ -24,9 +20,13 @@ pub(crate) mod cmdq; pub(crate) mod commands; mod fw; +mod logbuffer; mod regs; mod sequencer; =20 +use logbuffer::LogBuffers; +pub(crate) use logbuffer::RetainedLogs; + pub(crate) use fw::{ GspFmcBootParams, GspFwWprMeta, @@ -77,10 +77,6 @@ pub(crate) fn dev(&self) -> &'gpu device::Device { } } =20 -/// Number of GSP pages to use in a RM log buffer. -const RM_LOG_BUFFER_NUM_PAGES: usize =3D 0x10; -const LOG_BUFFER_SIZE: usize =3D RM_LOG_BUFFER_NUM_PAGES * GSP_PAGE_SIZE; - /// Array of page table entries, as understood by the GSP bootloader. #[repr(C)] #[derive(FromBytes, IntoBytes)] @@ -101,49 +97,6 @@ fn init(view: CoherentView<'_, Self>, start: DmaAddress= ) -> Result<()> { } } =20 -/// The logging buffers are byte queues that contain encoded printf-like -/// messages from GSP-RM. They need to be decoded by a special application -/// that can parse the buffers. -/// -/// The 'loginit' buffer contains logs from early GSP-RM init and -/// exception dumps. The 'logrm' buffer contains the subsequent logs. Bot= h are -/// written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE. -/// -/// The physical address map for the log buffer is stored in the buffer -/// itself, starting with offset 1. Offset 0 contains the "put" pointer (p= p). -/// Initially, pp is equal to 0. If the buffer has valid logging data in i= t, -/// then pp points to index into the buffer where the next logging entry w= ill -/// be written. Therefore, the logging data is valid if: -/// 1 <=3D pp < sizeof(buffer)/sizeof(u64) -struct LogBuffer<'a>(Coherent<'a, [u8; LOG_BUFFER_SIZE]>); - -impl<'a> LogBuffer<'a> { - /// Creates a new `LogBuffer` mapped on `dev`. - fn new(dev: &'a device::Device) -> Result { - let obj =3D Self(Coherent::zeroed(dev, GFP_KERNEL)?); - - let start_addr =3D obj.0.dma_address(); - - let pte_view =3D io_project!( - obj.0, - [build: size_of::()..][build: ..RM_LOG_BUFFER_NUM_PAGES *= size_of::()] - ) - .try_cast::>()?; - PteArray::init(pte_view, start_addr)?; - - Ok(obj) - } -} - -struct LogBuffers<'a> { - /// Init log buffer. - loginit: LogBuffer<'a>, - /// Interrupts log buffer. - logintr: LogBuffer<'a>, - /// RM log buffer. - logrm: LogBuffer<'a>, -} - /// GSP runtime data. #[pin_data] pub(crate) struct Gsp<'gsp> { @@ -165,9 +118,7 @@ pub(crate) fn new(pdev: &'gsp pci::Device) -> impl PinInit) -> impl PinInit(pub(super) Coherent<'a, [u8; LOG_BUFFER_SI= ZE]>); + +impl<'a> LogBuffer<'a> { + /// Creates a new `LogBuffer` mapped on `dev`. + fn new(dev: &'a device::Device) -> Result { + let obj =3D Self(Coherent::zeroed(dev, GFP_KERNEL)?); + + let start_addr =3D obj.0.dma_address(); + + let pte_view =3D io_project!( + obj.0, + [build: size_of::()..][build: ..RM_LOG_BUFFER_NUM_PAGES *= size_of::()] + ) + .try_cast::>()?; + PteArray::init(pte_view, start_addr)?; + + Ok(obj) + } + + /// Copies the contents of this buffer into memory that does not belon= g to the device. + /// + /// A buffer the GSP never wrote to yields an empty vector, as it hold= s nothing worth keeping. + fn snapshot(&self) -> Result> { + // Offset 0 holds the "put" pointer, which the GSP advances as it = appends entries. It is + // still zero if nothing was ever logged, which is all that is tes= ted here: a buffer that + // was written to is copied whole, and making sense of "put" is le= ft to the decoder. + let put =3D io_project!(self.0, [build: ..size_of::()]).try_c= ast::()?; + if put.read_val() =3D=3D 0 { + return Ok(VVec::new()); + } + + // ORDERING: LOAD->LOAD ordering needed to order the "put" read be= fore the data read. The + // GSP has normally been stopped by the time this runs, but a boot= that timed out can leave + // it still appending. + dma_mb(Read); + + let mut snapshot =3D VVec::zeroed(LOG_BUFFER_SIZE, GFP_KERNEL)?; + io_project!(self.0, [build: ..]).copy_to_slice(&mut snapshot); + + Ok(snapshot) + } +} + +/// The log buffers of a GPU, for as long as it is bound to the driver. +pub(super) struct LogBuffers<'a> { + /// Device the buffers belong to. Also names their debugfs directory. + dev: &'a device::Device, + /// Init log buffer. + pub(super) loginit: LogBuffer<'a>, + /// Interrupts log buffer. + pub(super) logintr: LogBuffer<'a>, + /// RM log buffer. + pub(super) logrm: LogBuffer<'a>, +} + +impl<'a> LogBuffers<'a> { + /// Allocates the three log buffers of `dev`. + pub(super) fn new(dev: &'a device::Device) -> Result { + Ok(Self { + dev, + loginit: LogBuffer::new(dev)?, + logintr: LogBuffer::new(dev)?, + logrm: LogBuffer::new(dev)?, + }) + } + + /// Creates an initializer exposing these buffers under a directory na= med after their device. + pub(super) fn scope(self) -> impl PinInit, Infall= ible> + 'a { + let dev =3D self.dev; + + // PANIC: The module data cannot be gone here. It is published bef= ore the driver is + // registered and taken down after it is unregistered, so it is th= ere for as long as + // probe() can be called. + let log_parent: &debugfs::Dir =3D crate::debugfs_data() + .expect("module data not initialized") + .root(); + + log_parent.scope(self, dev.name(), |logs, dir| { + dir.read_binary_file(c"loginit", &logs.loginit.0); + dir.read_binary_file(c"logintr", &logs.logintr.0); + dir.read_binary_file(c"logrm", &logs.logrm.0); + }) + } + + /// Preserves whatever the GSP logged, so it can still be read once th= e GPU is gone. + /// + /// The buffers are DMA allocations of the device and cannot outlive i= t, so their contents are + /// copied into memory owned by the module and exposed through fresh d= ebugfs entries. Those + /// live until the module is unloaded. + /// + /// Does nothing if `gsp_keep_logs` was not set when the module was lo= aded, as there is then + /// no directory to put the copies in. + fn retain(&self) -> Result { + // Copying is only worth it if there is somewhere to put the resul= t, but the lock is + // dropped right away: what follows allocates 64 KiB three times, = and no other device + // should have to wait for that. + let Some(data) =3D crate::debugfs_data() else { + return Ok(()); + }; + + // The directory is taken here and the lock dropped again right aw= ay: what follows + // allocates 64 KiB three times, and no other device should have t= o wait for that. + let Some(dir) =3D data.retained_logs().lock().dir.clone() else { + return Ok(()); + }; + + let logs =3D RetainedLogBuffers { + name: CString::try_from_fmt(fmt!("{}", self.dev.name()))?, + loginit: self.loginit.snapshot()?, + logintr: self.logintr.snapshot()?, + logrm: self.logrm.snapshot()?, + }; + + // Nothing was ever logged, so there is nothing to keep. A copy fr= om an earlier run of + // this device is deliberately left alone: logs from a run that fa= iled are worth more + // than the silence of one that did not. + if logs.loginit.is_empty() && logs.logintr.is_empty() && logs.logr= m.is_empty() { + return Ok(()); + } + + // Take every allocation that can fail before the previous copy of= this device is + // dropped, so that running out of memory here cannot leave it wit= h no logs at all. + let scope =3D KBox::>::new_unin= it(GFP_KERNEL)?; + + let mut retained =3D data.retained_logs().lock(); + + retained.gpus.reserve(1, GFP_KERNEL)?; + + // An earlier run of the same device may have left a copy behind, = and its directory + // carries the name about to be used again, so it has to go first.= Nothing below can + // fail, so the replacement is guaranteed to take its place. + retained.gpus.retain(|gpu| *gpu.name !=3D *logs.name); + + let scope =3D scope.write_pin_init(dir.scope(logs, self.dev.name()= , |logs, dir| { + if !logs.loginit.is_empty() { + dir.read_binary_file(c"loginit", &logs.loginit); + } + if !logs.logintr.is_empty() { + dir.read_binary_file(c"logintr", &logs.logintr); + } + if !logs.logrm.is_empty() { + dir.read_binary_file(c"logrm", &logs.logrm); + } + }))?; + + retained.gpus.push(scope, GFP_KERNEL)?; + + dev_dbg!(self.dev, "GSP-RM log buffers retained\n"); + + Ok(()) + } +} + +impl Drop for LogBuffers<'_> { + fn drop(&mut self) { + if let Err(e) =3D self.retain() { + dev_warn!(self.dev, "failed to retain GSP-RM log buffers: {:?}= \n", e); + } + } +} + +/// Copies of the log buffers of a GPU that is no longer around. +struct RetainedLogBuffers { + /// Name of the device the buffers came from, which also names their d= irectory. + /// + /// A copy rather than a reference to the device, so that a GPU that i= s gone does not stay + /// allocated for as long as its logs are kept. + name: CString, + /// Contents of the init log buffer, empty if it was never written to. + loginit: VVec, + /// Contents of the interrupts log buffer, empty if it was never writt= en to. + logintr: VVec, + /// Contents of the RM log buffer, empty if it was never written to. + logrm: VVec, +} + +/// Log buffers of GPUs that are gone, and the debugfs entries exposing th= em. +/// +/// The copies live under a `retained` directory of their own instead of n= ext to the entries of +/// the GPUs that are actually bound, so that a device coming back does no= t find its name taken. +pub(crate) struct RetainedLogs { + /// Parent directory of all copies. `None` unless retaining was asked = for. + dir: Option, + /// One entry per GPU. + gpus: KVec>>>, +} + +impl RetainedLogs { + /// Creates an empty set of retained log buffers, retaining disabled. + pub(crate) const fn new() -> Self { + Self { + dir: None, + gpus: KVec::new(), + } + } + + /// Creates the directory the copies will live in, enabling retaining. + /// + /// Does nothing without `CONFIG_DEBUG_FS`, where a [`debugfs::Dir`] i= s a zero-sized type and + /// the copies could never be read back. + pub(crate) fn enable(&mut self, parent: &debugfs::Dir) { + if !cfg!(CONFIG_DEBUG_FS) { + return; + } + + self.dir =3D Some(parent.subdir(c"retained")); + } +} diff --git a/drivers/gpu/nova-core/nova_core.rs b/drivers/gpu/nova-core/nov= a_core.rs index 0f8501c26e05..1ea61a2ebf4f 100644 --- a/drivers/gpu/nova-core/nova_core.rs +++ b/drivers/gpu/nova-core/nova_core.rs @@ -7,6 +7,7 @@ driver::Registration, pci, prelude::*, + sync::Mutex, InPlaceModule, // }; =20 @@ -44,6 +45,11 @@ /// driver registration. #[pin_data(PinnedDrop)] pub(crate) struct DebugfsData { + /// Copies of the log buffers of GPUs that are gone. + /// + /// Declared before `root`, as the copies live below it. + #[pin] + retained_logs: Mutex, /// Root directory of the driver in debugfs. root: debugfs::Dir, } @@ -51,8 +57,18 @@ pub(crate) struct DebugfsData { impl DebugfsData { /// Creates the shared data and publishes it, so that [`debugfs_data()= `] can hand it out. fn new() -> impl PinInit { + let root =3D debugfs::Dir::new(c"nova-core"); + + // Deciding here, rather than when the first GPU goes away, keeps = the teardown path of a + // device from having to create anything. + let mut retained_logs =3D gsp::RetainedLogs::new(); + if module_parameters::gsp_keep_logs.value() { + retained_logs.enable(&root); + } + pin_init!(&this in Self { - root: debugfs::Dir::new(c"nova-core"), + retained_logs <- kernel::new_mutex!(retained_logs), + root, _: { // SAFETY: Module initialization runs once and before the = driver is registered, so // nothing can be reading `DEBUGFS_DATA` while it is writt= en here. `this` is where @@ -66,6 +82,11 @@ fn new() -> impl PinInit { pub(crate) fn root(&self) -> &debugfs::Dir { &self.root } + + /// Returns the copies of the log buffers of GPUs that are gone. + pub(crate) fn retained_logs(&self) -> &Mutex { + &self.retained_logs + } } =20 #[pinned_drop] @@ -119,6 +140,12 @@ fn init(module: &'static kernel::ThisModule) -> impl P= inInit { description: "Nova Core GPU driver", license: "GPL v2", firmware: [], + params: { + gsp_keep_logs: bool { + default: false, + description: "Keep the GSP-RM log buffers in debugfs after the= ir GPU is gone", + }, + }, } =20 kernel::module_firmware!(firmware::ModInfoBuilder); --=20 2.55.0 From nobody Fri Sep 25 11:10:09 2026 Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60D143B8135 for ; Sun, 13 Sep 2026 18:38:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789324693; cv=none; b=FUFBqHSZpF5wrYZbk0zADxfeu1T4PfmUrjmVzAADLKkn6gOqoDIRo3QXq5SdAmjGk47dWrWO0xMOv4aGBVBHF7CH0m2Ee9aDe+xpGFwhVhqQRp3v1OFzM28yL0sZXX9u7GFaDcq7kjI04A5AsPXlES3MxCDJ2fEH6NQ+I0fRdwM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789324693; c=relaxed/simple; bh=+cBOmdfMRpo1mKp8ZhzrQDLiYCnzdvGdytXX8AvmtTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XXMYMtddlVAFp4kJmYGyplhHiHLzw9nlTMr8hKydc+Ia+lA2CIzVIuQtFTKzTKwj3GfI9uQH+Y3HWuIf/4WIe9iJuHEpvJnm8R9eROWoWrZfMKlthFtx6Krk+c8fMBNhzI2GFHwVv9ARZ/wQZGyXbA7GzyyUObbebihcZH7YKZo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SR1wEWcu; arc=none smtp.client-ip=74.125.227.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SR1wEWcu" Received: by mail-pj2-f12.google.com with SMTP id 98e67ed59e1d1-398beb616f5so487012a91.1 for ; Sun, 13 Sep 2026 11:38:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789324691; x=1789929491; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=f8egR4dMbs0feNIgyGf2N4bg3BFUD4E92PNKo+p8ClY=; b=SR1wEWcuD/mMkbPBWlrbP1D/60M9OmaU+Yy/7tVisKQLnzOTSoVWuPxnoKWbGwnugN 986M0OeCBZwK7RlX4NB62la4e2CQdZrbe73yaLoUk39tJGJmjRk4ISV3tT5C2QmVhSN4 5mx1XMfwjH3tZ1eEvAYwy03ooRcEIte5tKgSJ7+5bWtdM4kwz8+eCXjuU2waR9ogO+wl cNSFes2SaPXPL851/HISNQplGpyjdGFVPOpCms1bqaFvQrVLeppdBOmmvgUvyuOcJzX5 IHc37Lgk4M4dPqdtWxZpikcV5bUdttnpt7mibyYuMfVLVuTd2/AP5umnW7c5vu/cc3gw GSHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789324691; x=1789929491; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=f8egR4dMbs0feNIgyGf2N4bg3BFUD4E92PNKo+p8ClY=; b=nY7qsttFzkjM/aci4aS+5WIAq+avC19FFoFUTIM56Hy0/Gg/Qn3FPQFNw8etN6FvUF YtL9gjG69ENCdyJ2vtBuiGit/y4K0q98P/uo3zn7kCgkTdKmtkPM1drQl9PHYUO4rO13 VScALbT0UiYanjoyXhVp7ytbDo+pyMAzPH96wBJBQE3ux9jCubAqys0Zua64RYgS2NKF PWibehB1rsr85X9OGexQOg/bJODB/pLRK5DDInaaCBEPfTKXlvte6TBXgHZIJ4/s2WEK 7eU4prOdlQ9mteYomJ24io7ajcPHbJ6CxSB3UG+wi7xzpkHBGayyNKSAvnBe7GYfhMXH jFPA== X-Forwarded-Encrypted: i=1; AKwUvBwusGrCU6VGTk5IPt/ZTMUulw25eQOyeLELIuoaw7/84FBL2BaWvGUraJFyrS3nIsRWaBubwHeLrAFO7n8=@vger.kernel.org X-Gm-Message-State: AFuF++lyp//hmKsB0m5ciVq8+uem2aXUiMWa77sPwui77qUmCutWoz0y TkOisjwnHSY3GnAfd9DmweGAgUdAIF/jwGLY0ukhstRFOCkektR8wxMO X-Gm-Gg: AYBFou2Aihbr/5g45k2RpQVB3lnA2dQazxOvamki1ICiiQw1Kl74XrNh/KlH384l2gP 1og71fLtPYTSb/dXIhpI58R086vMjUJtQ1jxUSvHYgq9c1yHwZnlQeouRWqMgTH3XYMvGj6oZOt qfg1hOvHjOct5uUaF4cPWN8RIJ8fybzD4mUuXv9xn5IiCmydJ/Tp2FxcOEpKbRiw1LNP12HmhmJ eYro/2jQVf9h+9S486Fl+5KhMy2gWvPLsBbDdnLx2fAxaBTeQfimN1JhrYqm5MO0DRqKT7l4z1q NQ68jsviGSE2Dj6U4IUlecQPs0pIO/Vaz1pX9Mi+6fv5HM0HgznL2HENqsopl46xoWDzdfiTbvp ykN26JCwi83/vpRyvtLfsek6i0iOr4+LDQ5H8GtCJSAOIku3uvjhIqLqiCLLNfxPSR9Brt35dLM CA0mNEhAAVpiKS7Jl2ZUr0kTcZLzSczOh94/ldIG12gULrxvf5MFsISDSAQDTQT6ywS/BcJ/wuS fa3RspUfGRk0jV1gA== X-Received: by 2002:a17:90b:50cf:b0:38e:c232:9d2c with SMTP id 98e67ed59e1d1-39dd54d7dd3mr3783446a91.2.1789324690644; Sun, 13 Sep 2026 11:38:10 -0700 (PDT) Received: from localhost ([95.190.112.239]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39d9b08165fsm4118447a91.1.2026.09.13.11.38.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 13 Sep 2026 11:38:10 -0700 (PDT) From: Vladislav Zaharov To: dakr@kernel.org, jhubbard@nvidia.com Cc: acourbot@nvidia.com, aliceryhl@google.com, ttabi@nvidia.com, gary@garyguo.net, nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Vladislav Zaharov Subject: [PATCH v4 3/3] Documentation: nova: remove completed GSP log buffer task Date: Mon, 14 Sep 2026 01:37:34 +0700 Message-ID: <20260913183734.134307-4-vladazaharova2018@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260913183734.134307-1-vladazaharova2018@gmail.com> References: <20260913183734.134307-1-vladazaharova2018@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Exposing the GSP-RM log buffers through debugfs is implemented, and with the gsp_keep_logs module parameter they now also survive a failed probe, which was the part of the task that was still missing. Assisted-by: Claude:claude-opus-5 Signed-off-by: Vladislav Zaharov --- Documentation/gpu/nova/core/todo.rst | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/Documentation/gpu/nova/core/todo.rst b/Documentation/gpu/nova/= core/todo.rst index a01c362b1be0..28fc7329d56b 100644 --- a/Documentation/gpu/nova/core/todo.rst +++ b/Documentation/gpu/nova/core/todo.rst @@ -141,18 +141,6 @@ Implement support for instmem (bar2) used to store pag= e tables. GPU System Processor (GSP) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D =20 -Export GSP log buffers ----------------------- - -Recent patches from Timur Tabi [1] added support to expose GSP-RM log buff= ers -(even after failure to probe the driver) through debugfs. - -This is also an interesting feature for nova-core, especially in the early= days. - -| Link: https://lore.kernel.org/nouveau/20241030202952.694055-2-ttabi@nvid= ia.com/ [1] -| Reference: Debugfs abstractions -| Complexity: Intermediate - GSP firmware abstraction ------------------------ =20 --=20 2.55.0