From nobody Fri Sep 25 12:34:02 2026 Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8ED8F3F076F for ; Sat, 12 Sep 2026 07:18:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789197530; cv=none; b=Ib1WAqBY/0HdEtNN3Ogc+QAb1s5uv16JpoBuhgF5PW9CeZdWSVpuE2yU44iuU82JoGw4GExwWTvKvI1G/neRLNougIMJWj2izU0YhIfzCY2VDEsb+gkTI2l9baH19oHqQ+zaEozQ0ZQ4UdwbfHpfNw9VtfB/dqA4eUCaZjbGojk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789197530; c=relaxed/simple; bh=icmX+e8eV5T7lwdK+/Ae7/ixgFSBDIOa2J3Wav1g/+A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SmPpMoBjhMKcL83NwH41W3Z23j6AXv31iSWxRMOeut3v61jzHWBjW8BgCr2/0VEvacne8BjovffO64AaLXpFZsi9OIPrlahR/dnHvS5GBMU0zblgYcbcy4pEMHyDPl478fJ3fyW7bayK9lOQT14VETlmtAJSp+wC7OjReGennu0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=W7IP7bhH; arc=none smtp.client-ip=209.85.167.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="W7IP7bhH" Received: by mail-lf1-f53.google.com with SMTP id 2adb3069b0e04-5b4afc8465eso2466918e87.2 for ; Sat, 12 Sep 2026 00:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789197526; x=1789802326; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=57Wx5MSpf2ecnCSxZA6V8UzQ9jEpIiGdt0LOaHjR6Qk=; b=W7IP7bhHg4v8425hAcrGW4dD+vnuuEWnKcpWjpBdN2qfJSMSkUBPy3L04e0FQq65U8 KGJ8KJYP5qQnTjXvXcyjIv5wFW4C6HIJimoiDJnuKbVxzWMQJiZuCMOdKwrwFyPo1eE8 WkZbfSlanz42qiqaU0jDaiX6rHZjmP2TLFQ5YrJwz9H4WN3vTFQ9j/keyuqdAnwWJ2KU 19YNL7W1TRl/kTm9DFGtq2d4hLGNK3V0AbejEwpUALAt4QHbO805HbEtvdTn1L8CLgL4 zCWdX++2l+YllCJLwT9PF8a4PgscF250x9d+KtC1KSjoj910mBc4nf5vy5YuC+IPZ4NG 6DhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789197526; x=1789802326; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=57Wx5MSpf2ecnCSxZA6V8UzQ9jEpIiGdt0LOaHjR6Qk=; b=kDSwcHLCnutxzNQ7Lg48rTGc1Uprm6HHiLEeQoQ8EFUUnlKSRDyrxQXTVPrgVKvDFm 3nR1bO497mkRl3jXR+eFOJRgf7/TAWoSVJ0BMSn2+GBnGhuoYLmicc2OnNpS0gxo4OdF f+dd58FvHz73mtLrmh3AVbo8eerlU+m+HSY1mPC2hK3pyojV4r/HMavUS4zq1luDar+H KQlXpIeNcmREUW8P5WCy6ndH1Dq0DJqhdlKEgWOo68P/nmp9RoByLE7YUI8BkCz1zYDl GwhIKyBAEEA3I7GgxxRObgzQM+cwjGFEhhr8gmr2j9QC14g+4am/Z0JFfQh/mrKSFTbD XwpQ== X-Forwarded-Encrypted: i=1; AKwUvBz/t5YJtnH59A9bGuegF/iesBTJXeIPxMI/SofPzkYPDkNQ2edStYNw4WJ9XhqS88UmIlW8OLCMTeynmPA=@vger.kernel.org X-Gm-Message-State: AFuF++mTntB1XS3vYt+4bjsu+kQ8FxV+Bo523bJXtC48qWzUGGJnqghQ M+xgTmk/fLnG5UmU64QYO+oek8hQCKS+YDw4U8G3pcP9XriZgVKV4UMj X-Gm-Gg: AYBFou1OUiw0uwn+wNv4cmTu28/5tBvXrjb6jDKeslYXKiCy9pK2j1lXAhQ9XqoXxM9 PB+tnE0nkgTvTSikzZdtUshIdI1roSGPRErRTpuGaZW9og3VPwvChRF4RqDS4Jj1MpLdSu3gHdd JDGNLxw7mxwLny7nObUXMg/L9cqsvAc0XhMtJzkXt6N66saVB9htoRyb7sIvs/WW46dWa7+SsA2 1+C62BxEicJWPY54VHnSgZODVJJm0IwpfuwGWr1CkFJqJGMnMgwRZG0FLPOlJg1pnoktYcUBvOY 2fF0Ae33zVxRvQRjOWUQq6UZMNOmuJ+nOk/F8eAAkRUPtxoqOYn3xUwrHFo1QNe1xkeqVTtRweh DQiIdDiKd4YguWz4J8eNVg7GTGVCAJW2gHlJA5T1KWBsG4KE/RsX6OEJ39M4ROR4UfVSSwB9gni MHyrTnN4/h1GFOM/XxS7X61UGjZyZdEWZGzWwxizUFplEXfEuXZx2JSxVDYv+qJNBfyG4aNt/jD OUWCg== X-Received: by 2002:a05:6512:2246:b0:5ae:b7ca:33ec with SMTP id 2adb3069b0e04-5b8a02de7eamr1453366e87.9.1789197526260; Sat, 12 Sep 2026 00:18:46 -0700 (PDT) Received: from localhost ([95.190.112.239]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5b8a04571b0sm1096277e87.13.2026.09.12.00.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 12 Sep 2026 00:18:45 -0700 (PDT) From: Vladislav Zaharov To: dakr@kernel.org, jhubbard@nvidia.com Cc: acourbot@nvidia.com, aliceryhl@google.com, ttabi@nvidia.com, gary@garyguo.net, nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Vladislav Zaharov Subject: [PATCH v3 1/3] gpu: nova-core: build the debugfs guard before registering the driver Date: Sat, 12 Sep 2026 14:18:40 +0700 Message-ID: <20260912071842.622696-2-vladazaharova2018@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260912071842.622696-1-vladazaharova2018@gmail.com> References: <20260912071842.622696-1-vladazaharova2018@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" init() creates the debugfs root, hands it to a static, and leaves it to DebugfsRootGuard to clear that static once the module goes away. try_pin_init! builds fields in the order they are written, and an initializer that fails drops only what it has already built. The guard is written after the Registration, so a registration that fails leaves it unbuilt and its drop never runs. Statics are not dropped either, and the module is unloaded right after, so the "nova-core" directory outlives everything that could remove it. The next load then finds the name taken: debugfs_create_dir() returns -EEXIST, which Entry keeps as it would any other pointer, and every debugfs file of the driver silently fails to appear until the machine is rebooted. Build the guard first. Drops still run in declaration order, so the driver is still unregistered before the guard clears the static. Assisted-by: Claude:claude-opus-5 Signed-off-by: Vladislav Zaharov --- drivers/gpu/nova-core/nova_core.rs | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/nova-core/nova_core.rs b/drivers/gpu/nova-core/nov= a_core.rs index 1133c6ce5c55..11fe1d2858a9 100644 --- a/drivers/gpu/nova-core/nova_core.rs +++ b/drivers/gpu/nova-core/nova_core.rs @@ -47,7 +47,8 @@ fn drop(&mut self) { #[pin_data] struct NovaCoreModule { // Fields are dropped in declaration order, so `_driver` is dropped fi= rst, - // then `_debugfs_guard` clears `DEBUGFS_ROOT`. + // then `_debugfs_guard` clears `DEBUGFS_ROOT`. They are initialized t= he + // other way round, see `init()`. #[pin] _driver: Registration>, _debugfs_guard: DebugfsRootGuard, @@ -61,9 +62,14 @@ fn init(module: &'static kernel::ThisModule) -> impl Pin= Init { // cannot be any concurrent access to `DEBUGFS_ROOT`. unsafe { DEBUGFS_ROOT =3D Some(dir) }; =20 + // Fields are initialized in the order written here, and an initia= lizer that fails drops + // what it has already built, so the guard goes first: should regi= stration fail, its drop + // still takes `DEBUGFS_ROOT` down with it. Nothing would otherwis= e, as statics are never + // dropped and the module is unloaded right away, leaving a direct= ory behind that the + // next load cannot create again. try_pin_init!(Self { - _driver <- Registration::new(MODULE_NAME, module), _debugfs_guard: DebugfsRootGuard, + _driver <- Registration::new(MODULE_NAME, module), }) } } --=20 2.55.0 From nobody Fri Sep 25 12:34:02 2026 Received: from mail-lf2-f12.google.com (mail-lf2-f12.google.com [74.125.229.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 398563F1ACE for ; Sat, 12 Sep 2026 07:18:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.229.204 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789197533; cv=none; b=n75ostklyIOB/epjxT4ISXbl7l+wSbmWkT94p7LR0rIV7l0xBaRTQlF3CpyrA8S4QE5btzy00+R2BN03FbNrmEiebzxN6R8XuuQw4Y7o6vwCkg7XMg7R/lzee3miUNLCNUY7lHfALI/QeDLeN4f2ygnBSO/hapbmWu5yQrFIXsM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789197533; c=relaxed/simple; bh=XxXI80OwyGmv4bN+rIUNOLMs3E0LrjiW4rPKyX782Q0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CZRs/4SS+VlI4tMlTnbWYwC3O7zd2OO6tfMROR51LyaiXJr14KPq/eJ+kYXRrC0WKCuM9dyT0EYgbSVxft2xQKFbkIZyPgxW84s8fnzAetyTz/t0Xg6IUSmFKUjJd8njffJIZG2LT26gNa0cb6wLqjGvZV5k9Ptg5kG77ks0RN4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CGNZuUFj; arc=none smtp.client-ip=74.125.229.204 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CGNZuUFj" Received: by mail-lf2-f12.google.com with SMTP id 2adb3069b0e04-5b7be8dbabfso225640e87.2 for ; Sat, 12 Sep 2026 00:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789197528; x=1789802328; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=3uM5UvQub7UUt4dzxjz2gzlJaZCMP9QvmISolGD0vvk=; b=CGNZuUFjS0I7oUc5DVZfK4NZRYRV+myz+suFvCuDMR3W7YsIpCDCFyKhKRNFFyInxP MCswHXjV/hQU9ISeeJ0gaXD8vpIgMdaHCLNyTtSfqLMySZzPH5zqxd2AKLfrZvcBgpX7 Bx1t/NFKZ56cFz62VvQv0hB9VcAg9DGy5Gp62yd5zZr9Dt9e6uGl1Pg1V/IhgtylnsDb XMA33EksiiNet2KYto9qDOCQDTRPIntoTKo1cpGSoPHieqm1/xlhli72RUpDfBzAnkEF iShvTPqstmJ3utHGJpoSFM8wZ6g1FkM1JuEuXkxHIq02rZuA0aJxu4nKOfxduBPtEV6e ZbQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789197528; x=1789802328; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=3uM5UvQub7UUt4dzxjz2gzlJaZCMP9QvmISolGD0vvk=; b=GGBaOEUUqvZrRvyEA4l0qa8ie+tU1saGaFm1eIRXY0h6yCi/82HAbe/8AMdYqRZAxQ 8Anf/nPrJXdAi8Sw7KvmR0ysDyzwlEqeuURma4GeSc4EKxTbN/NM90uzq+5zUbFMn+Lt vJ442irHe2eDTUgr7THkIZaYCJPT3ZyGQ0npx+G7ZFQ00opoALoUa6Fiv01RC3gYjH0D Nhhj1377vnRZbUE5C9Oa6rAn8CVT52LxRKW0CwWVmFUa2YtUjlwm+8xKjJ1XinaBPh7z y036GI5FJHJcacbxsqm4yMAFkB1ugEP/R62Er6Qtk3qw8uWZYSQoC/cyqCRFOMrBJ97N xpiA== X-Forwarded-Encrypted: i=1; AKwUvBwNUMTPS1LkhDUnfooyzQ+ZVjM4vzQgcex2qccdlIts8M8FrNF4m6vAT+kENmyde0sN4PtMk2MJxbBF7qg=@vger.kernel.org X-Gm-Message-State: AFuF++nI+HQcgk3q56PIYz1GWolbzBVqrAHWZ/cNqJDBFxfhGapFc+3q X07KUrW28ai116H40Y8weNlp1UWa+5do/eM79e/ss5HSqFyhBsv1GeGh X-Gm-Gg: AYBFou1LHdCDxFUjqec2bpvE8vnOZL+jcQ84EGa/FR9Vc9A2QTLuMT95Yn4I8GTxXFX znSt+valeV9C5Swdk5UpYGChEuji5wQYH2fuqVOOCvX6N6BMfVEsxH7OcJXHGCxhXLl2/OVpTrE x8MAWWHACz9T3MolVpfyAlkiEEzIgZZZH105XUPImxH58XsvYurUy/GGsnGc22hJSTxATIS+cNV 3n9qQZ6bc3ZQxITZRz4Vzo0logrDruTz5t8+TLtfKZ/o+rQjOYgZYgSpdoCdEMavp2mD5Ie+Q3D f/rdeTUYzhXS3eqmNnZ2Tl2Sl+jaJImQK/giboFjGMglPudVoas0LeM5AY3+25DguaJi7l+bvxG Kh5CMerSl30uhjPm8xG8aUxL0c3EvsrpD7Y8TOaiA7+AYp2aF+IV5IokoKxZ4EKiykVMWhGCj6P sKmSSXAVKGhAXJrgRUV4iWE9NnHC9nE4OKCFGAZ/WobTNW0xQQZ3CtopljCOmORXIK6Gn1Mytir X8rWhqtVrrWEaWW X-Received: by 2002:a05:6512:15a0:b0:5b6:180e:3d6d with SMTP id 2adb3069b0e04-5b8a8ebe61fmr274523e87.9.1789197527758; Sat, 12 Sep 2026 00:18:47 -0700 (PDT) Received: from localhost ([95.190.112.239]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5b8a04571b0sm1096277e87.13.2026.09.12.00.18.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 12 Sep 2026 00:18:47 -0700 (PDT) From: Vladislav Zaharov To: dakr@kernel.org, jhubbard@nvidia.com Cc: acourbot@nvidia.com, aliceryhl@google.com, ttabi@nvidia.com, gary@garyguo.net, nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Vladislav Zaharov Subject: [PATCH v3 2/3] gpu: nova-core: gsp: retain the GSP-RM log buffers after unbind Date: Sat, 12 Sep 2026 14:18:41 +0700 Message-ID: <20260912071842.622696-3-vladazaharova2018@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260912071842.622696-1-vladazaharova2018@gmail.com> References: <20260912071842.622696-1-vladazaharova2018@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The GSP-RM log buffers are exposed through debugfs, but the Scope that owns them lives in Gsp, inside GspResources, inside the Gpu built by probe(). They are DMA allocations of the device and cannot outlive it, so the entries go away as soon as the GPU is unbound - and, more to the point, as soon as probe() fails, which is exactly when the log of a GSP that did not come up is the thing one wants to read. Add a gsp_keep_logs module parameter. When it is set, dropping the log buffers copies whatever the GSP wrote into memory owned by the module and exposes the copies until the module is unloaded. A buffer whose "put" pointer is still zero was never written to and is skipped. The GSP has normally been stopped by the time the buffers are dropped, but a boot that timed out can leave it still appending, so a DMA read barrier orders the read of the "put" pointer before the copy. The copies live in a "retained" directory, created during module init rather than on first use, which keeps the teardown path from having to reach for DEBUGFS_ROOT. Keeping them out of the directory used by bound GPUs also means a device coming back does not find its debugfs name taken by its own history; nouveau, which recreates the entries under the name of the GPU that just went away, has that problem. While at it, move the log buffer code out of gsp.rs into gsp/logbuffer.rs. Assisted-by: Claude:claude-opus-5 Signed-off-by: Vladislav Zaharov --- drivers/gpu/nova-core/gsp.rs | 100 ++------- drivers/gpu/nova-core/gsp/logbuffer.rs | 267 +++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 32 ++- 3 files changed, 315 insertions(+), 84 deletions(-) create mode 100644 drivers/gpu/nova-core/gsp/logbuffer.rs diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs index 25ea43f1cbe9..1a1eb7f37075 100644 --- a/drivers/gpu/nova-core/gsp.rs +++ b/drivers/gpu/nova-core/gsp.rs @@ -12,11 +12,7 @@ CoherentView, DmaAddress, // }, - io::{ - io_project, - io_write, - Io, // - }, + io::io_write, pci, prelude::*, // }; @@ -24,9 +20,13 @@ pub(crate) mod cmdq; pub(crate) mod commands; mod fw; +mod logbuffer; mod regs; mod sequencer; =20 +use logbuffer::LogBuffers; +pub(crate) use logbuffer::RetainedLogs; + pub(crate) use fw::{ GspFmcBootParams, GspFwWprMeta, @@ -77,10 +77,6 @@ pub(crate) fn dev(&self) -> &'gpu device::Device { } } =20 -/// Number of GSP pages to use in a RM log buffer. -const RM_LOG_BUFFER_NUM_PAGES: usize =3D 0x10; -const LOG_BUFFER_SIZE: usize =3D RM_LOG_BUFFER_NUM_PAGES * GSP_PAGE_SIZE; - /// Array of page table entries, as understood by the GSP bootloader. #[repr(C)] #[derive(FromBytes, IntoBytes)] @@ -101,49 +97,6 @@ fn init(view: CoherentView<'_, Self>, start: DmaAddress= ) -> Result<()> { } } =20 -/// The logging buffers are byte queues that contain encoded printf-like -/// messages from GSP-RM. They need to be decoded by a special application -/// that can parse the buffers. -/// -/// The 'loginit' buffer contains logs from early GSP-RM init and -/// exception dumps. The 'logrm' buffer contains the subsequent logs. Bot= h are -/// written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE. -/// -/// The physical address map for the log buffer is stored in the buffer -/// itself, starting with offset 1. Offset 0 contains the "put" pointer (p= p). -/// Initially, pp is equal to 0. If the buffer has valid logging data in i= t, -/// then pp points to index into the buffer where the next logging entry w= ill -/// be written. Therefore, the logging data is valid if: -/// 1 <=3D pp < sizeof(buffer)/sizeof(u64) -struct LogBuffer<'a>(Coherent<'a, [u8; LOG_BUFFER_SIZE]>); - -impl<'a> LogBuffer<'a> { - /// Creates a new `LogBuffer` mapped on `dev`. - fn new(dev: &'a device::Device) -> Result { - let obj =3D Self(Coherent::zeroed(dev, GFP_KERNEL)?); - - let start_addr =3D obj.0.dma_address(); - - let pte_view =3D io_project!( - obj.0, - [build: size_of::()..][build: ..RM_LOG_BUFFER_NUM_PAGES *= size_of::()] - ) - .try_cast::>()?; - PteArray::init(pte_view, start_addr)?; - - Ok(obj) - } -} - -struct LogBuffers<'a> { - /// Init log buffer. - loginit: LogBuffer<'a>, - /// Interrupts log buffer. - logintr: LogBuffer<'a>, - /// RM log buffer. - logrm: LogBuffer<'a>, -} - /// GSP runtime data. #[pin_data] pub(crate) struct Gsp<'gsp> { @@ -165,9 +118,7 @@ pub(crate) fn new(pdev: &'gsp pci::Device) -> impl PinInit) -> impl PinInit(pub(super) Coherent<'a, [u8; LOG_BUFFER_SI= ZE]>); + +impl<'a> LogBuffer<'a> { + /// Creates a new `LogBuffer` mapped on `dev`. + fn new(dev: &'a device::Device) -> Result { + let obj =3D Self(Coherent::zeroed(dev, GFP_KERNEL)?); + + let start_addr =3D obj.0.dma_address(); + + let pte_view =3D io_project!( + obj.0, + [build: size_of::()..][build: ..RM_LOG_BUFFER_NUM_PAGES *= size_of::()] + ) + .try_cast::>()?; + PteArray::init(pte_view, start_addr)?; + + Ok(obj) + } + + /// Copies the contents of this buffer into memory that does not belon= g to the device. + /// + /// A buffer the GSP never wrote to yields an empty vector, as it hold= s nothing worth keeping. + fn snapshot(&self) -> Result> { + // Offset 0 holds the "put" pointer, which the GSP advances as it = appends entries. It is + // still zero if nothing was ever logged, which is all that is tes= ted here: a buffer that + // was written to is copied whole, and making sense of "put" is le= ft to the decoder. + let put =3D io_project!(self.0, [build: ..size_of::()]).try_c= ast::()?; + if put.read_val() =3D=3D 0 { + return Ok(VVec::new()); + } + + // ORDERING: LOAD->LOAD ordering needed to order the "put" read be= fore the data read. The + // GSP has normally been stopped by the time this runs, but a boot= that timed out can leave + // it still appending. + dma_mb(Read); + + let mut snapshot =3D VVec::zeroed(LOG_BUFFER_SIZE, GFP_KERNEL)?; + io_project!(self.0, [build: ..]).copy_to_slice(&mut snapshot); + + Ok(snapshot) + } +} + +/// The log buffers of a GPU, for as long as it is bound to the driver. +pub(super) struct LogBuffers<'a> { + /// Device the buffers belong to. Also names their debugfs directory. + dev: &'a device::Device, + /// Init log buffer. + pub(super) loginit: LogBuffer<'a>, + /// Interrupts log buffer. + pub(super) logintr: LogBuffer<'a>, + /// RM log buffer. + pub(super) logrm: LogBuffer<'a>, +} + +impl<'a> LogBuffers<'a> { + /// Allocates the three log buffers of `dev`. + pub(super) fn new(dev: &'a device::Device) -> Result { + Ok(Self { + dev, + loginit: LogBuffer::new(dev)?, + logintr: LogBuffer::new(dev)?, + logrm: LogBuffer::new(dev)?, + }) + } + + /// Creates an initializer exposing these buffers under a directory na= med after their device. + pub(super) fn scope(self) -> impl PinInit, Infall= ible> + 'a { + let dev =3D self.dev; + + #[allow(static_mut_refs)] + // SAFETY: `DEBUGFS_ROOT` is created before driver registration an= d cleared + // after driver unregistration, so no probe() can race with its mo= dification. + // + // PANIC: `DEBUGFS_ROOT` cannot be `None` here. It is set before = driver + // registration and cleared after driver unregistration, so it is = always + // `Some` for the entire lifetime that probe() can be called. + let log_parent: &debugfs::Dir =3D + unsafe { crate::DEBUGFS_ROOT.as_ref() }.expect("DEBUGFS_ROOT n= ot initialized"); + + log_parent.scope(self, dev.name(), |logs, dir| { + dir.read_binary_file(c"loginit", &logs.loginit.0); + dir.read_binary_file(c"logintr", &logs.logintr.0); + dir.read_binary_file(c"logrm", &logs.logrm.0); + }) + } + + /// Preserves whatever the GSP logged, so it can still be read once th= e GPU is gone. + /// + /// The buffers are DMA allocations of the device and cannot outlive i= t, so their contents are + /// copied into memory owned by the module and exposed through fresh d= ebugfs entries. Those + /// live until the module is unloaded. + /// + /// Does nothing if `gsp_keep_logs` was not set when the module was lo= aded, as there is then + /// no directory to put the copies in. + fn retain(&self) -> Result { + // Copying is only worth it if there is somewhere to put the resul= t, but the lock is + // dropped right away: what follows allocates 64 KiB three times, = and no other device + // should have to wait for that. + if !crate::RETAINED_LOGS.lock().is_enabled() { + return Ok(()); + } + + let logs =3D RetainedLogBuffers { + dev: self.dev.into(), + loginit: self.loginit.snapshot()?, + logintr: self.logintr.snapshot()?, + logrm: self.logrm.snapshot()?, + }; + + // Nothing was ever logged, so there is nothing to keep. A copy fr= om an earlier run of + // this device is deliberately left alone: logs from a run that fa= iled are worth more + // than the silence of one that did not. + if logs.loginit.is_empty() && logs.logintr.is_empty() && logs.logr= m.is_empty() { + return Ok(()); + } + + // Take every allocation that can fail before the previous copy of= this device is + // dropped, so that running out of memory here cannot leave it wit= h no logs at all. + let scope =3D KBox::>::new_unin= it(GFP_KERNEL)?; + + let mut retained =3D crate::RETAINED_LOGS.lock(); + + // The module may have been unloaded out from under us while the c= opies were taken. + let Some(dir) =3D retained.dir.clone() else { + return Ok(()); + }; + + retained.gpus.reserve(1, GFP_KERNEL)?; + + // An earlier run of the same device may have left a copy behind, = and its directory + // carries the name about to be used again, so it has to go first.= Nothing below can + // fail, so the replacement is guaranteed to take its place. + retained + .gpus + .retain(|gpu| gpu.dev.name() !=3D self.dev.name()); + + let scope =3D scope.write_pin_init(dir.scope(logs, self.dev.name()= , |logs, dir| { + if !logs.loginit.is_empty() { + dir.read_binary_file(c"loginit", &logs.loginit); + } + if !logs.logintr.is_empty() { + dir.read_binary_file(c"logintr", &logs.logintr); + } + if !logs.logrm.is_empty() { + dir.read_binary_file(c"logrm", &logs.logrm); + } + }))?; + + retained.gpus.push(scope, GFP_KERNEL)?; + + dev_dbg!(self.dev, "GSP-RM log buffers retained\n"); + + Ok(()) + } +} + +impl Drop for LogBuffers<'_> { + fn drop(&mut self) { + if let Err(e) =3D self.retain() { + dev_warn!(self.dev, "failed to retain GSP-RM log buffers: {:?}= \n", e); + } + } +} + +/// Copies of the log buffers of a GPU that is no longer around. +struct RetainedLogBuffers { + /// Device the buffers came from. + dev: ARef, + /// Contents of the init log buffer, empty if it was never written to. + loginit: VVec, + /// Contents of the interrupts log buffer, empty if it was never writt= en to. + logintr: VVec, + /// Contents of the RM log buffer, empty if it was never written to. + logrm: VVec, +} + +/// Log buffers of GPUs that are gone, and the debugfs entries exposing th= em. +/// +/// The copies live under a `retained` directory of their own instead of n= ext to the entries of +/// the GPUs that are actually bound, so that a device coming back does no= t find its name taken. +pub(crate) struct RetainedLogs { + /// Parent directory of all copies. `None` unless retaining was asked = for. + dir: Option, + /// One entry per GPU. + gpus: KVec>>>, +} + +impl RetainedLogs { + /// Creates an empty set of retained log buffers, retaining disabled. + pub(crate) const fn new() -> Self { + Self { + dir: None, + gpus: KVec::new(), + } + } + + /// Creates the directory the copies will live in, enabling retaining. + /// + /// Does nothing without `CONFIG_DEBUG_FS`, where a [`debugfs::Dir`] i= s a zero-sized type and + /// the copies could never be read back. + pub(crate) fn enable(&mut self, parent: &debugfs::Dir) { + if !cfg!(CONFIG_DEBUG_FS) { + return; + } + + self.dir =3D Some(parent.subdir(c"retained")); + } + + /// Returns whether copies are being kept. + pub(crate) fn is_enabled(&self) -> bool { + self.dir.is_some() + } + + /// Releases every copy and the directory holding them. + pub(crate) fn clear(&mut self) { + self.gpus.clear(); + self.dir =3D None; + } +} diff --git a/drivers/gpu/nova-core/nova_core.rs b/drivers/gpu/nova-core/nov= a_core.rs index 11fe1d2858a9..557cc611f3fc 100644 --- a/drivers/gpu/nova-core/nova_core.rs +++ b/drivers/gpu/nova-core/nova_core.rs @@ -33,11 +33,21 @@ // TODO: Move this into per-module data once that exists. static mut DEBUGFS_ROOT: Option =3D None; =20 +kernel::sync::global_lock! { + /// Log buffers of GPUs that are gone, kept around until the module is= unloaded. + // TODO: Move this into per-module data once that exists. + unsafe(uninit) static RETAINED_LOGS: Mutex =3D gsp:= :RetainedLogs::new(); +} + /// Guard that clears `DEBUGFS_ROOT` when dropped. struct DebugfsRootGuard; =20 impl Drop for DebugfsRootGuard { fn drop(&mut self) { + // Retained log buffers own debugfs entries below `DEBUGFS_ROOT`, = so they have to go away + // before it does. + RETAINED_LOGS.lock().clear(); + // SAFETY: This guard is dropped after `_driver` (due to field ord= er), // so the driver is unregistered and no probe() can be running. unsafe { DEBUGFS_ROOT =3D None }; @@ -58,15 +68,25 @@ impl InPlaceModule for NovaCoreModule { fn init(module: &'static kernel::ThisModule) -> impl PinInit { let dir =3D debugfs::Dir::new(c"nova-core"); =20 + // SAFETY: Module initialization runs exactly once, and before the= driver is registered, + // so no probe can have touched `RETAINED_LOGS` yet. + unsafe { RETAINED_LOGS.init() }; + + // Creating the directory up front is what makes retaining possibl= e without reaching for + // `DEBUGFS_ROOT` later, from the teardown path of a device. + if module_parameters::gsp_keep_logs.value() { + RETAINED_LOGS.lock().enable(&dir); + } + // SAFETY: We are the only driver code running during init, so the= re // cannot be any concurrent access to `DEBUGFS_ROOT`. unsafe { DEBUGFS_ROOT =3D Some(dir) }; =20 // Fields are initialized in the order written here, and an initia= lizer that fails drops // what it has already built, so the guard goes first: should regi= stration fail, its drop - // still takes `DEBUGFS_ROOT` down with it. Nothing would otherwis= e, as statics are never - // dropped and the module is unloaded right away, leaving a direct= ory behind that the - // next load cannot create again. + // still takes `DEBUGFS_ROOT` and the retained copies down with it= . Nothing would + // otherwise, as statics are never dropped and the module is unloa= ded right away, leaving + // directories behind that the next load cannot create again. try_pin_init!(Self { _debugfs_guard: DebugfsRootGuard, _driver <- Registration::new(MODULE_NAME, module), @@ -81,6 +101,12 @@ fn init(module: &'static kernel::ThisModule) -> impl Pi= nInit { description: "Nova Core GPU driver", license: "GPL v2", firmware: [], + params: { + gsp_keep_logs: bool { + default: false, + description: "Keep the GSP-RM log buffers in debugfs after the= ir GPU is gone", + }, + }, } =20 kernel::module_firmware!(firmware::ModInfoBuilder); --=20 2.55.0 From nobody Fri Sep 25 12:34:02 2026 Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71EB73F1AAD for ; Sat, 12 Sep 2026 07:18:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789197534; cv=none; b=Lqj11HDP69TcC9rDYgsjhlzIFkvnvz/d96EIr4JUJUy6gSfgpEUMPRB+Mnt/b0hNiZUYAbZmAIgmAdl/mn4W893H38evUYkwHMEpmeR2PTefLsrRaYZ9mwLcvlaLtgcee9zmOcsjse3SdlFaje5uYrL0FHL2SNhiNX5e8UgaTWY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789197534; c=relaxed/simple; bh=RNy0Oc1b5YVJpZS5AO65ioCXwPG1K+v8mTvUlX+rTyw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HgQjZ9pKp1IcZAsgMa/SazeowSXNEsNHUybvDNCtMVlMYfiEkBzMr407Bb0nqExbS+Y6iZbNsfX9BTQTAnxm2QZrk9LBhqLOH2qcr4Gi/nDrACB32qHKCe16UwSWDgkFYLgXXLSH4Y+gvWSZTYfqyLND5oiLJVklWmGmqgc1Bag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=i+tBeMaL; arc=none smtp.client-ip=209.85.167.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i+tBeMaL" Received: by mail-lf1-f41.google.com with SMTP id 2adb3069b0e04-5b76515c0d7so2126620e87.0 for ; Sat, 12 Sep 2026 00:18:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789197529; x=1789802329; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NgkFZajAQJmzi//1l9aIpRfIFvgCaKKT8i3Z0CDYaWs=; b=i+tBeMaLpJsVRd5NtZFvwfpAYPw2IkUNxdL7HRV+1mzv4sR+y8lf2sh9y9YarFY9re bzopurCtaZLwNiIZXYHIEHb2byDa2my79AB/wPdW7I4hg1NDO/IrE6g+6OxgobWoNn6B Winja3vcyuDrdaXJIv7unmnS6624r0lVIs7PZIgrw01NHA2Uic6fNYiPwHUpiDRpFpl6 GvTRAKx44XjGES0M3D/vKuiE6sZyuGActsqMdpA6S3hRRnMYrj1iKlBu0lwQ74LTm1PN tlVN/kMqCwn9oVy9ugz/Vs4RKU8BSYmRODMHd9IoIuc1OzEs6jCgUnbtwIwC0qHXhJZq JIAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789197529; x=1789802329; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NgkFZajAQJmzi//1l9aIpRfIFvgCaKKT8i3Z0CDYaWs=; b=XXL3nEARwtkaBKaGX7ky7h2/A/sQhKytWZBCHY2rHKKD7efz0jC8JIaFJRg8ZQ1R96 U/ahneXFr1Py0xjvJj1hCsN4tTtQWysA8K9XPtLp16omz9qMbsX+HV8Mdwg2C0OunyKT fRXZ74yrJiavkXn5c5vjIiJUSvWmXv4tW5tI8XWhWuyB0k5yCIyIKn0Dkfr9zVOrLC4i oFd24snxMaGzKlzrBquYwB09XdfSNZ1LnVgtSDXl0WnXtOkhNuNsqk6rM66aigbmN0O6 JhENAv/BK9Ebp8ncUagLHOTImUIcUTmqCffmPzPSTm8t5mIbhXxMFFd0lJRVgdXm4ldn A55g== X-Forwarded-Encrypted: i=1; AKwUvBzKhk/tAiXC8UXl5DPWJwTtYRsGS0JixelmJ3WQhRGQrYRAc+Dgz/VC+y9Jaq2ddAMnK4Or/slBCo9pXyw=@vger.kernel.org X-Gm-Message-State: AFuF++lNxlLFA4NiGW0b+i67n2SeDy31jUrq9oZqrUKid4qnsl40xEEe tGXp2F2E/PrUxthbf7gIwIVjoNOxdfz4XdgJRwIzLDO7tm+bh19vUQ2j X-Gm-Gg: AYBFou31VllDD+lu7rtYkrTkSwFpGnJqOqCrUgqsUAc6u1pqThZbQgCJZoWCHE9OfIS dziuHEfCVaFB22oM8zOkouxinLE0VRjEbOxbDXHc2aBZcsEH2G3CI5gTiwVGo16CrF0Q4kUGvZY ZkI5b4BZE37kD6IHMUpr8YPqqpYIdB0583rkedNktoo+6dgei2u4avtSFZdn8fDgbauAf1Tyuii qyAxWkr7VGS9JXTaYT1WrXJ5rICvVkl31NVuLLtrBCP2QYtz8rWKDdWHmWx7XNW0jCj+qAb1vwh 5n5JFymvigjxFajkC0vD0kAzSZsdUjHtE9mtPSX93QmHb4xFHV9zFlMw44nM2QicTBj2OhOhRkT Gpu1bV3nKUqGuWqN8NaeXxLOrzxHwQD7ZgPkClgcFxi907w/+wf2OWdn+YsJc00vd9IznmF6uuK rUunB3Q+v3R8o/mBBZuB4pm0fPFisZqinXbeYvs+8tbBbrVS4SiGBCvas9jvZkrY6UgRgYFtHhA S7Njw== X-Received: by 2002:a2e:a596:0:b0:3a4:68f1:8bee with SMTP id 38308e7fff4ca-3a59108ac42mr14680391fa.6.1789197529066; Sat, 12 Sep 2026 00:18:49 -0700 (PDT) Received: from localhost ([95.190.112.239]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5b8a04571b0sm1096277e87.13.2026.09.12.00.18.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 12 Sep 2026 00:18:48 -0700 (PDT) From: Vladislav Zaharov To: dakr@kernel.org, jhubbard@nvidia.com Cc: acourbot@nvidia.com, aliceryhl@google.com, ttabi@nvidia.com, gary@garyguo.net, nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Vladislav Zaharov Subject: [PATCH v3 3/3] Documentation: nova: remove completed GSP log buffer task Date: Sat, 12 Sep 2026 14:18:42 +0700 Message-ID: <20260912071842.622696-4-vladazaharova2018@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260912071842.622696-1-vladazaharova2018@gmail.com> References: <20260912071842.622696-1-vladazaharova2018@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Exposing the GSP-RM log buffers through debugfs is implemented, and with the gsp_keep_logs module parameter they now also survive a failed probe, which was the part of the task that was still missing. Assisted-by: Claude:claude-opus-5 Signed-off-by: Vladislav Zaharov --- Documentation/gpu/nova/core/todo.rst | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/Documentation/gpu/nova/core/todo.rst b/Documentation/gpu/nova/= core/todo.rst index d5130b2b08fb..cae0578d32f1 100644 --- a/Documentation/gpu/nova/core/todo.rst +++ b/Documentation/gpu/nova/core/todo.rst @@ -141,18 +141,6 @@ Implement support for instmem (bar2) used to store pag= e tables. GPU System Processor (GSP) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D =20 -Export GSP log buffers ----------------------- - -Recent patches from Timur Tabi [1] added support to expose GSP-RM log buff= ers -(even after failure to probe the driver) through debugfs. - -This is also an interesting feature for nova-core, especially in the early= days. - -| Link: https://lore.kernel.org/nouveau/20241030202952.694055-2-ttabi@nvid= ia.com/ [1] -| Reference: Debugfs abstractions -| Complexity: Intermediate - GSP firmware abstraction ------------------------ =20 --=20 2.55.0