From nobody Fri Sep 25 01:20:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B7444071CA for ; Thu, 17 Sep 2026 19:09:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672159; cv=none; b=pnKN3DnQRvixM5ZcxE1oQvffdIO8A+dsvzg+eww0JFvQDJHsPXn9wgwVVCW3/boAPHvXeibwQg2LbAUIzLTj2k59v8GhQlFJRAA4XU0nB5OcjfpbRsXjYxn2FWBzdmZoIdGTi1Cy3+Mzn47DC2G9N+yhbpXwXmcTkXteiri3CJU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672159; c=relaxed/simple; bh=eHOw8F3IkRIGtKhm/M6ooAyD9lgu+cTocEyxzD8OK1A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LIVk2PWwK1FFvjXkOYz2JJaJLmkcEicPZGvtLA8vJr/S9MUHKDyXedgwn4FD/EQNkmyHSeSuqTOFwHmQQY7pG+ZFu+ObYN6HOUY2rSJJQcOSAR3SQJnYRZ7zz9xJhF086bHWPX9zV+AYL1q8e9l3PqjY5ejxpIYoTtUU0uKGUG4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=V0U6YKAQ; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="V0U6YKAQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789672156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UMYaLnjAcPpkFYhDeWh0e8dcKcn1Q9rTNxagCwidSss=; b=V0U6YKAQCG1f9yVsRvnJwbJjZeABVSIIdnamRnG25dz9LQf3/1wz6nr83zfi7mQV6NV+qU X+E/8n0onvhf4BoeF/IH4SJeMdPdghe7R0NX/xBzjzNzb+IwsefhKSAZFZcsSTBygFcDeU FeXhyYO0bwHpBP5Lw+7uhlDk+x6V59Q= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-260-cJpuEwPpMwKswTyH-jZSyQ-1; Thu, 17 Sep 2026 15:09:10 -0400 X-MC-Unique: cJpuEwPpMwKswTyH-jZSyQ-1 X-Mimecast-MFC-AGG-ID: cJpuEwPpMwKswTyH-jZSyQ_1789672147 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8B46E18301D0; Thu, 17 Sep 2026 19:09:06 +0000 (UTC) Received: from GoldenWind.lan (unknown [10.22.64.177]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0574319AC047; Thu, 17 Sep 2026 19:09:03 +0000 (UTC) From: Lyude Paul To: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: Dave Airlie , "Timur Tabi" , "Andy Shevchenko" , "Mohamed Ahmed" , "Maarten Lankhorst" , "Kees Cook" , "Simona Vetter" , "David Airlie" , "Thomas Zimmermann" , "Maxime Ripard" , "Mel Henning" , "Hongling Zeng" , "Danilo Krummrich" , "Lyude Paul" Subject: [PATCH v6 1/5] drm/nouveau/gsp/r570: Add support for INTERNAL_GCX_ENTRY_PREREQUISITE Date: Thu, 17 Sep 2026 14:54:25 -0400 Message-ID: <20260917185916.1089621-2-lyude@redhat.com> In-Reply-To: <20260917185916.1089621-1-lyude@redhat.com> References: <20260917185916.1089621-1-lyude@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" OpenRM's runtime PM handling looks a bit different then nouveau's, one part in particular that differs from us: OpenRM actually consults GSP to ask whether the GPU should be allowed to enter Gc6 and/or GcOff before runtime suspending the GPU. In the event the card isn't ready, runtime suspend is simply delayed for a few seconds before retrying. Implement the command used for querying GSP about this, NV2080_CTRL_CMD_INTERNAL_GCX_ENTRY_PREREQUISITE, and check to ensure that the GPU is ready for runtime suspend in nouveau_pmops_runtime_suspend() using this query. If the GPU can't be runtime suspended, update the last busy counter of the device and then return -EBUSY from nouveau_pmops_runtime_suspend() - essentially delaying the runtime suspend process by whatever autosuspend_delay_ms is set to. Signed-off-by: Lyude Paul Reviewed-by: Dave Airlie --- V2: * Remove accidental return of bitmask from nvkm_udevice_gcx_ready() * Use gsp variable in nvkm_udevice_gcx_ready() more. * Remove unused GSP_GCX_ENTRY defines V3: * Fix error checking V4: * Fix layering violation * Remove structs from r535, this hadn't been added yet! V5: * Fixup another mistaken duplicate of the INTERNAL_GCX_ENTRY_PREREQUISITE struct V6: * Fix another accidental mismatch between the GSP objects used for ctrl_rd vs. ctrl_done. Not a clue how this got back in. drivers/gpu/drm/nouveau/include/nvif/cl0080.h | 10 ++++++ drivers/gpu/drm/nouveau/include/nvif/device.h | 1 + .../gpu/drm/nouveau/include/nvkm/subdev/gsp.h | 2 ++ drivers/gpu/drm/nouveau/nouveau_drm.c | 13 +++++++ drivers/gpu/drm/nouveau/nvif/device.c | 13 +++++++ .../gpu/drm/nouveau/nvkm/engine/device/user.c | 35 +++++++++++++++++++ .../gpu/drm/nouveau/nvkm/subdev/gsp/base.c | 10 ++++++ .../gpu/drm/nouveau/nvkm/subdev/gsp/priv.h | 1 + .../drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c | 28 +++++++++++++++ .../nvkm/subdev/gsp/rm/r570/nvrm/gsp.h | 7 ++++ .../gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h | 2 ++ 11 files changed, 122 insertions(+) diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl0080.h b/drivers/gpu/dr= m/nouveau/include/nvif/cl0080.h index ea8267e0d8da2..9e639df1da467 100644 --- a/drivers/gpu/drm/nouveau/include/nvif/cl0080.h +++ b/drivers/gpu/drm/nouveau/include/nvif/cl0080.h @@ -4,6 +4,7 @@ =20 #define NV_DEVICE_V0_INFO = 0x00 #define NV_DEVICE_V0_TIME = 0x01 +#define NV_DEVICE_V0_GCX_READY 0x02 =20 struct nv_device_info_v0 { __u8 version; @@ -55,6 +56,15 @@ struct nv_device_time_v0 { __u64 time; }; =20 +#define NV_DEVICE_GC6_READY BIT(0) +#define NV_DEVICE_GCOFF_READY BIT(1) + +struct nv_device_gcx_ready_v0 { + __u8 version; + __u8 pad01[6]; + __u8 ready; +}; + #define NV_DEVICE_INFO_UNIT (0xffffffffULL <= < 32) #define NV_DEVICE_INFO(n) ((n) | (0x00000000ULL <= < 32)) #define NV_DEVICE_HOST(n) ((n) | (0x00000001ULL <= < 32)) diff --git a/drivers/gpu/drm/nouveau/include/nvif/device.h b/drivers/gpu/dr= m/nouveau/include/nvif/device.h index 7877a2a79da9b..ce2fadcb05d9e 100644 --- a/drivers/gpu/drm/nouveau/include/nvif/device.h +++ b/drivers/gpu/drm/nouveau/include/nvif/device.h @@ -22,4 +22,5 @@ int nvif_device_ctor(struct nvif_client *, const char *n= ame, struct nvif_device void nvif_device_dtor(struct nvif_device *); int nvif_device_map(struct nvif_device *); u64 nvif_device_time(struct nvif_device *); +int nvif_device_gcx_ready(struct nvif_device *); #endif diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h b/drivers/gp= u/drm/nouveau/include/nvkm/subdev/gsp.h index 64fed208e4cf8..cd10c37053695 100644 --- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h +++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h @@ -495,6 +495,8 @@ nvkm_gsp_event_dtor(struct nvkm_gsp_event *event) int nvkm_gsp_intr_stall(struct nvkm_gsp *, enum nvkm_subdev_type, int); int nvkm_gsp_intr_nonstall(struct nvkm_gsp *, enum nvkm_subdev_type, int); =20 +int nvkm_gsp_gcx_ready(struct nvkm_gsp *gsp); + int gv100_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct= nvkm_gsp **); int tu102_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct= nvkm_gsp **); int tu116_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct= nvkm_gsp **); diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.c b/drivers/gpu/drm/nouvea= u/nouveau_drm.c index 4d1ad718e09b7..298b940296c73 100644 --- a/drivers/gpu/drm/nouveau/nouveau_drm.c +++ b/drivers/gpu/drm/nouveau/nouveau_drm.c @@ -1148,6 +1148,7 @@ nouveau_pmops_runtime_suspend(struct device *dev) { struct pci_dev *pdev =3D to_pci_dev(dev); struct nouveau_drm *drm =3D pci_get_drvdata(pdev); + struct nvif_device *nvif =3D &drm->client.device; int ret; =20 if (!nouveau_pmops_runtime()) { @@ -1155,6 +1156,18 @@ nouveau_pmops_runtime_suspend(struct device *dev) return -EBUSY; } =20 + // Check if the GPU itself is ready for runtime suspend, otherwise mark a= s busy and check + // again in a bit. + ret =3D nvif_device_gcx_ready(nvif); + if (ret < 0) { + NV_ERROR(drm, "Failed to query GCX readiness (returned %d)\n", ret); + return -EBUSY; + } else if (!(ret & NV_DEVICE_GCOFF_READY)) { + NV_DEBUG(drm, "GPU isn't ready for suspend yet, delaying...\n"); + pm_runtime_mark_last_busy(dev); + return -EBUSY; + } + nouveau_switcheroo_optimus_dsm(); ret =3D nouveau_do_suspend(drm, true); pci_save_state(pdev); diff --git a/drivers/gpu/drm/nouveau/nvif/device.c b/drivers/gpu/drm/nouvea= u/nvif/device.c index 24880931039f6..1be9fbe6cb703 100644 --- a/drivers/gpu/drm/nouveau/nvif/device.c +++ b/drivers/gpu/drm/nouveau/nvif/device.c @@ -38,6 +38,19 @@ nvif_device_time(struct nvif_device *device) return device->user.func->time(&device->user); } =20 +int +nvif_device_gcx_ready(struct nvif_device *device) +{ + struct nv_device_gcx_ready_v0 args =3D {}; + int ret; + + ret =3D nvif_object_mthd(&device->object, NV_DEVICE_V0_GCX_READY, &args, = sizeof(args)); + if (ret) + return ret; + + return args.ready; +} + int nvif_device_map(struct nvif_device *device) { diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c b/drivers/gp= u/drm/nouveau/nvkm/engine/device/user.c index 23d11d8221cb6..f78e6b9b42923 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/user.c @@ -27,6 +27,7 @@ =20 #include #include +#include #include #include =20 @@ -189,6 +190,38 @@ nvkm_udevice_time(struct nvkm_udevice *udev, void *dat= a, u32 size) return ret; } =20 +static int +nvkm_udevice_gcx_ready(struct nvkm_udevice *udev, void *data, u32 size) +{ + struct nvkm_object *object =3D &udev->object; + struct nvkm_device *device =3D udev->device; + struct nvkm_gsp *gsp =3D device->gsp; + union { + struct nv_device_gcx_ready_v0 v0; + } *args =3D data; + int ret =3D -ENOSYS; + + if (!gsp) { + args->v0.ready =3D NV_DEVICE_GC6_READY | NV_DEVICE_GCOFF_READY; + return 0; + } + + nvif_ioctl(object, "device gcx ready size %d\n", size); + ret =3D nvif_unpack(ret, &data, &size, args->v0, 0, 0, false); + if (!ret) { + nvif_ioctl(object, "device gcx ready vers %d\n", args->v0.version); + + ret =3D nvkm_gsp_gcx_ready(gsp); + if (ret < 0) + return ret; + + args->v0.ready =3D ret; + ret =3D 0; + } + + return ret; +} + static int nvkm_udevice_mthd(struct nvkm_object *object, u32 mthd, void *data, u32 si= ze) { @@ -199,6 +232,8 @@ nvkm_udevice_mthd(struct nvkm_object *object, u32 mthd,= void *data, u32 size) return nvkm_udevice_info(udev, data, size); case NV_DEVICE_V0_TIME: return nvkm_udevice_time(udev, data, size); + case NV_DEVICE_V0_GCX_READY: + return nvkm_udevice_gcx_ready(udev, data, size); default: break; } diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c b/drivers/gpu/d= rm/nouveau/nvkm/subdev/gsp/base.c index 9ba1316831e7e..e475d0e8fa7bf 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.c @@ -20,6 +20,7 @@ * OTHER DEALINGS IN THE SOFTWARE. */ #include "priv.h" +#include =20 int nvkm_gsp_intr_nonstall(struct nvkm_gsp *gsp, enum nvkm_subdev_type type, i= nt inst) @@ -47,6 +48,15 @@ nvkm_gsp_intr_stall(struct nvkm_gsp *gsp, enum nvkm_subd= ev_type type, int inst) return -ENOENT; } =20 +int +nvkm_gsp_gcx_ready(struct nvkm_gsp *gsp) +{ + if (!gsp->rm->api->gsp->gcx_ready) + return NV_DEVICE_GC6_READY | NV_DEVICE_GCOFF_READY; + + return gsp->rm->api->gsp->gcx_ready(gsp); +} + static int nvkm_gsp_fini(struct nvkm_subdev *subdev, enum nvkm_suspend_state suspend) { diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h b/drivers/gpu/d= rm/nouveau/nvkm/subdev/gsp/priv.h index 71b7203bef507..b07797813b049 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/priv.h @@ -93,6 +93,7 @@ void r535_gsp_dtor(struct nvkm_gsp *); int r535_gsp_oneinit(struct nvkm_gsp *); int r535_gsp_init(struct nvkm_gsp *); int r535_gsp_fini(struct nvkm_gsp *, enum nvkm_suspend_state suspend); +int r535_gsp_gcx_ready(struct nvkm_gsp *gsp); =20 int nvkm_gsp_new_(const struct nvkm_gsp_fwif *, struct nvkm_device *, enum= nvkm_subdev_type, int, struct nvkm_gsp **); diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c b/driver= s/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c index b45781cd0dfdc..89b801c1e6098 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c @@ -10,6 +10,7 @@ #include "nvrm/gsp.h" #include "nvrm/rpcfn.h" #include "nvrm/msgfn.h" +#include "nvif/cl0080.h" =20 #include #include @@ -215,6 +216,32 @@ r570_gsp_set_rmargs(struct nvkm_gsp *gsp, bool resume) args->bDmemStack =3D 1; } =20 +int +r570_gsp_gcx_ready(struct nvkm_gsp *gsp) +{ + NV2080_CTRL_INTERNAL_GCX_ENTRY_PREREQUISITE_PARAMS *ctrl; + int ret =3D 0; + + ctrl =3D nvkm_gsp_rm_ctrl_rd(&gsp->internal.device.subdevice, + NV2080_CTRL_CMD_INTERNAL_GCX_ENTRY_PREREQUISITE, + sizeof(*ctrl)); + if (IS_ERR(ctrl)) + return PTR_ERR(ctrl); + + if (ctrl->bIsGC6Satisfied) + ret |=3D NV_DEVICE_GC6_READY; + if (ctrl->bIsGCOFFSatisfied) + ret |=3D NV_DEVICE_GCOFF_READY; + + nvkm_debug(&gsp->subdev, + "GCX ready status: GC6=3D%s GCOFF=3D%s\n", + str_yes_no(ctrl->bIsGC6Satisfied), str_yes_no(ctrl->bIsGCOFFSatisfied= )); + + nvkm_gsp_rm_ctrl_done(&gsp->internal.device.subdevice, ctrl); + return ret; +} + + const struct nvkm_rm_api_gsp r570_gsp =3D { .set_rmargs =3D r570_gsp_set_rmargs, @@ -223,4 +250,5 @@ r570_gsp =3D { .xlat_mc_engine_idx =3D r570_gsp_xlat_mc_engine_idx, .drop_post_nocat_record =3D r570_gsp_drop_post_nocat_record, .sr_data_size =3D r570_gsp_sr_data_size, + .gcx_ready =3D r570_gsp_gcx_ready, }; diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/gsp.h b/d= rivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/gsp.h index c458569af9d72..2814629fddd22 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/gsp.h +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/gsp.h @@ -639,4 +639,11 @@ typedef struct GSP_FMC_BOOT_PARAMS } GSP_FMC_BOOT_PARAMS; =20 #define GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100 (14 << 20) // = Hopper+ + +#define NV2080_CTRL_CMD_INTERNAL_GCX_ENTRY_PREREQUISITE (0x2080a7d7) + +typedef struct NV2080_CTRL_INTERNAL_GCX_ENTRY_PREREQUISITE_PARAMS { + NvBool bIsGC6Satisfied; + NvBool bIsGCOFFSatisfied; +} NV2080_CTRL_INTERNAL_GCX_ENTRY_PREREQUISITE_PARAMS; #endif diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h b/drivers/gpu/= drm/nouveau/nvkm/subdev/gsp/rm/rm.h index e9ac47d86b69a..86970129ad96a 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h @@ -40,6 +40,7 @@ struct nvkm_rm_api { void (*drop_send_user_shared_data)(struct nvkm_gsp *); void (*drop_post_nocat_record)(struct nvkm_gsp *); u32 (*sr_data_size)(struct nvkm_gsp *); + int (*gcx_ready)(struct nvkm_gsp *gsp); } *gsp; =20 const struct nvkm_rm_api_rpc { @@ -174,6 +175,7 @@ int r535_gr_chan_new(struct nvkm_gr *, struct nvkm_chan= *, const struct nvkm_ocl int r535_gr_promote_ctx(struct r535_gr *, bool golden, struct nvkm_vmm *, struct nvkm_memory **pctxbuf_mem, struct nvkm_vma **pctxbuf_vma, struct nvkm_gsp_object *chan); +int r570_gsp_gcx_ready(struct nvkm_gsp *gsp); extern const struct nvkm_rm_api_engine r535_nvdec; extern const struct nvkm_rm_api_engine r535_nvenc; extern const struct nvkm_rm_api_engine r535_nvjpg; --=20 2.55.0 From nobody Fri Sep 25 01:20:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55A844071CA for ; Thu, 17 Sep 2026 19:09:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672170; cv=none; b=E4M+l/Fw1DExOpxxmfEyniaNKKl3k1VhOHCRBCxEjnTlv+EbYxd7+ugqIYUG8j2UObHWxOYTVttPGb5BP1k+6mTvpbKgRedOD52MIa2gK2YrV6OODbvaQYu4UIHLUkBqCulb4pmFDLd5wAL0+u0jd6eWYdarYch9Y4ggbUVh0v4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672170; c=relaxed/simple; bh=ajC0yqseWTsig6cspQ6QSlFq5KT5f55YPoWFqDQGo1o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qITLdHcMIFuW7Jvl8kPCjTbzZZWNA/W5mPe1xY27Px/fkzRmxv6GT9D9WxyiPC/EnmRJpZ9I+HJzfXCvauMsKTW/sSXmCIU/sNt2j7CeL1oEkyNFw4spBeXtlM/eeLNuC/76CWwgipEvA18tw7vIw5jLogj8NpkfOFlPEZHQhSE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=eVLg/8E3; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eVLg/8E3" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789672168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OxFbtdfQR1qpPytMLnVathlzGu5CVvUPYFWb0M6LybY=; b=eVLg/8E3BsxyX9mjWk4XlhCjFeZQjU8ENsVpGGd3SxtvkzDmr9HPwiRSK/i3rjrSADuYmh WUV+vnmRhDFHZMCPHB0O/cFlevuozU3/NzTmz4/MpoC6jw61woIx3PIHPHcSxZ0lCobHLs aqXiLEizuU2rllVnd64KnIhKzNU9xlA= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-479-fZ6HeGJ0OtGQv0ljUpcoVw-1; Thu, 17 Sep 2026 15:09:22 -0400 X-MC-Unique: fZ6HeGJ0OtGQv0ljUpcoVw-1 X-Mimecast-MFC-AGG-ID: fZ6HeGJ0OtGQv0ljUpcoVw_1789672160 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9F1CF1955DE1; Thu, 17 Sep 2026 19:09:20 +0000 (UTC) Received: from GoldenWind.lan (unknown [10.22.64.177]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 172CE19AC047; Thu, 17 Sep 2026 19:09:17 +0000 (UTC) From: Lyude Paul To: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: Dave Airlie , "Timur Tabi" , "Andy Shevchenko" , "Mohamed Ahmed" , "Maarten Lankhorst" , "Kees Cook" , "Simona Vetter" , "David Airlie" , "Thomas Zimmermann" , "Maxime Ripard" , "Mel Henning" , "Hongling Zeng" , "Danilo Krummrich" , "Lyude Paul" Subject: [PATCH v6 2/5] drm/nouveau/gsp/r535: Add support for MEMSYS_GET_STATIC_CONFIG Date: Thu, 17 Sep 2026 14:54:26 -0400 Message-ID: <20260917185916.1089621-3-lyude@redhat.com> In-Reply-To: <20260917185916.1089621-1-lyude@redhat.com> References: <20260917185916.1089621-1-lyude@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" This is a GSP structure describing various characteristics of the memory management system that GSP provides. Start by fetching it during driver load, but don't do anything with the information we get from it just yet. Signed-off-by: Lyude Paul Reviewed-by: Dave Airlie --- V3 * s/nvkm_gsp_rm_ctrl_get()/nvkm_gsp_rm_ctrl_rd()/ V4: * Remove mistakenly duplicated struct .../gpu/drm/nouveau/include/nvkm/subdev/gsp.h | 4 ++ .../drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c | 17 +++++++ .../nvkm/subdev/gsp/rm/r535/nvrm/gsp.h | 45 +++++++++++++++++++ .../drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c | 8 ++++ .../gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h | 1 + 5 files changed, 75 insertions(+) diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h b/drivers/gp= u/drm/nouveau/include/nvkm/subdev/gsp.h index cd10c37053695..ed5c6e0e68d3c 100644 --- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h +++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/gsp.h @@ -156,6 +156,10 @@ struct nvkm_gsp { struct sg_table fbsr; } sr; =20 + struct { + bool use_raw_mode_comptagline_alloc; + } memsys; + struct { struct nvkm_gsp_mem mem; =20 diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c b/driver= s/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c index 94925f1590ea4..63aa30f947471 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c @@ -1782,6 +1782,23 @@ r535_gsp_fini(struct nvkm_gsp *gsp, enum nvkm_suspen= d_state suspend) return 0; } =20 +int +r535_gsp_get_static_memsys_info(struct nvkm_gsp *gsp) +{ + NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS *ctrl; + + ctrl =3D nvkm_gsp_rm_ctrl_rd(&gsp->internal.device.subdevice, + NV2080_CTRL_CMD_INTERNAL_MEMSYS_GET_STATIC_CONFIG, + sizeof(*ctrl)); + if (IS_ERR(ctrl)) + return PTR_ERR(ctrl); + + gsp->memsys.use_raw_mode_comptagline_alloc =3D ctrl->bUseRawModeComptagli= neAllocation; + + nvkm_gsp_rm_ctrl_done(&gsp->internal.device.subdevice, ctrl); + return 0; +} + int r535_gsp_init(struct nvkm_gsp *gsp) { diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/nvrm/gsp.h b/d= rivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/nvrm/gsp.h index b6683a5bf870c..7b10b7548c575 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/nvrm/gsp.h +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/nvrm/gsp.h @@ -782,6 +782,51 @@ typedef struct NV2080_CTRL_INTERNAL_INTR_GET_KERNEL_TA= BLE_PARAMS { =20 #define GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE ((48 << 10) * 2048) // = Support 2048 channels =20 +typedef struct NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS { + /*! Determines if RM should use 1 to 1 Comptagline allocation policy */ + NvBool bOneToOneComptagLineAllocation; + + /*! Determines if RM should use 1 to 4 Comptagline allocation policy */ + NvBool bUseOneToFourComptagLineAllocation; + + /*! Determines if RM should use raw Comptagline allocation policy */ + NvBool bUseRawModeComptaglineAllocation; + + /*! Has COMPBIT_BACKING_SIZE been overridden to zero (i.e. disabled)? = */ + NvBool bDisableCompbitBacking; + + /*! Determine if we need to disable post L2 compression */ + NvBool bDisablePostL2Compression; + + /*! Is ECC DRAM feature supported? */ + NvBool bEnabledEccFBPA; + + NvBool bL2PreFill; + + /*! L2 cache size */ + NV_DECLARE_ALIGNED(NvU64 l2CacheSize, 8); + + /*! Indicate whether fpba is present or not */ + NvBool bFbpaPresent; + + /*! Size covered by one comptag */ + NvU32 comprPageSize; + + /*! log32(comprPageSize) */ + NvU32 comprPageShift; + + /*! RAM type */ + NvU32 ramType; + + /*! LTC count */ + NvU32 ltcCount; + + /*! LTS per LTC count */ + NvU32 ltsPerLtcCount; +} NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS; + +#define NV2080_CTRL_CMD_INTERNAL_MEMSYS_GET_STATIC_CONFIG (0x20800a= 1c) /* finn: Evaluated from "(FINN_NV20_SUBDEVICE_0_INTERNAL_INTERFACE_ID <= < 8) | NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS_MESSAGE_ID" */ + typedef union rpc_message_rpc_union_field_v03_00 { NvU32 spare; diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c b/driver= s/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c index 89b801c1e6098..ea38a94211f4b 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/gsp.c @@ -138,6 +138,14 @@ r570_gsp_get_static_info(struct nvkm_gsp *gsp) } } =20 + ret =3D r535_gsp_get_static_memsys_info(gsp); + if (ret) { + nvkm_error(&gsp->subdev, "Retrieving static memsys info failed\n"); + return ret; + } + nvkm_debug(&gsp->subdev, "memsys: Use raw mode for comptag allocations? %= s\n", + str_yes_no(gsp->memsys.use_raw_mode_comptagline_alloc)); + return 0; } =20 diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h b/drivers/gpu/= drm/nouveau/nvkm/subdev/gsp/rm/rm.h index 86970129ad96a..17480d4e527ac 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h @@ -176,6 +176,7 @@ int r535_gr_promote_ctx(struct r535_gr *, bool golden, = struct nvkm_vmm *, struct nvkm_memory **pctxbuf_mem, struct nvkm_vma **pctxbuf_vma, struct nvkm_gsp_object *chan); int r570_gsp_gcx_ready(struct nvkm_gsp *gsp); +int r535_gsp_get_static_memsys_info(struct nvkm_gsp *gsp); extern const struct nvkm_rm_api_engine r535_nvdec; extern const struct nvkm_rm_api_engine r535_nvenc; extern const struct nvkm_rm_api_engine r535_nvjpg; --=20 2.55.0 From nobody Fri Sep 25 01:20:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F26D4C8C68 for ; Thu, 17 Sep 2026 19:09:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672183; cv=none; b=Ilkpr/8XhujODITF45MvroGFt8BgW/MsdokiIKozeIWd+9ludYcG/TV+ngx0hmsVt+YlHmAEj/O9wjcK/G78Bx4jCEoQfLsNZflBK6IxpJz1IMxSeAppFkfHD1dPDLpkxav7AalBtn23RlpBZ8AKQcEWaJaA8pvoQPnGPzRyrCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672183; c=relaxed/simple; bh=ALxb9IdrEGsWfm98e5hYbr4TB9v46uqiCFIO73hJR/w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eyMBCTIdOljQ38K372a8W+QepX+UKtMM1Qxq0CDQZ68Yr9Xz2DyXa7a9ix8TD9BOv5heI7141erkY7pzvgSoSFoZbPKE3T33sUGo9FTjdlBlaZbWlLTe9hVX0kFzRDRgl/VJMhYO5bpqPf1kIWAFh1zKnzRWRg9JbVPj4D4K8tQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=S6DzhVQM; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="S6DzhVQM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789672181; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VRIqDmgykK4M700vzuRX8fM0npjQeUSDWwklvXV0FfY=; b=S6DzhVQMCPxrJie2oMgjUIj/upqotdotYm7iBKt9YR2IPt+8KDLupP1tGIBy1biR0eAihN NsgXVv6TQ8CKnvvvhZZ/uRleh3TwI9hlQ+6Nx70kzRrQEmv6Rb0HuXJsdqlqPT8d+8vEQD ZRd8jneB0sJG1CgoQXD3TYYpActIJFY= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-680-gr0xAXXDPtidSIf5jagLow-1; Thu, 17 Sep 2026 15:09:37 -0400 X-MC-Unique: gr0xAXXDPtidSIf5jagLow-1 X-Mimecast-MFC-AGG-ID: gr0xAXXDPtidSIf5jagLow_1789672175 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E69681964CE3; Thu, 17 Sep 2026 19:09:34 +0000 (UTC) Received: from GoldenWind.lan (unknown [10.22.64.177]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 55B3D19AC047; Thu, 17 Sep 2026 19:09:32 +0000 (UTC) From: Lyude Paul To: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: Dave Airlie , "Timur Tabi" , "Andy Shevchenko" , "Mohamed Ahmed" , "Maarten Lankhorst" , "Kees Cook" , "Simona Vetter" , "David Airlie" , "Thomas Zimmermann" , "Maxime Ripard" , "Mel Henning" , "Hongling Zeng" , "Danilo Krummrich" , "Lyude Paul" Subject: [PATCH v6 3/5] drm/nouveau/gsp/r570: Add comp mode workaround from issue #3172217 Date: Thu, 17 Sep 2026 14:54:27 -0400 Message-ID: <20260917185916.1089621-4-lyude@redhat.com> In-Reply-To: <20260917185916.1089621-1-lyude@redhat.com> References: <20260917185916.1089621-1-lyude@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" One of the things that OpenRM does right before initiating fbsr is apply a special workaround (nvidia issue #3172217) which temporarily disables raw compression mode on the GPU. It is later re-enabled after resuming with fbsr completes. Since we don't currently save the compbit backing with fbsr, this shouldn't currently make any functional difference in the suspend/resume process. But it will be required for implementing support for saving and restoring compbit backings from the GPU. Signed-off-by: Lyude Paul Reviewed-by: Dave Airlie --- V2: * Fix gsp object mismatch in r570_memsys_enable_raw_comp_mode() * Clarify debug message a bit in r570_memsys_enable_raw_comp_mode() V3: * Enable raw mode before rescheduling FIFO channels V5: * Send NV2080_CTRL_CMD_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE using gsp->internal.device.subdevice, not gsp->internal.device.object. .../nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c | 48 +++++++++++++++++++ .../nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h | 6 +++ 2 files changed, 54 insertions(+) diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c b/drive= rs/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c index af5aa5065c3dd..f73d9b29e891d 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c @@ -26,6 +26,35 @@ r570_fbsr_suspend_channels(struct nvkm_gsp *gsp, bool su= spend) return nvkm_gsp_rm_ctrl_wr(&gsp->internal.device.subdevice, ctrl); } =20 +static int +r570_memsys_enable_raw_comp_mode(struct nvkm_gsp *gsp, bool enable) +{ + NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS *ctrl; + int ret; + + ctrl =3D nvkm_gsp_rm_ctrl_get(&gsp->internal.device.subdevice, + NV2080_CTRL_CMD_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE, + sizeof(*ctrl)); + if (IS_ERR(ctrl)) + return PTR_ERR(ctrl); + + ctrl->bRawMode =3D enable; + + ret =3D nvkm_gsp_rm_ctrl_wr(&gsp->internal.device.subdevice, ctrl); + if (!ret) + nvkm_debug(&gsp->subdev, "memsys: Raw compression mode %s\n", + str_enabled_disabled(enable)); + + return ret; +} + +static bool +r570_need_raw_comp_war(struct nvkm_gsp *gsp, struct nvkm_device *device) +{ + return (device->card_type =3D=3D GA100 || device->card_type =3D=3D AD100)= && + gsp->memsys.use_raw_mode_comptagline_alloc; +} + static void r570_fbsr_resume(struct nvkm_gsp *gsp) { @@ -33,6 +62,7 @@ r570_fbsr_resume(struct nvkm_gsp *gsp) struct nvkm_instmem *imem =3D device->imem; struct nvkm_instobj *iobj; struct nvkm_vmm *vmm; + int ret; =20 /* Restore BAR2 page tables via BAR0 window, and re-enable BAR2. */ list_for_each_entry(iobj, &imem->boot, head) { @@ -54,6 +84,13 @@ r570_fbsr_resume(struct nvkm_gsp *gsp) vmm =3D nvkm_bar_bar1_vmm(device); vmm->func->flush(vmm, 0); =20 + /* Re-enable raw mode if it was previously disabled */ + if (r570_need_raw_comp_war(gsp, device)) { + ret =3D r570_memsys_enable_raw_comp_mode(gsp, true); + if (ret) + nvkm_error(&gsp->subdev, "Failed to re-enable raw comp mode\n"); + } + /* Resume channel scheduling. */ r570_fbsr_suspend_channels(device->gsp, false); =20 @@ -104,6 +141,17 @@ r570_fbsr_suspend(struct nvkm_gsp *gsp) /* Stop channel scheduling. */ r570_fbsr_suspend_channels(gsp, true); =20 + /* Temporarily disable raw mode to prevent FBSR restore operations from c= orrupting + * compressed surfaces. Required for ampere and ada. + * + * Nvidia bug #3172217 + */ + if (r570_need_raw_comp_war(gsp, device)) { + ret =3D r570_memsys_enable_raw_comp_mode(gsp, false); + if (ret) + return ret; + } + /* Save BAR2 allocations to system memory. */ list_for_each_entry(iobj, &imem->list, head) { if (iobj->preserve) { diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h b/= drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h index 8af432375f7a2..9050a8274b273 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h @@ -16,4 +16,10 @@ typedef struct NV2080_CTRL_INTERNAL_FBSR_INIT_PARAMS { NV_DECLARE_ALIGNED(NvU64 sysmemAddrOfSuspendResumeData, 8); } NV2080_CTRL_INTERNAL_FBSR_INIT_PARAMS; =20 +#define NV2080_CTRL_CMD_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE (0x20= 800a6f) /* finn: Evaluated from "(FINN_NV20_SUBDEVICE_0_INTERNAL_INTERFACE_= ID << 8) | NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS_= MESSAGE_ID" */ + +typedef struct NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PA= RAMS { + NvBool bRawMode; +} NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS; + #endif --=20 2.55.0 From nobody Fri Sep 25 01:20:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C36D4DA9A0 for ; Thu, 17 Sep 2026 19:09:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672199; cv=none; b=B1PiBZnhkploSsc3FIa+RWpIdpdZbG3VzVAQN6Okdcg3G9jbQ9Igd+dbvgRVVNZoM2lL4KimsTNFvmPbBhnjzbKlpmPT/4KV2KMixizAVcSNji+xpTT437jTk7VxWyG7NOWzSE4hH8mGZN7Mu9Bj1607nKAvFfqzgcofIUXNFiU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672199; c=relaxed/simple; bh=UHwBIlgGdXn7fbe3PU+LcElWhuv2EUNW+E4BuXEHjI4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CjbBwhpuml+82t2VetKgkfzdSBmX4NWYII3FImaSLd1P6hlV3avHVRk8/3BPj60lhIilnY16hqzarPmT+tGhoeyApliQYVs0IT3Te0+DHEePA4q+kogd5fr64v/GfUdd5DGbkaglhhmsjebIam1b3qM8AIHoCXvXKovx7wYm8wc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cbrJaHpF; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cbrJaHpF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789672195; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RHd/Dk5Z3xxUsB7VKRWoo8mcH7HGTcC9gaNXWpM+1jM=; b=cbrJaHpF/uSRsRncZmytH2GqC8XSTBIoidwzeKBASRG6XT3awvl66lCFrjYJmEaoaHR6la ZfBl9jOjsZf8AhBuNAxcNwnRLnckFzhSzRvpOtV23F7PIZDzFqVxIyZ1X55DDg8/q+Y1TH cnat5kj6U6hwM6gw/6SK4DtUj8icTuk= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-460-jXrflQxCNBGwIkgJlFvpow-1; Thu, 17 Sep 2026 15:09:51 -0400 X-MC-Unique: jXrflQxCNBGwIkgJlFvpow-1 X-Mimecast-MFC-AGG-ID: jXrflQxCNBGwIkgJlFvpow_1789672189 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 12F64183019B; Thu, 17 Sep 2026 19:09:49 +0000 (UTC) Received: from GoldenWind.lan (unknown [10.22.64.177]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 85A2519AC047; Thu, 17 Sep 2026 19:09:46 +0000 (UTC) From: Lyude Paul To: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: Dave Airlie , "Timur Tabi" , "Andy Shevchenko" , "Mohamed Ahmed" , "Maarten Lankhorst" , "Kees Cook" , "Simona Vetter" , "David Airlie" , "Thomas Zimmermann" , "Maxime Ripard" , "Mel Henning" , "Hongling Zeng" , "Danilo Krummrich" , "Lyude Paul" Subject: [PATCH v6 4/5] drm/nouveau/gsp/r570: Start saving comptag backing stores Date: Thu, 17 Sep 2026 14:54:28 -0400 Message-ID: <20260917185916.1089621-5-lyude@redhat.com> In-Reply-To: <20260917185916.1089621-1-lyude@redhat.com> References: <20260917185916.1089621-1-lyude@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" One of the portions of OpenRM's fbsr process that we never implemented is the saving and restoring of comptag backing stores. This isn't strictly necessary for fbsr to work (as long as we don't specify bEnteringGcOff =3D 1), but implementing it brings us much closer to matching OpenRM's fbsr process - which means we can rely on things being well tested on Nvidia's side. Now that we have the required driver workarounds in place and fetch the required information from GSP's memsys on driver load, let's implement support for this by fetching the required space for the compbit backing stores and adding it to the amount of memory that we allocate for fbsr. With this, we should be able to safely enable bEnteringGcOff in fbsr. Signed-off-by: Lyude Paul Reviewed-by: Dave Airlie --- V2: * Fix object mismatch in r570_fb_get_compbit_store_size() (rm_ctrl_done and rm_ctrl_rd() were using different objects by mistake) * Fix GSP object mismatchs * Don't return the size of the compbit store directly from r570_fb_get_compbit_store_size(), this makes it impossible for us to return errors since it's a u64. .../nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c | 27 ++++++++++++++++++- .../nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h | 23 ++++++++++++++++ 2 files changed, 49 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c b/drive= rs/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c index f73d9b29e891d..a93a39ef7c690 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c @@ -26,6 +26,23 @@ r570_fbsr_suspend_channels(struct nvkm_gsp *gsp, bool su= spend) return nvkm_gsp_rm_ctrl_wr(&gsp->internal.device.subdevice, ctrl); } =20 +static int +r570_fb_get_compbit_store_size(struct nvkm_gsp *gsp, u64 *size) +{ + NV0080_CTRL_FB_GET_COMPBIT_STORE_INFO_PARAMS *ctrl; + + ctrl =3D nvkm_gsp_rm_ctrl_rd(&gsp->internal.device.object, + NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO, + sizeof(*ctrl)); + if (IS_ERR(ctrl)) + return PTR_ERR(ctrl); + + *size =3D ctrl->Size; + + nvkm_gsp_rm_ctrl_done(&gsp->internal.device.object, ctrl); + return 0; +} + static int r570_memsys_enable_raw_comp_mode(struct nvkm_gsp *gsp, bool enable) { @@ -135,7 +152,7 @@ r570_fbsr_suspend(struct nvkm_gsp *gsp) struct nvkm_device *device =3D subdev->device; struct nvkm_instmem *imem =3D device->imem; struct nvkm_instobj *iobj; - u64 size; + u64 size, compbit_store_size; int ret; =20 /* Stop channel scheduling. */ @@ -152,6 +169,12 @@ r570_fbsr_suspend(struct nvkm_gsp *gsp) return ret; } =20 + ret =3D r570_fb_get_compbit_store_size(gsp, &compbit_store_size); + if (ret < 0) + return ret; + nvkm_debug(&gsp->subdev, "fbsr: Compbit backing store size: 0x%llx bytes\= n", + compbit_store_size); + /* Save BAR2 allocations to system memory. */ list_for_each_entry(iobj, &imem->list, head) { if (iobj->preserve) { @@ -174,6 +197,8 @@ r570_fbsr_suspend(struct nvkm_gsp *gsp) size =3D gsp->fb.heap.size; size +=3D gsp->fb.rsvd_size; size +=3D gsp->fb.bios.vga_workspace.size; + size +=3D compbit_store_size; + nvkm_debug(subdev, "fbsr: size: 0x%llx bytes\n", size); =20 ret =3D nvkm_gsp_sg(device, size, &gsp->sr.fbsr); diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h b/= drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h index 9050a8274b273..cb3e448415b42 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/fbsr.h @@ -16,6 +16,29 @@ typedef struct NV2080_CTRL_INTERNAL_FBSR_INIT_PARAMS { NV_DECLARE_ALIGNED(NvU64 sysmemAddrOfSuspendResumeData, 8); } NV2080_CTRL_INTERNAL_FBSR_INIT_PARAMS; =20 +#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO (0x801306) /* finn: Eval= uated from "(FINN_NV01_DEVICE_0_FB_INTERFACE_ID << 8) | NV0080_CTRL_FB_GET_= COMPBIT_STORE_INFO_PARAMS_MESSAGE_ID" */ + +typedef struct NV0080_CTRL_FB_GET_COMPBIT_STORE_INFO_PARAMS { + NV_DECLARE_ALIGNED(NvU64 Size, 8); + NV_DECLARE_ALIGNED(NvU64 Address, 8); + NvU32 AddressSpace; + NvU32 MaxCompbitLine; + NvU32 comptagsPerCacheLine; + NvU32 cacheLineSize; + NvU32 cacheLineSizePerSlice; + NvU32 cacheLineFetchAlignment; + NV_DECLARE_ALIGNED(NvU64 backingStoreBase, 8); + NvU32 gobsPerComptagPerSlice; + NvU32 backingStoreCbcBase; + NvU32 comptaglineAllocationPolicy; + NV_DECLARE_ALIGNED(NvU64 privRegionStartOffset, 8); + NvU32 cbcCoveragePerSlice; +} NV0080_CTRL_FB_GET_COMPBIT_STORE_INFO_PARAMS; + +#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO_ADDRESS_SPACE_UNKNOWN 0 = // ADDR_UNKNOWN +#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO_ADDRESS_SPACE_SYSMEM 1 = // ADDR_SYSMEM +#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO_ADDRESS_SPACE_FBMEM 2 = // ADDR_FBMEM + #define NV2080_CTRL_CMD_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE (0x20= 800a6f) /* finn: Evaluated from "(FINN_NV20_SUBDEVICE_0_INTERNAL_INTERFACE_= ID << 8) | NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS_= MESSAGE_ID" */ =20 typedef struct NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PA= RAMS { --=20 2.55.0 From nobody Fri Sep 25 01:20:28 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68D523DD85E for ; Thu, 17 Sep 2026 19:10:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672213; cv=none; b=qt003FAZWZOqOMxw3PcxIJ9CEfI9qgdz1VLj7wtweaB5WCv3uTt7qsgHz9JTcRLPqFVTq9JvPTbruX/4l3giVoZytXOUCiOjTj7kAY2v7zJc7ynqoq/HC5exAgz/K1MYB5Xv6/fVJzHLNiMzpZl/5rQCcLfhE2iMLeKklFKex/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789672213; c=relaxed/simple; bh=XBQni7Y/Y0NSihYgwyZDcK5nopDCnmlSDhhpeREHqXQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u5RcZSzYxIaZtUxO8V4zh6dG93prilmkn/1za1jbyWKYivpKidas9NAUxsEH+LBpkcPjeVw0Ttlds0/Fh7SloI5nDxRIyBfHPqBdkFHWZgrE6j5aRIIzA605Ft2QGbFvXK5moBJ4f/6Yax94nyru1PxHmQ81Ry82UPRMjfxkEAc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DI010hBd; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DI010hBd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789672211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4ZLSwLuQl37xTTXPl3smRd+44VFQmLk3YryksgVUFqw=; b=DI010hBdGWbPTvwnrKYYkAq5GiuZNWBgr2OBAUMbzkvnjxU+azR+SIRqG5yRbL0lP5KxWA LFoATZsL+aWPFFX89TpzsAMZBHhEyMV0XxCX9xVYC7JOGkQVyaZudjzl5hPaXeG47qkYAG eSqAZWbVbAoZp1KWP3bJdnGsVi4VhSw= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-661-L08vBYwrMvevo6MKfy0e6g-1; Thu, 17 Sep 2026 15:10:05 -0400 X-MC-Unique: L08vBYwrMvevo6MKfy0e6g-1 X-Mimecast-MFC-AGG-ID: L08vBYwrMvevo6MKfy0e6g_1789672203 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5188818301BA; Thu, 17 Sep 2026 19:10:03 +0000 (UTC) Received: from GoldenWind.lan (unknown [10.22.64.177]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id ACB8919AC047; Thu, 17 Sep 2026 19:10:00 +0000 (UTC) From: Lyude Paul To: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: Dave Airlie , "Timur Tabi" , "Andy Shevchenko" , "Mohamed Ahmed" , "Maarten Lankhorst" , "Kees Cook" , "Simona Vetter" , "David Airlie" , "Thomas Zimmermann" , "Maxime Ripard" , "Mel Henning" , "Hongling Zeng" , "Danilo Krummrich" , "Lyude Paul" Subject: [PATCH v6 5/5] drm/nouveau/gsp/r570: Enable Gcoff in fbsr again Date: Thu, 17 Sep 2026 14:54:29 -0400 Message-ID: <20260917185916.1089621-6-lyude@redhat.com> In-Reply-To: <20260917185916.1089621-1-lyude@redhat.com> References: <20260917185916.1089621-1-lyude@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" Now that we're properly saving the compbit backing stores on fbsr init, we can start setting bEnteringGcOff =3D 1 again without things breaking, which brings us closer to following the exact same code-paths OpenRM does for fbsr. Signed-off-by: Lyude Paul Reviewed-by: Dave Airlie --- drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c b/drive= rs/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c index a93a39ef7c690..469e7eed1d6f8 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/fbsr.c @@ -135,7 +135,7 @@ r570_fbsr_init(struct nvkm_gsp *gsp, struct sg_table *s= gt, u64 size) ctrl->hClient =3D gsp->internal.client.object.handle; ctrl->hSysMem =3D memlist.handle; ctrl->sysmemAddrOfSuspendResumeData =3D gsp->sr.meta.addr; - ctrl->bEnteringGcoffState =3D 0; + ctrl->bEnteringGcoffState =3D 1; =20 ret =3D nvkm_gsp_rm_ctrl_wr(&gsp->internal.device.subdevice, ctrl); if (ret) --=20 2.55.0