From nobody Wed Oct 8 09:24:00 2025 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45D0A23B60B; Mon, 30 Jun 2025 10:41:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751280100; cv=fail; b=TpT/mv9zgccHWLsUFEXu37TSpETF0AbQM5NzNKgZiIEwme4l3cNje1iJLTfEtar/AcgOmFwnfMZhJbex37Itc8eqfGMqzfE1YSuhagU22qtKeW8j610d6TL22i5FFQa10KNvFKeFE+/d0QU9PkDpjUjbG5zQlrN/9B/MQ1BuszY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751280100; c=relaxed/simple; bh=QMDHeyn1MyOFBNBKN0ZG778Cq9Lb6oc7F925UcUy2dk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GFUrf9NjFLeAOzrCzVhnwEeuNeD0vfn+Bb1J+iF+1Es93xwTZFiyPy10p6gb6gt3ZALkjxLF43ZN3VHT/+sLeUc1MX0b6rNqREmEK66GjFQqFMqXuew3OTcUt21x3KbB1RgeitjZCCPq+cVIibhU228sS0dw2HQL2DSHxDPBq8Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=aW3iDEif; arc=fail smtp.client-ip=40.107.93.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="aW3iDEif" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OEu7kVsKNGvB11OccxY/DOlpMCDhe7ffa6rk+kLpOWTZ3iwAKV1e8ViuTGLfyeCEgHdtrbr3Vv0Em0UzJCCpy0n9pyTnPksT3hqBP2OSEcIHtzzWwhhmNFGsDp6uYKMUGdc4H7K9JNSLSqr9xLcBRrHBhlmxw9yOLIVRg4BkIDXj/0LtGH9tG1Gpa23UJ+6CWnWSNBvxhKwePyqjKIgMdNHxZnGbQq0wDA7OUTxyA3rBckYIieCqO6jQaq0J0pEs9FzsI88uQ/xJ+m9JcfRAb07hmKnr4Zh/scJK1OLGvC0HWhUrJB7tx5HJPqxfyEjnx2Ou7f/1bnjXyX3z6efGDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KCseoRpCc1/28UrqFzQEy0Xjo4AGVgrSDtiCzhWtbe4=; b=nXUcHEEt9INi+NrauSsAfawjaAYB/LniIIdXUZMlXvgMxTEue+tUrGyKZq6BFsznNsaMfVq45/OgDWvM0L9v9H+Xppn6kS8VZbfAHj3aqrNylKMt1HEF4sn2Vs0xzFE3P/k4uSYMky/eA/pcWRUccMtb6RA/CkOSV3tIYVhna9rcABNO3xj6ot2EUAe/MGq8DkgaQ5/DgMul10WXUu/Nl95GyyvE9KeEMkz6qAPey3J5tSZnIS0k7u2PqXQGVcI26e47qqf/WRd52YnKAgUUOJgqkUZBBRAyO+7mRj4dVoiXLRiTkDOqd7/qZMgnNlYGBB4bR64Bv94U2w6w1fLXLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KCseoRpCc1/28UrqFzQEy0Xjo4AGVgrSDtiCzhWtbe4=; b=aW3iDEifdZ8Stbt5K7SwIzzlO8XVWWxXghJjKlYIkyJFpd1kufsOLnhsNPstXWdqddSTCwYnR1P7wa2pVYHWdh+dxC2CJ0zAnr9HyZVnYTUMbsIGyfBfm1hTX4cfmkwjiOQgT592DFdP8ygsG7GztuC3XJh2FHh9rykmyNE60MI= Received: from SJ0PR03CA0242.namprd03.prod.outlook.com (2603:10b6:a03:3a0::7) by DS0PR12MB8454.namprd12.prod.outlook.com (2603:10b6:8:15e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.29; Mon, 30 Jun 2025 10:41:36 +0000 Received: from CO1PEPF000042AD.namprd03.prod.outlook.com (2603:10b6:a03:3a0:cafe::91) by SJ0PR03CA0242.outlook.office365.com (2603:10b6:a03:3a0::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8880.30 via Frontend Transport; Mon, 30 Jun 2025 10:41:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1PEPF000042AD.mail.protection.outlook.com (10.167.243.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8901.15 via Frontend Transport; Mon, 30 Jun 2025 10:41:36 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 30 Jun 2025 05:41:30 -0500 Received: from hjbog-srdc-41.amd.com (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Mon, 30 Jun 2025 05:41:26 -0500 From: Samuel Zhang To: , , , , , , CC: , , , , , , , Samuel Zhang Subject: [PATCH 1/3] drm/amdgpu: move GTT to SHM after eviction for hibernation Date: Mon, 30 Jun 2025 18:41:14 +0800 Message-ID: <20250630104116.3050306-2-guoqing.zhang@amd.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250630104116.3050306-1-guoqing.zhang@amd.com> References: <20250630104116.3050306-1-guoqing.zhang@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: None (SATLEXMB03.amd.com: guoqing.zhang@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AD:EE_|DS0PR12MB8454:EE_ X-MS-Office365-Filtering-Correlation-Id: 515221fd-15b5-45e6-8fd2-08ddb7c2b010 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?cIoIgczDmW8cFTRZ6IGwPAAnYipGjI1yy77c6j75L71tIxoWZ3W2Q83topnk?= =?us-ascii?Q?BMDwfUjIyFb6xE503C91Hy5bdluz37HF4M0f7Q9CVbL42ebhOCfid1mhUBn9?= =?us-ascii?Q?Mz5PBkup1o5Uq6TQ8s1d1MhX6viChLfSI9PKf49LYFxijeGzrA0XG+waLHCp?= =?us-ascii?Q?XBT0gSujjPiSvIvk1VbUFtfMv3xzEfM/3CGIoS7RANPl17510ZmrLNN56Zuo?= =?us-ascii?Q?wncjNBLc1YukqdVuURNOkFyOz0/gSkojaXyII1OAwnlPrwzAGhZA6e4MqVbp?= =?us-ascii?Q?TtHeEa5jgD+zh7rQCwiEGhJezXO1wm9dOpkpNksqnjDIINCqIreJ9MUuJwki?= =?us-ascii?Q?5xWFWCrm6V/oYctpiiKlghXXzYeFev3jpQnnmw+0G6INY35hlWK8FhIcWXPP?= =?us-ascii?Q?k0Mfh26C/V5oepw0kI2sqbhxBliUMabUTAyBupupVbPGg5ugXrHLV/TCJdH+?= =?us-ascii?Q?L8oOhL8O85GjaCi72m+boyjjui+fSefA5Mkj68k2ofITB+Hp2VKoEkQrWEDb?= =?us-ascii?Q?iMHC61f3JvXwSxbQSJL4xHeKcx3Rn6nCZHPhZDGl1MY/9kGvDKWraiDYzgtU?= =?us-ascii?Q?iyd4hwwG8UvIezw1a0ZIq1Z5xIYARV3W/0S3kWR617Y05VWl2J7cwVy9c5I0?= =?us-ascii?Q?yONP1vzg+mgLRwSgkSNgG1UrtcsLW8JrXuqW0WBXYyIjvkxUxXq89ShUoOwK?= =?us-ascii?Q?AxzkT6LNC7pbzP4jPCtl8INKlwgXM2asW7N+LQOUE3gJIW+mOgdMOXR+TQjm?= =?us-ascii?Q?JdN14pRr1/G1rZ+1h+7NrTBJLJgsU1iExyN3R7ZWRZNhJkLfTJV3b7zAQMo5?= =?us-ascii?Q?Zu5iqMKgvxlct6y/dbSO8Khm6a3vVsFo3doQT1Xf5zxQ0leAwqlblsO280Qh?= =?us-ascii?Q?qcIbCwzNn1LJYW6cWkiSvt1mytQwG1sYHjLNtUFsgj7q3huSSNTiXFO0afP6?= =?us-ascii?Q?TnYrWb/hqGTUsNIfw4OI2Ak3oR+krua+GDzJlHngO/g0pQLFlHma2SwKFajr?= =?us-ascii?Q?klw0PwFFzyG6yOMueiflvXZmC0KRdHXej3PYKgn0fWE5qI+LGkWJfEHJaWdv?= =?us-ascii?Q?U5yP7sxqx7Wbh6PD/C3WNoKWpZN6K4wGxuQcRZFo1t+6fKS/E6Jt0vPT8BVB?= =?us-ascii?Q?ZrSqqRu0EvQctk2p1Vq0hHBTa8Rf78XWUcF1+BSvmCCzWCoj3Lk3TY63RNan?= =?us-ascii?Q?ughKnNnfkTJzLhsk4SoTN4IrBcKPg6pd7w73gui0dx9hUuzbeiSOfoejQrBI?= =?us-ascii?Q?ilwj/VHTgpdcVK7Y0f2rSGeVxXuTRlWSto+UkJycMWKs8XkHjGH0FNNpnryF?= =?us-ascii?Q?pHwdbbfLrYF9O1eKRKIDT6zM74e5Kc9IyMEBxqHxJMMxoMmqsI90ETLOcrH6?= =?us-ascii?Q?OLYk9SW/KfX3X5esbAi0dpAkw6oCFoJ3TM/+dYkn/Q0W6EZrXwUcmRbqazhm?= =?us-ascii?Q?EwSJzppOeoVYLYG0VZukPfC3RCKF7Ei12AhxskMYgmOPEiy3yrBBCHO6vd2c?= =?us-ascii?Q?rtQ+c7rndgpe0b1PRBqIrFAw61UHcEZZ4/Hv?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(1800799024)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2025 10:41:36.0516 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 515221fd-15b5-45e6-8fd2-08ddb7c2b010 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8454 Content-Type: text/plain; charset="utf-8" When hibernate with data center dGPUs, huge number of VRAM BOs evicted to GTT and takes too much system memory. This will cause hibernation fail due to insufficient memory for creating the hibernation image. Move GTT BOs to shmem in KMD, then shmem to swap disk in kernel hibernation code to make room for hibernation image. Signed-off-by: Samuel Zhang --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 13 ++++++++++++- drivers/gpu/drm/ttm/ttm_resource.c | 18 ++++++++++++++++++ include/drm/ttm/ttm_resource.h | 1 + 3 files changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/= amdgpu/amdgpu_ttm.c index 4d57269c9ca8..5aede907a591 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -2889,6 +2889,7 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo, int amdgpu_ttm_evict_resources(struct amdgpu_device *adev, int mem_type) { struct ttm_resource_manager *man; + int r; =20 switch (mem_type) { case TTM_PL_VRAM: @@ -2903,7 +2904,17 @@ int amdgpu_ttm_evict_resources(struct amdgpu_device = *adev, int mem_type) return -EINVAL; } =20 - return ttm_resource_manager_evict_all(&adev->mman.bdev, man); + r =3D ttm_resource_manager_evict_all(&adev->mman.bdev, man); + if (r) { + DRM_ERROR("Failed to evict memory type %d\n", mem_type); + return r; + } + if (adev->in_s4 && mem_type =3D=3D TTM_PL_VRAM) { + r =3D ttm_resource_manager_swapout(); + if (r) + DRM_ERROR("Failed to swap out, %d\n", r); + } + return r; } =20 #if defined(CONFIG_DEBUG_FS) diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_r= esource.c index fd41b56e2c66..07b1f5a5afc2 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -534,6 +534,24 @@ void ttm_resource_manager_init(struct ttm_resource_man= ager *man, } EXPORT_SYMBOL(ttm_resource_manager_init); =20 +int ttm_resource_manager_swapout(void) +{ + struct ttm_operation_ctx ctx =3D { + .interruptible =3D false, + .no_wait_gpu =3D false, + .force_alloc =3D true + }; + int ret; + + while (true) { + ret =3D ttm_global_swapout(&ctx, GFP_KERNEL); + if (ret <=3D 0) + break; + } + return ret; +} +EXPORT_SYMBOL(ttm_resource_manager_swapout); + /* * ttm_resource_manager_evict_all * diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index b873be9597e2..46181758068e 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -463,6 +463,7 @@ void ttm_resource_manager_init(struct ttm_resource_mana= ger *man, =20 int ttm_resource_manager_evict_all(struct ttm_device *bdev, struct ttm_resource_manager *man); +int ttm_resource_manager_swapout(void); =20 uint64_t ttm_resource_manager_usage(struct ttm_resource_manager *man); void ttm_resource_manager_debug(struct ttm_resource_manager *man, --=20 2.43.5 From nobody Wed Oct 8 09:24:00 2025 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2080.outbound.protection.outlook.com [40.107.92.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E5BB259CAE; Mon, 30 Jun 2025 10:41:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751280104; cv=fail; b=jsCZ8IbkRHgRTByjQ6sIWMawq9VSCHO5RTaB3Pvv4wSkG3+Vhs45egokcBdD5scLTO7lEwQxwg9TBpGD+HR6P0qRWvcr9r8jfd7ApdO2ycHnSjzJup57QqvpBPYAMGWqPfTkepyazuHcSz5/JzPG+bANvb/00UkWKMrWsJBnJCM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751280104; c=relaxed/simple; bh=qo7I3ndgrr/FKHPIEo+QwhVI9j+Qu7m08ZNUvI7eHco=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=X+gDZrTyBpfkUFV8ipVLZQGqDD4hr2cP/3Q3JkaYSVkaoLYqnXqoK79RZASZdVZGOKa7DvKqCxbDijUk5SA4AfEWE62s+MY6Iqd+jDs4OK2epK9HNCJoZzf/BPl8hLgVlaybq3eO16dzVF6/uPmcBLQatmN6cpThMvXKk0io3Mo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=VfEZ5c2u; arc=fail smtp.client-ip=40.107.92.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="VfEZ5c2u" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QVuFRBppVchSfZSeK3geucfn5LjJLtAHYAlWW+1l/5ed4gDgv39ntjG0fnkVN2XHNnv3JFsOXt0K99vDgm/FduWsOv0QSaXhfHA0XSJbWzaTs4fz95lE8BU5ZHck1obouSFN/qOcn/BSd7DWnMAQccVPUEt2GQlj53OqVtihNiC8ZUronhWDN80XRSfUIX8IOW8S19pExy03GyuBuznAXx4O6ppDjrBGk3zi5ac73LJcoZU1die00N0SuJPn4fpLGus1udJx/3yLCtqT1Fye94YMy/jWc9dWx9uWGixnS7NbZFEED5JwQg3AeMLD/viodcX8nuKsLftjFz9250VqRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IjitZ801ujEKLUDcwxCjZHouyz0X/iHMWMPwrlHhPzI=; b=fPcBZTYN4asBXVzw/UBixzW7UA8L/cpHcaOAPeun0OpjPE3sN3Qer2+ZyKfiHpoHa3tG+DGlRR9vvRxn8WIPn3OWTQFXNPVqnzuF+jmPNoLP88GWt0ATLDHQtygfxporHLYHAOaUvm+JiI9lm7UxouwQ6FH9+YsN5DVm6hDhAEsVtrrQoMRlSyMiKF2Q0f0Q1b9RNMw7E+ReLKPgn4U8yf0DP2o2sRnHlecRK8g4+bl6Ny3PuhHhlrvskCwCb5I1J785unSZ2FF2PKbQGJ45Ji6LD+/vIGP69CMQzlZgJaHJmfPlMRw87/hGDqE6X2RdCrEy7w40/lQtHAvKqfq+YA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IjitZ801ujEKLUDcwxCjZHouyz0X/iHMWMPwrlHhPzI=; b=VfEZ5c2uWju+XoFueQ95KSzXjAwGN6jo+OeHFSZbt5zyMwUeOfoVhSoeF89Voi4HNlTfDWDlkn0VXaBi7iI8zNijyvpPKO7FywZIINaw1/VEI1Sm560FU0WtJcoj/QfYXSKvQ1F2Cj6t9E7abvx0t6VS8uoO/FJ6Y2VpHrXT0h8= Received: from SJ0PR03CA0269.namprd03.prod.outlook.com (2603:10b6:a03:3a0::34) by SA0PR12MB4461.namprd12.prod.outlook.com (2603:10b6:806:9c::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.27; Mon, 30 Jun 2025 10:41:39 +0000 Received: from CO1PEPF000042AD.namprd03.prod.outlook.com (2603:10b6:a03:3a0:cafe::e2) by SJ0PR03CA0269.outlook.office365.com (2603:10b6:a03:3a0::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8880.29 via Frontend Transport; Mon, 30 Jun 2025 10:41:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1PEPF000042AD.mail.protection.outlook.com (10.167.243.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8901.15 via Frontend Transport; Mon, 30 Jun 2025 10:41:38 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 30 Jun 2025 05:41:33 -0500 Received: from hjbog-srdc-41.amd.com (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Mon, 30 Jun 2025 05:41:30 -0500 From: Samuel Zhang To: , , , , , , CC: , , , , , , , Samuel Zhang Subject: [PATCH 2/3] PM: hibernate: shrink shmem pages after dev_pm_ops.prepare() Date: Mon, 30 Jun 2025 18:41:15 +0800 Message-ID: <20250630104116.3050306-3-guoqing.zhang@amd.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250630104116.3050306-1-guoqing.zhang@amd.com> References: <20250630104116.3050306-1-guoqing.zhang@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: None (SATLEXMB03.amd.com: guoqing.zhang@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AD:EE_|SA0PR12MB4461:EE_ X-MS-Office365-Filtering-Correlation-Id: 4c7cdf9d-e649-4017-171a-08ddb7c2b1b1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Cl40Nlh/S52VhsOgrbyUrmGDp1kSL4U50NvogG45yprhcuXzUyaOnZrm3XW1?= =?us-ascii?Q?cKv1hfH7VVYvUxGfYiwKxfclAwL3XC2qVW0MWv5A65R0ZCjUqOgfCGrAcMos?= =?us-ascii?Q?tEKJgqN0cbUe+8LGRnzlwJS/LPoD83j2JNGEW+3t3wv2pW1JrQgMEUon/Qq+?= =?us-ascii?Q?2lq+xhEShYIOSuS5jr13BlKfmBLM3DywlLtjSISGBl+EqiOny/EMXAh9iESG?= =?us-ascii?Q?YvjyP+KaH9JT0NZ71LIXnSeJTJt27DNMmH2KEpjqjeprB9sRvBDgjL0mGyl2?= =?us-ascii?Q?L1dMbeKDAksn84aK9fY6MezSt9dV4QgeIoNWGYFrZRSXvbjjQza0/KqBZmI5?= =?us-ascii?Q?3VWd2pKCPYQZg8wBX9LNK4RrAslmr9Nrdwif1dsOAGw7LyYymFwr2v6pcLb5?= =?us-ascii?Q?fiYOdzuX4aYGoIaWhXfNLY5thTGvgd6QggC1+bD8z3hXEBbNgAxVKKAGmzl9?= =?us-ascii?Q?6MbBnbF49shJfds2BMWY3mW0PxXZ0iViStURFPq0/d4QFxrkXdScZGvclSJ1?= =?us-ascii?Q?0Te0/uDjK8t7aB47N3T9kOEpiGhScnOFM6zJnM6HxCnC3QfrIQshlonWjlny?= =?us-ascii?Q?nRfG1jyHbi8+OZmmuBzoRcK1biPJQZin2qI8i4OcU8m/e6sNsixFekMpP9WH?= =?us-ascii?Q?PGEcW0rkCJ/qifIpDzgaZGFf6gXkJjFcyzhYVRuwvJ0J0NtcO33Sg5cuYYDM?= =?us-ascii?Q?XAat7iC22N4i+E+4Zx/5DutaUSmooDgHG28jiICJI6Qs9FA0g5vLvPr0x519?= =?us-ascii?Q?Mpv1I24nzEKMXZ6y6UUi482EvYy7qQ/qsxsDlpRc/0Tp+i2RoTpS8Dl7y0Cv?= =?us-ascii?Q?zGBaJQQcOHNfZQyVQ6HPMWY84m8pcyChCRJTcsWQ+i0De83EF5n7mHnxbYbo?= =?us-ascii?Q?KDRyYyMgfQYekxZsTU3yHOJ3pi9lbdTKAlWZlG4QzJMPnjAKQfIFjAWk2aoB?= =?us-ascii?Q?nKOjWKh0WbeDvvp76RGAiPQHFrAA9fsWxRxTNLrVuflF4926sH0nO14K+BbO?= =?us-ascii?Q?LoT6SfYgClNheQ0Btxokaqpg6Pp9gQ7TFmVg8REf+fY+lRRL8WUx5YR9tNGG?= =?us-ascii?Q?IzU/9E/++BO1tT0uiJ8oq74Lg/T1V6bqzMCGR6b0JPaU6/6xZmkYJyUswCA6?= =?us-ascii?Q?zAGg0NAxu4u+HRTwudKBI5Fk4UIHKWZ/g6UMCy2wYXlxo9r8FKcaObnWEVUV?= =?us-ascii?Q?RKIS9ZEPBQKqQ/N5CKAqYBF1cY9iBqgCKm3hoKj9ZEoRSd97C14liku02YNn?= =?us-ascii?Q?o6le0Eb5Tzno/rQwDp38xv/F2oX8nVVqb6klnQRevhzMTJ6RIfjSZxHoNWUo?= =?us-ascii?Q?XevoiEJNPufCQP07ckO1yvBLE6pRd4bv04qd0p7kKzd2oiKN8+p1lKaRZf0/?= =?us-ascii?Q?ddrP/eaBLtsMpcqYEtl5AJH85+i0C3lnV+54azH08QleigmKxtXooi/wTA/h?= =?us-ascii?Q?OCsUornVBO8w87wk3iFcQEpW7zRuJpJQvB/7Xr7s32ixsU2JG9r8WJgMzSMT?= =?us-ascii?Q?Yq7DuqvrhMCaK+f/6JcDVelFkcmIGR5g5drf?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2025 10:41:38.7858 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4c7cdf9d-e649-4017-171a-08ddb7c2b1b1 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4461 Content-Type: text/plain; charset="utf-8" When hibernate with data center dGPUs, huge number of VRAM data will be moved to shmem during dev_pm_ops.prepare(). These shmem pages take a lot of system memory so that there's no enough free memory for creating the hibernation image. This will cause hibernation fail and abort. After dev_pm_ops.prepare(), call shrink_all_memory() to force move shmem pages to swap disk and reclaim the pages, so that there's enough system memory for hibernation image and less pages needed to copy to the image. This patch can only flush and free about half shmem pages. It will be better to flush and free more pages, even all of shmem pages, so that there're less pages to be copied to the hibernation image and the overall hibernation time can be reduced. Signed-off-by: Samuel Zhang --- kernel/power/hibernate.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c index 10a01af63a80..913a298c1d01 100644 --- a/kernel/power/hibernate.c +++ b/kernel/power/hibernate.c @@ -370,6 +370,17 @@ static int create_image(int platform_mode) return error; } =20 +static void shrink_shmem_memory(void) +{ + struct sysinfo info; + unsigned long pages, freed; + + si_meminfo(&info); + pages =3D info.sharedram; + freed =3D shrink_all_memory(pages); + pr_debug("requested to reclaim %lu pages, freed %lu pages\n", pages, free= d); +} + /** * hibernation_snapshot - Quiesce devices and create a hibernation image. * @platform_mode: If set, use platform driver to prepare for the transiti= on. @@ -411,6 +422,8 @@ int hibernation_snapshot(int platform_mode) goto Thaw; } =20 + shrink_shmem_memory(); + suspend_console(); pm_restrict_gfp_mask(); =20 --=20 2.43.5 From nobody Wed Oct 8 09:24:00 2025 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2067.outbound.protection.outlook.com [40.107.100.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED105257427; Mon, 30 Jun 2025 10:41:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751280110; cv=fail; b=rQBdezxD9xph1EP6RDEjxf0pvxskgNrRaYUI7vu4i4lsP40b9Ma/q2c75ka3TVkBr3XNl41yZPeQO7u7NXR77UMYtIbkV/rqAw9xbvgjlZ9MKbvX6bfmxDnOKq/k8xjMPSX/zlNllgS2DmJxy3djmcS4B5VX7B25Yeh3eWXVVhc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751280110; c=relaxed/simple; bh=NjDg5yhjRlq+5r+MDaAhAint4+Zf+EsIeHD/HCkGdsI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DGpQ5z+WaRW2MNmZ3l7XWSVui3GU/voBpDUgJ/phpRo9YOSgYZUBmo/kJnlDsANbjF9i9iGFZUTg7e1G+lqZ8e7iZ/gZMvkcznyFg+zMuMTAQtqaiEVqHY+Z84Kgxk4Q3AAxKWwAwAXw0agzSQm5RF3ML8yMjGb0pgsZwaEZJVc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=1AMjU8Is; arc=fail smtp.client-ip=40.107.100.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="1AMjU8Is" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BobdrmNHzpqrFHeuditosSFG3q3sumCCdBmGPvQX6Jbe5nHGb3rDzGTBjSDSn04LeY2O7j+BIbW0dIXLEXpWQLN4uhoLXH/apdZtSDy1NCeT7KPPXnBpMvrq6bMHY4VLUJ1j9en+oMO7w13VzU6HxuYtPiJf4bonhPOp9gvI5tE++/zvHUHJNh3MUcr77zEUSW8lI/1UwE7XQ+/LgKLBWkE8bbOp97qrKeVgzhlkfxF5Mf2Y32bHW0XTmxK4jC+CGJSxfxOyOvrdFvsj+JZeMBc3kKg1s+n94uOP9Rvw6KZdn30l8zsag+wyv5URWsL0C+TGtc/7K3piG6eV6B/Q4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z4L7Ww+hhD42gc5GQN0DToyDYofS1ZmQRzIw0dKVfFs=; b=gxeXYmQgA5TRkcW337vl5d5benkhZVn4d/d3Tz1+hg6vVqz8wpJYQf8j8+ARpFcrqQLSRL4kv+Dzz9YZKzDTcgAqIZr6YKw+TQwf/ztU7ZyHzPMHHIDrpe4GZpMVClyhsN7Dn5WqusrSe3RjWkbB3vEC5q81t8Pt3gSZEey3Z45/YlwudzK1CC7qdlJzB9KIFwxYxxEhlinwOHw2ekbD9S/2uk9eh+JmA8Bih3XZfWOhWCntxneterkAbTna835eN81zwlMsrSFcO7PCL+1RbLCsCtNVWwOlD4YwNp7utuvrIfD/hq3BsJfLkNsfXAfWcwc8iJkIM+wYeb1am9XkAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z4L7Ww+hhD42gc5GQN0DToyDYofS1ZmQRzIw0dKVfFs=; b=1AMjU8Isl/gg5wj3SZVvVuOZhrAr7sGH0koOCkro0j9loMc+JtTtHDD0M6z/6rt0TwQ+KZUSBGGRkaZYD+GLFwkN5BA/T6AvyiKE1nmR9GirO2YJEo6IPP7N8vDBYJ1tJPCmlCKwadyFGjnRb57B3EecG78+qWeobH4wi5xK3xs= Received: from SJ0PR03CA0270.namprd03.prod.outlook.com (2603:10b6:a03:3a0::35) by SJ5PPF000ACABD1.namprd12.prod.outlook.com (2603:10b6:a0f:fc02::984) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.29; Mon, 30 Jun 2025 10:41:44 +0000 Received: from CO1PEPF000042AD.namprd03.prod.outlook.com (2603:10b6:a03:3a0:cafe::e0) by SJ0PR03CA0270.outlook.office365.com (2603:10b6:a03:3a0::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8880.29 via Frontend Transport; Mon, 30 Jun 2025 10:41:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1PEPF000042AD.mail.protection.outlook.com (10.167.243.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8901.15 via Frontend Transport; Mon, 30 Jun 2025 10:41:44 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 30 Jun 2025 05:41:37 -0500 Received: from hjbog-srdc-41.amd.com (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Mon, 30 Jun 2025 05:41:34 -0500 From: Samuel Zhang To: , , , , , , CC: , , , , , , , Samuel Zhang Subject: [PATCH 3/3] drm/amdgpu: skip kfd resume_process for dev_pm_ops.thaw() Date: Mon, 30 Jun 2025 18:41:16 +0800 Message-ID: <20250630104116.3050306-4-guoqing.zhang@amd.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250630104116.3050306-1-guoqing.zhang@amd.com> References: <20250630104116.3050306-1-guoqing.zhang@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: None (SATLEXMB03.amd.com: guoqing.zhang@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AD:EE_|SJ5PPF000ACABD1:EE_ X-MS-Office365-Filtering-Correlation-Id: e379d11f-4fb9-49aa-d7f3-08ddb7c2b4d9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?HY6ZSg9aOegnZWnSOd/13F/BBalC7+jgn+CZeIVwmWXYHgMOaxYd4E6Lug6S?= =?us-ascii?Q?SVQhi0IRrWKI01PSMUz1Wp874LvMITNjjp+X/BgOl4BZLjSUkCd7384aj22w?= =?us-ascii?Q?yN7o1168iy5bomZdF4Rh4ASAGeiUcmuMvk1h+jGXkBddjTtRfBN0qNSA15T0?= =?us-ascii?Q?0eBWEo+WbmUr4JW/bx2oOCGH2ZsvQ8wr6/7FGZrnwIjFdWGnj9Je3Sdcofy3?= =?us-ascii?Q?I8CJ58Ct0khRMbq6e2HMBNBOS2teKOB1CfyjlSZoJN7dzTpb4d8HeovV/9gi?= =?us-ascii?Q?3e9o5Ev3eF/TRyMCfW2F1fZrv00c+xL8MXSfYl0jF2+AxnYl6TqLFeB+oLTV?= =?us-ascii?Q?8TW3waArgP9SC4wtWncBLLzgJb9BNzkOYV22x2onHnltZT09RqGV2Tx4i3Nm?= =?us-ascii?Q?Ri43TxM/G/jUKQKMIHlk6ywKx1+dfsuJdELj1VYg+y8JnFATCL/qnO2bXXHa?= =?us-ascii?Q?lNcXZhI9JyycuawzicDb6g2iZgOhsAqEeFqrh2Q0n5JT5K0S++/Rpoz6rLRy?= =?us-ascii?Q?eFxfayXbbUt5IPCN4bzAXh0FEd7MiRPnVYIjUpLCG7orJZZCiLElmC5zGih8?= =?us-ascii?Q?VvB7cKon2kVHflf9nKSkZXm1qNmW/3rsC/4hcUpb0JvM3ix5ncySlP0IhsqJ?= =?us-ascii?Q?pzy0bRtqheee0wmBWWuGhiKKETG2raUrNvJuXZ3uI2eCmSLXPHyJD3swK9bP?= =?us-ascii?Q?QDR++BBLkuH/g57sEOcdcTdH+IxgtqM7Rcvd3lWgH0hQ+JOkWvJyCT/ajfiW?= =?us-ascii?Q?y5yCzN/goY7iKJeYdYGc6AvctRrbjexEEUjHAb3OTFYTrECCtPelejVYevGz?= =?us-ascii?Q?H8uIkQ9Qbu38yaw8iKbk68KHsWgYKu6OGWfstjEHdradzxS9sjH8J59XwLUJ?= =?us-ascii?Q?kaW/P2oKhIcGRM+iASYFDuMJKFoNOk4E/PJlsao/3isxLRpWKa+97i0amjPW?= =?us-ascii?Q?jjqbYTVoxolAZHOemaLS1ab3MmQyjeCt1/o+SJpFVVj2vz0avxwz1jqqXdHF?= =?us-ascii?Q?uusdRUPP5+wggLCMDlRhzmd0JvbmH8vYG/UjiM8bCjhV3xkJvpEJYCDqHwzg?= =?us-ascii?Q?2EJoFLsqR1ebtT9ugVVDeSMJH02n+fn+MGDDG4+PulqcQECPf0SWX3wDtbwi?= =?us-ascii?Q?mdYnPV1/qGo8/LJtnFdCc4IsRGlR5oBM4dCD9gzaWkl0lqPAHsL+AE47Viyb?= =?us-ascii?Q?wj0qeGj6kXzj0GWIRgag3rcvXcpOJk/Qf5GEiMx8HyRaFys/dUakC7qUHSfY?= =?us-ascii?Q?Y9XyJc8oQn2Q6h8y0jhTndM1M8+TRHUqkPL+l7H2H9vTU+9lhAbXoxx+Bhf0?= =?us-ascii?Q?IKsfiHBXDWGHXVLB69EjvioTFsViMRr1LRbGlT4x/uDwXZr36WUmHaDNr6Vt?= =?us-ascii?Q?6UBvaclpKW/mgk8jJGF/SeHoJrtJ7XNbUejaywXtUHVf50FCOTuBIP+lLPP4?= =?us-ascii?Q?SYLseOZeyOCIwdKy6C0itii7nly48vSHG8qFaWBoaW+zva+XYn+RBWGI/ytA?= =?us-ascii?Q?231NgTEvCBxchDgAsk4QAtq9gJXfAou3PG1G?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2025 10:41:44.0826 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e379d11f-4fb9-49aa-d7f3-08ddb7c2b4d9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AD.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF000ACABD1 Content-Type: text/plain; charset="utf-8" The hibernation successful workflow: - prepare: evict VRAM and swapout GTT BOs - freeze - create the hibernation image in system memory - thaw: swapin and restore BOs - complete - write hibernation image to disk - amdgpu_pci_shutdown - goto S5, turn off the system. During prepare stage of hibernation, VRAM and GTT BOs will be swapout to shmem. Then in thaw stage, all BOs will be swapin and restored. On server with 192GB VRAM * 8 dGPUs and 1.7TB system memory, the swapin and restore BOs takes too long (50 minutes) and it is not necessary since the follow-up stages does not use GPU. This patch is to skip BOs restore during thaw to reduce the hibernation time. Signed-off-by: Samuel Zhang --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/a= md/amdgpu/amdgpu_device.c index a8f4697deb1b..b550d07190a2 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -5328,7 +5328,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool= notify_clients) amdgpu_virt_init_data_exchange(adev); amdgpu_virt_release_full_gpu(adev, true); =20 - if (!adev->in_s0ix && !r && !adev->in_runpm) + if (!adev->in_s0ix && !r && !adev->in_runpm && !adev->in_s4) r =3D amdgpu_amdkfd_resume_process(adev); } =20 diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/= amdgpu/amdgpu_drv.c index 571b70da4562..23b76e8ac2fd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -2734,7 +2734,9 @@ static int amdgpu_pmops_poweroff(struct device *dev) static int amdgpu_pmops_restore(struct device *dev) { struct drm_device *drm_dev =3D dev_get_drvdata(dev); + struct amdgpu_device *adev =3D drm_to_adev(drm_dev); =20 + adev->in_s4 =3D false; return amdgpu_device_resume(drm_dev, true); } =20 --=20 2.43.5