From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011054.outbound.protection.outlook.com [40.93.194.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9E373451CD for ; Wed, 9 Sep 2026 04:00:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.54 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926404; cv=fail; b=fY0kA7RDbwSa/YZGqDtpKl3CLoZq5bBvG62j4CzUbfMSRWHGDRsBmHhXrDAQ7aJmiOUifh2AsJqLqCfQ+dGzq0EsTNLb3BHeUOrhsJaOZ0ySXLnBrYufaaaM0/3zZXqvsQ1yG837og+iYYJUJtvAap3jKgVYInsTxliOrqrJD4k= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926404; c=relaxed/simple; bh=5iJTHokIpR1WeJNB+J5NP/bQbCxEpJAqN8bf3UE836o=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=BngLcThMZYOFUdzg2FQFDDEznzC8kFy62I+uqgHZjC7Qf4UOHPE1Sea9e2I8UjG2mxniRIr6L+9HbnAVWOg+4707zJ/9QZM/kxCd5U2hLlO11upu3PXZbe3keXLidkZp1W6xjqiMP8B+OPIGt4VpxgHfeY+7CPur3SJMqbcnw9A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=utCkomOB; arc=fail smtp.client-ip=40.93.194.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="utCkomOB" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=phgji0G9jTC+DezsAen9qNDmm1U7GdgtZL0/LYo1MtzGebF/2m0+mw0JlMfQTAY2AuYDeVs1HAd3jEUH9eWjkAy2jd5pDuhOo9K0dOQc1PbcEFwCu03IyguGnKQecxMsc6Fw9NeC6MpyB9wvu1nq1jOrRCI2WcLhYccMzTmhT21Uy4yW6Te3GKLA3AFBjYEphz1EhlUDIHajlOHjHrjbxha9vMu8Iky5MJjrk6XHjNXZ+RwsTTaLYIUFhK06AF+tD31bOYM0ZpDN0+blOXsT8Z4HOl7PyOe9bUgHmmK1hGp9zdSMGQqU472Cg6ZKMzM8Cd4FA1oLisV+eBE0TyaZ9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0SvUjPA8yU+ZMwmFYnNYcsRElXwkoEXowwonemw1s6Y=; b=THVysiggPXh7Te/qiu3CYzWhb4eczT9zJlzgWfE9pVrW9JcGCL2hn6SHlkocm98LD7JR7FIXZZX5XszPpEX3jqrjxZ3/Mmtxm+VAoY1JaPJbDUJfNpRmQrInWARRkkHNevspVW4NrQRZIDL7HeVxk9e/1YWguy8BTzFt1x6hYOgnHymYYVrwrJXgtvsvTEnlsjdJJcBmrOgusS8JVK+ISt9RXJ2N5tQKTFss2en1Mu/5dXC70sU4PvDjhgXkif7qRAcnhvbOQvy8WZQHyDw7OJzJ15072zVGYtedkFFlldQPeLqzoxNx7CxDr/VGMCEW7GbyWh3ylB0u5EfrqZ/ckA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0SvUjPA8yU+ZMwmFYnNYcsRElXwkoEXowwonemw1s6Y=; b=utCkomOBYgHUyzRCc7BZ6r4lxFDV+uboxGjosYEboJJtGqb3weoyI0T1opLI+wGwoHz4oF0oizyQzzAlytSX5VDXohP4ow3xCVZo1GnfJhxO3D1XJELY8G1eIpeXemj18CTlCHgVVU+CYsn1hVUwPnMrhFMNg2viVqKqXQ6Sc/hgRx6vyHkuazShfIYqxGpwJwmonv/q5EvKGZ7k/U503W2n7e5VJhezUFTWFJL1d3cTdUvqvJ17DlOSkF4h/ungGzuDlkF3VE99h6pnOlcozGrPukgaghZ/hd2iV/2i0jXfHg1AY6q7F/mB2ZWDzrwFb3GNxUZ0He4Sahy7754o+w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 03:59:59 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 03:59:59 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:39 +0900 Subject: [PATCH 01/16] gpu: nova-core: mm: Add common types for virtual memory management Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-1-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P286CA0107.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:380::7) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 6dfd9330-981d-449f-7c48-08df0e26d101 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|6133799003|3023799007|10067099003|11063799006|5023799004|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: 1CvOB9D3r93+tzRkqEhZy7L8ndAZIZVs8/E2ZhDVQ9A6xNId+Iwilht/1itguyXsxKMXa19LtrWBMSGuEjBbV3kUFDxfq8NA2LIAjui29BnLmpesEPsks5a58CpXYUgKFOU00zRGY53I2Rdihj3o1rCo9dZuTyPlaz941CwALOJU+Xi6nsxF1EAatENlQjrZJqNbNRJgaD8tfT0jokxLjcpcUQ6Ns7DL1hu1a5Yhk2wbdgOc/J65h+V3LV71sjsUrv23lgcggmCAVrmFChIxz0T1C1GWruM4+o0vSpSQWrn0xKuldAjkgq0o6STnWFx306ZbEu1XzxqZJH5/EFtSKjDme4tvMkAShpl+xjZsRWEYKn8b9aVg7Q4pzCfdE1Mgs6Vi/feVHtHR9kazGHedvjXFSeCu+juGG4FwnR9DgRJRSeDjrRLRA8iIk3f4TFRHEU5WU3XkpRXGguLcZyZHy/Ngy+Rsr8RnXUBYH1jFUq7qanHHqP2CqnyJ16DD+VYlQCyFj5K5ZnHUqFrERwZ4Y4cpA5GXIjkFp159D/SQIJc38KYPNVrzO4rMUNsNhEfCH+0filXS1Bz7Z4qvl3D3WAAVVy4sUHgSxmvupMcuN/b55wKq8DvhQ8vKz+pB6tIs4mrAoWT801SO6jh5ngjRG8xi+W+W09DYQbgfOWQYrfE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(6133799003)(3023799007)(10067099003)(11063799006)(5023799004)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cWdzTURKdzdhd1kzdnk3Q0ZaNWxZcnoxV0hUc3h4TVY0Qnd0RXFVM0JvOVI1?= =?utf-8?B?ZEZSYVhQREswTDFFM3JZV1NFYVB2ZnFodWpvZGhJeld3SzQ0QS9MU0crRDlD?= =?utf-8?B?Z2FiUXREQjFLTVRWQUFRanBZT0hVN0k0em9nTGxWdkpUQ1Z1dGd2TEpKRG9j?= =?utf-8?B?SjRzRlBNeFowSDR5aGhOYUVhMnRzL20rdHBocW5rVDZXQkNxVUt5NFVPQVhT?= =?utf-8?B?RWt4aXBIbCtjVlJNVmpRTXRMS3FyNmpDSjNjcE5nSmMxTTJKTm9MRXc5Um45?= =?utf-8?B?NlJwUjdxcUJwMlBLNDF2ck1SN1pNZW5jNWZPK3VTa3JISXhkQzF3akk5M2ho?= =?utf-8?B?cGJLV3VYdXZzQ1JPM01HbHlnbFFYK1l0TDB0U1daVG1XYVJBZ2ttVDVRZ0pl?= =?utf-8?B?V3F4Y0dwUUhFSUZkU0xhdDRoVUZ1cFB1NzVYWTg4Z09wTGZSUm1YOFlKRGFh?= =?utf-8?B?RTBYVFJhbUlTazFHVWdDRGNVNWIzVUVHU3djQ2tTMVFCd2h6WUlPblpTTzZT?= =?utf-8?B?dHBKVUljUUJOMGxRVkhsSDdadHhxS00yK2FHSDVEVFhoOW9YM21mZW9yQi9Z?= =?utf-8?B?RWh4bitOV2hlbXJMZmNrVjBIM0dmdGZWcnl2TUhSTFFOVktDaEIrVHFDeDRH?= =?utf-8?B?UHFQalFkaTU4TFRmSk9mMGphOU1KNEpucmo2VE1rTUpIdm5nZTBhV1psZ2hY?= =?utf-8?B?QXc3Sk9WdjlRRGlGZUF1NkQzVVQyRllHdS84d0QzUzl1WmNhKzdpUlBKYjVQ?= =?utf-8?B?Ly9zaUk1U1NjV1EyMlN5d3ZTVXR5ZkJJamsxajQ3K1pZdi9Bdk9VTHhlc1Nr?= =?utf-8?B?LzJmZDBtY3gySFNrQmdYOUxORGtIVHJYREV4T1YyaGVmK2lIaE1TdXJvSVkv?= =?utf-8?B?dmU2WGtoUkV6UnJiQXpUYmlHajR5c01KNytzbjBCUmZrMWE1anlrd3MwWVJr?= =?utf-8?B?ZERlODlpd1NCUmtGUlRWK2k0bTBLdE8xSlBOOU05Uy9sNnVyL1hvUE5vOGZG?= =?utf-8?B?QVE5cDBaT1NOYUJXTnRjV0p0RGFzOTV6NVkrMjJUSlFOZTA1SC9KbGZJRFVZ?= =?utf-8?B?REs0ei9Eb0hGenhuR0F1N2JKcHlZck1jUHFCYTNxeXB2OSt1VldFR2Z3MWRW?= =?utf-8?B?d2IvTnQwaUJ4dXo2Ti9zdWZaOFZwajVGQWxhMzZmNDBCdE9OY3YrZ3pUNHJX?= =?utf-8?B?VzBXazlmalNaVlFuRWJsQU1CL1N0RjdySDBacTd0ZDQxL2EzVlFrdGtrM1ha?= =?utf-8?B?ckFBUTNaaDdSd0F0T0dKV3Y1NjI5MXVHRDd1ditLMGlRVjdYRlljTStkLysr?= =?utf-8?B?QlRna0MyWkprSm9pV29NUWFtbmZmaEx0QTFSMFZkWVB4SGt1N0lPOGw5OEJC?= =?utf-8?B?RkJMWXFpUUlMa0FmWWw5NjJFWVNMM282cTIrUTRUYWgyRnJ6ODhFMWtWK08y?= =?utf-8?B?akJ1N0ttNkQxOHNGYTBxcnJ3MnEydGxXeTQrRzEvWWFvUGtCMHhPTjFybUFT?= =?utf-8?B?ZStqVVdIOVRHc3cwM0ptSFZzbVRJRDBMcjYwT0l3UDVaaVhQZWU2T20yN1Y1?= =?utf-8?B?czYwa1AyVHBOYXZNQUJzTVFTVXZFdmpxV1BhRHhRMXdHVkozeTRUa1RvUXlM?= =?utf-8?B?aU5Ta3JIbnMvUkltNWpuRjlRMDloNmxxLzJyMll5dG8xczdhampYcXZVVndn?= =?utf-8?B?eXI0bk8xdEdITkE5WkJtL0lCbURIMUFXc3poeVZKbm1uYk1qU0ZFZGNPN1hv?= =?utf-8?B?SEI2U2MzcWJ4SUlCK3NDSFRjN2pTRkQ0RGdYTVZ1b2J1ZDErcTVhMUFHN25J?= =?utf-8?B?NjhqSmwxTmkwaUVsT0tZZERZVmc3dWJGQVE0ZmxEVVhINXNMdm1ULytJZzdD?= =?utf-8?B?a0pRR2FBSEdHbzI0TGY5SFY0VzNDRUZVNlRQYVRxWG9mN3IyZ0NaWTd0U3lr?= =?utf-8?B?UFJPT2hWeFNkcVVCZTJoUWJySHNOdFJmT3Q1T3F1eGFsWHBUWTd6MXNrSy9u?= =?utf-8?B?bkxSU01JVTA5VGFuVVNqS21rZVZ4M0o5UFZmcUJYcllVMk13NG5zbXZjdWlI?= =?utf-8?B?OGlsQzFvRmNYSFVIeHJnNk1aNVlUcGRMTUlYUjBZbTdDdkZvRVZCeWIrNXkw?= =?utf-8?B?U0VxY3VWcy9pTWdlelJjbUs3NWxiR1Buem96THJqVkU4bnNOd28zM1NiWlhj?= =?utf-8?B?NGxKaFRyaE1WclJNNnR1OHNydWlCRU1EdU54aUtCcGc0Z0prRnZBa05scUtT?= =?utf-8?B?ZXVoV1I1TlUvL3dOZEowRW1nVlkwZ3ZFZXRvZmlSMWRmNHFaSlRERUt2L25I?= =?utf-8?B?RnMwS21rWC9IakNnTVAvdVJReS80MVA3WVNreWVwWWp3QzdLS0FSUkRwVFJy?= =?utf-8?Q?TpFw0HILwCidCn7ri35YtcW5BRK1zn8oWyajc/674+l8O?= X-MS-Exchange-AntiSpam-MessageData-1: uT2mi4cAVO+qwA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6dfd9330-981d-449f-7c48-08df0e26d101 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 03:59:58.9035 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: AuqZB9GqzdjFyZgeeMERafRHelMjYajTKFpp1CBPKb2nhscS5OfN4/d7YW3ky4sIVe+rKkr/bXJEFnDnUWXK2w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add common virtual memory memory management types: `PAGE_SIZE` constant, `VirtualAddress` bitfield type, `Vfn` (Virtual Frame Number) type, `Pfn` (Physical Frame Number) type. Signed-off-by: Joel Fernandes [ecourtney: restore Pfn and its VramAddress conversions on the raw API] [ecourtney: make dead_code expect unconditional until the self-tests patch] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm.rs | 142 ++++++++++++++++++++++++++++++++++++++++= +++- 1 file changed, 141 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs index 9e4338c7c393..4234ba4d596a 100644 --- a/drivers/gpu/nova-core/mm.rs +++ b/drivers/gpu/nova-core/mm.rs @@ -3,7 +3,34 @@ =20 //! Memory management subsystems. =20 -#![cfg_attr(not(CONFIG_NOVA_CORE_SELFTESTS), expect(dead_code))] +#![expect(dead_code)] + +/// Implements `From` conversions between a frame-number type and `Bounded= `. +/// +/// Each MMU version module should invoke this for the specific bit widths= used by that version's +/// PTE/PDE bitfield definitions. +macro_rules! impl_frame_number_bounded { + ($type:ty, $bits:literal) =3D> { + impl From> for $type { + fn from(val: Bounded) -> Self { + Self::new(val.get()) + } + } + + impl From<$type> for Bounded { + fn from(v: $type) -> Self { + Bounded::from_expr(v.raw() & ::kernel::bits::genmask_u64(0= ..=3D($bits - 1))) + } + } + }; +} + +/// Implements `From` conversions between [`Pfn`] and `Bounded` fo= r bitfield interop. +macro_rules! impl_pfn_bounded { + ($bits:literal) =3D> { + impl_frame_number_bounded!(Pfn, $bits); + }; +} =20 use core::{ fmt::LowerHex, @@ -11,12 +38,15 @@ }; =20 use kernel::{ + bitfield, fmt, + num::Bounded, prelude::*, ptr::{ Alignable, Alignment, // }, + sizes::SZ_4K, // }; =20 use crate::{ @@ -58,6 +88,9 @@ fn pramin_mut(&mut self) -> &mut pramin::Pramin<'gpu> { } } =20 +/// Page size in bytes (4 KiB). +pub(crate) const PAGE_SIZE: usize =3D SZ_4K; + /// Physical VRAM address in GPU video memory. #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] #[repr(transparent)] @@ -124,6 +157,113 @@ fn sub(self, rhs: Self) -> Self::Output { } } =20 +impl From for VramAddress { + fn from(pfn: Pfn) -> Self { + Self::from_raw(pfn.raw() << 12) + } +} + +bitfield! { + /// Virtual address in GPU address space. + pub(crate) struct VirtualAddress(u64) { + /// Offset within 4KB page. + 11:0 offset; + /// Virtual frame number. + 63:12 frame_number =3D> Vfn; + } +} + +impl VirtualAddress { + /// Create a new virtual address from a raw value. + pub(crate) const fn new(addr: u64) -> Self { + Self::from_raw(addr) + } +} + +impl From for VirtualAddress { + fn from(vfn: Vfn) -> Self { + Self::zeroed().with_frame_number(vfn) + } +} + +/// Physical Frame Number. +/// +/// Represents a physical page in VRAM. +#[repr(transparent)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub(crate) struct Pfn(u64); + +impl Pfn { + /// Create a new PFN from a frame number. + pub(crate) const fn new(frame_number: u64) -> Self { + Self(frame_number) + } + + /// Get the raw frame number. + pub(crate) const fn raw(self) -> u64 { + self.0 + } +} + +impl From for Pfn { + fn from(addr: VramAddress) -> Self { + Self::new(addr.into_raw() >> 12) + } +} + +impl From for Pfn { + fn from(val: u64) -> Self { + Self(val) + } +} + +impl From for u64 { + fn from(pfn: Pfn) -> Self { + pfn.0 + } +} + +impl_pfn_bounded!(52); + +/// Virtual Frame Number. +/// +/// Represents a virtual page in GPU address space. +#[repr(transparent)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub(crate) struct Vfn(u64); + +impl Vfn { + /// Create a new VFN from a frame number. + pub(crate) const fn new(frame_number: u64) -> Self { + Self(frame_number) + } + + /// Get the raw frame number. + pub(crate) const fn raw(self) -> u64 { + self.0 + } +} + +impl From for Vfn { + fn from(addr: VirtualAddress) -> Self { + addr.frame_number() + } +} + +impl From for Vfn { + fn from(val: u64) -> Self { + Self(val) + } +} + +impl From for u64 { + fn from(vfn: Vfn) -> Self { + vfn.0 + } +} + +impl_frame_number_bounded!(Vfn, 52); + #[cfg(CONFIG_NOVA_CORE_SELFTESTS)] pub(crate) mod selftest { use core::ops::Range; --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011054.outbound.protection.outlook.com [40.93.194.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C57E23115BC for ; Wed, 9 Sep 2026 04:00:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.54 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926408; cv=fail; b=PLf7Lys8xmttFWBASixHMTsAKAz/AXyNa/FXof76UpB3Oih0NM9iOvSqBJ1AkbayBxNeXEQiAnOzbPr/twqkfXCMrpitrOTGkCf6oSL988fb6e7fujPdQhnjpBDNQuKtLu9Tl90knCbk4wRJb46NqUSXdNxHNYT43qwZnyjjpTg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926408; c=relaxed/simple; bh=WCmFRqaCV7T+SQhE1CfbAA3myP99oMmflJTdJ9/NImw=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=sBpPih7YPYe6fEKy6ng5wUK2Upe0ogaG9DsjHZ4tFcxgH1vFxL5jwlME0bOp/iPQJfQyjkAPMyy+JrT7id96gEMh+2L479mUIT5SVaI7JlR2ACHUAhDptCM4QxeLqrkUd7CEWpx8Bg8t3PAcEA6MLut4ngHTVvdT2upsbjE5yOI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=FUAA+dc2; arc=fail smtp.client-ip=40.93.194.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="FUAA+dc2" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DsJjmNy60QU75s2yWqjgrEoNm1TKxeXy+XcaI0aSrCslrKtEQHsqasQiCBfmGmY8ebOxHymPKa0eq08twcSidFcgH8nvzMvw4hyc3R0KaPnn3XzcsuqgVnWkP0c0HMujj3Jdu+caaDGVIrSjrO3SPfM2oQ+bnz47s0TuHuJvBnRi2piSdMkC15IJCWrPklQCSwAaNcH/fWQnIg+dMkpTqKV+YJ1WXHvhfKHTqwjhdIuRUlqw8jsnTlxEMshodRq70fu8eQzmh5lIWgemLjdxAv3UK0qelT3HQGjmOcIMVhfZcKoc/yQh5d9hjwcEKkGdTTqX9mo9OMVV0XmxIP689w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mYwS/DY3xF9WwS5trYXkgbt/9bKHvMuSbyCswNXh5os=; b=pfZrooE2Qsr9vtfXnPolFHtnoby47B1tmFgymUD8lv3WE2+FwzXp9csQVibUBtHL2nXbCKRSgN9EvbjQ4tTyFUdsnMbEghPTohEqtGNletQhvbNPctkbDfcLEIsPH4dWaiuM7aqirCUAuazK7bwHTjmIrwxkJKGuuR0XfxRH7OWFinfIhsMNuLkupdbBzsHrH43muuPjpzyEBOHrxD1Ug7VSzKQQPN7YK/y+R49hUWq3yJN2kzHSsdslkCJX1/rxZc9i3gc42u6+hw0jRheD1fi9krDxzoYC5pohAJofqSCiKgo4vFHHuKJ8dIMMKXHf7JCmttZKCstjsPgY4XGM2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mYwS/DY3xF9WwS5trYXkgbt/9bKHvMuSbyCswNXh5os=; b=FUAA+dc2brroVMAbbXVk5OUMj6grcFrr4Z8tDB4xU/r8rSTtvJSlGFdfD2eWvgX5QKXogmSLpbswnQjljoK4SbJjm+l5VWLDQLJ8eWbzTW7jvFPvGszoE71Trk83uY1s+8LIQgDI4DUxX5qCTtTQN8pGO9pyc3Yg3oz75ijVUzxtHC/rhujm9CM0/RfuyEcI1swNDbODsrlYoHFOwrM1cv3fkyzOXOGisst5UZJ1zJrRI1SS85LnXhFIxDq92K9MkMDWqxnKz5TsxNOZa6U9dmqLZb7wN7ixOn4VLRTMbBD3PeBA4yawxL9M6xVO4t50FT/KuoOpMAvqJW8wDmATdg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:02 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:02 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:40 +0900 Subject: [PATCH 02/16] gpu: nova-core: mm: Add buddy allocator and TLB to GpuMm Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-2-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TYCP286CA0019.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:263::13) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 6a25cc5a-e013-40f0-5a33-08df0e26d320 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|6133799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: GxyAn0VvxK33vH4wuHVpYH80yCpdg88g3gARD9DC4BztR3WDxpnH/fgLaqvwwKM9H5FQH1IfX9V/nxEwFzQsJGY1o9hI1W6UB3jGqOMHHtBcK+1zln4FT26MVIgLUs02mTYIYAFZ6wKTVTi1jcqi9o3cn16C70HqRvrEDAdqznl8kZ2f9a+w7XTF40pQ7ubH9Jr2XFkLFw5sAMR6R19EFFxDGJsjunkheL1MQbRmdAg9qnkEHqZcGCo8J2/BX4q99zsQzPGI98Vkc1tY/HGr+Fcipq780ndzg+07cSr+0tTYPDnsdCqNAoKJgkaE5SqYuU/KNtEJNR3fB6ntvsc71OgZoDF89YjDBrd7iz8hLWYgt7+BnIJzBothcHUVvaZOWCaNEyAaoVyw5cmH046CqGoMlOONA3XVrq9rl3zvRAvhXgjmyeDlhRmtOxfiESHnWnrH2THlO49A6PMRDORtxltBAkUaB+VpEvT5RugubIr4ahCVW57Hz5QGFhkTZOlVT6NZHn+LLiIhDZU633lH3T91bnY6haBU5gtuCAPTG20Am4WLOIqGhJ27pU3DfdRkHWRQaa9ndsOT4SD4KDqFTEQGmI/r0eQRlV2c3LnP38/YfwrmUw5xORxRBS7Xgn9gTOWVeC6GAOo54LasyoqcO5hTM71rRfT156Er5DKM7XU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(6133799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YVNoSUVMRlI2SFQ0akhKaDlZUFFyaWxCb3lMSlpnb0d1ZWFYZ2tVVm54ckx6?= =?utf-8?B?bGRqeXNiSDU0dEkya1hsdXFjV295eUNNOWNteFpFNlIvQjZwc2cyUWZpWjQw?= =?utf-8?B?ZmhxUWxXWXZubnBXMThtQmRmcGcxcncrOXBqY3dOVUFIR3B5Yy9HNXJmTEtB?= =?utf-8?B?T0pxM1JhMHVOTStNNWFXUE5TK2R2WUQ2ZEQ1NkVnQ0xNSGxXSEhEVTI3T1o5?= =?utf-8?B?ajZvV3lXN21sN2VQTDdDME1DbnI0a0RCUFZWYzJkQ25zaUs1aGVsOXNXR1lz?= =?utf-8?B?UDdlR3psQVYrYmZvMlhSU1VacW9DT0F1LzAyMUZsS0RlSjZRVXhpZDBYaEVT?= =?utf-8?B?VmxGZXZ0MElaMWx3cGp1b0lnTUliVUE5MUthWE96L0hQYlkreitFTEdqdzQ3?= =?utf-8?B?RFE2SGxqVEMwbjd5WHhwRGlvTXBjVnF1RXJ2U0JteXJsWkVLRFFkNU9NQS9y?= =?utf-8?B?eG9qbXZoNE92ZlkxMGxNd3FWeGhmT1BDbkJFNXZramFnS1NuanU4RmNPc1Fr?= =?utf-8?B?NGZHK2plVmZPUlNRL0U5NHJoY3pwS3dxTDNvam5LQnVEU3lFWlNxRXlvVTdM?= =?utf-8?B?OGhnT2VLb3Uzc2R1Y0ZWZ3czVlN4YTB2VkUrclRnU29ITzZhVEZBTWt4Y0hO?= =?utf-8?B?ei9XQ2liSGoxbDNWVzZHUkZzZ2k2Wk1MYzRURXNidkVSSmV5MjFJUE85VFdS?= =?utf-8?B?TUVURWdDd1VjazhPRDdMcG0rMytEaFRteWlMNk01eFlGQlpjSEE4ajhEMmF0?= =?utf-8?B?TUhucW8vUWlkR1lIWktZUzJuQ01SRWxxKzdkUlBGVXhtalFKbFFwcVYxYlZu?= =?utf-8?B?ckx1Ty9kaEo1V21XcFE0d3lwcHhUaFRFTnlBY2V2cGg5SHBaMGtiNTl6bkUv?= =?utf-8?B?eHd3K2x3K09sbnJnU2xVSUJGL2dZOTQ3bVpBRHYvZU9SU1YvY3FoUWY3Y2Z2?= =?utf-8?B?WEZzcFFLRnR1aU9qZ3luenU5S1B3SWdjTFVQNkR2UVR2TDFTcCt0OGUxenNP?= =?utf-8?B?SExsTVJZd01DdFhxdjgreWlYcmZhTVpmVS9WSitNL3h1VkFLVTQzK011TTg0?= =?utf-8?B?WUF2elZlRlRIdTZjWWpiS0F5RS94dzJDR1hEUVdWR3Z3M0lUT2FPc3BkTUli?= =?utf-8?B?OXdFWW9ucytQVWxNckd2TStla21iQ1lIMng3eEptS1hiRkZYMkM0TE4rYk9H?= =?utf-8?B?Y2kwaiswSVZzU2Fmc0hpcjVZcHJtWWhFY1Q3TEhxdG1pUDlCTUFJQW5hSzNt?= =?utf-8?B?NlhlTklRQW9QZmVMTWlHNGpXYUdnVVpCUkdmSXhuTWdEMklWblRUNno1ZWk3?= =?utf-8?B?ckEwcUMyZzZDYTFDZXlmTndjeXFHa2JUUTlQNUdqK0NxMXdGdSs3VloxYVhO?= =?utf-8?B?ekNQdWJyMnU2cDAyUFZjQzBrN1VOaG1rU1ptdVR5MVZ3YStQOWpUcWRwWUxE?= =?utf-8?B?S1BOMm1zQ3gzcnVDZHBoaVhGa2pIVFNxaTREbjR2L1c3MWRTRG96ZXB3Z3dq?= =?utf-8?B?amFjK1J2dUtXSlJ1R0QxQ0RCTWx0aTVQTDlEZVdKV2M2KzhjVjV1QXQxNk5P?= =?utf-8?B?NkpvS2JjZWZjY1RuU3I3cU5HYVA0NFNGYlhJeC9YL05ZU3N6aUNteGlsbVc4?= =?utf-8?B?ZjN2VjE1YTlTckE0VlExbmJmekJXUDNVQnc3K0VIbTlNODBRK1F4UzF2Q0ZQ?= =?utf-8?B?dkdsNmY3ZlMwYy9yVUUxMDdxMVRsM1Ezd0xWem9McjJHYlQ3YS91Q215YXVa?= =?utf-8?B?d24vcWdyZVkxVDBuQWdEMXdDYnNNdUpSNGczbUlVV0R5K1JYM1M3bWQ5VDll?= =?utf-8?B?VzlxVkE0TUpMbHR1UWN6L3kxSFM2eGh0aWx5NTlJL0V4dHpRbThDQ3FSNk0r?= =?utf-8?B?enI4bUowM3QvOStaOEFBWUY5MzI0ejB1eFRxODI4cmxHaWFBdUpVOWpFaU9P?= =?utf-8?B?RWFFa0JjVlVNVHA4cVVmbytGY3FiSnJhMVo1NUZNbVE0N0dmSEEvMExCdVNz?= =?utf-8?B?bVZnRHFuamM4R1BrUTdyT2FmcFIwSTJvOFRwQzJhVzRRaVVhVEJ3U2QxRlZI?= =?utf-8?B?L3BIdU43eEExVDBDTGo0S0lURDBQSS9nWFg3Wml4emtQOGdJWkczWFRTRkcz?= =?utf-8?B?UG0rZlkyZG9iVS9EeHp2WE9FRXlSM29BQktQRG1qM2puUEIvNHg4b3JWc2pU?= =?utf-8?B?LzVwSi9OVXZVMS9VNXNyTWc0dUJucEVUVDk3eERHTnRNQXA5cWN1U2E3TlVE?= =?utf-8?B?N204elhrWjJ3Q0wzOURJYSs2V1A4eTJIOE5ib1FQZFBBd0lMTHZLcDM1UWx3?= =?utf-8?B?VlhibkZBSDZXZnMxZ2h6dWM4djc1Lyt4aytWUnM0a3dRdGIvS2R4Q3BHVGl3?= =?utf-8?Q?qpomWDJ9ALQvI8Ooy7F5IJulEridfMDh4ld8tpfPEr4bE?= X-MS-Exchange-AntiSpam-MessageData-1: VpkF6NdJhSWObQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6a25cc5a-e013-40f0-5a33-08df0e26d320 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:02.5784 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Q+NtzZLQEJIQcMz/VJ/gXC10umqFnLgzofvN3nL+TVhLs9O0/Av2Q5dpQSR0vpOExMdKTkkqMZIbIxms9pREUQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Extend GpuMm with the remaining two memory-management components: - Buddy allocator for VRAM allocation. - TLB manager for translation buffer operations. PRAMIN was added in an earlier commit; this completes the centralized ownership model with accessor methods for each component. Signed-off-by: Joel Fernandes [ecourtney: rebase GpuMm for borrowed BAR0, pin Tlb, write regs directly] [ecourtney: size the buddy from the first usable FB region, drop dev_info] [ecourtney: update for the VramAddress raw API and typed register base] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/Kconfig | 1 + drivers/gpu/nova-core/gpu.rs | 27 +++++++-- drivers/gpu/nova-core/mm.rs | 24 ++++++++ drivers/gpu/nova-core/mm/tlb.rs | 120 ++++++++++++++++++++++++++++++++++++= ++++ drivers/gpu/nova-core/regs.rs | 68 +++++++++++++++++++++++ 5 files changed, 234 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/nova-core/Kconfig b/drivers/gpu/nova-core/Kconfig index cb7f0b00f796..1934f17baa8b 100644 --- a/drivers/gpu/nova-core/Kconfig +++ b/drivers/gpu/nova-core/Kconfig @@ -5,6 +5,7 @@ config NOVA_CORE depends on RUST depends on !CPU_BIG_ENDIAN select AUXILIARY_BUS + select GPU_BUDDY select RUST_FW_LOADER_ABSTRACTIONS default n help diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs index a52a3d4d86af..b797472279d3 100644 --- a/drivers/gpu/nova-core/gpu.rs +++ b/drivers/gpu/nova-core/gpu.rs @@ -6,11 +6,16 @@ device, dma::Device, fmt, + gpu::buddy::GpuBuddyParams, io::Io, num::Bounded, pci, prelude::*, - sizes::SizeConstants, // + ptr::Alignment, + sizes::{ + SizeConstants, + SZ_4K, // + }, }; =20 use crate::{ @@ -422,11 +427,21 @@ pub(crate) fn new<'a>( }, =20 // Create GPU memory manager owning memory management resource= s. - mm: GpuMm::new( - bar, - gsp_resources.spec.chipset, - VramAddress::from_raw(gsp_static_info.total_fb_end), - )?, + mm: { + let usable_vram =3D gsp_static_info.usable_fb_regions.firs= t().ok_or(ENODEV)?; + let buddy_params =3D GpuBuddyParams { + base_offset: usable_vram.start, + size: usable_vram.end - usable_vram.start, + chunk_size: Alignment::new::(), + }; + + GpuMm::new( + bar, + gsp_resources.spec.chipset, + buddy_params, + VramAddress::from_raw(gsp_static_info.total_fb_end), + )? + }, }) } =20 diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs index 4234ba4d596a..202bb8f8ebed 100644 --- a/drivers/gpu/nova-core/mm.rs +++ b/drivers/gpu/nova-core/mm.rs @@ -40,6 +40,10 @@ macro_rules! impl_pfn_bounded { use kernel::{ bitfield, fmt, + gpu::buddy::{ + GpuBuddy, + GpuBuddyParams, // + }, num::Bounded, prelude::*, ptr::{ @@ -54,16 +58,23 @@ macro_rules! impl_pfn_bounded { gpu::Chipset, // }; =20 +pub(crate) use tlb::Tlb; + mod hal; mod pramin; mod regs; +pub(super) mod tlb; =20 /// GPU Memory Manager - owns all core MM components. /// /// Provides centralized ownership of memory management resources: +/// - [`GpuBuddy`] allocator for VRAM page table allocation. /// - [`pramin::Pramin`] for direct VRAM access. +/// - [`Tlb`] manager for translation buffer flush operations. pub(crate) struct GpuMm<'gpu> { + buddy: GpuBuddy, pramin: pramin::Pramin<'gpu>, + tlb: Pin>>, } =20 impl<'gpu> GpuMm<'gpu> { @@ -71,6 +82,7 @@ impl<'gpu> GpuMm<'gpu> { pub(crate) fn new( bar: Bar0<'gpu>, chipset: Chipset, + buddy_params: GpuBuddyParams, total_fb_end: VramAddress, ) -> Result { // PRAMIN covers all physical VRAM (including GSP-reserved areas @@ -78,14 +90,26 @@ pub(crate) fn new( let vram_region =3D VramAddress::ZERO..total_fb_end; =20 Ok(Self { + buddy: GpuBuddy::new(buddy_params)?, pramin: pramin::Pramin::new(bar, chipset, vram_region)?, + tlb: KBox::pin_init(Tlb::new(bar), GFP_KERNEL)?, }) } =20 + /// Access the [`GpuBuddy`] allocator. + pub(crate) fn buddy(&self) -> &GpuBuddy { + &self.buddy + } + /// Access the [`pramin::Pramin`]. fn pramin_mut(&mut self) -> &mut pramin::Pramin<'gpu> { &mut self.pramin } + + /// Access the [`Tlb`] manager. + pub(crate) fn tlb(&self) -> &Tlb<'gpu> { + self.tlb.as_ref().get_ref() + } } =20 /// Page size in bytes (4 KiB). diff --git a/drivers/gpu/nova-core/mm/tlb.rs b/drivers/gpu/nova-core/mm/tlb= .rs new file mode 100644 index 000000000000..cc862e8159a1 --- /dev/null +++ b/drivers/gpu/nova-core/mm/tlb.rs @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! TLB (Translation Lookaside Buffer) flush support for GPU MMU. +//! +//! After modifying page table entries, the GPU's TLB must be flushed to +//! ensure the new mappings take effect. This module provides TLB flush +//! functionality for virtual memory managers. +//! +//! # Examples +//! +//! ```ignore +//! use crate::mm::tlb::Tlb; +//! +//! fn page_table_update(tlb: &Tlb, pdb_addr: VramAddress) -> Result<()> { +//! // ... modify page tables ... +//! +//! // Flush TLB to make changes visible (polls for completion). +//! tlb.flush(pdb_addr)?; +//! +//! Ok(()) +//! } +//! ``` + +use kernel::{ + io::poll::read_poll_timeout, + io::Io, + new_mutex, + prelude::*, + sync::Mutex, + time::Delta, // +}; + +use crate::{ + bounded_enum, + driver::Bar0, + mm::VramAddress, + regs, // +}; + +bounded_enum! { + /// TLB invalidation acknowledgment scope. + /// + /// Controls how far the hardware waits for the invalidation to propag= ate + /// before clearing the `trigger` bit of `NV_TLB_FLUSH_CTRL`. + #[derive(Debug, Copy, Clone, PartialEq, Eq)] + pub(crate) enum TlbAckMode with TryFrom> { + /// Fire-and-forget: no acknowledgment required. + None =3D 0, + /// Wait for acknowledgment from all consumers, including remote G= PUs + /// reachable over NVLink. + /// + /// Globally is strictly required only during unmap or permission + /// tightening, because the backing memory may be reassigned after= the + /// flush returns and a stale TLB entry could let the GPU access f= reed + /// memory. For new mapping or relaxing permissions, a stale entry= would + /// merely cause a redundant fault and retry, so [`TlbAckMode::Non= e`] + /// would suffice. + Globally =3D 1, + /// Wait for acknowledgment from consumers within the local NVLink + /// fabric node only; skip cross-node ack. + Intranode =3D 2, + } +} + +/// TLB manager for GPU translation buffer operations. +#[pin_data] +pub(crate) struct Tlb<'gpu> { + bar: Bar0<'gpu>, + /// TLB flush serialization lock: This lock is designed to be acquired= during + /// the DMA fence signalling critical path. It should NEVER be held ac= ross any + /// reclaimable CPU memory allocations because the memory reclaim path= can + /// call `dma_fence_wait()` (when implemented), which would deadlock i= f lock held. + #[pin] + lock: Mutex<()>, +} + +impl<'gpu> Tlb<'gpu> { + /// Create a new TLB manager. + pub(super) fn new(bar: Bar0<'gpu>) -> impl PinInit { + pin_init!(Self { + bar, + lock <- new_mutex!((), "tlb_flush"), + }) + } + + /// Flush the GPU TLB for a specific page directory base. + /// + /// This invalidates all TLB entries associated with the given PDB add= ress. + /// Must be called after modifying page table entries to ensure the GP= U sees + /// the updated mappings. + pub(super) fn flush(&self, pdb_addr: VramAddress) -> Result { + let _guard =3D self.lock.lock(); + + // Write PDB address. + self.bar.write_reg(regs::NV_TLB_FLUSH_PDB_LO::from_pdb_addr( + pdb_addr.into_raw(), + )); + self.bar.write_reg(regs::NV_TLB_FLUSH_PDB_HI::from_pdb_addr( + pdb_addr.into_raw(), + )); + + // Trigger flush. + self.bar.write_reg( + regs::NV_TLB_FLUSH_CTRL::zeroed() + .with_all_va(true) + .with_ack(TlbAckMode::None) + .with_trigger(true), + ); + + // Poll for completion. + read_poll_timeout( + || Ok(self.bar.read(regs::NV_TLB_FLUSH_CTRL)), + |ctrl: ®s::NV_TLB_FLUSH_CTRL| !ctrl.trigger(), + Delta::ZERO, + Delta::from_secs(2), + )?; + + Ok(()) + } +} diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs index bf1f3a97c632..9978fb2803b0 100644 --- a/drivers/gpu/nova-core/regs.rs +++ b/drivers/gpu/nova-core/regs.rs @@ -10,6 +10,7 @@ sizes::SizeConstants, time, // }; +use pin_init::Zeroable; =20 use crate::{ driver::NovaRegisters, @@ -26,6 +27,7 @@ PFalconRegisters, PeregrineCoreSelect, // }, + mm::tlb::TlbAckMode, // }; =20 // PBUS @@ -471,3 +473,69 @@ pub(crate) mod gb202 { } } } + +// MMU TLB + +register! { + base: NovaRegisters; + + /// TLB flush register: PDB address lower bits. + pub(crate) NV_TLB_FLUSH_PDB_LO(u32) @ 0x00b830a0 { + /// PDB address bits [39:8]. + 31:0 pdb_lo =3D> u32; + } + + /// TLB flush register: PDB address higher bits. + pub(crate) NV_TLB_FLUSH_PDB_HI(u32) @ 0x00b830a4 { + /// PDB address bits [47:40]. + 7:0 pdb_hi =3D> u8; + } + + /// TLB flush control register. + pub(crate) NV_TLB_FLUSH_CTRL(u32) @ 0x00b830b0 { + /// Invalidate every VA in the PDB selected by `NV_TLB_FLUSH_PDB_L= O/HI`. + 0:0 all_va =3D> bool; + /// Invalidate TLBs for all PDBs (ignores `NV_TLB_FLUSH_PDB_LO/HI`= ). + 1:1 all_pdb =3D> bool; + /// Restrict the flush to the HUB MMU's TLBs; skip broadcasting to= the + /// per-GPC L2 TLBs. + /// + /// The GPU MMU has a two-level TLB hierarchy: + /// 1. The *HUB MMU* sits at the top and serves memory requests fr= om + /// "host-side" engines: the host/channel interface, copy engin= es, + /// display, and BAR1/BAR2 accesses. + /// 2. Each GPC (Graphics Processing Cluster =E2=80=94 the block t= hat houses + /// shader cores / SMs) has its own L2 TLB that serves requests= from + /// the compute and graphics engines inside the cluster. + /// + /// When set, only the HUB TLBs are invalidated. This is a perform= ance + /// optimization for flushes that only affect HUB-side mappings (e= .g. + /// BAR1/BAR2 windows), where fanning the invalidation out to every + /// GPC's L2 TLB would be wasted work. Must be false when flushing + /// mappings that may be cached by compute/graphics engines. + 2:2 hubtlb_only =3D> bool; + /// Invalidation acknowledgment scope. See [`TlbAckMode`] for deta= ils. + 8:7 ack ?=3D> TlbAckMode; + /// Write 1 to kick off the flush. Hardware clears this bit when t= he + /// flush completes; reads as 1 while the flush is in progress. + 31:31 trigger =3D> bool; + } +} + +impl NV_TLB_FLUSH_PDB_LO { + /// Create a register value from a PDB address. + /// + /// Extracts bits [39:8] of the address and shifts it right by 8 bits. + pub(crate) fn from_pdb_addr(addr: u64) -> Self { + Self::zeroed().with_pdb_lo(((addr >> 8) & 0xFFFF_FFFF) as u32) + } +} + +impl NV_TLB_FLUSH_PDB_HI { + /// Create a register value from a PDB address. + /// + /// Extracts bits [47:40] of the address and shifts it right by 40 bit= s. + pub(crate) fn from_pdb_addr(addr: u64) -> Self { + Self::zeroed().with_pdb_hi(((addr >> 40) & 0xFF) as u8) + } +} --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011054.outbound.protection.outlook.com [40.93.194.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFC032E040D for ; Wed, 9 Sep 2026 04:00:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.54 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926411; cv=fail; b=LmSIo4y6uvfoejN/8gnarBSB9gOL2iRxmzcWXkLI3fLI5pH2E3kalbwxmpovXdr1/IXSoaoqnYJH7i8Ej2wlfAu3fhk4CncBDdJ96n9s5cU6UHsCqi+wY4dAO4AgLWdytJdwKCTEfU/Vj4KQ3XBjfc1eM8a0E4yRh3brZfdynkE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926411; c=relaxed/simple; bh=B9udrx+Eu/wdDgHz7Jy2a0erhwpZ+2R3Wgos7UPa8VQ=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=a9EkMuKbCXhhhOKFu7cA5Bl7qkadGJkC8xPYbXTv6FCb0FiqjjGMg20UQGtrdD8YI3b0i+VSJiizJCwy9O3Ecdr/azJaHugMINMMNO13druA/cG4qtaXDbWc4PdgQPlZxYCxUq4/ljPrmil3SFM9GACGE6PwEu3Qeg3KOAAXa3c= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=h+h2Tyuc; arc=fail smtp.client-ip=40.93.194.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="h+h2Tyuc" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=V3t6RgffyrgDsTUPPqIrGIwjQv671YneQFYAVOb49buB+7uDiPeSJSk4lQCUSe023OoA7IH9bC4qaSO/xQuQKRWF/WpZD1h9x1XMMTwtYNgZae2KjfnK0a3vhj58d0WzqZMUB6QM5uRTx1nZlQ0Ax6be5lbYEVigOEAjSRidbM5OtAA/aMuQn8T7+odkq7tJT93BwCWfvglFl4SY/AdOfmXtBlkhD5fWXx9i/XFBYLa8BdueP+h2paC3ej7yMPV0DFVWiagwwYpAsp46x7bAteZ9buTGGqdCrjY6oScqbfNAIv4iARg1xx0Ap28kCJ7Nu8VGA9KifD71ZGdMId2Sgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RLMUmfNIdwkun3p2fu/8VzvsJYUV7eIu5MgcHJGCq4M=; b=AKq7HD3TD/jtVeDZ5yu0vvW3fS/645JW1CZCy+VmgejlNqEMJXQLoRRsFQ2pCGUslsSiN7QplFb+xvO/wRTR5pe3xk5l2Spiiil2impSBAm5H+3BwFhwJPRycN7lV3fP4p8obQXHf0AA8x5lDSM6Nk8RYZPDzQMLhki1bY8ZlTCxoqQOsVqAfjbGWJiHfjVN0BCjtzMfkuy4xOF6nq9ADYJmOs52cPb7+tajjQ4hqGUTAyOZ8tODrHAJmlJ3EoydcCAHAWd9UyJv84hIPHAeDWMeqUnEAuWsXYzyNTGcylj4vmGzfu7RRK636LiNxZNBsS8+YW1YAtSkdric2kQO8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RLMUmfNIdwkun3p2fu/8VzvsJYUV7eIu5MgcHJGCq4M=; b=h+h2Tyuc4Su0DP7kyqWs9zPrsXyQ9xUCFJ9muils6BeJ30ggDZdsAybqufJs9BtzHzKV2v7wKHU98djxFfzOcmtAITBxV9jgac1eoATXnq5XHaFpXNu3hSGw6KV+QYfkRQWjMMfoSs2XB8of1qGiru/ZRfjhJpknkMGovQNTZZBfhhusRB0rxGE/CCdhZfkku3w+sO25HvE5GoWB/M/GbH4XFJOZBpn/0lThroPAgV0TL1Nx0Jln8LCBFcmsXosHET7OrzsqUDzMp2yD8AynrP7Ljd0fDoZRxgwzZR+J8PCYhvzn9O4Jg79fXBmCzQfImLk3VntkOiqa7ReIdoUS1g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:06 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:06 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:41 +0900 Subject: [PATCH 03/16] gpu: nova-core: mm: Add common types for all page table formats Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-3-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TYCP286CA0104.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:2b4::20) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: f288ae5e-60f5-4d82-0428-08df0e26d577 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|6133799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: m4MdYnAoMSgV0vioB4UOga9NMSSMqkXHe8PYLM5XUmJaKM0iXiRK9b5p0+M9YFdSVV2SVMZnmT98M5f+1Vu9jrPwe7pUzN2COKWCiDTONwH51D75bbC9MJv0Pj9aw7AZzbGoCRcsnqkMQesBBsOsIa/FMFFL+viVNyZXXgSNrsrZo8rGx898sAYc3RgtZ462szE7sIdBQ80//Tjt3hK4s0ryX376gcUAcIiolfYACqrMYbloUHIR8gd3EupTZHIziiPBQAIvkba15shqI/M0xRnneiN+Ksuy2UnqovPJNXuLnAEZcBHbjXdZMjN/u7FMPVQvqMVbvjO/AnK7MD5yIWXsveAohX4UggfzeGjMNqsUfGe8WD8edmVvtzmlNijTcitb9TSxqgw5PaMZ7YuCZC3zwLfYhpCPtHPqlkSN6rX73Q0uXG2zuGn0Nik6gsLNeC4/ZXMDN9+eE4tSm3ovUFk43dvANdUP4kHHupJNM1g3mnKZBrruYxN/E/24OmbK9O2zZ97TeNHW7ihLM20mB1G76n3PnPIGiu4xZ8ddFChKIiT9F9C3R/ZFWjRsHmuAcRBErRU/wTBPkw50G0fImzR3mpNFCCnkVl7Dbmk+gzvuWcZ2/MZtOS8Soc86BO+Hw5WiYxLPNUO/FwjG5jUam22VSw8NJmAGsgpVV45e+2I= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(6133799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?akFwcmhQZ3NjWFVSMzlaVWlkSUJGd1hpOThLVXBneitJY2RlYnQxalFmMGxy?= =?utf-8?B?VzB1NGMvOE8wL05taTROYjIxdk01alZrN0RNMFp6VmFWbFo0Z1UxMHcxRFk3?= =?utf-8?B?SzRJNWd5RU9BRzBoM2RJK0czZkRqY01JQnAvWlJEVVRQaXlrSzJ0OUkzbXJ0?= =?utf-8?B?Q3RjTys2RnpzUVhrQ1QwMnFLRDVLY2RWMnRnc0pYYmtzdU94TnRWYkNXaG82?= =?utf-8?B?RGNMQVp6U0xCaXNCeEdoR21PN0FLTkoxdFRTcEVKeGU2cWVRTUEzbUpHMDhv?= =?utf-8?B?Vzhuc1U0dnBOUXJGOGZSUU0yUVpjNjVkTHRxbmpqTVRxVm00QnE2SjhqRkRu?= =?utf-8?B?WVlTcWt1eDVFbk5HNU10a05ubS83N3hCeVJLOSs5U3BoLy9YRTkwM0QxcEl4?= =?utf-8?B?aHYrNUVQaEkzR0g1aUE5d1FoNHZiQmcvMzBPOEJzbFNnSXpwYUlBMTJZT1gy?= =?utf-8?B?WGtqZjcwZFByc1ZlbVl6SnR5djE1blcrUTlyaWtFcWJlN0lGYUNMeFNybWEx?= =?utf-8?B?MmJPMmd6ZFEvOGxFblpRR2NyMGsvYmJ5czFIVTFpMDdsSGJrWHhFL0h1dlJs?= =?utf-8?B?aU1nZktpTXRaRFoxdlMvcWZWc1ozWTZ5dU1wcndpQy82UjRoNDcxaDY2UC9u?= =?utf-8?B?UEpXTE5mU2dHRmp1U2Jwb0RDQ1c0QmtBVXZyQmNLUWN4bzBhN2xkb3pxY01a?= =?utf-8?B?dFA5Wmk0RHhXNXo0bmR1dzZ1VWVqRk96bmNKdmxYMkFaampjOEJOUzBtRlJK?= =?utf-8?B?SWpKSG02MzB0WXkzdFk3bDR6aXQyY05LSHVHVEE0VVA2T01jUUJFUlJ4bzhl?= =?utf-8?B?MTF0dVExcERVNmdxdTdETDYxY0EyZS83V3RVZ1MvNnlieG0vdk5mUVVnVzlq?= =?utf-8?B?dUVqM3prRHpOWnhhTk1iT0RNbkR5QmYwVW5sR0Z0TExVQ1c4djVtSEZmSG90?= =?utf-8?B?bSt3ZHRSbndmZ1ZUdFV0OWlKZW4zc0hTSlZQbUYvL3dkcCt6UUpLRmlJa0pi?= =?utf-8?B?bnJWZ3lEQU5jcEViZWovUlZ3R3dTR0lNdnpyTFd5SEJQS0E1QnlFZjA5ZTVD?= =?utf-8?B?dytMcEF2YlY3V2dtRkMzdTJMSFNDYU1qVnp0RTh0VjlmYUZJcktISjVBeUMr?= =?utf-8?B?SFZpNStYcnJqTTdTWDJET0NGN3h3KzRaWVlianBkNU5scElqS1c5QnVDN2xM?= =?utf-8?B?alF6dE82WGdZSnBzMnhhK0l2bG40TWpkUlBKT1k0ZmFxSVY5aXhCaVFTYUZz?= =?utf-8?B?Tk5ERFYxeG5ISVZwOTJ4RFdqdFpXclkwcDhlMkJyWldvYjFrWVBEZG9HTFZh?= =?utf-8?B?VUxjQzJXR3U0eVRWaVVKSDU2K2RDanNQVisya0lDREY3U1NHTVZDUHdUNTVs?= =?utf-8?B?N1BoZnFSRmlid21vaUlpOWM2Y0UzS3diSVVoeXQvbkFlNm16QzlucllFU3Jp?= =?utf-8?B?NGtxclR0NmJYM2RSc3ZLRkJ2MXo3ZThKMy92R1Q5RlpEdFNIbVRISnQ2SWdF?= =?utf-8?B?aHhpTkducjdyNXBCbnF5d1hQaWc2aHArcU9jV3pzNzROcW82MUtRSnVMbzVL?= =?utf-8?B?ZE5SRWhqWHV6aVkraTJlTjdqdU1wclhacEpRZDUxZkdaa1BIa3ZuN0w1YnQw?= =?utf-8?B?SStFNkNvK0ZUREpuNnBxU3h4MTk2TUQwbHhyVy9nY3RJTmlxTjM5bmIzWGlH?= =?utf-8?B?NGIwdlJLSFlUWWNuQXZ6QmpLNVBxMWhwRmJSc2Zub1lGY3N5UkN4c2d2TmVN?= =?utf-8?B?R3NJLzVmVE8zRHRyQS9QTWVadzRBa2M5Vkwxb092TjlkNTNMVTJEencyODU4?= =?utf-8?B?Z0cyM0hVdGJVZGRsalpjcG41TERDNGtlSWNwaC85d0RscnpyRlZSVjVtR21v?= =?utf-8?B?TURCSEU4VEtBOWtKUGttRUdaakNMLzJtRnNpWlJzWEZxUUlpdzkzb0ZROWVr?= =?utf-8?B?QldFMUw4V3J1R0ZIM2FCTzRONVZucnExNEpWc01uWC9SbkNBdUJHYVpOTkVq?= =?utf-8?B?dURpTVNCZVNKbWtMd0pIZE9aWWZsK2VRcWk3bFF2S0hZT3EvUG1QaEVDWWh2?= =?utf-8?B?bDAxQVVhaXBlNGJFNkxHbmV0Vng1T2J3bUxjNE1pVnd4SFpQRkNKVjkyZG1a?= =?utf-8?B?SmdUc0c5Q1RxMCs3VmtDbVNuMm1rbG1LcG5oeW52WEdqS25mRzFXWmQ3Zzd0?= =?utf-8?B?b3NSTnlyeXp3TW8zanRwVEl2OXQ4L200d2pLaHMxb3ZJekNtWGNmZ1kzWFVp?= =?utf-8?B?elB4SDBpd05lZFU2ZHFSN0F2aGZMbi9PdU1MUnBPYWppUVFndVU0Z2ZjL2lG?= =?utf-8?B?WWtlaDdaNjYwSkl2MndlM2FHZW1rQm9MMm1LQUNCSTBBbTlOUi9jbEN5MndX?= =?utf-8?Q?mLG/ft/477KAUK/cflyvlnva1kd5YC6Cn6xIccMKLtYeD?= X-MS-Exchange-AntiSpam-MessageData-1: yZk6VCX6dIKjJg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f288ae5e-60f5-4d82-0428-08df0e26d577 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:06.3850 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1ZSxIY5U9KZEhOwqY0/JE/1ohQOis2SprFBkxhc8MWDKtWeqRrajCRze2cUc6zD+gREFyxp4VHABSWxRb5lBYQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions. Signed-off-by: Joel Fernandes Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/pagetable.rs | 158 ++++++++++++++++++++++++++++++= ++++ 2 files changed, 159 insertions(+) diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs index 202bb8f8ebed..377cc9d44702 100644 --- a/drivers/gpu/nova-core/mm.rs +++ b/drivers/gpu/nova-core/mm.rs @@ -61,6 +61,7 @@ macro_rules! impl_pfn_bounded { pub(crate) use tlb::Tlb; =20 mod hal; +pub(super) mod pagetable; mod pramin; mod regs; pub(super) mod tlb; diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs new file mode 100644 index 000000000000..ed0f3d731c63 --- /dev/null +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -0,0 +1,158 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Common page table types shared between MMU v2 and v3. +//! +//! This module provides foundational types used by both MMU versions: +//! - Page table level hierarchy +//! - Memory aperture types for PDEs and PTEs + +#![expect(dead_code)] + +use kernel::num::Bounded; + +use crate::gpu::Architecture; + +/// Extracts the page table index at a given level from a virtual address. +pub(super) trait VaLevelIndex { + /// Return the page table index at `level` for this virtual address. + fn level_index(&self, level: u64) -> u64; +} + +/// MMU version enumeration. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum MmuVersion { + /// MMU v2 for Turing/Ampere/Ada. + V2, + /// MMU v3 for Hopper and later. + V3, +} + +impl From for MmuVersion { + fn from(arch: Architecture) -> Self { + match arch { + Architecture::Turing | Architecture::Ampere | Architecture::Ad= a =3D> Self::V2, + Architecture::Hopper | Architecture::BlackwellGB10x | Architec= ture::BlackwellGB20x =3D> { + Self::V3 + } + } + } +} + +/// Page Table Level hierarchy for MMU v2/v3. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(super) enum PageTableLevel { + /// Level 0 - Page Directory Base (root). + Pdb, + /// Level 1 - Intermediate page directory. + L1, + /// Level 2 - Intermediate page directory. + L2, + /// Level 3 - Intermediate page directory or dual PDE (version-depende= nt). + L3, + /// Level 4 - PTE level for v2, intermediate page directory for v3. + L4, + /// Level 5 - PTE level used for MMU v3 only. + L5, +} + +impl PageTableLevel { + /// Number of entries per page table (512 for 4KB pages). + pub(super) const ENTRIES_PER_TABLE: usize =3D 512; + + /// Get the next level in the hierarchy. + pub(super) const fn next(&self) -> Option { + match self { + Self::Pdb =3D> Some(Self::L1), + Self::L1 =3D> Some(Self::L2), + Self::L2 =3D> Some(Self::L3), + Self::L3 =3D> Some(Self::L4), + Self::L4 =3D> Some(Self::L5), + Self::L5 =3D> None, + } + } + + /// Convert level to index. + pub(super) const fn as_index(&self) -> u64 { + match self { + Self::Pdb =3D> 0, + Self::L1 =3D> 1, + Self::L2 =3D> 2, + Self::L3 =3D> 3, + Self::L4 =3D> 4, + Self::L5 =3D> 5, + } + } +} + +/// Memory aperture for Page Table Entries (`PTE`s). +/// +/// Determines which memory region the `PTE` points to. +#[repr(u8)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub(super) enum AperturePte { + /// Local video memory (VRAM). + #[default] + VideoMemory =3D 0, + /// Peer GPU's video memory. + PeerMemory =3D 1, + /// System memory with cache coherence. + SystemCoherent =3D 2, + /// System memory without cache coherence. + SystemNonCoherent =3D 3, +} + +// TODO[FPRI]: Replace with `#[derive(FromPrimitive)]` when available. +impl From> for AperturePte { + fn from(val: Bounded) -> Self { + match *val { + 0 =3D> Self::VideoMemory, + 1 =3D> Self::PeerMemory, + 2 =3D> Self::SystemCoherent, + 3 =3D> Self::SystemNonCoherent, + _ =3D> Self::VideoMemory, + } + } +} + +// TODO[FPRI]: Replace with `#[derive(ToPrimitive)]` when available. +impl From for Bounded { + fn from(val: AperturePte) -> Self { + Bounded::from_expr(val as u64 & 0x3) + } +} + +/// Memory aperture for Page Directory Entries (`PDE`s). +/// +/// Note: For `PDE`s, `Invalid` (0) means the entry is not valid. +#[repr(u8)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub(super) enum AperturePde { + /// Invalid/unused entry. + #[default] + Invalid =3D 0, + /// Page table is in video memory. + VideoMemory =3D 1, + /// Page table is in system memory with coherence. + SystemCoherent =3D 2, + /// Page table is in system memory without coherence. + SystemNonCoherent =3D 3, +} + +// TODO[FPRI]: Replace with `#[derive(FromPrimitive)]` when available. +impl From> for AperturePde { + fn from(val: Bounded) -> Self { + match *val { + 1 =3D> Self::VideoMemory, + 2 =3D> Self::SystemCoherent, + 3 =3D> Self::SystemNonCoherent, + _ =3D> Self::Invalid, + } + } +} + +// TODO[FPRI]: Replace with `#[derive(ToPrimitive)]` when available. +impl From for Bounded { + fn from(val: AperturePde) -> Self { + Bounded::from_expr(val as u64 & 0x3) + } +} --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011041.outbound.protection.outlook.com [40.93.194.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 960AC3537E0 for ; Wed, 9 Sep 2026 04:00:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926414; cv=fail; b=lqL7fphKLkZys2m50fVfkIo5KzBV/zkhMLYtBJvc6Xf9ybSD1+ok/sd0R+LA8u3buHV32eYWH82xOi/1Z1lgtcYrcrzzec06O6uazz7Bcjx47eCOMKcwXEkoO6wd6Gp/xD0nsooo9WLHiwRoLOiapcOoXNBUFaqVleDBcNeScL0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926414; c=relaxed/simple; bh=GwFIS50Ch8DJtE9jEVr3O1oZHTAvBCIgTzivvl8Zdv0=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=L0zXwP04DoSrgbmTTAD7IZ/wKsrJNQrzENa2AwrGAOVrl9W8I7/opDHhxiTjkttA5za0P3A25kxhuKkX4joOUEOPeV+cphMFXx5RTIKDWxgq+EdlxWsnZpSNCTagPjrdRvz84ycXctyFH271WSeftc/hlx/aGvuOigipi+WUaQQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=pzgWq3IW; arc=fail smtp.client-ip=40.93.194.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="pzgWq3IW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pWtYX1wCFOXkLuCQPJtH6+O/HBjkx5A6xrKSApJk9RMcOECII0yzqKSDJOFgYj8p5tnL4/S21RJpMpZ8p7DXDg67xRKyiMH3gIab8aEkSxUdXpfRV8lJxZ+ygZSI3DlNpGR8PTKfFD+y+SRqEYiF7AQg4zgulrrirJ/3EXy1pqgTg4ySadyuBQoRdRmdP7XeXCPDTq9On8V0FvBqYKhy7M5KUV2rgAKoBVvfaidjK9F4NI1hT7IrlZ0wP1nq86iIHd1KIZY6FaazhZZpL8jaGz/A4XXQQWSQMJ+4/9lOpajnuSBSc5GLrWIE8GiPhMzVi5NkNH+nTY6I8xAfyWmx8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ibCIauwU7qaVH7uCQs9Y2szGfAw/OArMlvmGbF1dy74=; b=TR+PyPv4NIdwVmMMR3hHMxsBv1K40nqM7i4vUXSKHf3LOIQdiALYmwEVL+b3Bk33apYjCp/QSd0LB9oiLF0YlOTul/+o8IE2pWeFw314mApwOTErtpNFhk82fYsLtzVVXMrwubbGdSfq89MFpclNV8bGXghiRYa59B/8kq0J4l79cxKHJ9xV8GRK3C6JHTW8shNvyNqbzT+SrfYSyQwHtXMMrKG7r2iJ9CIzi3NKV8ga2QF3cW1BSyWVrUtlCFG98dcmEjdpwKbDPIXLnmcIjjc++EXgWn/PX0ryS3+iw2H0+bb5iTBmCtjpcTn5u7j6gn/xXl3ubbAiunjODQwdEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ibCIauwU7qaVH7uCQs9Y2szGfAw/OArMlvmGbF1dy74=; b=pzgWq3IWQhZsw6wH4bqFlv2roQv/RhGwHEbRrcbvGVZgMnKy2oYPugj81J5ZtiIKygh55efzQCTxI9ea3POdKa4YEXlNDs8Pi9qIBYtyuObdbEQYz6nT07RcCuP/Oq9OGONluJ6GtYErRGBcYk4skNYOQOHCiemDfTF43T4H1stF+/JKUWg4t1L61B6k/gppciuIFwvsAkYNPJdDcX9UZtmNxpfWYtMi4Dr6N1yWfQMBNZrzA5jXMDUQ03lauxWcqiuNO4WCw7mnc6alYJPe8b/xqzOLeJ9CL9EaDxrP4A2w2gge83iNmHQqnjfgqMOUH3syMvNsk9ub2IK12Xqrdg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:10 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:10 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:42 +0900 Subject: [PATCH 04/16] gpu: nova-core: mm: pagetable: Add PteOps trait Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-4-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TYWP286CA0018.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:178::8) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 24a38e10-20f3-4080-acfe-08df0e26d7c2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: wBVANaYUyHGV72NAPHp/9T/VHWVuSFa5OXVSfVp8Yvk7kxXpbgOrU3R9cTv5clvH+VPK6E0g40OVKhT9f4zdoZWfI1uY8VcF/urqbge5rfRPuynPlHMQJtUrQbjApw5f5hyVO39Z1vgZnN6PyCVo6M59m+wMUZL+iIPRYCgqQ9fVZnPhFqDDMwuhvU9i+kmYknm9PHASbPd3yhyNpYNnsHPUECkcIPGZ/OgCoVumqFA6kepDPgdVQ2iDkq6vygoITDBmG4I4PbN4H/kczHQC/jyE9aRuhHi7gdzEjNXWSSFemzeUfwEInmRIGzUDU8a2y+uBjlVp/TrU9ceezLSxv6tVVTuDfGAN0oPbQ87Bl+UPeDXQDFJl22IuiP8eWT3Utj38PjGE/vfpt/aIshin9CUa8llqreqzUN8Dx/x1VlKKIDwpVlvDh3ZxQ2+fjjMGmVhItwlgXHFN8NXAfEP0estBIOWbuAPhCcJzuooVHz/z1hKkAUCnXIR3yAINxqofkXudMm7reuljBIt7XK/vYN00iO/s1afFhw8dXsD0hT5FsAGJT0D+t/w2T4nO3qr/oPL3QB6pSPGaS3G/PzbuQeN49UoR58rDAhUuVTr93bBBwCWd1xE2U6g6gMco5qaBs3ESKtFO/hlyHcb5ige4lLZxLUrv461mZlCpSCpkPnQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SmtFZ0NCK3FnZmFNOXNRRmxUQldyODZGNmY1bG1VeDViMzljSDZOYWpSdFpE?= =?utf-8?B?dXMrdHovdGU3L2ovRVdwd2FNNitoTEN5dE43WDZQaWVVb0UzMHc2WjJRVW8v?= =?utf-8?B?TU9NVnhtUU5reVBMam9OZHlkNTV0NkR4T3h0U3NMWVRRNTZqVkhBL0hPSDlP?= =?utf-8?B?eUtYc0o5WkJjTHBEQWlWbmU3dFpaRG5iNEs0VTJxLzhmaGxjY1hXaVVKN3Fm?= =?utf-8?B?dXVrYnhuc05rek9jRU8wcDU2Ynp0TWdiL2tud1ZFM0cyS1dyOGtyMmYxNmFB?= =?utf-8?B?UDhNdWRVUFZaWWZZOHE3Uk9CNFhsc3VHN2MzQ3VzWk14bnpCd3BTeWlBcjU0?= =?utf-8?B?TitFWWx1L1ZXZFN6cEtiR01zckQ0bGNHejFLeVBLQldENzdLMjF5VkhYYlVT?= =?utf-8?B?QW92aHNvV1VwWlpXc3J4ZzJIajBaZDhGRjExYzdoSDRMM252QlNUV3dQRlhl?= =?utf-8?B?MTJSRTkrWlc5dlQ0ZUQ1bGg4SVhiRXBTN21QRnQ1U1Vma0RKL0xKSy93aS9Q?= =?utf-8?B?SEcwZkd6QXppOVFSV004ZVRSQ2FEZEN4cWsxNURKRmxHMlVwR21YSDVhR2hj?= =?utf-8?B?azNaOFkwZUdUWlRUMERiNTFwckZzUVo4UEQ3a3dCUE1GUE9aMStEYkRHMXYx?= =?utf-8?B?VDN4N2l2aXlDYU12blBuNlVObEZoTHBKYUFEUWdVUW0zTFc0UHRBbzN0SkhK?= =?utf-8?B?TlVDcUNBS3Y2bnBSRDZYQ09neUVkeE5EZHVPSCt4Y1F1Q1lVc0huYi92M2FM?= =?utf-8?B?ZDhSVFphWVB4cGZvRS94YXd6YjRxTTFMYUtOai9ISTBjQjliZnMvS0wwRW1K?= =?utf-8?B?Y1RES0hncDZDQzNKdjlvL1h0VUQ4cnN0bUpoK2lCZVBnMzNSSU03RysreG5V?= =?utf-8?B?S21CNWhsdlkzcWdaOVlFNWVIb2hKQVZVbUhZWEpBTkVPUk13K0Z6Ti9nc2J1?= =?utf-8?B?TlZWaHZTb1R2L2pKakJUeDlNeUQ0WE1sRjZKdHRXWm9oQUtDVWI0cDBTcm85?= =?utf-8?B?Q05CekFtMUMxelJpeU0yeFBMMHNHV0dVQ2J1SmdySkh6ejZpQjlUK3lIWGVs?= =?utf-8?B?dEpRdnU1MVZjTy9rU2l2bnl0dlZEZDNvcVNXeHNPSlVocGdEbDhyb1psN1B2?= =?utf-8?B?TGdtMGx0MkJaVko3Q1NHNVNEWHNnSXpNOTNkZU1xQzVXTks3UEFHS29sNndr?= =?utf-8?B?VXdwdFRYRnZJOHJUaWZpdVVGR0l5YmorQmcyS05NYlJPOXdSUFRTWjNzUXRE?= =?utf-8?B?ZUZEYzJCeEVQYXRFQ29TZlRFVFI5WjRZM2g4TUg5V3llZ1UrTG05UHdmdVRT?= =?utf-8?B?akhtaGxTV1FHejJzUFZwajVMVTVibWdlb0tSdlhCVlNreWl1SlQwYUNVdDlE?= =?utf-8?B?TVFjVzN1engyVTV0cTBZT3pEbmVaT09naGlvNlo2ZmpaN3Jxc05RbGJIT2kx?= =?utf-8?B?Rnd1VnNpZ2k2a2lobXlkNWZvSnFibVp0NGI1K1krRkhpOVpDSVREajJiNDBm?= =?utf-8?B?MjY4VFR1cGgrWUhpMDVkYVlqdEVCa3RDYitCVDNmU2dmbHR4ei9PeDRDaUU4?= =?utf-8?B?Z1l5T29QcUYvYS83NEVrSmZJeE9QWTdnWHpDd2Q1Mkt2RUp4M0hXWnJwcjdI?= =?utf-8?B?YmhsekpwTDFXVkxMOXcycVE3dE0rY20vZE54ZEJ5emkrSmRSWEpVWHUwWHBH?= =?utf-8?B?Yi9pN1dRMnFURm5CcTRYMjI0MGhTQWhjVkw1ckZmV1lua1BudjhZOC9SdC9J?= =?utf-8?B?U3JMOUlhYWZqZXJ4ZktlVDBNa0pYZ3JJQ3o5RnVscFE3ZTRjMEVBOGtpSkdz?= =?utf-8?B?UkNFOG1zS2RmdFZRWHk2aTQ0WXJXUUNtN2YvUkpDcHJoNU5wekJ4OHVrRU1o?= =?utf-8?B?OEZLRnVSOWpzSExvVzZ0TnkrKzFrNlhWckFDMldrRlRYd3pwV1RiQ2s2ZDZD?= =?utf-8?B?UDVReFk0Y3BnMWtPTDJmLzNKREUvUUovN2dJUlFnWFlIRDdhZERrMnl0QkNN?= =?utf-8?B?M3lsa1JnZnpsMjZRSlZOR0Q4MUpWTW03d2JPS2c1ZGg0TklDMmQwTnNxL0lk?= =?utf-8?B?SWtyMEhpeVFwdEN1SmhURWh4VjJHWW1hZ1B3MzJkREd0ajBwSkRleG0yaDM4?= =?utf-8?B?Wm5QWTI3MERxYnBTY1premhBSWxHYUZ5bG5GNTlPT0dTKzZYZjl5bHVXdVE4?= =?utf-8?B?ckJnRXliUi82TVJHbW5IdEZUekRYUlVVbTZ2ZmU3a0hwOUhJM2tFR096WVl0?= =?utf-8?B?K3c1K2RTKzc1eDNRYU9wV0hoQmZLbHI1K2hnUnVRQ2NnbCtuWVVLUjFJVkQz?= =?utf-8?B?VUlZL1h2L0EvU2pqNlljaEFMRTBqTzlsQ3ZJNmpma1ZQYU1GQlZ1aVJTeEQx?= =?utf-8?Q?fER3Jb6crh0x8k8a/yjzm5sLg5zYRPyyDTZHDceBkpmik?= X-MS-Exchange-AntiSpam-MessageData-1: MLmjxuG24KouMw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 24a38e10-20f3-4080-acfe-08df0e26d7c2 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:10.2905 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LMnbG3USEdZy5U+eRLZJfuaxOj2vAZpTnNHaOwh9X87njulmwkFBk6FJOa7zG9qaSsnP0P8NuDZ3oOIRt6LKwA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Introduce a trait for GPU Page Table Entries (PTEs). New `read()`/`write()` helpers are provided that go through a `Pramin`). The forthcoming MMU v2, v3 PTE structs will each implement `PteOps`, allowing the later page-table walker and mapper to call PTE operations. Signed-off-by: Joel Fernandes [ecourtney: replace PraminWindow accesses with typed MMIO views] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 46 +++++++++++++++++++++++++++++++= +++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index ed0f3d731c63..a23370b4b62d 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -8,9 +8,18 @@ =20 #![expect(dead_code)] =20 -use kernel::num::Bounded; +use kernel::{ + io::Io, + num::Bounded, + prelude::*, // +}; =20 use crate::gpu::Architecture; +use crate::mm::{ + pramin, + Pfn, + VramAddress, // +}; =20 /// Extracts the page table index at a given level from a virtual address. pub(super) trait VaLevelIndex { @@ -84,6 +93,41 @@ pub(super) const fn as_index(&self) -> u64 { } } =20 +// Trait abstractions for page table operations. + +/// Operations on Page Table Entries (`PTE`s). +pub(super) trait PteOps: Copy + core::fmt::Debug + Into { + /// Create a `PTE` from a raw `u64` value. + fn from_raw(val: u64) -> Self; + + /// Create an invalid `PTE`. + fn invalid() -> Self; + + /// Create a valid `PTE` for the given memory aperture. + fn new(aperture: AperturePte, pfn: Pfn, writable: bool) -> Self; + + /// Check if this `PTE` is valid. + fn is_valid(&self) -> bool; + + /// Get the physical frame number. + fn frame_number(&self) -> Pfn; + + /// Read a `PTE` from VRAM. + fn read(pramin: &mut pramin::Pramin<'_>, addr: VramAddress) -> Result<= Self> { + let val =3D pramin.window_at::(addr)?.view().read_val(); + Ok(Self::from_raw(val)) + } + + /// Write this `PTE` to VRAM. + fn write(&self, pramin: &mut pramin::Pramin<'_>, addr: VramAddress) ->= Result { + pramin + .window_at::(addr)? + .view() + .write_val((*self).into()); + Ok(()) + } +} + /// Memory aperture for Page Table Entries (`PTE`s). /// /// Determines which memory region the `PTE` points to. --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59A2D357CED for ; Wed, 9 Sep 2026 04:00:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926429; cv=fail; b=mptb2Wk4HEC0MPgES5q34ib6IWSMIPFENlhAKKCjLTU/OFWEx8xmH7tuN0t32JOEBs/WtOs++geavYD1u4wsHdNm/FkNE4IOeHjD44suXer9HmfqBglc4WDmgsbNgPEWHnaeFk0YzSSdvkWHIAXrtP6vARKTwD5igeema13NbcQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926429; c=relaxed/simple; bh=gWbIrbeHXoqAcRbg2BMzkPrHYGdIuHhaQCCnbheXHY4=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=ScMgt/Id7BOjlW0G5rsBedjjAiv97D2sxNkGrFDPxBj3hAllePooDdi5onLRVLHqe1jdWDE6jDhWyCrbj0DDRCKdF741A85KIJPNCkP117f6KYr9Qz/oUGCmyUmyMBxCRWsRZMnipHWR1D5OYee7E2MalgebnAWaHVBZBrPtdAU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=RFdi4aPf; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="RFdi4aPf" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=v+MH1O7Yi1sZKujudg80aZlcfumhig6AZuj5tILZy4Nv20ocMJ2+Kyp5nHtDLqwfqNoUMSAvRyKHv4Wng7hnunNwU6taq/+1AGmaOxQQgohoekEniDLw9CB69OfjEhoPQVxxnqgRYZKGfYJAAkcKpf9wXHelNn9QbnuaJ/5MpHOD//kYC/VkqPTn32eLtj2TG1ZFK3EXVvh/oND1qg0yALp2hbl3VoO99iRlfsb8vq1v7hcadX4QIGJaDnICrXE4VkhxltsLL1GGE6go7IaweqjVSJ4d1dqqXR25RwnBJcQsiCmacLBaAf/wZzI5Gw7mitw6njj/X1rQJSn2D1T9Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Gq6PNDh30qXGFAhiOvrT9JLq0bhmSqC5EjeaLC+1qqw=; b=q+GHY3Vr/+/urt6sWV/uPFZrIxT9oc9RsuV4jxDbgDFzp3utBmBKElimX8/rmy97YLJ/d6s6Qzs44L1KCqeqD+ukKXSHGnWg/+nPiZscyf+yc7kRx/ANr9/mmNzQEUfMXZjGtAPQZvJ9zsCHZ/OavgDMLCDAUzJOop7qjQavr1X3i5M/AIb0i3RNeB94aLBNSYcKO6PoDCClvxmkQ8WtTx+qpWHS5OE8Tkz6gfHmS8s2ohGNhYJLIvkkv5erStO8cRSq6CduY7WDjp+YCUp+2px9qLujhCpte1ejGscCjP7wmzfelH2btZ8djPZ0BLIjv2lR8gHMqBhuvN2K7JSYIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Gq6PNDh30qXGFAhiOvrT9JLq0bhmSqC5EjeaLC+1qqw=; b=RFdi4aPfajydgVY1GfE9X7JxIXRRze4hx/46M+AVfkTOyZgFejlxi0cmF7lq2kFru7cLBe9oxyWQDd0vLHXWVBJtcQd9+9QKgDzyemdUXjV9IOWuCfjvDWFY4BxWNYZKe25csSoBlaQYoyvtEIkNfMDO1rntyu9wBENRayokbb90HKMrxktSNIJNYyEa7MWdM1HR+HCI7SqugPXVYoro6bwOhMOKxwWlHmbZoTr05dZJwLnlFPebcXXTeiL4W53RdehhETUlBGcAqoQh03Jch3fH0N+paTkNmJ7CkpM/djVYC2OynXiZ/TRklfDUcBlAAPJ+8lZt2TW/Xk1fHEDGbA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:14 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:14 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:43 +0900 Subject: [PATCH 05/16] gpu: nova-core: mm: pagetable: Add PdeOps trait Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-5-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TYCP301CA0071.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:7d::18) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 38360d87-26f1-4a79-ee5b-08df0e26d9f7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: hrgrg/PSLDQlcSMiFWLxH2aU0lP6Qf9OKmPNf2F1AyDPjElDtFKFp6uRE6YF3IC+d7kY8VC0rdosc1Mn0wUrO2kIsohvZholkB+Qw03NHTNkpMaKXiLJjAFX9msVnJHZo6tpKfm6LAvOLmzRmDDIex7VHdGlXvYLftaXavK/dcheT6mosjjYwBF/NUUykcvGwQi+wDSomiiCXGnxKmj+uf9V6wLAxpKskZzzj5QjsCu/Q4glWJRBevkoLR0aFoiY2oiVE3YPs4pyZyeh/LhpZ90eRDuS7+2RHnwxAag8QV+eegXEgrsSp1cKNP+0zzGapVLN+5jKPQXUZ0DR53WPkTmmhIiAsLcImt/HSQRVFAv/P8xc655LoQfucAUStn08I6g/XsuM7D1t2lwV9x7Q/5f6fHtZR4IZqy+WLX03PdCJLSK/8zafR38b0uDcB7Qi9RmKGGgkYvU8IMfjYsAATJH1W3ybqlaUSCmDNou4GnqrDZtjb3blOiFf+2KehSaNwzBu6lv0/cKGjbdgERuZRNXXUCIZFahk3YT6Qtcu8RtSgYEziAuiMTu48mt8g7jHh0r0UUzAHEGyNCjSj5K/RP6JTBlaiqDRkXKb9AhK3lPsRrsZ1Lg0fSa1QN07Ye2sSgruU/Q8/ORSjyEYj9Is8XwheqI97KA3WUgRmub3/vg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VGd3V0hwU2VGYzJHdHdyTG5scUR4djltZWZBZ05IdTFzYXR0RndHYi9oKzFY?= =?utf-8?B?WStLMUs4YkVFZC9pMlpETGtBVHN6UUs0NTFJVm0vWXM4M2hKYjc4UXVnd3dW?= =?utf-8?B?b2N4cnNqamRLT3praFpqZ1J5SGhNWlVsYnU4N0cveW94SGF0SkI0OU5SVmVh?= =?utf-8?B?aU1saHJXUnFyMmlmUWp1L2VSOTBwMHJlemxKM2tjZ3pkZ2JwdG1hTDdQMXd2?= =?utf-8?B?VVgrandTSm1TMUxIRE80d1NmeEM4Z1RZMy9udjdiaFpkL0NWbHBrZ1UwSS94?= =?utf-8?B?ZDIvVFFIaVNLKzVtRm91MTUxUmNIQlM0aFd1VkRVaFdLTUVpUHIxTG1sdzBk?= =?utf-8?B?eVFwdXg4WUQ0VGZtbGJJNDJPMDVxZUV2OU1CaEtwREFZOGE1djk1N1plYTlT?= =?utf-8?B?TTZXcVI0a0w4amtaRlNqRXJkZ2xGUGpMdENsSnZEcjVYV09LV3VpY3V6aVFG?= =?utf-8?B?WXRvSGVCaFl3d2ZwdktHTE9lUkpWYzJvZ2lyWWtqUEJPRHU0c3VBcDNBY0Yz?= =?utf-8?B?RHhIcGN2TUVvcXNwWllyVkZaTlhZcERwTytlVnVuRURnTERPSG9LcU0vL3I0?= =?utf-8?B?b2pKUENxL0VPb2NwUktzVFljK2pTa0xrUTRpeWtpWmJVTFFHQ0pMUDNNK1FK?= =?utf-8?B?Z2kySGhtTlRUcnJBZ0NQUnBuZnpVNHBkTzdJUFNKV1h1MlhhMVp5alZucUdy?= =?utf-8?B?Q2J5R1RmTy9kTDQyYjd3T3NLb2svOFhEYSttRDJlVFJCd1BjQ3E4NHIwZFUy?= =?utf-8?B?NkM5KzNKVzZHNW5aVm1FNy9UdHZKSnRVMFRseU4vNzdRVmhLdzZ5aHc3MTNQ?= =?utf-8?B?SmxNTlh0NmZ0RDNwSXNJYkVwV1pQcS9UY0xQYmNXREJZK0VHT1pheXpZY09R?= =?utf-8?B?amJ0aVl5SndhZ054K0IxWjRBS3QzWXpJczU1ZVhyUHdLd3hPNm9Gd1hFMXFN?= =?utf-8?B?V1JSdTF6SXBaNDNDY0djZjJ6alNPWEJXa2xFam1EZTVacjNpZUNpbElFcVRz?= =?utf-8?B?WTFVK2prK0lRdlNLNFJmN1JFa0VPaEdiYnZEK0ErMEp5bzltZzZIbnI0dDVJ?= =?utf-8?B?SkVvZTExOE91TllmNDk3bE5HV0NEMkN2anF5TmxpNGwwSk5kem0rWVVNN2Jp?= =?utf-8?B?RkhLbHpJS3N6eS9BVHhLRnpjaTUzUEl6QkJsYmFvUllMTHVPejJpUHdVN0Zt?= =?utf-8?B?MDRXT3pWRnA5MHFydnpKN0FaVjV1VjFud2ZEeldXVEowS2JiZW9pV2M3Y1lI?= =?utf-8?B?VlBwUWl2RVhsVXZKR0JwcmN3RlZKaHBoenFjSmkwTjU0OVowaEdXOHNNQlVI?= =?utf-8?B?TTFJQm52ZEw3MG1JYzNBdnlvN24wK3FDQlJ1SjdFbFpUQzA3aFJ2NU1hU2I2?= =?utf-8?B?OTVsSlF5VHI1YlU2VWw3azhsdjQ4ZlMvanE3N0I0a242ZzdpTWMwdk0rYURD?= =?utf-8?B?UkVKRUI5YXFtQW5Jd204NWJIZ01ERzYvaHFxakwrREhHY25mL0RaOUpnT2Fs?= =?utf-8?B?Z2l5OUllbE1ZSmExZkp1WWVMdUxDKzB0RjBYRzBiLzNDeGFXc2tVUlhYMnd3?= =?utf-8?B?VXFyRjBVY2duQ0FEb0RlOWNJd2ZEVW5jUHhlb21ONUdOSWJpT1BIaWFlbjd1?= =?utf-8?B?WXAycGFGek9UbEI5Q3dYcFIyWlR0Rk1XNkg5R1VwTE04dXJacld6SEZQNko3?= =?utf-8?B?akdGRVk3eWN6QTQwVGtUNFFnZG80T3Z0ck8xQmgzM2FNQkt0bS90ZVZYUTk3?= =?utf-8?B?UXBtUmY5NWQ2NGxKZStVOFdrbnFEY1FYTytuS2FWOG5GbytqU2hpR3dZSlJB?= =?utf-8?B?NnBnbEQvc2FINml0SWcvRUFLVjBSUTNVeXVJSmh0aWxJL25UampoZFYzc29y?= =?utf-8?B?bnVra1hKUi9sRU5XOUNscW1oekVNaGhVcGpFb1RwczY4RDRGUm1GbDBieTVu?= =?utf-8?B?amlJSnJxRFNyUG5rSWVhcy9vcU5sdkpBV1lMRTV6Z1lOaWtNek9SeTR4RFk2?= =?utf-8?B?SVpPTXpFd21ROGxKalVwM1RoNUZkQ0xPNU8yL3JscHM4dEJONzlTQ2ZUZVJp?= =?utf-8?B?c2VrSkIwVmRMZ29FNGFJa0lveS9uTTM1QVhxTllKSU8yQkVYMTdSWE1nMTZ5?= =?utf-8?B?ZGVEMXdLQXdqVVA4UEw5ODZOOGs3YUh3dW9wY0lSaExqNlU1MUxaaTUxMzNY?= =?utf-8?B?ektiVnVQazRScXBmbzVUU0RqK1JabFNObnVWQXp6ZlJHeTZmb2htK1VLT25Q?= =?utf-8?B?LzNNOG40cnRuNVZhazIvNzR2NDFPKzhlK1V5ZnUrRjdrelNkcEtKRnpndW5Y?= =?utf-8?B?YWdYS21XK05DYi9PcTc0QUFkM0ZObDhHZG45U2lsK2owVjJnRzYvcTRCMkVt?= =?utf-8?Q?h/0Yv5xgDBDpVPC19YkNfzwzaXr7hzyv7lgNwg19aiIyq?= X-MS-Exchange-AntiSpam-MessageData-1: /Y1xekYY8rtMww== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 38360d87-26f1-4a79-ee5b-08df0e26d9f7 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:13.9822 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: r4lW3mBGYqrK+KVwgttuntSOpmyMyIyZr3zNSmO+272RlxkW5GxeJqT69d+Y7wMw/KpVXh2ryP8Qdq10eMYIcQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Introduce a trait for GPU Page Directory Entries (PDEs). Default `read()`/`write()` helpers via a `Pramin` are provided. The forthcoming MMU v2, v3 PDE structs will each implement `PdeOps`, allowing the later page-table walker and mapper to call PDE operations. Signed-off-by: Joel Fernandes [ecourtney: replace PraminWindow accesses with typed MMIO views] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 41 +++++++++++++++++++++++++++++++= ++++ 1 file changed, 41 insertions(+) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index a23370b4b62d..14436fa6adf0 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -128,6 +128,47 @@ fn write(&self, pramin: &mut pramin::Pramin<'_>, addr:= VramAddress) -> Result { } } =20 +/// Operations on Page Directory Entries (`PDE`s). +pub(super) trait PdeOps: Copy + core::fmt::Debug + Into { + /// Create a `PDE` from a raw `u64` value. + fn from_raw(val: u64) -> Self; + + /// Create a valid `PDE` pointing to a page table in the given apertur= e. + fn new(aperture: AperturePde, table_pfn: Pfn) -> Self; + + /// Create an invalid `PDE`. + fn invalid() -> Self; + + /// Check if this `PDE` is valid. + fn is_valid(&self) -> bool; + + /// Get the memory aperture of this `PDE`. + fn aperture(&self) -> AperturePde; + + /// Get the VRAM address of the page table. + fn table_vram_address(&self) -> VramAddress; + + /// Read a `PDE` from VRAM. + fn read(pramin: &mut pramin::Pramin<'_>, addr: VramAddress) -> Result<= Self> { + let val =3D pramin.window_at::(addr)?.view().read_val(); + Ok(Self::from_raw(val)) + } + + /// Write this `PDE` to VRAM. + fn write(&self, pramin: &mut pramin::Pramin<'_>, addr: VramAddress) ->= Result { + pramin + .window_at::(addr)? + .view() + .write_val((*self).into()); + Ok(()) + } + + /// Check if this `PDE` is valid and points to video memory. + fn is_valid_vram(&self) -> bool { + self.is_valid() && self.aperture() =3D=3D AperturePde::VideoMemory + } +} + /// Memory aperture for Page Table Entries (`PTE`s). /// /// Determines which memory region the `PTE` points to. --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD68F299959 for ; Wed, 9 Sep 2026 04:00:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926431; cv=fail; b=AiyDP4iNeDI3njG5Vo3zBrbbmvCPDbl5X1mVGHyHalD1fkvez3z6m2JacxyxKS/VnD/62eeA/8Jcmapm/5LirQ2yWjD3I+uNmkM/PsRV/hHu3TdnAsfqJLI4jHHfzOxw2kP2gqNI0ia5Nsz9vLvc6WjPw7qNhub9yAmhiGifyMw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926431; c=relaxed/simple; bh=/EE17Sdksc0nbIVGPYk37MiVjQnfSmDa8FvkvtlHKW4=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=fU2Cy1i0Pqhuu2NljNdywrFm+7aGwAf/nlWKYobni2lmBrQ+5i4yplTw/i5weV6/UwPV4jLxZVbAP6nP65PnX1kAOAJIQj3QMUc4OcFEW1p+EdeaDt2/U1kkPfnN7WQamo4NFC1FePF98UdGYrX9yZqSxfW7xKkNDBYamzoYmhE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=XV30cD2t; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="XV30cD2t" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ACZuHCDd05sHXf+rn3YEiklLQ48OVS1jNlJrz5F7WbD4AK3HxVkLnf+geo3Fd8nabRX9/bH9Co/P9Z+3TcXQlI2En4xHA5nrfrEkIu0nEYxQXtyS9TWHmw3MSEmX6OLHBIDP+tH3eqlIaf5cd98QBfPWDy+/5GF84wIsawi4J5sYVm4pWaxw//DJmYbpUvZZzTNP5eCO29ivS2KcplsB+LPQHOjpixZqgGAHQUuHhg70A5BNuVAd3N1kvhl/cuJof3I+xHzJhH0p25kfp1akhqNYcXs6C6/yAaZaKFkwNmTSvgQtbf3WqBD4EqdUuvatuOVvOwalHYMi2raYH76lJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YX+dhapqTB6yMbeGniaEdwx0sxAYyQ7UwaIEktbkCn0=; b=JKlLTsQCmvMnsr5ccsf2dLWnOA0Af6iBgTvxHV4WlHUZs7bPpy22GXldDGwe+1LbxKRwa8xOxAAF1P50mIULrcHbVE+75u7/KOe51yeDaLBykmLlLEd0wzMsLcZFoIOhYs80pPD7lrcEW8ko0pP6FJ5A/KJhfp4WKYw7welEZ3cpCOgbcYPDS0ljfyh2hzi+QpyqjQIro/KkaHOh8huEO0mTKFcjn9iMGsdIt5q8HFeU+zcejZ60kCCB29K4EByD+TWzbc/Kok5Bk2ix7TDL0kSafEOkC4RM9BImp+xb64sYvdLaTIoCM//sQ4IgNmLsm0ztdzs2EGIe41ivB67Wlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YX+dhapqTB6yMbeGniaEdwx0sxAYyQ7UwaIEktbkCn0=; b=XV30cD2tGU66c/KRWj7Zbjx8ftPoajJYsVLT3Dy0Z8+KOIX+PTrpcPg0dTVGsDofNFYtcVhGII2jer5BXuveahWc1NnytMSo9Y098IBSGBiHnvxu+EY0LLslPGAD0l2A8KIUL1nBKel4RH+yQXYtaIgOmOSnNPT2CZa0P+UxviF6Qs7X0EmWA2F/ntBj8c5frC1LXZAG/6k8rb+1yWr4qczMjcabgO0XcSyoJuEPbNxjjnmmklrnm9Hsux6KJY8PrygHXWqaj93VJPBxSSX+uqRX8b0fzi0RK9wOqlewLzbDhxS3LEiZBFY2vji6GrmC97mRK2EhIlS/4W6ebUuxcA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:17 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:17 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:44 +0900 Subject: [PATCH 06/16] gpu: nova-core: mm: pagetable: Add DualPdeOps trait Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-6-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TYCP286CA0069.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:31a::17) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: e0e75ee2-c702-4b92-6590-08df0e26dc00 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: CpjniXI9XTSMvUc5adYcwOXBfyakrh5tQtRA0Os2EGaJXAFrJmigmKZGb5s8lFWCFbFw5qLbdAie3PXUM4fS0Rtn2z3lQ0GRr7/VkFyJfwzUbBeEbCn2FMffvos1V59mAdebrov751/HHhPdAqVNxdW8vGl9mc23HpQaIOdFdDphVpCDFpw69QA/t0PZbLrtCzugFK7V58EyMrZ9/wCOWRxoDEI7d9CbNk2s9uIK7jr6maZKVPaUrGaGoQi+D2klbPVJfeR5lYG+tWijCF3Lvw43giQEDfaaCyHvVbiMT4T3Thhuk4L7CdISBemSAqHT/xf1kgGK4cDHlCykorv3+VQ8wP6Kx0WviHPAKrSD2ZUB5roQcJA322Q0n/gqjF0ZcqSPgFf4979UTSD3DH1Xara38qQdYm6zeuVmVVoicSuKRFemXBIl543hqfMzrAgJdfqrKzsJI3XVrty82/zmr5+kv6wpZICcLIoiyktc7xknUUsoN5xN++u47Jqc6pgiP0mg6y/MQjeyZDyyxRms3DPL/y9M0EtPRfbmmtHTEV/67Y7NhwmSmzl2sNkvHFlXbKKK+/ny7mm4SunYVQUDDqW0MeL7xVFx0ugYYIgU69jwddMOc2xSc5Jqo4AgF/N0eJT7Z1YfXRrQA9VXXeTvo+G9Tfxx2YxQzJ+bbDEtDtE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NjFTWTlTRmJabFBjbGZRaWI3RC9WYkkvMW9WaDJhTkVyQzA1NEFBcFNxaUJ4?= =?utf-8?B?Y2R2d0hKcnhESG1zOWpvdmdJTVNEN1hTdUtkRlRHVktCaU0wamx6S0lXbmhT?= =?utf-8?B?TnFyMDBPcFRRR25KTnIzUVBVN0k2VldBaGEyOUo0OTd2a1JjbVU3cmU5bFpZ?= =?utf-8?B?UFpFa0tZUkhhZ0NnMGVtTVByaSswOGl1WWxFdUx4NitOVkVibWhBOFVzdWMz?= =?utf-8?B?ZGQ2Q3FlVjNlRVhSWFJ3R0JBWURhOTNKUGgyTmpkRDEzTGpIVGx0a0twOHFr?= =?utf-8?B?TlR3U0ZwWjRENFo1REc1djJhdFVMbUpicHpuRFRVdUEwVlJ0Q1JMRWR1U0Ir?= =?utf-8?B?UWsxaGNZZUo5S2JEVGdLRVo0UWd0RWZaeTZtNU9wbXBFVjh5aUpEUW01QzFZ?= =?utf-8?B?enZaMlUwYy9nLzVkcHZTaS9ZN05QUkxqc2dzYmk5SUVtd25zTTJxalozOURZ?= =?utf-8?B?eTBuWklmcnJZOVJ6MFBUUlZ0eGR1SklEOGVsanlMdlFhQWtTSk9xUzJjQU5w?= =?utf-8?B?NUs4dnkwSGYzOU9XMS9paDdRSnpsSGFEVlpyWVhYT0U1RXBwek1iSHFIOEZv?= =?utf-8?B?aHdxM3N6empNL3ZSRzY1THVSbVRLejRpMTNOV0M1TlRxajhYV3dBYXFpYzVX?= =?utf-8?B?VnM4OW15aFlhUUNUMjR6VkQyUnQ4QnVzRTVCOWlJTU0rTmIySGlOc2VENGZK?= =?utf-8?B?eFFUbGRNTHZhanJXVEFtdkZIMmxTWHNkZGxrWmxadWo3TytxWFVPbmY0c2Nh?= =?utf-8?B?T1FKaHoyMlU3NHJ0MVE4NGVyYjFxc2NuQW5OYkJtSVhTdFY2bHR6WEpUaCs2?= =?utf-8?B?QUduekk2czVpTnRKQzRVM1FUK0xnTlcxc1RMZThsNHhERUhrY2szd0w1YmVZ?= =?utf-8?B?WU4wcjZ2ZzR0ekE2clFzWnFjdGhzbi9OSFkrYVRkT1dwckowWXlIQ0ZxckJn?= =?utf-8?B?NndZSWlHTFBQN2ZMWmpYTXQyWk9FdG84WTZQME95V3E0Z0ZuUDhrazhJNXg5?= =?utf-8?B?LzAwdnkrK2JwN2VXNnFVTjJqcnJWcTdENVRYVFExZ0trRUF4VEdCZlgwYUly?= =?utf-8?B?TkdvcjV4dUxlWC8rc2ZKNDdkckxuK2lYZ2VwbVAzekNrRXFkb2NzYUFPMkN4?= =?utf-8?B?bHljVDZBN0pldzk0ZjhQbzBVUmVTcjBNNjZLNDBtMkdXaHFJbzVzV01Valhp?= =?utf-8?B?YytoRE1tUTBEczBUL1FaYkJieDJ5bGdSeG93NTJPRUZIb2dkKzdKL29sMDcy?= =?utf-8?B?V01QbDlRSEtFenhkd0xHMUlHL3kzNWJGR21ScmQ1QWIzdGFUaFNNZzRkZ1py?= =?utf-8?B?TWFFR0k3QlBYbjFETytUL0owZ1EyTUY3WnZsT3p6eVg0UUs4VHRJRVhTS0pP?= =?utf-8?B?UGtOMEpIUHRQTWhTWUFROFBzZS92Yngzb3dwM1pqNDlBWXdBL0pwQjUxUGF6?= =?utf-8?B?b09BYXRKUmE0c1R5ZFhpRWVMOU9OYkdxbS9Na3ppRWFzRG1LRXY3MzBWUHhj?= =?utf-8?B?TXcwSncwd1Y0dHAvY0tIeDhwMVhsaER1dmhLWkZ4VG1jKzQza29JZ0RxbWJo?= =?utf-8?B?YnYxYkRmdi9QNzRaQVlmWFNFaGNSL1F4K056TUhpdVR4cVNISUtTdmhaNTBs?= =?utf-8?B?dmlsR0djKzlqVk1peTBvUFJIRW5HYVlvNk1RbTdRSFptQ2REcXpaOXRJWXNI?= =?utf-8?B?cHFVN0QwQXRMUktGbkN0cFhJZGQ0ZU1tRXRLQ1NQNlZyaW93WWt0eUNkcDVV?= =?utf-8?B?U0ZRS3JTNEt0ckk2VmxwNE9iMU1JaEJLTHJiY1Jldm9mRDdsTTluSjNwdHBp?= =?utf-8?B?MFdQMnRQK2RVd2Zad3hSNlBGUnJuejFISXo2QVJXTHM2RUxVakF2NXdmUUU3?= =?utf-8?B?cTMvS0xEUmo4bXl5OE5VSUtIUjkvT0hCWXVBdU5NUGdvWm5rdUhVd05ReDFM?= =?utf-8?B?ZUcyZTBkcnVVL0NMYlNQMGtLejFaN2dHMVB2OEZJczF4NFd1dkc3R3UyTjZZ?= =?utf-8?B?Y1paUGhDZit5UE5WVER1dHYzZEtVbkhOdnVzdzlmeXRhck9FMjZxZUVkVThU?= =?utf-8?B?MWd4eEp5eXJBZlBKSjVmTUVZWVdmdDA2d21vL2FQcUYyZ2FQdityR2ZodFZU?= =?utf-8?B?Y1dvSDFRNWdNVHV0Mng5OS9qS2E5bnVlekxaVWdSeUVjWW9xSXdoZXpqbHBu?= =?utf-8?B?RWlQY1hoZDNoQW1DQzg1aWoycGtaWTlsdlkyUTlLTmNiMy9JcmJWOVRLaUUx?= =?utf-8?B?NFYzUklUMGpDNWtTRGd5ZWltK1lpcmZtY1d3emMrQVRScGYyVU1xbTVNRVlP?= =?utf-8?B?OFpGeXJ3YU5IWERobU5oR3E5Q1Y1elRWN2dSb0xDYk8yV3BWOHVKcG5INFhL?= =?utf-8?Q?ulHN58londoU0XXR+y5Px6+7kQdU2nV89UGMVZyJmGb4I?= X-MS-Exchange-AntiSpam-MessageData-1: 1cnjRUmHeogBIg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e0e75ee2-c702-4b92-6590-08df0e26dc00 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:17.3472 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pI2D32hN6IaEivQeoy0lnG2KUd/c2kzLUhF3r//VCQXCSaHwluOvDW/FXNtlyGbOZ0PScc6TMIUvt63Ov8gE/A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Introduce a trait for 128-bit Dual Page Directory Entries. The `read()`/`write()` helpers issue two 64-bit accesses through a `Pramin` to load/store the 128-bit value. Signed-off-by: Joel Fernandes [ecourtney: replace PraminWindow accesses with typed MMIO views] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 41 +++++++++++++++++++++++++++++++= ++++ 1 file changed, 41 insertions(+) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index 14436fa6adf0..550145d56e3d 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -169,6 +169,47 @@ fn is_valid_vram(&self) -> bool { } } =20 +/// Operations on Dual Page Directory Entries (128-bit `DualPde`s). +pub(super) trait DualPdeOps: Copy + core::fmt::Debug { + /// Create a `DualPde` from raw 128-bit value (two `u64`s). + fn from_raw(big: u64, small: u64) -> Self; + + /// Create a `DualPde` with only the small page table pointer set. + fn new_small(table_pfn: Pfn) -> Self; + + /// Check if the small page table pointer is valid. + fn has_small(&self) -> bool; + + /// Get the small page table VRAM address. + fn small_vram_address(&self) -> VramAddress; + + /// Get the raw `u64` value of the big PDE. + fn big_raw_u64(&self) -> u64; + + /// Get the raw `u64` value of the small PDE. + fn small_raw_u64(&self) -> u64; + + /// Read a dual PDE (128-bit) from VRAM. + fn read(pramin: &mut pramin::Pramin<'_>, addr: VramAddress) -> Result<= Self> { + let lo =3D pramin.window_at::(addr)?.view().read_val(); + let hi =3D pramin.window_at::(addr + 8)?.view().read_val(); + Ok(Self::from_raw(lo, hi)) + } + + /// Write this dual PDE (128-bit) to VRAM. + fn write(&self, pramin: &mut pramin::Pramin<'_>, addr: VramAddress) ->= Result { + pramin + .window_at::(addr)? + .view() + .write_val(self.big_raw_u64()); + pramin + .window_at::(addr + 8)? + .view() + .write_val(self.small_raw_u64()); + Ok(()) + } +} + /// Memory aperture for Page Table Entries (`PTE`s). /// /// Determines which memory region the `PTE` points to. --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D602F3537E0 for ; Wed, 9 Sep 2026 04:00:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926433; cv=fail; b=bUM5ANWyKXRPUYPEMZMhYE6AstO9wNBUgYwBKVVgitvomibVHAQbO6ZnHt0GrEuWfnQgdjFLqFdfFqIhR0BLlPT/8mQyhEnPJmAn1opmN58rrnuLwDDdSJ1tE10XvttGqbm4bssopAQ9x0KPfZK6koMoGJ5cyAyhjC1zQ3kY91Y= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926433; c=relaxed/simple; bh=cJ5XfpdgYnGDFg/FS/jhOK4Gyyd584UkYTPTLCipNoQ=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=LVzjXHjqywY8Ox5lNHNxQuZuLsuqMH/mBaFeECyx/xf52vxgKulUnZpgShcjr9HZ4Ypa5ogoLwsGq3w5wz6a7TVjYFENfx3eEvcXhC5+vKpDw+/lYCqSnLkngpGc5nQiMNNbPHCPsQBakrzwX4zvP2CjMxsZBPfXc9a9etxk3Uc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=TZykc5+F; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="TZykc5+F" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=E4V2DCm1zd5L9vWUemU/VSyv+peIgMhZ1xbSfBQgScyCjYh1LLyuH4XXIuMTJHKTtu0KiJiUuY9Eak55du5GtiBD5DQ21quxouLmEnmTUfqFLhprT/jqworVXZfiJVPwBiWMAnmWOS2jpbvWz7NwdhNKo597SFzgRtR2A3IDO9pMysCBYkxPbx1CcJ1ve7GA/Bxkwtjs1CQkz8oMnDQV4MUNu7exLLLbv85tafPUmUfTCv+O0FLz8QAH2fAzJoXoyPny+WryoW+ZFPCUn29Yqg54jKcs5E7rD36gQheUENm5/45t1TdZjDtZrefr+by30umOJzfDyfHD7mHwsfbtXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Q5K5IdJkFyZm7uxVc/dlhsh4UAnBWNtHKqqpSwd32m0=; b=IUHKrWQk+XgVFSZqFp9mZOZeCxQwpL9CFQcUlEwU32spt+fGCGBttIN1l7RfTWoJVO8p11nXy4soSGWqfYRQvKvBZQX3tVu/o4b0eAzO8X1p9E70os8HaDoQF5fYfrIYmuZ5mR3Sc/8WMEHz/a7p6RF6hxImCwi3LVHtS57bGZaoC4oBDXx/u5csJdaXcftzh+v5xwbJ9Dhmjl6CVB8XfWsWcC6vZzn743tyOU/vVV+Q8nakjr2QQPLy+tLW8ZzuLuw7bL4WBdqAmRqJOkCyNAOPPeiNDiZvkUYZx4+Dn2HdF54ZFLxsUgh97vg8euLJ4yPAhvyzJEOjRPKq3+H4YQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Q5K5IdJkFyZm7uxVc/dlhsh4UAnBWNtHKqqpSwd32m0=; b=TZykc5+FTC1WzpXl83dWssD74AS0gx901KGSuFDGFooHTzlekqowXWaA9G9J/oBBtsJI8g3pRrDuwamE8hO+OHhvGOoEz2xzs2swaggKa7hAbECIn8cmprpp7F/nqXGevD6Io8n8FmiYeYANP73LXg2i1p+QUiIbwkiGMOnLbMQlE7z7GhCNVSw2SLFSMVo+3pxsObLl5kIXnH17V9G4yTRqYdZBBWx3Wdm5KJ1fQFAKP1CRpyU1GcCnRU03kC4OQ7P9JAhjXnR5W5wu30cLjIyse6zNIJM33a1RRbwYZ59I+qZaLC1vntOd/oAmovqwqAOjddKdFAchKZMOccXodA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:20 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:20 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:45 +0900 Subject: [PATCH 07/16] gpu: nova-core: mm: Add MMU v2 page table types Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-7-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P301CA0099.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:37b::15) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 09c2ac75-72c4-45c8-2397-08df0e26de16 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|10067099003|11063799006|5023799004|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: l8q5xr0QvthHGCKOkWkFYfMOn9tNO13t/sRRY4i1kWYx2u1qTg+UIn/JlZfgVnsBA+iGvyKi3neBARTBX5zXpwXgRlNzV23ho/H/6oWx/nCgex9TuKo0NrbnpFdJKuE5zkUqXaF4RamuE9PkBeaQI3fFy7W71oGmCl6h0cr5fnr2xwFSCnk8NsdXMey+mb3QyIdW0OkHCOsCX8/SE/MLHBn48XkwUsJWkpVaDYIAvFjMhV1Nv7wN7RueigiyBS2s5v+q5FBGdk+oDNN3rnKhU5wJ6JhyNz3kun87Gbu+uAqm19xykdzgOFvag5l5aQB/U+VYr7kgVzbzpbJ5I0my3uTLo6s7AIGhVW78iJZfRYDFYxaUj3yvvv18cRGRZ/uuLsU6FVwEGiASXvaRZ34BgpLKtTbEytNA5xaFAPfKAgu0X6lRH5wwlBhaCDRZXyGcM/b9zPRQudV2S3VvneW4wWcAzlf8pTuXh7lrzvKCQqsOBy2tXTs72zpNF+1sEd4l7cEbvUqvTxAE89SG/Z1uhknu78kCWLBdJkqLgev3hkGnn8dlcgyXjLSkxnTA6JabTXLVEhzNVSaT1veYmkx5VQeGE2Yk7vgGIoji5nrp/XhMJReFaOyk8ifLI1CfCA1yxZc1dVWbnmpoBJv82p4lrtgMbagpM1CMYB4wmkIrjDM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(10067099003)(11063799006)(5023799004)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?T3l5dTA0cXBkTGNEZTltcHkzN1pSa0RjRXo0NjNhalFQL0NkTCs2M0F5SG9L?= =?utf-8?B?V1ZXN0h0Vm9QVVpad052bU5OSElIYmVoUFMweU9IQmhZNXFRRHh2SVQxWFZE?= =?utf-8?B?WUE5ZS90QjMyZ05iS1dDb0VUYVhmQnJZQXBZYklVTW5aSkxkM0NMTEZsZzBv?= =?utf-8?B?VkNyM0tPanNNem15bnpLVW4rU3A0eW94bkdYajRKMFFnaWliZlpFVzRTK0py?= =?utf-8?B?TWFaV1hFZURhNE5lUjlRb0UzVEdPWEZUaGpGUkhmTFlXN2NIbDZJbFk3ZjIw?= =?utf-8?B?Z3ZrN1RoZFFTbGJOdEJXZ25aWmltVmFFUzZsUTJ1S2VxZHdHWHdjaENwaGlW?= =?utf-8?B?M2VnRkZWQjVaQm9WTXV4Y1kwL0NDdWtTeElrT1FsbEpLWlNtN3RudHhyaGFE?= =?utf-8?B?b3dPR21lOTlXQVFGNnFyKzhBMEFWOXlSOEZzQUNwZ2ZGYUdVR0ZsNyt6cTRx?= =?utf-8?B?aEcySVZkNkhsaGtQTko1Wm40Q0oxR1UzQmNoVDlUSS8rS0J6SFNNanVlWnlL?= =?utf-8?B?d1lFSkZQdWoxVG1mckVISnFPUWJoK0prZnV6L1U4cnlWa0t3NVNxcDJkZHpR?= =?utf-8?B?b2pQSzlDelNpenppMG54dld6RTFSQmJJbkFxLzZwYmh6a0dIUVVyYW5GbGxU?= =?utf-8?B?RS9wc3psZDdGS1FEYVpHbkFtWUlpOEJNRFJKbjF0WDRPYUQyamF0N0g3eS9E?= =?utf-8?B?eFpSMk9iUEV2a2FnL0RFbjVicTVKVlVqa0hkM1ZoZEVwN1FIRnNFUXBmTHJC?= =?utf-8?B?R2pUMVdsc053c250WkMvcGVOMk5BNzBMVS92NGY5NmtqZGtEVTlPSm1rUlVC?= =?utf-8?B?M1dMdmRENXBPLzdkYnlIWjNtREdjaFkrc2ZqZEFXeHkvZEsrOWNpYmFNbHdN?= =?utf-8?B?anlSNGdtMTJDclMxaVVNU3kvUHRURHVyazBHVWlxNWlaSzNIS0RRMXNCSVVH?= =?utf-8?B?bXJSd1g2dzljSStjZXhURkVaa3pvVzRWVFdPRDVNaU1yeWpIamNBR3dORWpB?= =?utf-8?B?Tnk1Q25jYlpxRnM2S3dNVWg2dUgxQS80R2pHUnRHVTFBTlhTYmlYdS9FdWpu?= =?utf-8?B?VENjdzc3eGEyWHpPUlM2Q0xQcExTUkJ6bWU3NGY5RFRWdHJOa3hMTk8wNDZN?= =?utf-8?B?NTF1TWN1NzFhQzlLcnJmN3FHVXFqcTg4WVgvc0pleTM5cW9kRzlOQTRrZzhU?= =?utf-8?B?RmFOZDNBQ1hmMXA1UlBwcm9uVTZKQkFDSkg3aHJYQVdHdGZiQUZlbW1lY0F1?= =?utf-8?B?eDAwRmhiSWtvOS8xNk5qT0FoM0IrWkRsVTlrV2x1WWpVQ0dodERHSTRXZlFt?= =?utf-8?B?TTVQRjJjRGRodnl1VGJsbHNVWXNhSDhwanZKcVUwWndqb3RWbUxaRENueEZF?= =?utf-8?B?cUQwTXl3Tk9QTWk1QUtiTHJicXp0MllhcGwwVU1ZTWFKcTRFV3hzUDYyakM2?= =?utf-8?B?U0J6M2NQZHZDQUgyUFR6N3Z6RGZMV2xGUURXR1JiQ096bjNsZ2IrUVdna3V2?= =?utf-8?B?SGVhY3M1eFZFMkZOb09pbkUyVnJJRjJlcm5HT1M3RnNTZFQwZmlhM0d4cTA5?= =?utf-8?B?ZHpGNVoySjFiUnJMT2tqS2RKaHQxNW9CUWhFYTBrVTRIM29Xam9XZ3dPb2dq?= =?utf-8?B?UzFzSllDdkJCclRuNWRHRXVidldIaVNDdTNVYkFTZSs3Q2M5QURsRXIvOWRw?= =?utf-8?B?S005TGhaRnJYaWdIMkIzdVM3cUx5S0srTTg4ZCtOczdKVkNGMjNETm5VVkll?= =?utf-8?B?TmptTklGVWMzUGdKcHloOEFkMkdLMFN0TitVNncxdU4yS0xKb2dXNWZOcXc1?= =?utf-8?B?Qm9tMnl1Ymlkano2SDQyYTVWTDZ1Q0lSVCtqRDJhN3c4czNuUTY1TWtoaDBP?= =?utf-8?B?a2htUlZYSDhxUmFwR0tmWkErNlJjdnZOZDlMemRPMlRYN3pzMmdzTnJSR0pP?= =?utf-8?B?dm1SeGNPdnZGa2lxdEtkWEZxZERSYkFheFBkWG15SHBmTFVZOEZoRVpwOUNS?= =?utf-8?B?WUtnYW5DMzBHaXJoODVWSkM5VEJEdytURTB4VWhESEJpRFdON1gxbkJ3TzRS?= =?utf-8?B?RnY4d3lCSVI2b0hGWGlIMi9odGJTMVhTbHlFTllUZjFheWhlUDBsQmRoQVp4?= =?utf-8?B?a3lmYUtTbzlTbmo4dmRjVlJMTUlYZXhpRjh1U1V3UW5kY1RITjJ5bnRBcmRy?= =?utf-8?B?cUJ1Vi9WQ1V1NVdUcHRWZVFLWkxrVk9MUmRsb1BzZ3h6MjNPUUxBK2JWa0tV?= =?utf-8?B?emoza0UvNWllbHkvOWRQUXpudmE3SmNKODhSK1h6WHpsUFlpR200MDFmWlZD?= =?utf-8?B?UEZGQXpURUFpUHBCN3oxTUVvdFRQRHZMKzFBaGxRWjJwTWlyN3V3RCtlREpB?= =?utf-8?Q?utzDg66waFsWsBNIP3penLyeQrfz8h8UNh30h/nsF3Zl+?= X-MS-Exchange-AntiSpam-MessageData-1: RaSk+N0YhKeEdQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 09c2ac75-72c4-45c8-2397-08df0e26de16 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:20.8522 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UeoD2OJXSfZMQzoW7ReExe1YuRtYGsCnDT5LE0DnqjVudm/UzEZ2ROkSKsU3YXwVem5kY8EbSZjqYOMSAF2Umw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add page table entry and directory structures for MMU version 2 used by Hopper and later GPUs. The `Pte`, `Pde`, and `DualPde` types each implement the `PteOps`, `PdeOps`, and `DualPdeOps` traits introduced earlier in the series, providing the version-agnostic API used by the forthcoming page-table walker and mapper. Signed-off-by: Joel Fernandes Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 2 + drivers/gpu/nova-core/mm/pagetable/ver2.rs | 271 +++++++++++++++++++++++++= ++++ 2 files changed, 273 insertions(+) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index 550145d56e3d..df8e7e7d7327 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -8,6 +8,8 @@ =20 #![expect(dead_code)] =20 +pub(super) mod ver2; + use kernel::{ io::Io, num::Bounded, diff --git a/drivers/gpu/nova-core/mm/pagetable/ver2.rs b/drivers/gpu/nova-= core/mm/pagetable/ver2.rs new file mode 100644 index 000000000000..089e5cc2bfc3 --- /dev/null +++ b/drivers/gpu/nova-core/mm/pagetable/ver2.rs @@ -0,0 +1,271 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! MMU v2 page table types for Turing, Ampere and Ada GPUs. +//! +//! This module defines MMU version 2 specific types (Turing, Ampere and A= da GPUs). +//! +//! Bit field layouts derived from the NVIDIA OpenRM documentation: +//! `open-gpu-kernel-modules/src/common/inc/swref/published/turing/tu102/d= ev_mmu.h` + +#![allow(dead_code)] + +use kernel::bitfield; +use kernel::num::Bounded; +use pin_init::Zeroable; + +use super::{ + AperturePde, + AperturePte, + DualPdeOps, + PageTableLevel, + PdeOps, + PteOps, + VaLevelIndex, // +}; +use crate::mm::{ + Pfn, + VirtualAddress, + VramAddress, // +}; + +// Bounded to version 2 Pfn bitfield conversions: +// 25 bits for video memory frame numbers (bits 32:8). +impl_pfn_bounded!(25); +// 46 bits for system memory frame numbers (bits 53:8). +impl_pfn_bounded!(46); + +bitfield! { + /// MMU v2 49-bit virtual address layout. + pub(super) struct VirtualAddressV2(u64) { + /// Page offset [11:0]. + 11:0 offset; + /// PT index [20:12]. + 20:12 pt_idx; + /// PDE0 index [28:21]. + 28:21 pde0_idx; + /// PDE1 index [37:29]. + 37:29 pde1_idx; + /// PDE2 index [46:38]. + 46:38 pde2_idx; + /// PDE3 index [48:47]. + 48:47 pde3_idx; + } +} + +impl VirtualAddressV2 { + /// Create a [`VirtualAddressV2`] from a [`VirtualAddress`]. + pub(super) fn new(va: VirtualAddress) -> Self { + Self::from_raw(va.into_raw()) + } +} + +impl VaLevelIndex for VirtualAddressV2 { + fn level_index(&self, level: u64) -> u64 { + match level { + 0 =3D> *self.pde3_idx(), + 1 =3D> *self.pde2_idx(), + 2 =3D> *self.pde1_idx(), + 3 =3D> *self.pde0_idx(), + 4 =3D> *self.pt_idx(), + _ =3D> 0, + } + } +} + +/// `PDE` levels for MMU v2 (5-level hierarchy: `PDB` -> `L1` -> `L2` -> `= L3` -> `L4`). +pub(super) const PDE_LEVELS: &[PageTableLevel] =3D &[ + PageTableLevel::Pdb, + PageTableLevel::L1, + PageTableLevel::L2, + PageTableLevel::L3, +]; + +/// `PTE` level for MMU v2. +pub(super) const PTE_LEVEL: PageTableLevel =3D PageTableLevel::L4; + +/// Dual `PDE` level for MMU v2 (128-bit entries). +pub(super) const DUAL_PDE_LEVEL: PageTableLevel =3D PageTableLevel::L3; + +// Page Table Entry (PTE) for MMU v2 - 64-bit entry at level 4. +bitfield! { + /// Page Table Entry for MMU v2. + pub(in crate::mm) struct Pte(u64) { + /// Entry is valid. + 0:0 valid; + /// Memory aperture type. + 2:1 aperture =3D> AperturePte; + /// Volatile (bypass L2 cache). + 3:3 volatile; + /// Encryption enabled (Confidential Computing). + 4:4 encrypted; + /// Privileged access only. + 5:5 privilege; + /// Write protection. + 6:6 read_only; + /// Atomic operations disabled. + 7:7 atomic_disable; + /// Frame number for system memory. + 53:8 frame_number_sys =3D> Pfn; + /// Frame number for video memory. + 32:8 frame_number_vid =3D> Pfn; + /// Peer GPU ID for peer memory (0-7). + 35:33 peer_id; + /// Compression tag line bits. + 53:36 comptagline; + /// Surface kind/format. + 63:56 kind; + } +} + +impl PteOps for Pte { + fn from_raw(val: u64) -> Self { + Self::from_raw(val) + } + + fn invalid() -> Self { + Self::zeroed() + } + + fn new(aperture: AperturePte, pfn: Pfn, writable: bool) -> Self { + let base =3D Self::zeroed() + .with_valid(true) + .with_aperture(aperture) + .with_read_only(!writable); + match aperture { + AperturePte::VideoMemory =3D> base.with_frame_number_vid(pfn), + // Sysmem PTEs use VOL=3D1 to bypass L2 for cache coherency. + AperturePte::SystemCoherent =3D> base.with_frame_number_sys(pf= n).with_volatile(true), + AperturePte::PeerMemory | AperturePte::SystemNonCoherent =3D> { + kernel::pr_warn!("MMU v2 PTE aperture {:?} not supported\n= ", aperture); + Self::invalid() + } + } + } + + fn is_valid(&self) -> bool { + self.valid().into_bool() + } + + fn frame_number(&self) -> Pfn { + match self.aperture() { + AperturePte::VideoMemory =3D> self.frame_number_vid(), + _ =3D> self.frame_number_sys(), + } + } +} + +// Page Directory Entry (PDE) for MMU v2 - 64-bit entry at levels 0-2. +bitfield! { + /// Page Directory Entry for MMU v2. + pub(in crate::mm) struct Pde(u64) { + /// Valid bit (inverted logic). + 0:0 valid_inverted; + /// Memory aperture type. + 2:1 aperture =3D> AperturePde; + /// Volatile (bypass L2 cache). + 3:3 volatile; + /// Disable Address Translation Services. + 5:5 no_ats; + /// Table frame number for system memory. + 53:8 table_frame_sys =3D> Pfn; + /// Table frame number for video memory. + 32:8 table_frame_vid =3D> Pfn; + /// Peer GPU ID (0-7). + 35:33 peer_id; + } +} + +impl PdeOps for Pde { + fn from_raw(val: u64) -> Self { + Self::from_raw(val) + } + + fn new(aperture: AperturePde, table_pfn: Pfn) -> Self { + let base =3D Self::zeroed() + .with_valid_inverted(false) // 0 =3D valid + .with_aperture(aperture); + match aperture { + AperturePde::VideoMemory =3D> base.with_table_frame_vid(table_= pfn), + // Sysmem PTEs use VOL=3D1 to bypass L2 for cache coherency. + AperturePde::SystemCoherent =3D> base.with_table_frame_sys(tab= le_pfn).with_volatile(true), + AperturePde::Invalid | AperturePde::SystemNonCoherent =3D> { + kernel::pr_warn!("MMU v2 PDE aperture {:?} not supported\n= ", aperture); + Self::invalid() + } + } + } + + fn invalid() -> Self { + Self::zeroed() + .with_valid_inverted(true) + .with_aperture(AperturePde::Invalid) + } + + fn is_valid(&self) -> bool { + !self.valid_inverted().into_bool() && self.aperture() !=3D Apertur= ePde::Invalid + } + + fn aperture(&self) -> AperturePde { + Pde::aperture(*self) + } + + fn table_vram_address(&self) -> VramAddress { + debug_assert!( + Pde::aperture(*self) =3D=3D AperturePde::VideoMemory, + "table_vram_address called on non-VRAM PDE (aperture: {:?})", + Pde::aperture(*self) + ); + VramAddress::from(self.table_frame_vid()) + } +} + +/// Dual `PDE` at Level 3 - 128-bit entry of Large/Small Page Table pointe= rs. +/// +/// The dual `PDE` supports both large (64KB) and small (4KB) page tables. +#[repr(C)] +#[derive(Debug, Clone, Copy)] +pub(in crate::mm) struct DualPde { + /// Large/Big Page Table pointer (lower 64 bits). + pub(super) big: Pde, + /// Small Page Table pointer (upper 64 bits). + pub(super) small: Pde, +} + +impl DualPde { + /// Check if the big page table pointer is valid. + fn has_big(&self) -> bool { + PdeOps::is_valid(&self.big) + } +} + +impl DualPdeOps for DualPde { + fn from_raw(big: u64, small: u64) -> Self { + Self { + big: PdeOps::from_raw(big), + small: PdeOps::from_raw(small), + } + } + + fn new_small(table_pfn: Pfn) -> Self { + Self { + big: PdeOps::from_raw(0), + small: PdeOps::new(AperturePde::VideoMemory, table_pfn), + } + } + + fn has_small(&self) -> bool { + PdeOps::is_valid(&self.small) + } + + fn small_vram_address(&self) -> VramAddress { + PdeOps::table_vram_address(&self.small) + } + + fn big_raw_u64(&self) -> u64 { + self.big.into_raw() + } + + fn small_raw_u64(&self) -> u64 { + self.small.into_raw() + } +} --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0D00352C3C for ; Wed, 9 Sep 2026 04:00:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926436; cv=fail; b=Fy+XjQcynVS9BLPalBX7rJ55e6JYxUwfNbUqgpAefEv6ruqraFI3HDEC3krXoyMI3YoYobCsj60p0ErCUF+PNsULRSRt9VBBtDSjudIkofxcWopr1Jme/zPsDbcUS6evmYgAg9qTv2yXyzAN5lCt15dMU/Km8NQgLpFQVkwxKC0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926436; c=relaxed/simple; bh=dPuvEaLDVPbQv867JIDAuvop23ckm33SD6H58y1rgrE=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=gM8ODreV6loJ2HJdjWdrvxHb3uUzTM8Sa0VBBUKGjBOIH+qTtBbqrdoq1hUv87UnJfAULPdnP7X7X+kvx01z0h51NOYRGGuaze5P1SolHfIT19msCBieUlKkRZkRtdzcn0Vucyu7xYO5kY48ORL71q43gYjWe7tAhwESrvCOg/s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=QEBDylgx; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="QEBDylgx" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Dgkra/KRqqEQ1DcON27n7sJxZiAKuwJjIZ8prRNidYd++ngs3Wj/qyaSq9KOPcHQ6Vfe5utdlHt9e/OU6rX4GFAPbNUERiig9dNvxdFBox9vsY511zYs5Qn7Yr8SjMTSMzFIFomSlfWpTQKGZQfgJgKSptvxvUEZXbcQr/pghmPi3euGuawBylmR8WuzNQrVX08Jw7zcMmWUdb6PJlNogcfPINcss7q85FXjtwafGlT2Kirkwc9kSmh7PnO4UCEk9QgABcFp404MgwS/DxhFtEUzlPWD1CeQ0UeU6CGlrGufVl1BxOSukiPtvkqRMu/8+i2D/U1IkYJ2isRm91lgIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bmIFw/eodPAV3vr4OEmuhzdcv+p+yzWvHEbfI/QoUA0=; b=AxSjXBuWv0yXKn8dRuSkUDVba07cGrpYbKbAgoFbpNHtTeZOCqrIXUkkwIJJeAMIF5JpAU7gHBIHvDFtiKg/bUuyvB1HeC7udYAzAhufiSHB6RQFsSn08zmVl8aIGH/NK/k/WlXZOmQWkHF5i3hcr2OOrMxr9RDH9DXiBmWsM9MnPW1R3ZF7UTCuIRHFGmM+ZYAuF+CGA4GzEXtVVREfwy/fLACJI5NAwyh6GM8r9vJbORAtsaI+QEUsb7sYnCaPWDFO5jap33X4YS9SzMJaxPYU+VEzIAQ6sqcd+p39Z/g5nIrtjlHR1kGJj7UNmPOvWkM2Oh5pp2iTFO5F2ijBJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bmIFw/eodPAV3vr4OEmuhzdcv+p+yzWvHEbfI/QoUA0=; b=QEBDylgx1fvU37eotBhozm+pDag3f++kPiEX6mP1KRTxexf70JDSKCAxCDNtPzRHC9N71+ibpdWQ97UndxF3hIfjAWVmGj4kMX1ri99pRmAtUB9xVWjWZNkQvkU3JDsy4Fho8iTkFC9ZBCUf66aAIwP8he4L7UUc4+g4iOlSUYvZC8QLIjWuvKSo6kG7SlJbGMReQP4kJaQ+tjM/0S5mCvhaERJRK6IlKXQtKgZ3cRUuGO50G4R8LVY1UCSza5g1vUPInYWu8VGPpnpGOzwPb9BNUtrszSpmWIU1LxR5HaIzPL1jS61UkHmu58CX6bUJ4K9VwGX8EmtQAykxBMGQfQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:24 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:24 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:46 +0900 Subject: [PATCH 08/16] gpu: nova-core: mm: Add MMU v3 page table types Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-8-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P301CA0105.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:37b::6) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 87e14154-f6d6-4741-838a-08df0e26e02c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|6133799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: bRXwPFXI29yiFiCqfXgXlWBCrVrzjoaRgMXAHhiKZZMzcADbVwsnoWY9wUiLpQL8P6b+jmfikVqCPRxJ7AKi0Hd5ExhGMM6TnUB4YKiqBSJx9lashc1omhykgwVA2OwVZqt87mRKhWuXGjkHAu+pXZuLhIsCY2vM1e2GoyiduwWSTW+rVZ5Dx3vIz00cbMygMdno3rEfo+AtnLUlrMnmZe1AZ54QWR2/LwcqQ70pv58sm4bvgzeNeN+N5DmpZGvbFTJwmDI8FY//O+jdUW0F1bg19TqAZlxpqBdRyNYNDR53Ifvbmf9po8oDfUbNo5GVTUnsJcUBagkoog14KJOIZ+QFbfObRMplRxVl8oX/amfPW9nQHmXRmHNZkK728pG0CeklYobMHvZD9GvJS38cGrhbRJWcJ/3tbpr9lfMVLcCAL42Kx3L4uHXc5rQIeumNegczCZPEpNqxXq+yzjlgDKP57o0hpvHbGiRI8qwtPVO5EkKzQWUcWPtcNBLrRLEAV0MSWri/7s9z0A9xcUo5pVFXqSvqWegBldvljWxWqN09IYfvsiMjm/uBbgkCXd5+Boomv0yc3jBVPs+1fTfkoVv59VcOvLkFtXjvVsQ/FOVUPPFUtLE5flpOiYC9CJo7i8DwlMi6zq3L33XXMDjifQ5UNEvsVDZaxcZyuyuJzFM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(6133799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aXBrdC9LN0VvZTU5YjNCZSsyM1l0Ti96SkVuUkJKUU51NkNvc1EzMzdMNkVS?= =?utf-8?B?YWF3S21KdGM1Sm5BLzQ2UDN4QTltdjV5b2VpWE0xUlY1VXVUS05yWGZYRkhY?= =?utf-8?B?aVdncTRuYXZmVENpd0FXU3FGbW5OWUI1ZTI0UkVscy8xV0Rrc29YNHhkaGE4?= =?utf-8?B?NFczcUpadmhoVzlMUnU1ODdIdzhQZ0szWlRuZ3lGVTJpeXJRRFVabXhIQnZC?= =?utf-8?B?SEhpTmR1ekhhL2xRMmRzSmZhOE4xMTlUNHFGYVJCMmNUVGw2SmRvdW00S0dR?= =?utf-8?B?Z0lXbUNpQXlnbnVtU0R5elpOckRWOWdUdkxGenVHSVJsamxNd2hFU0gzZnVH?= =?utf-8?B?WU1lQ09UOFQzY1dORFozZmRLNE4yTm1BTE9XVDFoNm0rekNSQks1YllZaHJh?= =?utf-8?B?Y3VVR3ZxdlFSZFNuUm13WkNKZE1TaytUcGpxVVRjcmlKUWlBcGR2dWR6amZs?= =?utf-8?B?Ym9rYWVFQVZGeDBYSUdXK20vdFN4Ykh1cXdDNWgwaHVqMk1LdHo1eGxkaXFG?= =?utf-8?B?Q3pjQlhqK2FmSU5TdGxkUlZOajFwMENoQUQ0MHRVcE8yTHB0UDRISk9iWm9D?= =?utf-8?B?NTV2MFh3NWo4VkxDVGk4L1lGd3RTN0YxdzB3Q0NvVXlrSTJvSHlzTGF2NnJX?= =?utf-8?B?ak1vZkRkdUQxYVFXRVJZamtseWJ5UlptaFhrbVpzbi9iOEhzNlV5L3dBeXB4?= =?utf-8?B?dVB0aWRzSVd4R055YVZtdHhSSTQzelNFMnVaa3ZoYTg5YWF3WWdMUnJuR2d0?= =?utf-8?B?Zm9xMzd0V2w0TEdSSHJmYVFCRzdxWmVzQU1sMlJJL3VSbkJwR3M5dCsyVUt4?= =?utf-8?B?RDFSbUZqUGxyUXBvaTNSNjBJQjVoaUFHdG5pVTlVZk9zVFhOcTlDcktoeXo0?= =?utf-8?B?TWw3UVhDL1hnNjVoTFJkaGo2TzByN0dzRzdzSDE1R2N4SjNseFY4aEtQeUw0?= =?utf-8?B?aTQweFZURC83U1VRb3NTQXZoSEhXYnZNanZDdjVXeTROQXFFcmhlYlJieEkw?= =?utf-8?B?MTZ5L09wdWd0endkTlpFd3dqcTFJdVhrc1VZR2s1Z2t6NUpSVWtLSVY0UXNp?= =?utf-8?B?eG5hU00ydUFISEpMcHpNS1BJTzkxY2tzaDJKZzVueldaT1lzaW5Fck8xeHdV?= =?utf-8?B?bjZYYVhUOUNPZ1F0K2hwTm1yK3dHOFg2TUZmMTJ0U2REQVlIK05TaXJjWEZS?= =?utf-8?B?YnBUOStISmk0NDhOcFVwOERacW5FV045MGFRdHg0RlRGVk11U2tJTTNVRnI3?= =?utf-8?B?N1NGbFRYZFlwclFPZnJmTENOOFZRUHJCZmsxNVZEdVdNSEdTbHBkekZnRWRX?= =?utf-8?B?d01OU1ROMHdsUHVZSnlEczBxUHNTRXArVzlBcXZ5b1Z0S0ZGSjRpOFdtVWdC?= =?utf-8?B?Y2p2Q1RZd2ZGWUFZOGFqdjROT2tzR291YXltWjB2RFh5OXh3cmV3eTBmQ0lE?= =?utf-8?B?cFY1ZUJwelhiMVN3eURtTlZXREFjQzFUMUJpSmo3ZDY2OEtoZVh5Mm1GNkh2?= =?utf-8?B?STk1Tm5jZGZ3UzZXc3N6aUM0Qk1SSnM0ODNtYlN3SmxDZFFSWlFpL2RGcjZi?= =?utf-8?B?VFZxdGR3TWtObUNUdTNCdDRMZExqY2ViaFhrVGs1NEF5RVVza3ZBU0cvN3Bu?= =?utf-8?B?RHFBZnBLUVpjSC9UcnhDaDJsdHU2Z3Q3U00ySzlWY1FKYlZwaVdoSFpCMjBx?= =?utf-8?B?NVdYUmZGZHhIZlJuT1FHUDRXS3gzZ0gzckF4WU1PaUJEUmFoNm1ycE9PTHhz?= =?utf-8?B?UGtBTUorT1NITEJoMSs1azRWQldGNnVGbWJaUGxOeVlpdE5vMGUxclZrVzA4?= =?utf-8?B?M2hFendLNmFjb2w0OC9mNk1weit4SUpjS0pRSmhIdmpHWVdUQmpCbU5mYUdK?= =?utf-8?B?NEFJRk52S05peTZXZVdWOTlxYU15RVRra0ZpNk5BdkJMZSthRGdMcHNZd3VP?= =?utf-8?B?RG9xbitreUdDZUlEdVNYOU5LTTJBdWtyOFlYR0FvcFZWREp1WlkrRmttWVdn?= =?utf-8?B?TkY4aU5mTTFqQjJpWVBKZGZGcytlTkRUZm9qVjMzTG9TdUJnQ2RiVE1ZeDVB?= =?utf-8?B?VjJQWFEyTVNiTGc5NFlRdFhzWVdBZ1FCSDJqM0RVajlyWU5xd0Q4T291TTBV?= =?utf-8?B?RzRCODBaanlnMnJLNC9kY1RWbytuMnc2VWMxbFFlb3ZmT3RQVUlteElMVFdp?= =?utf-8?B?LzU2VXJDMGR6TFNKb2xEQ1ZSMW1Rd1c4ZDlFV0hZVzVLYkRnbmNVWG1XL1M0?= =?utf-8?B?TDArT1NWUTVDaXNPNjd0aXkvMXJ5SWRVYzBlRllKcHNqT3pCNWpKSUpqWEdy?= =?utf-8?B?d3czOHFVSnIyT0tlZTFGNkI1WFczKzZLVlJGZlZwM0lhcXdLRlpnN1RGVnkx?= =?utf-8?Q?NCYe+yy8p8T/HLslTNZnCWIZ5MJsjWRLCKcrZwt3RMhKV?= X-MS-Exchange-AntiSpam-MessageData-1: bkCJ2Wg6Ref2/g== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 87e14154-f6d6-4741-838a-08df0e26e02c X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:24.3750 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xaEKg6qEO2Kyj4xaFShJkSilIU2pvCDGxno0C2DnJIPBwTAeVf5g/F0tns0/rZQ6EYqBXtioTkO9h2SCxhvi1Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add page table entry and directory structures for MMU version 3 used by Hopper and later GPUs. The `Pte`, `Pde`, and `DualPde` types each implement the `PteOps`, `PdeOps`, and `DualPdeOps` traits introduced earlier in the series, providing the version-agnostic API used by the forthcoming page-table walker and mapper. Signed-off-by: Joel Fernandes [ecourtney: update for the VramAddress raw API, apply rustfmt] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 1 + drivers/gpu/nova-core/mm/pagetable/ver3.rs | 417 +++++++++++++++++++++++++= ++++ 2 files changed, 418 insertions(+) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index df8e7e7d7327..128bfc0d277a 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -9,6 +9,7 @@ #![expect(dead_code)] =20 pub(super) mod ver2; +pub(super) mod ver3; =20 use kernel::{ io::Io, diff --git a/drivers/gpu/nova-core/mm/pagetable/ver3.rs b/drivers/gpu/nova-= core/mm/pagetable/ver3.rs new file mode 100644 index 000000000000..80c06463de88 --- /dev/null +++ b/drivers/gpu/nova-core/mm/pagetable/ver3.rs @@ -0,0 +1,417 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! MMU v3 page table types for Hopper and later GPUs. +//! +//! This module defines MMU version 3 specific types (Hopper and later GPU= s). +//! +//! Key differences from MMU v2: +//! - Unified 40-bit address field for all apertures (v2 had separate sys/= vid fields). +//! - PCF (Page Classification Field) replaces separate privilege/RO/atomi= c/cache bits. +//! - KIND field is 4 bits (not 8). +//! - IS_PTE bit in PDE to support large pages directly. +//! - No COMPTAGLINE field (compression handled differently in v3). +//! - No separate ENCRYPTED bit. +//! +//! Bit field layouts derived from the NVIDIA OpenRM documentation: +//! `open-gpu-kernel-modules/src/common/inc/swref/published/hopper/gh100/d= ev_mmu.h` + +#![allow(dead_code)] + +use kernel::bitfield; +use kernel::num::Bounded; +use kernel::prelude::*; +use pin_init::Zeroable; + +use super::{ + AperturePde, + AperturePte, + DualPdeOps, + PageTableLevel, + PdeOps, + PteOps, + VaLevelIndex, // +}; +use crate::mm::{ + Pfn, + VirtualAddress, + VramAddress, // +}; + +// Bounded to version 3 Pfn conversion. +impl_pfn_bounded!(40); + +bitfield! { + /// MMU v3 57-bit virtual address layout. + pub(super) struct VirtualAddressV3(u64) { + /// Page offset [11:0]. + 11:0 offset; + /// PT index [20:12]. + 20:12 pt_idx; + /// PDE0 index [28:21]. + 28:21 pde0_idx; + /// PDE1 index [37:29]. + 37:29 pde1_idx; + /// PDE2 index [46:38]. + 46:38 pde2_idx; + /// PDE3 index [55:47]. + 55:47 pde3_idx; + /// PDE4 index [56]. + 56:56 pde4_idx; + } +} + +impl VirtualAddressV3 { + /// Create a [`VirtualAddressV3`] from a [`VirtualAddress`]. + pub(super) fn new(va: VirtualAddress) -> Self { + Self::from_raw(va.into_raw()) + } +} + +impl VaLevelIndex for VirtualAddressV3 { + fn level_index(&self, level: u64) -> u64 { + match level { + 0 =3D> *self.pde4_idx(), + 1 =3D> *self.pde3_idx(), + 2 =3D> *self.pde2_idx(), + 3 =3D> *self.pde1_idx(), + 4 =3D> *self.pde0_idx(), + 5 =3D> *self.pt_idx(), + _ =3D> 0, + } + } +} + +/// PDE levels for MMU v3 (6-level hierarchy). +pub(super) const PDE_LEVELS: &[PageTableLevel] =3D &[ + PageTableLevel::Pdb, + PageTableLevel::L1, + PageTableLevel::L2, + PageTableLevel::L3, + PageTableLevel::L4, +]; + +/// PTE level for MMU v3. +pub(super) const PTE_LEVEL: PageTableLevel =3D PageTableLevel::L5; + +/// Dual PDE level for MMU v3 (128-bit entries). +pub(super) const DUAL_PDE_LEVEL: PageTableLevel =3D PageTableLevel::L4; + +bitfield! { + /// Page Classification Field for PTEs (5 bits) in MMU v3. + pub(in crate::mm) struct PtePcf(u8) { + /// Bypass L2 cache (0=3Dcached, 1=3Dbypass). + 0:0 uncached; + /// Access counting disabled (0=3Denabled, 1=3Ddisabled). + 1:1 acd; + /// Read-only access (0=3Dread-write, 1=3Dread-only). + 2:2 read_only; + /// Atomics disabled (0=3Denabled, 1=3Ddisabled). + 3:3 no_atomic; + /// Privileged access only (0=3Dregular, 1=3Dprivileged). + 4:4 privileged; + } +} + +impl PtePcf { + /// Create PCF for read-write mapping (cached, no atomics, regular mod= e). + fn rw() -> Self { + Self::zeroed().with_no_atomic(true) + } + + /// Create PCF for read-only mapping (cached, no atomics, regular mode= ). + fn ro() -> Self { + Self::zeroed().with_read_only(true).with_no_atomic(true) + } + + /// Get the raw `u8` value. + fn raw_u8(&self) -> u8 { + self.into_raw() + } +} + +impl From> for PtePcf { + fn from(val: Bounded) -> Self { + Self::from_raw(u8::from(val)) + } +} + +impl From for Bounded { + fn from(pcf: PtePcf) -> Self { + Bounded::from_expr(u64::from(pcf.into_raw()) & 0x1F) + } +} + +bitfield! { + /// Page Classification Field for PDEs (3 bits) in MMU v3. + /// + /// Controls Address Translation Services (ATS) and caching. + pub(in crate::mm) struct PdePcf(u8) { + /// Bypass L2 cache (0=3Dcached, 1=3Dbypass). + 0:0 uncached; + /// ATS disabled (0=3Denabled, 1=3Ddisabled). + 1:1 no_ats; + } +} + +impl PdePcf { + /// Create PCF for cached mapping with ATS enabled (default). + fn cached() -> Self { + Self::zeroed() + } + + /// Get the raw `u8` value. + fn raw_u8(&self) -> u8 { + self.into_raw() + } +} + +impl From> for PdePcf { + fn from(val: Bounded) -> Self { + Self::from_raw(u8::from(val)) + } +} + +impl From for Bounded { + fn from(pcf: PdePcf) -> Self { + Bounded::from_expr(u64::from(pcf.into_raw()) & 0x7) + } +} + +bitfield! { + /// Page Table Entry for MMU v3. + pub(in crate::mm) struct Pte(u64) { + /// Entry is valid. + 0:0 valid; + /// Memory aperture type. + 2:1 aperture =3D> AperturePte; + /// Page Classification Field. + 7:3 pcf =3D> PtePcf; + /// Surface kind (4 bits, 0x0=3Dpitch, 0xF=3Dinvalid). + 11:8 kind; + /// Physical frame number (for all apertures). + 51:12 frame_number =3D> Pfn; + /// Peer GPU ID for peer memory (0-7). + 63:61 peer_id; + } +} + +impl PteOps for Pte { + fn from_raw(val: u64) -> Self { + Self::from_raw(val) + } + + fn invalid() -> Self { + Self::zeroed() + } + + fn new(aperture: AperturePte, pfn: Pfn, writable: bool) -> Self { + let pcf =3D match (aperture, writable) { + (AperturePte::VideoMemory, true) =3D> PtePcf::rw(), + (AperturePte::VideoMemory, false) =3D> PtePcf::ro(), + // Sysmem PTEs use uncached+no_atomic PCF for cache coherency. + (AperturePte::SystemCoherent, true) =3D> { + PtePcf::zeroed().with_uncached(true).with_no_atomic(true) + } + (AperturePte::SystemCoherent, false) =3D> PtePcf::zeroed() + .with_uncached(true) + .with_no_atomic(true) + .with_read_only(true), + (AperturePte::PeerMemory | AperturePte::SystemNonCoherent, _) = =3D> { + kernel::pr_warn!("MMU v3 PTE aperture {:?} not supported\n= ", aperture); + return Self::invalid(); + } + }; + Self::zeroed() + .with_valid(true) + .with_aperture(aperture) + .with_pcf(pcf) + .with_frame_number(pfn) + } + + fn is_valid(&self) -> bool { + self.valid().into_bool() + } + + fn frame_number(&self) -> Pfn { + Pte::frame_number(*self) + } +} + +bitfield! { + /// Page Directory Entry for MMU v3 (Hopper+). + /// + /// ## Note + /// + /// v3 uses a unified 40-bit address field (v2 had separate sys/vid ad= dress fields). + pub(in crate::mm) struct Pde(u64) { + /// Entry is a PTE (0=3DPDE, 1=3Dlarge page PTE). + 0:0 is_pte; + /// Memory aperture type. + 2:1 aperture =3D> AperturePde; + /// Page Classification Field (3 bits for PDE). + 5:3 pcf =3D> PdePcf; + /// Table frame number (40-bit unified address). + 51:12 table_frame =3D> Pfn; + } +} + +impl PdeOps for Pde { + fn from_raw(val: u64) -> Self { + Self::from_raw(val) + } + + fn new(aperture: AperturePde, table_pfn: Pfn) -> Self { + match aperture { + AperturePde::VideoMemory =3D> Self::zeroed() + .with_is_pte(false) + .with_aperture(aperture) + .with_table_frame(table_pfn), + AperturePde::Invalid | AperturePde::SystemCoherent | ApertureP= de::SystemNonCoherent =3D> { + kernel::pr_warn!("MMU v3 PDE aperture {:?} not supported\n= ", aperture); + Self::invalid() + } + } + } + + fn invalid() -> Self { + Self::zeroed().with_aperture(AperturePde::Invalid) + } + + fn is_valid(&self) -> bool { + Pde::aperture(*self) !=3D AperturePde::Invalid + } + + fn aperture(&self) -> AperturePde { + Pde::aperture(*self) + } + + fn table_vram_address(&self) -> VramAddress { + debug_assert!( + Pde::aperture(*self) =3D=3D AperturePde::VideoMemory, + "table_vram_address called on non-VRAM PDE (aperture: {:?})", + Pde::aperture(*self) + ); + VramAddress::from(self.table_frame()) + } +} + +bitfield! { + /// Big Page Table pointer in Dual PDE (MMU v3). + /// + /// 64-bit lower word of the 128-bit Dual PDE. + pub(super) struct DualPdeBig(u64) { + /// Entry is a PTE (for large pages). + 0:0 is_pte; + /// Memory aperture type. + 2:1 aperture =3D> AperturePde; + /// Page Classification Field. + 5:3 pcf =3D> PdePcf; + /// Table frame (table address 256-byte aligned). + 51:8 table_frame; + } +} + +impl DualPdeBig { + /// Create an invalid big page table pointer. + fn invalid() -> Self { + Self::zeroed().with_aperture(AperturePde::Invalid) + } + + /// Create a valid big PDE pointing to a page table in the given apert= ure. + fn new(aperture: AperturePde, table_addr: VramAddress) -> Result= { + // Big page table addresses must be 256-byte aligned (shift 8). + if table_addr.into_raw() & 0xFF !=3D 0 { + return Err(EINVAL); + } + let table_frame =3D Bounded::from_expr(table_addr.into_raw() >> 8); + match aperture { + AperturePde::VideoMemory =3D> Ok(Self::zeroed() + .with_is_pte(false) + .with_aperture(aperture) + .with_table_frame(table_frame)), + AperturePde::Invalid | AperturePde::SystemCoherent | ApertureP= de::SystemNonCoherent =3D> { + kernel::pr_warn!("MMU v3 DualPdeBig aperture {:?} not supp= orted\n", aperture); + Ok(Self::invalid()) + } + } + } + + /// Check if this big PDE is valid. + fn is_valid(&self) -> bool { + self.aperture() !=3D AperturePde::Invalid + } + + /// Get the VRAM address of the big page table. + fn table_vram_address(&self) -> VramAddress { + debug_assert!( + self.aperture() =3D=3D AperturePde::VideoMemory, + "table_vram_address called on non-VRAM DualPdeBig (aperture: {= :?})", + self.aperture() + ); + VramAddress::from_raw(*self.table_frame() << 8) + } +} + +/// Dual PDE at Level 4 for MMU v3 - 128-bit entry. +/// +/// Contains both big (64KB) and small (4KB) page table pointers: +/// - Lower 64 bits: Big Page Table pointer. +/// - Upper 64 bits: Small Page Table pointer. +/// +/// ## Note +/// +/// The big and small page table pointers have different address layouts: +/// - Big address =3D field value << 8 (256-byte alignment). +/// - Small address =3D field value << 12 (4KB alignment). +/// +/// This is why `DualPdeBig` is a separate type from `Pde`. +#[repr(C)] +#[derive(Debug, Clone, Copy)] +pub(in crate::mm) struct DualPde { + /// Big Page Table pointer. + pub(super) big: DualPdeBig, + /// Small Page Table pointer. + pub(super) small: Pde, +} + +// SAFETY: Both `DualPdeBig` and `Pde` fields are `Zeroable` (bitfield typ= es are Zeroable). +unsafe impl Zeroable for DualPde {} + +impl DualPde { + /// Check if the big page table pointer is valid. + fn has_big(&self) -> bool { + self.big.is_valid() + } +} + +impl DualPdeOps for DualPde { + fn from_raw(big: u64, small: u64) -> Self { + Self { + big: DualPdeBig::from_raw(big), + small: PdeOps::from_raw(small), + } + } + + fn new_small(table_pfn: Pfn) -> Self { + Self { + big: DualPdeBig::invalid(), + small: PdeOps::new(AperturePde::VideoMemory, table_pfn), + } + } + + fn has_small(&self) -> bool { + PdeOps::is_valid(&self.small) + } + + fn small_vram_address(&self) -> VramAddress { + PdeOps::table_vram_address(&self.small) + } + + fn big_raw_u64(&self) -> u64 { + self.big.into_raw() + } + + fn small_raw_u64(&self) -> u64 { + self.small.into_raw() + } +} --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5848B354AEB for ; Wed, 9 Sep 2026 04:00:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926438; cv=fail; b=ujUiaxoAoIzpIJ4+CzvoOLGe+0cxgzaPUqpYG+tkL1g5s/Xzi9GibPNT+l1BUaR7ibyRz30mK59KWFLLHMKgm1vNDC6uCIepHQ0N6rScFEf7Dt/7rgLP9+JE/dL0ei6SbSygi/kPYTmRzn1R9Kecu5mCwbcUi7lCDNo2gLNnIQw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926438; c=relaxed/simple; bh=ns/2Syrqy9+s61grDQpabJbKuuKM2dmvfhdfSKwD49k=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=adHaoaH4yIEYsRhLdH8UJmGN7qP/8Igxs5+08FuCmNKAimmGoNQnSTBiT6R2PIc1nnBAZ8PADkY3RUlrZq6siTH5ITLHfmIfXOPpX5lU3P+FQzf7gGQ8v0KtXrWMmL/xTyfRVbCVpoJMUOSmrAUehCmGxNewavKOwK05skQqFNk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=iFHhCpm5; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="iFHhCpm5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wTge47NSiFqEcvJQdxeX0SpzIOf1pQoy3Y9jwOL1qeUFVAqWJzZP6N0hIVKFv+mI9GBtdY8CPLrYl4laTGbOcfChn1QRUdHVsYs6Y7Q/+KA2C7CpgWvK8sF4YLYWwm85SHTFO1rl2rkdJyCyP0LVrY7rmNrZV2k/3fo8u5dZ+UPCYUEAGrRYMZlysG+XskQWvGbMHig1oVgsjqbc67jFwkjZqOLdUp0bxL2B8fZ373oKkGnb88qoPDTNoy9hLvjs9z+md48G/l95XyGahhBzODwV+mwUyvio6xDZRCkNpVdvK9Ds67OCT8Vp0Iaploii6ZQIj15TvBGvUmBR7WBHpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6kizmvt1KnkEGoqPZTjaTWb+I01TygJEhuaRv89eWZw=; b=jsa2pfKaT/tVp4P+IutcwIq0Pdk9c5bPs8KMlRWAq2giD/W3iWTTtHyeSa+ISsvgoUT85KS7YtWkD6MriqDa3R/EVBRbVFNlPn8nXswtGwDBulp547HGKlPY9Jc89hW6SgpxKS2+SWR3f+8l2uR0VjCDmYKDHd1ekB/C5ra8k5+kH9Z6+SK8433cBAKPJdIAOa/82T4hEj3O9n5uEB1zAo+wn1P0TzY2WKzCtSYqUYACdovqrxvVIfy/SEd+Gna2Un11K8snu0gqufbNw9McI2T2v0deYgXH6ArDvVXwunual/wTosT7f6+pXjVYGrFHeHTm+sNUhTP3invErjnf7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6kizmvt1KnkEGoqPZTjaTWb+I01TygJEhuaRv89eWZw=; b=iFHhCpm5Q5/hIJ0PsStlS5BDvToQ0eDu17H4+1KuLi+rZwjfWIBnIcV6N3DOrgAugMUoRxvwohN6Yfzaf7Qpjm4AIK9C5UGQmWMNnhmGbNzsAaBRVtDwUD1GNlKR2OWi43WcQIpMsjMqAk5cePkdJxM5CN5Ss5LRc8FBNrCqTpUfD5RdNBUheDZuhl3vVu/EykNtiQEQAty2fifDcuVFseDn06euN+H8rTl8XgwOp/mS7CjJVNHp5ahR0DfDfzZmSMkxVgebaQNhW5ip0UC/QzI2KgC3qm879jcKO05C5V0VDSwiajWdpnL46jI8kS4NVGC9DvwP4pWrRvbw3ml7rQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:27 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:27 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:47 +0900 Subject: [PATCH 09/16] gpu: nova-core: mm: pagetable: Add MmuConfig trait Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-9-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P301CA0031.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:2be::17) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: d6b91019-44fe-4927-aee4-08df0e26e22d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|6133799003|3023799007|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: VGjJO7cC/ArUeO75MJ5SnzPyRZSxPzSKIPO5QpGeZmfE6PcnVz9Z4PBCWas3oKDwnRbOg74dRnUOuwgbLgft4d9SWFbHufiRjHvulSX1sYZH/Z+g1cio1Xim9IFOhtkUtIMUum9IZF6vBUFQlDkHza4OGpvUf/h2MkdUMQdQnkabCoN/uZfQ0p2S2DLlWA45aXIJfHWzl+Ql/v21u7b9MBxNDTXyPHLduOAnoPGNDbDT0jnT/Jfbm8P9B1618VVV408AoZ4utdxwBVjlxApkab65+pHRHKEtZZUrCoki2cyTGKww7HETTYvNWlbrHeEHzqEicBdvGPy2t9AQK0dVM2b/ZeDnqGhOloCIgFPMmYjJP+1GTnI9dvUWN5R8PWRW3YaV8WQLIOcLvJUJhxziJRAVECr76qWmwIkxMjEuielUw9U8likvrCnKyftWGfmLFoRF/a2Fz0B0/IXLo3SQC2NV58qeT+Txwc25wS1CuAK7dDjFlRCIPRJh5lOTMuegD03Qvq2UhFW8xF1HA8hpC83QbuZ/H8CvN1z4JwWaHdaYLoeWeQEaV1MPD69jbPO8i22jEps2oez5M2uNIrjaELJCqIk56qq9YKFRZdXfoiuzGjPc+KIyvY2/NZgRwq6k6AKHGs81P78FP2T6b54DjM/kVCo6rHTXkdE7nV7VXa8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(6133799003)(3023799007)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UXltdDV0MCtNVGtYWE81c2lVaDlCdUJ1UitjWEEwNnhBTWcyVXUyVERpTWsr?= =?utf-8?B?cjNDUmwyZnVYdFFpZUw2T2tkZ1JCcFJneEV6T3A4MGpFdllQSW1iREJWK09Q?= =?utf-8?B?VnhRMUlmVjJ5b2haa0RUbjhCNmJjb09FLzkrYXBRSWI0WjBYQVdiaGZLSUMy?= =?utf-8?B?QlpHU0tUVjArZHBxMSs5MlhiUlJCMTRXY3Z0cnplOEZKbmhmUWh4ajQxK0tI?= =?utf-8?B?d21FWUZHZUdmclcyR1hCMG45WG9uK2w0MUxHb1l3a08yY25PMjhNUUdMSjQv?= =?utf-8?B?UWxzaW1vT09jSDZUOUJybnl2ZkRERzRka2EvMnFGd0I4TXVwQzUydlNZaXZW?= =?utf-8?B?QzJKVi9mbVpvV3VRUzM5blN5QWZqNVhWNjgrSHh2dGJYUTFGTDVVMyt0a3Ft?= =?utf-8?B?MjB2aG01cGNCYUlKejduYndLY25Bb0c2VkVGb05KbC9sZ1VyM2NiRDV1QmU3?= =?utf-8?B?WmtNdXhEbzNwc2xGZitqdzFLZUNlSEpzb3YzUzBjeE41YkdsODQwWGVDNlFa?= =?utf-8?B?VFV0ZlF1THA1YUt4L3djT0dmQ0tSUFNSdUtKVWJCNGtSanR6Sk96RWRwVzdr?= =?utf-8?B?RW1Ed05vZ3d3aEFjcHpsQ280S1pFUTZSNE5FK0VRbUptZVhRR0pNbkhkTGRG?= =?utf-8?B?VzJiZVNobGFJT2RBTUR0SkJUcXg5U3lVbVRmWUJKVmpuczdqQ3hyZTU0VThX?= =?utf-8?B?Y1ZBU2pZays3N1NMKzZKK0lBTjZMS2hIM2o2TEVtdllwci9WYmhWNU9VY0t1?= =?utf-8?B?eHIwNHpDdm9UUDlDaXk1QlFLYlloNStvNmVtZTVqWXJaVDlBa3FZTHFsY0Nt?= =?utf-8?B?ek1GSUVJYlBRbEJMYk93b0JhbVJXb0Y0aWVZTWxFNU9CLytOa2g5Z1lZS044?= =?utf-8?B?RGE5V25CU3dZZEVwTEFvZjBlbjFZN0xYSEFWRGw4YUxjbkhIbGpyWFRXajcw?= =?utf-8?B?NVFNK3hEd1gxWWZaQWZjVHQ1N3VMd3VtTG1TYTZDajN1SWFZMHI0cXh0MkZ2?= =?utf-8?B?OXhrS2FyU0NydmlTNkNYWkpMRFFqVzZJZUZWMlM0azg3S2xpcGR5V3NnQVVY?= =?utf-8?B?NUllQW4vOU5jQTNUcFBGK2dpWHRQangyNFVTMmEvNFFWRVpsV2MvY21jWjZ4?= =?utf-8?B?N00zbzBBNmFGVHdZdzNKR1VaR1p3YkkyY1AybFIxT3I5blJNNkh2aGlDVlRZ?= =?utf-8?B?bGsyMWJMVjFXdnd2NS9URUxtYmZ0Nm01djR1dU8zOGw3SlRsaGQ1cnBSS2FY?= =?utf-8?B?c3VtZWNES1NRY0MvSGMvRm5oeEtqT1VXYTVVdCtjRGx4djcxb2lPYlp1cjJK?= =?utf-8?B?TGxMRzVvdUh1ZE02c3prY2xacXV4a0JSeCs0NlIzWWRzS3crVWxRUUVDY2ox?= =?utf-8?B?bXhXMDFaSTR0OVkyd2Q2S3ZzcnpYSXhNSi9OTHdZblRkcEsxTWJwS0w4Ukxz?= =?utf-8?B?ZFJTSjVld053YjBYbWxWRU15K0ZVUFlVYXc3TXU4bXVFbGJxdk0rcEZsT0RC?= =?utf-8?B?R2RhRUkyYlRNZEhrSStGWFdHTVBtSWEzRHhlZ2plUENsYWZxWnBBQlNtdEVX?= =?utf-8?B?WkV2RWxRdkJCSTdhUkVOLy9UdTlyMGhFUmtIV0RWTEZZTElqTXE1OU13Q1lT?= =?utf-8?B?VEMybWJ2LzFyTWpZaFNZdVFBUUVkRWlINzRnMGZUSzg1TDZsR2tTWnE5S2wx?= =?utf-8?B?QXk4R0k4dGFpU21GV0NyRTZldTVQZ3gxL0pTL1MzMlNrOHBlMFRrUGMyRSt2?= =?utf-8?B?OG5uNUo1OVJTbWxqN1hEZ3o2QTI5a1BMU2ZyMWhBRDY5WXptTEVMR3BBZ1M3?= =?utf-8?B?SE0zQlVra0hFalh4NTlJRnJ0eE9tZTdDenk0VVBjS3BkSG9wUFhacjU0aWxJ?= =?utf-8?B?Q0R1NFJsZlJWQVRpbXZTYncwMTFNZ1JFRkdtZXk3RVc1UDRzTWJWSXJLcU1t?= =?utf-8?B?d1JaMWRkVFlOOXF0aG9Idlo2cW02UUFMbDFtM2FvL2M1dTZOTFhSTERFMjlX?= =?utf-8?B?ZUVsMmhIUGp6TStTQ2p2b2dCNjFPS3JZYnNFYk12Ti95c0RNbENoZ3BiMFh3?= =?utf-8?B?bVpZTUcxN3FJVHljYzRpbTlJQ3dEM0ZBYTZha3hRK3cyeURob2Nya05hdW9B?= =?utf-8?B?Z0dtbEl5eS8vamFKV2MxazJJbHhwY0lab3o4SVpiblh3UmpsTVprMUVIUWlM?= =?utf-8?B?b3FXbTJoZnUyeFIxUURtejRYK09KdHR4RnVFdDBIQnllZCt1SVRTb1ljZXpL?= =?utf-8?B?dnhSSklyd0dVc29haHFnZC9WUFhGR2FRdE9vcmwxK0NXNmxUQnRreEFUUlpB?= =?utf-8?B?RU5NOUhKZXFIUitVMEtDYVBzSTJaQzB4T2FBODJMOEhRdzhiYzh6SjlhWk01?= =?utf-8?Q?aa21PrrJYA5aS25uJ8GP+6XQ9oedcLybjgGfAoiY6qX6H?= X-MS-Exchange-AntiSpam-MessageData-1: pr9J+9vpvj+jnw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d6b91019-44fe-4927-aee4-08df0e26e22d X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:27.7138 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZuhDFzcl4FMVTB0xayumV7I0Z1WFpYx/PpnZwjxTSG6wpMaI1sux49sjMop8OraJUOqKUjhcWBjzzWyo5uL/yQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Introduce `MmuConfig`, the trait that ties the entry-operation traits (`PteOps`, `PdeOps`, `DualPdeOps`) together with the version-specific constants and helpers. `MmuV2` and `MmuV3` are zero-sized marker structs that implement `MmuConfig` for Turing/Ampere/Ada and Hopper/Blackwell respectively. Dispatch is fully resolved at compile time through these markers, so version-specific code is selected without runtime overhead and without wrapper enums. This enables version-agnostic page-table operations while keeping version-specific implementation details encapsulated in the `ver2` and `ver3` modules. Signed-off-by: Joel Fernandes Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 109 ++++++++++++++++++++++++++++++= ++++ 1 file changed, 109 insertions(+) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index 128bfc0d277a..d88499ed3d56 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -21,6 +21,7 @@ use crate::mm::{ pramin, Pfn, + VirtualAddress, VramAddress, // }; =20 @@ -213,6 +214,114 @@ fn write(&self, pramin: &mut pramin::Pramin<'_>, addr= : VramAddress) -> Result { } } =20 +/// MMU configuration trait -- encodes version-specific constants and type= s. +pub(super) trait MmuConfig: 'static { + /// Page Table Entry type. + type Pte: PteOps; + /// Page Directory Entry type. + type Pde: PdeOps; + /// Dual Page Directory Entry type (128-bit). + type DualPde: DualPdeOps; + + /// PDE levels (excluding PTE level) for page table walking. + const PDE_LEVELS: &'static [PageTableLevel]; + /// PTE level for this MMU version. + const PTE_LEVEL: PageTableLevel; + /// Dual PDE level (128-bit entries) for this MMU version. + const DUAL_PDE_LEVEL: PageTableLevel; + + /// Get the number of entries per page table page for a given level. + fn entries_per_page(level: PageTableLevel) -> usize; + + /// Extract the page table index at `level` from `va`. + fn level_index(va: VirtualAddress, level: u64) -> u64; + + /// Get the entry size in bytes for a given level. + fn entry_size(level: PageTableLevel) -> usize { + if level =3D=3D Self::DUAL_PDE_LEVEL { + 16 // 128-bit dual PDE + } else { + 8 // 64-bit PDE/PTE + } + } + + /// Compute upper bound on page table pages needed for `num_virt_pages= `. + /// + /// Walks from PTE level up through PDE levels, accumulating the tree. + fn pt_pages_upper_bound(num_virt_pages: usize) -> usize { + let mut total =3D 0; + + // PTE pages at the leaf level. + let pte_epp =3D Self::entries_per_page(Self::PTE_LEVEL); + let mut pages_at_level =3D num_virt_pages.div_ceil(pte_epp); + total +=3D pages_at_level; + + // Walk PDE levels bottom-up (reverse of PDE_LEVELS). + for &level in Self::PDE_LEVELS.iter().rev() { + let epp =3D Self::entries_per_page(level); + + // How many pages at this level do we need to point to + // the previous pages_at_level? + pages_at_level =3D pages_at_level.div_ceil(epp); + total +=3D pages_at_level; + } + + total + } +} + +/// Marker struct for MMU v2 (Turing/Ampere/Ada). +pub(super) struct MmuV2; + +impl MmuConfig for MmuV2 { + type Pte =3D ver2::Pte; + type Pde =3D ver2::Pde; + type DualPde =3D ver2::DualPde; + + const PDE_LEVELS: &'static [PageTableLevel] =3D ver2::PDE_LEVELS; + const PTE_LEVEL: PageTableLevel =3D ver2::PTE_LEVEL; + const DUAL_PDE_LEVEL: PageTableLevel =3D ver2::DUAL_PDE_LEVEL; + + fn entries_per_page(level: PageTableLevel) -> usize { + // TODO: Calculate these values from the bitfield dynamically + // instead of hardcoding them. + match level { + PageTableLevel::Pdb =3D> 4, // PD3 root: bits [48:47] =3D 2 b= its + PageTableLevel::L3 =3D> 256, // PD0 dual: bits [28:21] =3D 8 b= its + _ =3D> 512, // PD2, PD1, PT: 9 bits each + } + } + + fn level_index(va: VirtualAddress, level: u64) -> u64 { + ver2::VirtualAddressV2::new(va).level_index(level) + } +} + +/// Marker struct for MMU v3 (Hopper and later). +pub(super) struct MmuV3; + +impl MmuConfig for MmuV3 { + type Pte =3D ver3::Pte; + type Pde =3D ver3::Pde; + type DualPde =3D ver3::DualPde; + + const PDE_LEVELS: &'static [PageTableLevel] =3D ver3::PDE_LEVELS; + const PTE_LEVEL: PageTableLevel =3D ver3::PTE_LEVEL; + const DUAL_PDE_LEVEL: PageTableLevel =3D ver3::DUAL_PDE_LEVEL; + + fn entries_per_page(level: PageTableLevel) -> usize { + match level { + PageTableLevel::Pdb =3D> 2, // PDE4 root: bit [56] =3D 1 bit,= 2 entries + PageTableLevel::L4 =3D> 256, // PDE0 dual: bits [28:21] =3D 8 = bits + _ =3D> 512, // PDE3, PDE2, PDE1, PT: 9 bits e= ach + } + } + + fn level_index(va: VirtualAddress, level: u64) -> u64 { + ver3::VirtualAddressV3::new(va).level_index(level) + } +} + /// Memory aperture for Page Table Entries (`PTE`s). /// /// Determines which memory region the `PTE` points to. --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C74FE34F483 for ; Wed, 9 Sep 2026 04:00:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926441; cv=fail; b=ug+de3nnUS4GEe+MzLlmb4QIcIuMKlGunMv5DboFLPUICjYCQkqZdodxXyGiC7Zd8kmn6fURM90LV/waRbj1XuxcmvQbZaBSStv9IV51XwaYD4JJp7CwUbOXMdeEnENFwcOBO3RZG/yVLs1p5ipNM5TzD6EDq4a54Ii3kAZkhag= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926441; c=relaxed/simple; bh=aTu01KpaIhkErdhNSYZeOki+vv7/X5SdGP5clJJaVlE=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=YCelnBTKeaoxm5K8gJ/IMylBx/bKnhHbTyskHlTXQjnN6z6evUj358qYkrNNIgtlopUYSEP0xNQzBOKIsa4XluXg+3CR8+ED656OdWlypy6PD8vftpzoIZpjwiGYw2QEYsKhqWhMhlEzKM9rmNnlDf6kaCv1CuPmAsiIr3AoNgA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=YIKxN1/h; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="YIKxN1/h" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=je0Dc2nPHw/9axHfJFKbqN77inOo36vdDbgmyKrdFBAtMhSXaLTUA3pfTQB7pLWYIBycF1DXOcQ6hmru50qP5l3Pt/qr5PxlIu+83q+72m+DvhWhBekQ32yORmTWp16dCQoZfeb27tt7KGTTSs4/yA8LNwR1R/Z1Vt7iMCsn6RRt7HHH82ewQzV3DKnIQFtlUCLM2ng3MlOlQmF4cWdOEyfZLiRkITHPo7EoXXOF9ZmaGlg1nhEdy0s6XLQ309D+FXlrVcVx4AiJO3CQFdGKSQFy06/HjvBky9JGVkePc9ZdSRELxkcysv6ut+7s1+V0Ag5ROvP4xoXIIzO7UjH+nA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=za0ewrcmC0w1PS+XN4qhFTXRlioK4XsCh7x2BAt6fhs=; b=nixTuxASmlH8t5XYmXnNkHPvv6hix/D71hK7myK+vcbX2SfzvM4dUWMCazzWQN4NPcPVtuq0cIQolJQmTEyDwuCVyr+FaTv0U9BknKL00w9zD/zD/6i9syLEfkYuOCxtkWzR4eeXZS17O4yQVrEEZrKQ4Rd1gY1Wtks3awF10f7WOo5InFAThETijG+YPR5R3brwG25MPj3cOI3qaDmUm2BIM8MiTBsC0mcgv+KEy7o721OgVkpEq0f92TNcCT6oC6IVOSEgapeIQse2dShVT9I0RYKrfOQwlMlAVzP1PGyr9+Mgprx0vTBj9mV8mHlBbjYvAlLDSzyaHawFoyUozQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=za0ewrcmC0w1PS+XN4qhFTXRlioK4XsCh7x2BAt6fhs=; b=YIKxN1/hWDMlIrO/zm9ytxkWoKCYxHglG2AO9WW4r0pU7SpYrJyuiXrPMpopQaSdjFF9nogTMQMesGpdRkLlyvixZfbxlzyYt0uds6Towo0C5jbW16gpx677p4xal0T9MfDqM/7neO/Rpa27osYlyQkg9tqCRuP8CAhPEquMYI4EdR195EqYBJnX+F8/wPCCPC0T6g5BZYjUeZhZ+QHSLghw7CVBEnpKyuVlwl9+TggxjSJ/Ow+xZbF8SAqpD+tIgTwz5O12AwJz/E4lXHGBVQOXp0eLBP9xS+Ytr/W04of8irM803OCVOyWWSNKv0rTChYQGYc6OGaDm9iSSsJAew== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:31 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:31 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:48 +0900 Subject: [PATCH 10/16] gpu: nova-core: mm: Add page table walker for MMU v2/v3 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-10-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P286CA0052.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:36e::17) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: e99ab467-7aa1-46ba-aa6d-08df0e26e44f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: 9evwO4kx7JVOs3Op6cCoAUS10kK4GKnhrupTyRoLX4hJawwMEK/6vnsuGxZWqUOfUvNg9rGsNEufy1QoQ+WmgakRA8IrBP1BcLR1+MGTFsZRInn0kFZYwNJu23vK56Kgy3ZFGRsHmekmqtGNkBsasuCwdL8ZUiJE8t+X6B+2ZwM3LbMsIPt9tCn8TV5Qt2lkBskeQjIaFMwFabRSclsICTjMYA4OP1waWkg3pb9kYR014HPKbCB5mZxrH8Dk8MuDyEtlFLpjeE1SAH7YLztPXKlO0pPO9IeBQux4ciwzQkx+bruiFtimyUEjSu7IKM/W73yjjaVoMrMlvSVMtHR2XrGsZMwMe5AFqBxYONojL5qDoJdppbZ5rJ6bK7ZuO5YQvxvpENV32Kbc6FWsRSc76z64USeo08U7YN1KATIEdZZPAnseM/Vv6kb6vNq+9Fe/mnrt/X+TwrACwBELhXrH8tqXX/jeKx+x6LH4YgikvNslQsLB7Fi5HAQyd3w7U+Bq2TUaQenZrPVjC0CpSf6w7HzlsECqFu17Iohr2YiFhlQZpcm3ZoyGkVBXSqX9B5olqv0FV02YeshRfN4USTvQyxQZHxcaq5MhUqjO1TmpCxy6gRk95I0dU8+LtDQlmUcDN2yHBq/nrYWih5xw675vAPnWuDrNP0AF37OV4dW/CiA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Mm5qYmU2a0g1Y1dacmprK0Njc04vR1B1dG9ER2hyM1pxanhZb2JDVFRSRVBG?= =?utf-8?B?TTEvYWVoU25zUXpwc25BYVhMSVc5ZE9PVkw5QmJWQmRmZ3Z3WHZkNzBvUjl2?= =?utf-8?B?c3ZQd01sQTI5eGhXd3ZtalFkdmRIdHZpaWVXNFdXbEErY3ZCTXU3QVB1RWxW?= =?utf-8?B?cEFSRTBlUjh4L0hQNTFFeW5VNXhuSTkyeFdhTWVUV2N1UERDUFl0QjBFUG4x?= =?utf-8?B?c3c0T1Jxd3N3SllRM1ZCQWk3RTJCazliMEFDWm5uQ2VFcXN0dmdGd3cydi9j?= =?utf-8?B?QUJyS2NsWGxjbkRSZWlmRWFFY1NCTm9vK2FZWlMwNGRyMi9Qd0h5RGkyZXdP?= =?utf-8?B?YWdsTzZJTWNINjZCOTVLM0x2L3gvY3FpVXpFRDZXTHFFT3ExQTBGQktGYVgy?= =?utf-8?B?Z0Z1WTlYeE0rYWdJd1Jyb2tNVGdvUlJtUm9SNkhDM0EvUDdmbjRIMjNnYkpl?= =?utf-8?B?dUVyaWxKb2hQZXF2MHlHNE9zQW54VDNpa3AwWWZrUHR1eVRNMWVaWmpmVWRF?= =?utf-8?B?RnhDcE5xeTB5RWpsbFh3SlRtTy9oek9jYXkxVnFCd3BES0NkNzI0WkV6YzNj?= =?utf-8?B?ZUpmaXg5SFNGRFBNc1UrMUtxMjVTTXZocWZsOXJBK1hnQU9jemVsbC94TGhs?= =?utf-8?B?bTVPcVovNnBwSzVmWTlqWVhDQzBEeWtPa3RNdmtJeXBJMUxDNEtmdmowQ0RF?= =?utf-8?B?OWRMVWhUTFBwODVRR0F3ZDd2TW9odnRhcjMyT09HT0dhY3E0UTFDVzRwRW1J?= =?utf-8?B?SEM3QmMyTFcrb2QxZWQxRmFYcnAvVFNGWnpwNjBEbDcxNnJoSUVXUnRjeWxo?= =?utf-8?B?QXNlRXVKUjEyTHN2LzFvMkhPaklNdldoazF2eHovdTJ3NE9ZbjRZS1ZmY2p3?= =?utf-8?B?a2JFcXFZdytad0taOVZVRVd4eE83dllZczNjYXZub2pYMWRKUDgxcGg4OEt1?= =?utf-8?B?VG83MGdpZVNTOFJmeG13enlJa0ZCNlh4WUJSWS9kWTEyRDNvQlJLc2UzTm50?= =?utf-8?B?cDE5NFk3UFZUY3lKYzh0elRoUzJGdTQxck1heW5JNzd4UWc4M29sWkhLbjQ1?= =?utf-8?B?aGg3YjBtU2ZmZElpaE93T2NUTjltdHNWLytOb3creEFVc1RGaEl1ZzV6YmE4?= =?utf-8?B?T0dJMGM4a3VhNU44cU9ZMVNONmxNVjVpUkYwNXNkWFdkQTZkMEsyaGcvWlow?= =?utf-8?B?RUFEcnJkTllZTER0bTZXaGUvZUNhSi8xcVljaE84L2xIRUdJMGlSblg2RmpS?= =?utf-8?B?ZWRjblFBQTBzSjh0UTlWU1hVTndZelptd3lpNi9zMjRPYXExK2h5STVRUi8w?= =?utf-8?B?dWJUak1MNzBTUDlmQWEydll3dzRLa0RNN1dCRzlLdFYvZEJsamhueVoxMGNK?= =?utf-8?B?aTBtUGVEeENkTDRHbnE0VkliZ0tsZjJZazlyZDhTbEYrTDZaQkxKY3NRcllB?= =?utf-8?B?THRRRU93VS9RbFQ1TWF4YTVEUGNhbnFiTFo0NzBjS1ZaWE1kWHVBSVpHVWJ2?= =?utf-8?B?RTQwWXFmUnMrMUJROWRucUdDVUl2U0htQm9GZ2F0OEJtWjNDYVU1YnBod0VM?= =?utf-8?B?UzVLT01SVFJnTTdMZ1EwbUtGZlp3ZE1Iejhrc1JRKzVDWW9zNGxWNk1Od2NO?= =?utf-8?B?SnVjekJxZW9oN1J6aE1ER2RoSWxrRlZCVEpVQVQ1NEdMUWRTMURiQ2RucEhS?= =?utf-8?B?RnRWRjN3V3hwd2ZTRFhSK0pWSTFhUVg5aDFpbGx1WXltWE5KR1kxLzJFbzVz?= =?utf-8?B?VTRIREJYTlRQMDdUU3pXWVlOaVdoMWJKOWhXeDlvZ3QwSmZ0QXNmbzRjdUZq?= =?utf-8?B?WHlKOTJzcGdNVzBTVE9sNzBhMU9EbThDVUVJVFVoaGxSdDFKaENzRGROaXdK?= =?utf-8?B?ZmRKRHIyVThlUldERVg0d2dDa0VsMVBBcmc3bVNwOGs2VFdJT2lRRDh1QTV3?= =?utf-8?B?OGtmUGoyN0JOSmkxWlFIU0dzTWRrUG4zc3VQZDZCMTJJZTk3K3grMjBsU01m?= =?utf-8?B?TVlSYTIya1I2UEJraFBaNFBGNmR0WTN3K3FJR0IrUkxFM3dLM0MxQWFYamtn?= =?utf-8?B?NFZuWUEyV09jRG55eGtBcUhOeGhod3VPSHp5amJZUEZlSzF6aDM5S2REbDVG?= =?utf-8?B?QXNCRnRMSnFGeUwwamQ5Nnh6VGlGdm5tblFHd1FEVnBtVUVwUUNMV3I0N1ND?= =?utf-8?B?Nk1VekZuQ2d2bmZldmtYdHpwc0R6VHNkQmpxRUNJUnR3ZTVzVHo1STZRWWYr?= =?utf-8?B?Znhuay9hUEFBQ3FOTXVzZUsyaTRyZGluNklpbE8xQkUrbG5kUlY0aUk4NElR?= =?utf-8?B?cmh1ZHpaWFZOZFFlak94bjY3U3E3OVI4d3ZJZi9kRXF4bHYveDdtKzhVR1Rh?= =?utf-8?Q?9edVaqsPjk7d+JNi8E542X0UW0Vl8lT5ccIES5EI0TK+x?= X-MS-Exchange-AntiSpam-MessageData-1: QBNbhuHOIoc/XQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e99ab467-7aa1-46ba-aa6d-08df0e26e44f X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:31.3074 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: InTyCDq7wSyx7qwDw4C/LitXnuVgyaouISw42Mh1d5L6X1tUDHXwrBLYqQkuqpfa1otJIIi8h0RwAR48IAee3Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add the page table walker implementation that traverses the page table hierarchy for both MMU v2 (5-level) and MMU v3 (6-level) to resolve virtual addresses to physical addresses or find PTE locations. Currently only v2 has been tested (nova-core currently boots pre-hopper) with some initial preparatory work done for v3. Signed-off-by: Joel Fernandes [ecourtney: take mutable GpuMm and Pramin, not a bound device and window] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 1 + drivers/gpu/nova-core/mm/pagetable/walk.rs | 244 +++++++++++++++++++++++++= ++++ 2 files changed, 245 insertions(+) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index d88499ed3d56..99145846daed 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -10,6 +10,7 @@ =20 pub(super) mod ver2; pub(super) mod ver3; +pub(super) mod walk; =20 use kernel::{ io::Io, diff --git a/drivers/gpu/nova-core/mm/pagetable/walk.rs b/drivers/gpu/nova-= core/mm/pagetable/walk.rs new file mode 100644 index 000000000000..76c1729971f5 --- /dev/null +++ b/drivers/gpu/nova-core/mm/pagetable/walk.rs @@ -0,0 +1,244 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Page table walker implementation for NVIDIA GPUs. +//! +//! This module provides page table walking functionality for MMU v2 and v= 3. +//! The walker traverses the page table hierarchy to resolve virtual addre= sses +//! to physical addresses or to find PTE locations. +//! +//! # Page Table Hierarchy +//! +//! ## MMU v2 (Turing/Ampere/Ada) - 5 levels +//! +//! ```text +//! +-------+ +-------+ +-------+ +---------+ +-------+ +//! | PDB |---->| L1 |---->| L2 |---->| L3 Dual |---->| L4 | +//! | (L0) | | | | | | PDE | | (PTE) | +//! +-------+ +-------+ +-------+ +---------+ +-------+ +//! 64-bit 64-bit 64-bit 128-bit 64-bit +//! PDE PDE PDE (big+small) PTE +//! ``` +//! +//! ## MMU v3 (Hopper+) - 6 levels +//! +//! ```text +//! +-------+ +-------+ +-------+ +-------+ +---------= + +-------+ +//! | PDB |---->| L1 |---->| L2 |---->| L3 |---->| L4 Dual = |---->| L5 | +//! | (L0) | | | | | | | | PDE = | | (PTE) | +//! +-------+ +-------+ +-------+ +-------+ +---------= + +-------+ +//! 64-bit 64-bit 64-bit 64-bit 128-bit = 64-bit +//! PDE PDE PDE PDE (big+small= ) PTE +//! ``` +//! +//! # Result of a page table walk +//! +//! The walker returns a [`WalkResult`] indicating the outcome. + +use core::marker::PhantomData; + +use kernel::prelude::*; + +use super::{ + DualPdeOps, + MmuConfig, + MmuV2, + MmuV3, + MmuVersion, + PageTableLevel, + PdeOps, + PteOps, // +}; +use crate::{ + mm::{ + pramin, + GpuMm, + Pfn, + Vfn, + VirtualAddress, + VramAddress, // + }, + num::{ + IntoSafeCast, // + }, +}; + +/// Result of walking to a PTE. +#[derive(Debug, Clone, Copy)] +pub(in crate::mm) enum WalkResult { + /// Intermediate page tables are missing (only returned in lookup mode= ). + PageTableMissing, + /// PTE exists but is invalid (page not mapped). + Unmapped { pte_addr: VramAddress }, + /// PTE exists and is valid (page is mapped). + Mapped { pte_addr: VramAddress, pfn: Pfn }, +} + +/// Result of walking PDE levels only. +/// +/// Returned by [`PtWalkInner::walk_pde_levels()`] to indicate whether all= PDE +/// levels resolved or a PDE is missing. +#[derive(Debug, Clone, Copy)] +pub(in crate::mm) enum WalkPdeResult { + /// All PDE levels resolved -- returns PTE page table address. + Complete { + /// VRAM address of the PTE-level page table. + pte_table: VramAddress, + }, + /// A PDE is missing and no prepared page was provided by the closure. + Missing { + /// PDE slot address in the parent page table (where to install). + install_addr: VramAddress, + /// The page table level that is missing. + level: PageTableLevel, + }, +} + +/// Page table walker. +pub(in crate::mm) struct PtWalkInner { + pdb_addr: VramAddress, + _phantom: PhantomData, +} + +impl PtWalkInner { + /// Calculate the VRAM address of an entry within a page table. + fn entry_addr(table: VramAddress, level: PageTableLevel, index: u64) -= > VramAddress { + let entry_size: u64 =3D M::entry_size(level).into_safe_cast(); + table + index * entry_size + } + + /// Create a new page table walker. + pub(super) fn new(pdb_addr: VramAddress) -> Self { + Self { + pdb_addr, + _phantom: PhantomData, + } + } + + /// Walk PDE levels with closure-based resolution for missing PDEs. + /// + /// Traverses all PDE levels for the MMU version. At each level, reads= the PDE. + /// If valid, extracts the child table address and continues. If missi= ng, calls + /// `resolve_prepared(install_addr)` to resolve the missing PDE. + pub(super) fn walk_pde_levels( + &self, + pramin: &mut pramin::Pramin<'_>, + vfn: Vfn, + resolve_prepared: impl Fn(VramAddress) -> Option, + ) -> Result { + let va =3D VirtualAddress::from(vfn); + let mut cur_table =3D self.pdb_addr; + + for &level in M::PDE_LEVELS { + let idx =3D M::level_index(va, level.as_index()); + let install_addr =3D Self::entry_addr(cur_table, level, idx); + + if level =3D=3D M::DUAL_PDE_LEVEL { + // 128-bit dual PDE with big+small page table pointers. + let dpde =3D M::DualPde::read(pramin, install_addr)?; + if dpde.has_small() { + cur_table =3D dpde.small_vram_address(); + continue; + } + } else { + // Regular 64-bit PDE. Use `is_valid_vram()` because + // `table_vram_address()` only reads the VRAM frame-number + // bitfield; system-memory PDEs store the address in a + // different (wider) field and would be silently truncated. + let pde =3D M::Pde::read(pramin, install_addr)?; + if pde.is_valid_vram() { + cur_table =3D pde.table_vram_address(); + continue; + } + } + + // PDE missing in HW. Ask caller for resolution. + if let Some(prepared_addr) =3D resolve_prepared(install_addr) { + cur_table =3D prepared_addr; + continue; + } + + return Ok(WalkPdeResult::Missing { + install_addr, + level, + }); + } + + Ok(WalkPdeResult::Complete { + pte_table: cur_table, + }) + } + + /// Walk to PTE for lookup only (no allocation). + /// + /// Returns [`WalkResult::PageTableMissing`] if intermediate tables do= n't exist. + pub(super) fn walk_to_pte_lookup(&self, mm: &mut GpuMm<'_>, vfn: Vfn) = -> Result { + self.walk_to_pte_lookup_with_window(mm.pramin_mut(), vfn) + } + + /// Walk to PTE using a caller-provided PRAMIN manager (lookup only). + pub(super) fn walk_to_pte_lookup_with_window( + &self, + pramin: &mut pramin::Pramin<'_>, + vfn: Vfn, + ) -> Result { + match self.walk_pde_levels(pramin, vfn, |_| None)? { + WalkPdeResult::Complete { pte_table } =3D> { + Self::read_pte_at_level(pramin, vfn, pte_table) + } + WalkPdeResult::Missing { .. } =3D> Ok(WalkResult::PageTableMis= sing), + } + } + + /// Read the PTE at the PTE level given the PTE table address. + fn read_pte_at_level( + pramin: &mut pramin::Pramin<'_>, + vfn: Vfn, + pte_table: VramAddress, + ) -> Result { + let va =3D VirtualAddress::from(vfn); + let pte_level =3D M::PTE_LEVEL; + let pte_idx =3D M::level_index(va, pte_level.as_index()); + let pte_addr =3D Self::entry_addr(pte_table, pte_level, pte_idx); + let pte =3D M::Pte::read(pramin, pte_addr)?; + + if pte.is_valid() { + return Ok(WalkResult::Mapped { + pte_addr, + pfn: pte.frame_number(), + }); + } + Ok(WalkResult::Unmapped { pte_addr }) + } +} + +macro_rules! pt_walk_dispatch { + ($self:expr, $method:ident ( $($arg:expr),* $(,)? )) =3D> { + match $self { + PtWalk::V2(inner) =3D> inner.$method($($arg),*), + PtWalk::V3(inner) =3D> inner.$method($($arg),*), + } + }; +} + +/// Page table walker dispatch. +pub(in crate::mm) enum PtWalk { + /// MMU v2 (Turing/Ampere/Ada). + V2(PtWalkInner), + /// MMU v3 (Hopper+). + V3(PtWalkInner), +} + +impl PtWalk { + /// Create a new page table walker for the given MMU version. + pub(in crate::mm) fn new(pdb_addr: VramAddress, version: MmuVersion) -= > Self { + match version { + MmuVersion::V2 =3D> Self::V2(PtWalkInner::::new(pdb_add= r)), + MmuVersion::V3 =3D> Self::V3(PtWalkInner::::new(pdb_add= r)), + } + } + + /// Walk to PTE for lookup. + pub(in crate::mm) fn walk_to_pte(&self, mm: &mut GpuMm<'_>, vfn: Vfn) = -> Result { + pt_walk_dispatch!(self, walk_to_pte_lookup(mm, vfn)) + } +} --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 317DE3546E4 for ; Wed, 9 Sep 2026 04:00:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926443; cv=fail; b=hnR/Ey3ynlB50J6qABuL4AMn1lKspwLAOMt0NCV121oegdI29j4T0Fg/xR+vFPhoJUDIs3ITD75Z2N22WVp1pP1yE16tSIxZQLmXUzTGjC6LXNIpz5HsSkXW0w7CYUUhAShj29/0oO/SwahGNII+RUEZyTgVO9uFGfr4g2mDIUc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926443; c=relaxed/simple; bh=uTId/ZiEvBX/6TiSrFihtRlYhqMoZS+3pDKKnlGRdNs=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=TayxnSjCb79B3pehcliBPSSty64V9HItUGNV15jAI4pP+8AdjygRvlc+V4/lG6pt1b56H2156l7P73ISzF56QcjSdy5G7xmT9MFh/XdWAtamRbS/yxmwK73cYoIPLFO38bShswLQScJQjEy+nxGkUoaYNBq45VaBMFjU9JR2Yt8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=PcHAgtV9; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="PcHAgtV9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NC7+pRBw3alcPWj5ovLF0aK8nLed27Au1pP5Ah3A239M0PJ9SfAXIKXmFJLQbMsn3R21DMWqmWZsqhhk2nzN461cBkUhMupvM0361Tve3LIHscPk2SzlYTUUsVsj+U0/9W9Fcm1S4L+Dn0spBhkjp7m7BVQC6+gVppCvpnqahG5zi5gNOD2Dtiu3jKBsg1imKAYjiTotYyJxHorT4V6rDutAVHvjEsPyuC/qL4MTbKmoZg7pNGE91ppZCJgL/RtNTfVOx+dSeIvTS3ex7sprXCDF9r2YPrjoYEyEnVBacHhCZQ+X0iCUZfdfsuV8nivkzL3MOgn4Ss3ux9ieteZVEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CxpLOGB5kyKzh+ei6U32cS450DmPV4hqpgm+qmKUhgE=; b=WCmy0zrTiID8awOIxeRQKTUG0ZoOCc92BwvYDpkAF3DTyuXrt7Y/lO64koQ0EeQvRWoj7I82CW/7PNxJYNHXQwQbb2k8dj+Px7r3k1UnNlkvsC3ZBbr/HRkF/w8j7kYL2HZM1CwzspyjB/n1uvU5dI8YrXc8FtK80mtMV+EZCse1sQXGpzqVakRLfXNmL3aUnpNwPDGUq5Ek0te9tF0ZXPX+4Tv/RzReF+c0DUIy4UTvYdQA2F1VYhP+B0WjCub2w0OaRgS7mKrKWnapPlOxpAXQMCOo+StUapDpWd0s1XKuM1l5sQ9MAVMVW0K201TNIqLbCi3WfgavGihPniXZaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CxpLOGB5kyKzh+ei6U32cS450DmPV4hqpgm+qmKUhgE=; b=PcHAgtV9u+0kUitE/1VZSlFL3lvZsKCkdCvp6XbRDVDz1efdbe7cH0Ehm0IJRs706HRFJhVrJZDeoMQSEtyKrdl7hhFDKGtARyjyk3qYG17XWmW7GXCNxdmA7tnuDytheKTWs86IqZzM6+FaPV2SkmDHGWSXR/awzxEFiDRKKyCeJ7Sf7gx4BmTg3MoC0GHs2Urcn7uHPy22LASkJGHhgatEvyODfukZBB+WDZlYNQcq6g/cj7y5Ad0wSrJgmgcxBi3KoxQhmIkimI+cu/FlPaD/F3Zi5mpXTqGxjod+EdQE5iab+ooCIWWDUj/am069co/Ivui1yv+tR0wBPumTIA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:35 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:35 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:49 +0900 Subject: [PATCH 11/16] gpu: nova-core: mm: Add Virtual Memory Manager Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-11-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P286CA0088.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:369::19) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 94ef3e38-589d-4a54-217e-08df0e26e676 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|6133799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: kvoX/ZtOMZ57RIqipKsMQY68JYbW+IpvAkHBMkr/i6EAw+VeXHLZohlJJXkc40kyvoApjcGOmmvc68loZQoFaI1OjuZ19+gH25uFPqSAzNiywlM5mfbjABlhjDXNf+Sqm928adLroF3EeTy/lReMA/KB0XP6yeCIKxICJ47FJPLWINpalpA9tCC3tWhn3XIDToFx0LZwn2XrX2gJA9QF/ZhIyOzhRO+DRuTOS9LUE+awYrO2irutKJnSe9K3OswLIfPp0Hl1AbXPVMv+M7SA6SoXa/GQ5oMKgfcuMlpDI4hgr/+un+3+Hi35ZW0trzCFAwNjMBb3CYYS15MSC1nb9QgJUyIzkCemRnoQOgjBIvTSp/AsqtrFtyzHvR7Gkn3/sI1J5Q+azh8zkshGSv3QryDMC3TSxHW1BbNvLzzlYd/iQvdAK7pXFkykoDr1ORYVeHvHnUHnnEtjGhvSYsDpxUpy3PMSG/1O/xL3+T7iOOmiwmcv6zv2BcAiLttdJPWLZpjHIKVQofJ653pXrND/GlVxCL1uCUDgkMTrRy1LS6TJYRa0jQL4MtlEutFCTgh6v+jI5kkvu9I9fvLqjs21Kryi848SRcYOIdcP7KadMLjupUbOcBXxrYQSlo2aKnAH2OUtgi4YpNDOyyRSvzOSMLD6FoJUQRUlD3mb5VaQKDI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(6133799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MDdHbnpKU3A0b3o4WGNRWU9QMm1jMUJrdmt0T1BMaGNQcXhsTTVnVWpVeXB2?= =?utf-8?B?YkxVYVJuZ2NLMTN0YURjNTNPQmZoYTJ5cUkyWXRkMnIwam96MVgreis2cXN6?= =?utf-8?B?Rnhkb1NtcWFvSExpOUFuY1Q3aXFJWUJqRlN4SS85TU50S1czVDNzNHBaaXh4?= =?utf-8?B?ZVFiaXd3K0RMVFVxa0o2d3FwczRoQUN3ZnEyQW5FQ0xNUTJjNVdaRC9BSC9F?= =?utf-8?B?aHc3SVJxV1ZCcUE0b2hJbTV6MXNaSWdJb1Z0UTl4OVFOaWJoeXhsUGJoQUds?= =?utf-8?B?Z283b0VZOVJXWTN2WUVnMFNqeHBkU3pVdWoyVXNvYkxKd2dVcG03ekJYOHFl?= =?utf-8?B?dkQvV0xCa0xTZnFCVzlCYmQ3T2s0ZGZPM0Q2bnQvRnFXOWRrdkNyRS85R0Y0?= =?utf-8?B?SHd5bVRsTmFRNnhYaXhUNi85QUNDYjhZWDlsN2RVQ2JLbHY2cFlOTEh3eERX?= =?utf-8?B?QWsrT1lCNDhLY1hIcWZmd0o5eGgwdFY2NHArSUdDcVhzazl1OU5HRHk5RW5q?= =?utf-8?B?aGowejhVc0dNTGhaSVVxZmg3N3R0S3J6bU5HWTk1RHpGUVpDa3VuVjVLd2p1?= =?utf-8?B?UnRPcVBDUmpJZ3VaRGx0N3JEeTBjTVNlOHg5WmJvNnRvb3BVeHVoQUE3bW5P?= =?utf-8?B?N0hXSXRBdUEzUURIQ1JzNlJjcFNJOXdkNHVWSElIaXpKYml5QTFFaFBhWGtH?= =?utf-8?B?bkRsYWFlWjdLVzMzL1dsNWNFWitZVFpiM2xSVUQ4TlVMOUtWdVFJYjBweTlV?= =?utf-8?B?cmZvSWNLSEZWZVJPODFOWTM5Q1hRWk9SZExTb2NuZzVXVEU2L1M0cGUreXEx?= =?utf-8?B?OUgyMHUxeStoQ3d1WlhhSUtxSUJkdmlsTlF1SzFSVU53K05hZUh1SUJ5QVRZ?= =?utf-8?B?QnlWTWttNDVINm5TcXdiMXRHL3JjSVNPNTUxaWkxTVkwWWRucWRaZURuOEcx?= =?utf-8?B?Q0tjYUZTdk0rSzJuMXFJcXE3Z2lGM1FYRVQ4WDVCV25HSFBOWmMxM05DalBR?= =?utf-8?B?c1JWdkhhd3lIMlNDM2xGb1NQZ3lUZ1RHQU9lZ1E4aGZibGJwVEhtQXpieDJE?= =?utf-8?B?V2ZBeS9RMFlRZENVTEJmTUVEVjJ0Zm81MUtUbkYrUEhhSUxOTmZmQ2NNYnZG?= =?utf-8?B?Uy9OekFpS2N4K1V6U1JUNVJNTm5vMlBZQ1lRZ29rZ2ZnajdOK1JhYmJOUlh3?= =?utf-8?B?MHlTbnF4bisySE9uMkdTNHNYMXd1bkFTRzJaNVZaSnQ5bjF4aE5FeWZBbHBH?= =?utf-8?B?UjVLSDlRb0RvNmtEN1Y5d2dKMVRtMnhFR1FTZnQwREJzQnhPeC9hYzdUcW1x?= =?utf-8?B?MFpCUlE0R2c0ZldqSWErRnVaTUZ1RUJtNjFwMmlWSkY0NHZ2ZytEWWxkWm1v?= =?utf-8?B?SW5lQ1h2V1o2NENQTDZNcmJNLytuU1FrVHk2RjhPRE5jVnkraS9LR3lJc0k2?= =?utf-8?B?Um1rdVQ2bU1EU3hjNjdYdUJnYlYvR2lsUTJnTmNHaGw4SktLVnZrOGhhc3FH?= =?utf-8?B?Rm1MYnVmTVVEVEppdVlFeS9uTndkc0VZMCt4NmpZYVdKUjFqUy9oclcvNmJT?= =?utf-8?B?QzZKdDV0SzJueHBlNy9kMkNVMnhVNEl0eGN5WlF3VWwvckpZL2J4QWZpUzVh?= =?utf-8?B?UzFua21ZeDRJVzF1eGorZk4yTGJpRmhtcnlwczUxNDFMdUdveTkzamFxazBu?= =?utf-8?B?OVRPZFd0SVNyVE81ZHJwUUdHMjNzdVM0V2JaQ0lET3FrYldmRGJsaUg2Qlgr?= =?utf-8?B?NEpzdEtCTjRMOWdSMC9lVE16TXp4SmdRbDRWdk1uNFpzZVhhTldmeHhSR25P?= =?utf-8?B?Y1JRdGJqZDBYaWUxMDAwZjRkT2N3TjJqTjl6VXZCenFyNUp2TTJ6YzVwWEgx?= =?utf-8?B?TlVzbGZiODF2MmdEY0pCb3BTSTNQeFRuZEwzczdpNWwvdDc5R01iamNSUGtV?= =?utf-8?B?bFdBckhJWFRlVEYyK1RCK3pLbU43ZW9ySExoZmVlQ1ZLMDVQMVlZZFdHK00w?= =?utf-8?B?NzRwY2RmYmN1MDgrTTQxRjZUVE1WNFhvdEJJUUsxUllhWFdmaUFGODNQV25v?= =?utf-8?B?UE5PUzI3VjJTaWpNQ1M2dzVWdjJ1R1F5WUhpV0JhY1FjNXBHQlRWN1RmQnBE?= =?utf-8?B?UUNiTm1hQ3hNK3l4a2QzWmIvRU9xM0pPNjh2SVVmeDJTMmhtampKSGN5OW95?= =?utf-8?B?TmhLN2hSNXQydzIyUCtTcld3ZVU3cWkzL1NibHZleWNHd2RGOUh1ZnlMQmRG?= =?utf-8?B?VWNjMXFpMlVHdEN1bnQyVy9ITkM2K29uZldUaVlCTENldjVPSUxraStiNXRr?= =?utf-8?B?cnA3V3RMeFJNMThhcWFmR3I3QmRrOGFsN1p1VDVRWC81UXdsZmZkd2xUaElT?= =?utf-8?Q?9VY29vBoiSaaGwybbTQBXMs4AP75TU3h3Kr4MKA9ik6TY?= X-MS-Exchange-AntiSpam-MessageData-1: Ite3DYbrMO9P9A== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 94ef3e38-589d-4a54-217e-08df0e26e676 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:34.8980 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tngFR7jFsqChuY2OmldqlxUxHNTSjrFlkS0ReJpckVBtxjmv1q1BdA19U6TUIaUSD66T84zHXWqEl++wz+g0SQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add the Virtual Memory Manager (VMM) infrastructure for GPU address space management. Each Vmm instance manages a single address space identified by its Page Directory Base (PDB) address, used for Channel, BAR1 and BAR2 mappings. Mapping APIs and virtual address range tracking are added in later commits. Signed-off-by: Joel Fernandes [ecourtney: pass mutable GpuMm to mapping lookup] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/vmm.rs | 58 +++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 59 insertions(+) diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs index 377cc9d44702..2cf37254fbb9 100644 --- a/drivers/gpu/nova-core/mm.rs +++ b/drivers/gpu/nova-core/mm.rs @@ -65,6 +65,7 @@ macro_rules! impl_pfn_bounded { mod pramin; mod regs; pub(super) mod tlb; +pub(super) mod vmm; =20 /// GPU Memory Manager - owns all core MM components. /// diff --git a/drivers/gpu/nova-core/mm/vmm.rs b/drivers/gpu/nova-core/mm/vmm= .rs new file mode 100644 index 000000000000..a7911efb65a0 --- /dev/null +++ b/drivers/gpu/nova-core/mm/vmm.rs @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Virtual Memory Manager for NVIDIA GPU page table management. +//! +//! The [`Vmm`] provides high-level page mapping and unmapping operations = for GPU +//! virtual address spaces (Channels, BAR1, BAR2). It wraps the page table= walker +//! and handles TLB flushing after modifications. + +use kernel::{ + gpu::buddy::AllocatedBlocks, + prelude::*, // +}; + +use crate::mm::{ + pagetable::{ + walk::{PtWalk, WalkResult}, + MmuVersion, // + }, + GpuMm, + Pfn, + Vfn, + VramAddress, // +}; + +/// Virtual Memory Manager for a GPU address space. +/// +/// Each [`Vmm`] instance manages a single address space identified by its= Page +/// Directory Base (`PDB`) address. The [`Vmm`] is used for Channel, BAR1 = and +/// BAR2 mappings. +pub(crate) struct Vmm { + /// Page Directory Base address for this address space. + pdb_addr: VramAddress, + /// MMU version used for page table layout. + mmu_version: MmuVersion, + /// Page table allocations required for mappings. + page_table_allocs: KVec>>, +} + +impl Vmm { + /// Create a new [`Vmm`] for the given Page Directory Base address. + pub(crate) fn new(pdb_addr: VramAddress, mmu_version: MmuVersion) -> R= esult { + Ok(Self { + pdb_addr, + mmu_version, + page_table_allocs: KVec::new(), + }) + } + + /// Read the [`Pfn`] for a mapped [`Vfn`] if one is mapped. + pub(super) fn read_mapping(&self, mm: &mut GpuMm<'_>, vfn: Vfn) -> Res= ult> { + let walker =3D PtWalk::new(self.pdb_addr, self.mmu_version); + + match walker.walk_to_pte(mm, vfn)? { + WalkResult::Mapped { pfn, .. } =3D> Ok(Some(pfn)), + WalkResult::Unmapped { .. } | WalkResult::PageTableMissing =3D= > Ok(None), + } + } +} --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011033.outbound.protection.outlook.com [40.93.194.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48B4535674F for ; Wed, 9 Sep 2026 04:00:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926445; cv=fail; b=SO5FopD/O/1aNRj5GZSNER/wIVnLCFDA6DHpaSYDaS9HqJdSZ8YWFV6WI4pG6cNBo8n0LR5J36KU98dEALamDxqfk+o9Zy3Q3W1wACCcPJMMQOYsjNSKmoUWBQ4RISk9Y1qOjG+OfFIUXd7Kf5yyo1eeCN0yEPBdVFZ8x2ls71A= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926445; c=relaxed/simple; bh=smK4KPLVovxKH6LrlGI4QwCzB2p/268cIwbwNPEGMlY=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=csuFhF8IkuA0KQ8kwIhgRbDjnpbnV1szs/ZWXk0XMfgFz08dpg5Am8uDx7K9R4JU7B1Uo01156FmwIK6ugkfsoV7aD7LZYTLN4tnq6kCe+ELyRgt7PJlxZ4UWGpKh7u4YPSD69j58vghkBn4/cQ/aVBi2ji6v92+s199sqxRgoU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=d9a39H7A; arc=fail smtp.client-ip=40.93.194.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="d9a39H7A" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KDJ5j1Ew4qjqZ9C9/aq0oc3rktN2vuwbtACIXbMdAS2ujkycz8WR9vkHYau2mv/Ecs+jlRpj2kX6fZQoSBr+zY1/ZNgu3aliw7sM2LdCPxi9XkpKmkqqoOvceFf9OgaviJWRbMcxz5FIFONkhiRqldx/UQ1VCYZJRZW9gjd0KkdkIsOYfCmNHw+XcBbWOWUW+bY5bxYwdYdFxHfznJATxUiYDiOVRBVMwcr4e0QbwGZVxuh5MT801umU5FWEnGMKrWnEwFMDaK1mLoVWXmvmMDOgV7x1lfSd3CF097JN/5f3lk6tpMDWEoKcWjpZTjFUoFaDoga4wy+Cp4qf2AOhgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ge1rr+fDkU8vZvZvm2ZD4imc4OicC9I6WSnqFT3xFT8=; b=PIUZtWW98DxbjZIp8/s0xCaU1SgMIvhRW1j/xE3tCEflIjnJzCtU1JqIKMeaoszIFrMECzrI8MxplUjkVcuCd/PwTJiWd8cUfZZmKO8oNMJyvrle7ZmdUz3Zu2SUoSjU0vXwitPXJtQW5PQLPyiGavomwacwWdd12pNJVc/MdXfmwS582fvtDTHbcNs8QTcE8zsPNVV4YwC+0uc8/7ngnQQ+2uc4qDEpE52ZQfwgWQqpcMZ/WkDyzRWL9+GP4eiI6E428xV3M4VlRVRW8RnA8Zt1anQA1D7CBAhpubQDNP7yv3n46yfNHVRy2LHmm7/J4p+kJFgiQRsveA2/Ke3h0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ge1rr+fDkU8vZvZvm2ZD4imc4OicC9I6WSnqFT3xFT8=; b=d9a39H7AxOYAGRxZ/VSwDO6CTtL/DGx3GVIpq+RoGC6D4O6uDaZIaXAohrcH2L9Z3/H5bzSVf92rUOMCqF22k8wGVk5xJBcBKjcwevuOQOrUOzZpJjQKJ6GigIxwkoDHG5GJ9PGb1y24nn35cfrKI1fbQ14nJZAmsPAuXb6p7VYRi+I96dOjGIVHWa3m2TEnx8QgaRpmpmADpogbfCg0l9+XMw+SmNbFbI7hzuSFLducRVZDZ8cgN1xX26vIJNHb7bZAC0VTpwWEAgv8mlZahuevIdV/yf2O68mGOWLtrERCFYmYs3TEEJr/YYNPR6bhWa1pVCkFJeBqSRlnIbbVtg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:38 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:38 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:50 +0900 Subject: [PATCH 12/16] gpu: nova-core: mm: Add virtual address range tracking to VMM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-12-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TYWP286CA0003.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:178::22) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: f615b77c-b541-48e2-f173-08df0e26e8a5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: flC2NY0w7+x6HFQYe1hnW76S1EuUAelx2uFFbyP+Iawbvw7414q54oDZ0NoOxmLA0qtcPO606/ixA6uo1qUECPkISS2akK1TaySBPTS0OD9LNJarsRYA+CcFWSCU+BtpvBSCtq5Uz1fkfp27jgTOWdCVAB1Y+nobWQjcgZZcgMM5BpeHcs0HVFEO+x3AAR3GQBPQLVxZnGQmcjA4LJKnTRamaCqjD8oX377zjRWl3MMj4JKwkdTR15G8pN2Gf3AKzrTVXabnP+sdN+tcUR+DZfrC1AFYoh0ZE4+fDranYst5dZ7YuOfE/rcXZ3Gb6evB07hj39XoEtJX7p0BHlJr1XwPjnopEy6W1YRuvXKSH9Z9CUyD6McS1fQj2+AqfcMeUJ4lBw2WTmUUvmhC5avdCzNhc8jfmDC2w0E/FKMPIrh7/gM8gNsfafx8KOKi0nJOo0WQYAl49p3VLkWd58i7YN8WkhE1Jo7DnU2dzTCQfNmG0mKFf4pZl3J/2FrMujtg6xZHQa08vG75uFxtcbTA/Uxwy0klcFA0j8e7mmNEue+WPLUhETKS4r9NR5dhGrOqVOT2YBbB7evqy9QON2FrzcdrLfW4DPp5DkWplzDfW6Itx//VqGzi0ko9f/eKQpo3Lsi/GQW19c0hosqtb1pZJFpepfjms8OAZ3rm63eqboY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UXlpVUE0bktTOXhwRVBDVXp3VGpUM3Vvb0dzVHpDNXV1d1JpYWtWTjFwbmdG?= =?utf-8?B?OTFubE1VMzdrU2tRSXlzWmRhMm1Yekp5Q1MzbnJwMnZWMWt1Yll2SzQ0NlBS?= =?utf-8?B?eU5ReHRhUlpMVWkwbVRJRkY1RjU3MThiYkpVNmZNSFY5MlhacUF4MEV2Qk9q?= =?utf-8?B?WDlZZytyOUo4SnI0T2hzN1dMTnVyWWJWRnZ4eU5ET3c4cTZCdVI4NkF0MElk?= =?utf-8?B?M05wTldqdDMxZlhnMG0rUHBkV1ZtdWhuVkFSUE1TVVloT3FBVWRiWER2RS9y?= =?utf-8?B?Yk0rVm43dXB0ckVzank1MnZZTGN4b3MwZ05uSEhLdm1NcWJxNHhCOWZmaHJr?= =?utf-8?B?dlkzT3U2YzU0UlRWaXNjTnVDaEx6cDdoYnYwMkNIcUM4Y1I2V2VFcUxBZUpX?= =?utf-8?B?QWd4QktMenZxRk9zZWdrcGZsTzNncVY1MWxyR0tveVUzL0dlaUg2cUlVYlFX?= =?utf-8?B?M24xTXFCZDA5V01wS1kwdVF5ZDdVWCtndDVaMmlPU2c4WjBJemY1K1prZ0k2?= =?utf-8?B?UHk3MmNJYXU0Z093NDY2YmR3TTR1bStJbS9idFRpanU0NWFqcy96cm1CeDRI?= =?utf-8?B?bnZmR2ZNUnhSMTdiTUxjTS9OeGxZSmJDNE8wQXNUUGFyYlZLYmNTV25oTnd6?= =?utf-8?B?ZzJRbGNzOWc0aVplekN2RGhpbkJ6d1RlNE8vb0RyNlFiakdsd3hvdEhQS2lS?= =?utf-8?B?QWh4QldWMWV3eGtFd05xK01QMFpVVTRpS0NBQlMxN1UxTU4rVE1GVXpnK3Ir?= =?utf-8?B?cTRkalh0Y0xmczhaZE1vTFRndEZjb3BVOGp5QWZBUWFKV2hoT2p2SEFtVStF?= =?utf-8?B?OHMzbjUxaXcxZ0QyaW9uVFhKWnFmTmJ0TEIxc0hweGI0Z1hoTHFhelZHRjRn?= =?utf-8?B?RVl6dE5VK1NGOC9nR0NDMXNMRTNBNnNReGhJdUd5RVE0QjR1cmJmblY4Z0Vr?= =?utf-8?B?NGgzMFJ1ZzFwZUxXNTUwRkovcDdiS1lDUHB0SzVQYitQMzZEVTZFdWw3N2dM?= =?utf-8?B?RHBhMVUxUWFiTVpNZUJMa2VsNExyemwwMTVwa1NaeE5qeGVDWnlNejFTRFcz?= =?utf-8?B?a0lyOHN4U0pmS1YyUmJsMS9qUisycnAvWGRvcUdML0JGQzVDVy9TdWdEaVV6?= =?utf-8?B?M0lEYzY1Z0YramlIVjdHdFo3elBxN2RpUDRZZnBVTEdjTlFZYWhJWGk5MWc4?= =?utf-8?B?OFh6cHNrQTAwbzZKNlFQTG5tRW9xSFNFaThIak1DeTg1YVd2ZGw3MitYL0hL?= =?utf-8?B?QWtoa29qTHkvNEVWcmFjZUtwVzh4d3FQa1FITm9pVEVDcS93bzJ4ZzcxRWU3?= =?utf-8?B?ckJOUFpDUVJKM3RTL1hNbWdRWHF3L2VMMkxhVmRuMUMxWEcvZW5WdG5XSmI5?= =?utf-8?B?YjhjOUpKdTd0K0lhNDZnQkJybW9aRHNwNEdhUXNtT25aSFE2L0JtLzRiV0Js?= =?utf-8?B?UXdQWkdlcFBENUlQamo0YW9ONnVLd3NQL3laa1paZXEwMzVkU1A1SzMwVUI0?= =?utf-8?B?dDhjK2JPVm1jdERUd2phTy8razFKeURjRFJhODdkT1ZQbEJyNDBVU25CS2RO?= =?utf-8?B?OVRvekRreCtNNlpNV1JWNVh1NnljemR5ZGp1QWMyQ0VWc25TeDlHczdpd2dC?= =?utf-8?B?Uy9WYjAzSmw4NEcxMDFKSExDL002T0FrSzh3UGVYeFVveHpvYld6Y3NEbGxY?= =?utf-8?B?M25NWEsxNEZDdkQ4S3V1cTJWM0JWZXdUWE5yVlVvQVVqN2NUTWxIZkdkd1Bs?= =?utf-8?B?WVpyZzNHYitCeDd5MVpGWnF3RVBNM2dCNzh3OWdLcHFkb0xIMW5PSWtLRjhT?= =?utf-8?B?c05xeC8wbElvZGtPOEtaaUxublJOa2RvQzZodGlnUGIzVFEzcC9weWl3U2l2?= =?utf-8?B?L2JNQno5MVQya0h4ak85VS9mcDZKZ1lpRFFYSm8wQzlxMmx5cHR0Y2hZMzlz?= =?utf-8?B?TVA4NTY4aWlJUXd0VU94MXFwektSd2dlZkFKNDNMaytRQXM1ME41VWNGaHpN?= =?utf-8?B?TFZFaTFRT2VxSkRxYU0wYmxEamZKR0ZBQjZzKzJTdnRwT1Z3aGVVTmVIWm95?= =?utf-8?B?SkFYZFJhSXUzdlNyQzQwWjRCWUZMZmhzVUlXMWF1VkFoSm9Lenl1Mk5JYWhC?= =?utf-8?B?Z0tMSzRhOTRaNFNhaDBqaU1qNDl4a05JeGdYSXZITnZhLzFxUHJ6L1RiYm9H?= =?utf-8?B?RndPTTI0aFhvQUFIT1NCa0J3K2NTS2x2L0NSL3RwMDZWM3VWcjZ6Y24va2cz?= =?utf-8?B?ZFhodzdiKzEwRjZ2YTFLTHVQTWptM3VyWFdWUWlRVEFiSFhCZjdjeHpxUm1x?= =?utf-8?B?YzlZUTBJenJXc2RQaDlzaUxIbXZsYVJsZFNSV2E0eUU1eW51S3Z3bUcxNXh4?= =?utf-8?Q?whfXAvllwcFGnmmB6jjUUgrnOF2FUP8RdfmqMSHuy2G1d?= X-MS-Exchange-AntiSpam-MessageData-1: Bydp41r+w95Y7Q== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f615b77c-b541-48e2-f173-08df0e26e8a5 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:38.5757 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: miBzYIdV7ycWLr8PpUqdPcACO2lSd/i/PafgiXnhfysgOefq4qyFgMybDicUljHQF0G8cinb3ngNY0VGLfRCVQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add virtual address range tracking to the VMM using a maple tree allocator. This enables contiguous virtual address range allocation for mappings. Signed-off-by: Joel Fernandes Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/vmm.rs | 83 ++++++++++++++++++++++++++++++++++++-= ---- 1 file changed, 74 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/nova-core/mm/vmm.rs b/drivers/gpu/nova-core/mm/vmm= .rs index a7911efb65a0..0bcae29db4f2 100644 --- a/drivers/gpu/nova-core/mm/vmm.rs +++ b/drivers/gpu/nova-core/mm/vmm.rs @@ -8,18 +8,27 @@ =20 use kernel::{ gpu::buddy::AllocatedBlocks, + maple_tree::MapleTreeAlloc, prelude::*, // }; =20 -use crate::mm::{ - pagetable::{ - walk::{PtWalk, WalkResult}, - MmuVersion, // +use core::ops::Range; + +use crate::{ + mm::{ + pagetable::{ + walk::{PtWalk, WalkResult}, + MmuVersion, // + }, + GpuMm, + Pfn, + Vfn, + VramAddress, + PAGE_SIZE, // + }, + num::{ + IntoSafeCast, // }, - GpuMm, - Pfn, - Vfn, - VramAddress, // }; =20 /// Virtual Memory Manager for a GPU address space. @@ -34,18 +43,74 @@ pub(crate) struct Vmm { mmu_version: MmuVersion, /// Page table allocations required for mappings. page_table_allocs: KVec>>, + /// Maple tree allocator for virtual address range tracking. + virt_alloc: Pin>>, + /// Total number of pages in the virtual address space. + va_pages: usize, } =20 impl Vmm { /// Create a new [`Vmm`] for the given Page Directory Base address. - pub(crate) fn new(pdb_addr: VramAddress, mmu_version: MmuVersion) -> R= esult { + /// + /// The [`Vmm`] will manage a virtual address space of `va_size` bytes. + pub(crate) fn new( + pdb_addr: VramAddress, + mmu_version: MmuVersion, + va_size: u64, + ) -> Result { + let page_size: u64 =3D PAGE_SIZE.into_safe_cast(); + let va_pages: usize =3D (va_size / page_size).into_safe_cast(); + let virt_alloc =3D KBox::pin_init(MapleTreeAlloc::<()>::new(), GFP= _KERNEL)?; + Ok(Self { pdb_addr, mmu_version, page_table_allocs: KVec::new(), + virt_alloc, + va_pages, }) } =20 + /// Allocate a contiguous virtual frame number range. + /// + /// # Arguments + /// + /// - `num_pages`: Number of pages to allocate. + /// - `va_range`: `None` =3D allocate anywhere, `Some(range)` =3D cons= train allocation to the given + /// range. + fn alloc_vfn_range(&self, num_pages: usize, va_range: Option>) -> Result { + let page_size: u64 =3D PAGE_SIZE.into_safe_cast(); + + let start_vfn =3D match va_range { + Some(r) =3D> { + let num_pages_u64: u64 =3D num_pages.into_safe_cast(); + let size =3D num_pages_u64.checked_mul(page_size).ok_or(EO= VERFLOW)?; + let range_size =3D r.end.checked_sub(r.start).ok_or(EOVERF= LOW)?; + if range_size !=3D size { + return Err(EINVAL); + } + let start_vfn: usize =3D (r.start / page_size).into_safe_c= ast(); + let end_vfn: usize =3D (r.end / page_size).into_safe_cast(= ); + self.virt_alloc + .insert_range(start_vfn..end_vfn, (), GFP_KERNEL)?; + start_vfn + } + None =3D> self + .virt_alloc + .alloc_range(num_pages, (), ..self.va_pages, GFP_KERNEL)?, + }; + + Ok(Vfn::new(start_vfn.into_safe_cast())) + } + + /// Free a virtual frame number range back to the maple tree. + fn free_vfn(&self, vfn: Vfn) { + let vfn_index: usize =3D vfn.raw().into_safe_cast(); + if self.virt_alloc.erase(vfn_index).is_none() { + kernel::pr_warn!("free_vfn: VFN {} not found in maple tree\n",= vfn_index); + } + } + /// Read the [`Pfn`] for a mapped [`Vfn`] if one is mapped. pub(super) fn read_mapping(&self, mm: &mut GpuMm<'_>, vfn: Vfn) -> Res= ult> { let walker =3D PtWalk::new(self.pdb_addr, self.mmu_version); --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010013.outbound.protection.outlook.com [52.101.56.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71A4426ED33 for ; Wed, 9 Sep 2026 04:00:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.13 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926451; cv=fail; b=RtkWd6sNd8UkXQFWXf0jNAQMs4zaPSalLuWRzglMeRaxiNZpufvGY6HS2KrOvHoXRZGSh41ZVOCg7PmdZVoZ9Ty63n/kigjl75gvPSImPcUDT4HWaUvXiIlW/DaUbIk1nFZmMom+dFKVYARY9CaxZL8kclQFpzYH9RQ2NjihBrY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926451; c=relaxed/simple; bh=T50u+aOaV1mvTymmudI5PHv8bxKtzhxkun4b3P89LQc=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=FiBv6QbFuNwzyjTzCn85CG+nN/xhD/20CaQQLjHtsRLVGgXC7z1l6A/0PzeBimuf1H+JeTsKWB6ZRlacPIFX/afin5adbikC3mjZf8kWbBs1BvqY5GlFRUZ3nhqAtgxIVkN8g6/n7jOn4loDp0c66al3mBBXZfEdzwpq3cXSzvg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=U8XpV1Tj; arc=fail smtp.client-ip=52.101.56.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="U8XpV1Tj" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SOYXaqtw2thz3k/FA4EjdyJ6Dlxx+qFkgc8hNdX4YgRVeO1jatXZUXdmmNUglYhS7q4VUITwUQWfGcDL7Cgt/vLqSBaBdtvv0x7qTAYabiQDiGjIL7hzVv2g3StM3nO+IfOAL49hCbJFzoRr0MgVOyzdQmB+Pl7sLUgNucfJCCfSZSzmm65VfeosKF2GFPDkUAIWhcm+jq47KisBGpB47rJRSRqrv/lDAj6WCwpsX2RlPnV7jJMt+COQnGj3lI65T+0++X7QWvDH1LT7gJ0/2NebdKp4Ec/RPi6oe+tt8UzX9H5sidBuHFYCmGN0LXhCzS0YNxex3YQRk/nTyhuVzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sMLSFf5DDxqcX7aFG3bBS/if6gxHZWvMp3adY7mnJew=; b=fo7LnghBDSR0rKIejA12U+eWGQK1RJwj4O3D/awZBEKJTWi8uL8Z4bT42oOXQamKRcfGjonvjv4BQnjme3VQMSaDfRBLrpwi8L5EuU/u9H/0tbUgf1sWF7Gq1pJhuOhOaqCphLrNR5m0Q9AvELuZgYBtRPDJ0mq5uJgCRQPWucEg3MTwffw5HzWzo71e/u9V2DcGFB5oC5xl/Oujdf5XbCqbPFkFlgSbM3lZsqlt95HgQC+7FXmyGzdTmYCRT8zbjp/MqoNuFDpCKyj7Yl2uDmqrhv7eX789+VwKksIAit8TiId3hc9a+rKvQbjQGASt/pcniNUqF02f4KxqRPOVfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sMLSFf5DDxqcX7aFG3bBS/if6gxHZWvMp3adY7mnJew=; b=U8XpV1Tjz0s8aYtD1J+yq1lRTyk1B9tIvVnrYyrXQaysyosp1oJptap6Ce95qamKozY70tU2Nwc7z/7SEfTvFZmfa/Gs8l8bcdDcqncy1aDYBJI1DjGmRomAtFIvmH2MQmHM9qKUO/66bZ+qrkO9r/FdxTYfleq0NgHy/0flVc+mafLWKBY6QmFgUC8KGwp4j9gcNG33Z5k4DklH6QihKveBKsAmrsUh+TF2JGQ6Aw/JSP7KvUHxxkyc84WN4hAmDeeDqed92C+WaxEjqY/Logs/vCCeeYg18SSvo14vbf0bcrYbErm8J/MKx/pHrrQ1wuTB/PXJ1P52Oz8fGBctTA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:42 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:42 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:51 +0900 Subject: [PATCH 13/16] gpu: nova-core: mm: Add multi-page mapping API to VMM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-13-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4PR01CA0089.jpnprd01.prod.outlook.com (2603:1096:405:37d::16) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: b561070c-bd6d-4f9f-cf6b-08df0e26eadf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|6133799003|3023799007|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: h8fszSP49rV7UpCloLzxpIJIDBiwGtbOdNmEzAjE12+H9UblRCyN657H0xLcuAj7UNNjOpc1UgLZJC42qHAe11WIGNROhkUX6xXkN6E+RPIulGNADDL++kLSw37uESiytoWohoINWufYIfuMICOFnUREwNubyFevgcBE8otsATp2z2c6ycGekBknJwMJnVAAlBSEWYHQR4a55lxD086+Qs5RdCAzayPF3lh0MVWYBQPjAuqW22nH3mVsfmyb4EGcoG/rp5mgNaof38vpXEgRpMKXhcy07AQjn4Tbc1ru+bX4goPqm4pH7lFbhn32nQAGy7D7vNO8Ne2InOUg2S3BycdU2LRLgS6/UM6XeE5g6Vej90x9UTzZUKyya663dbEZMv5JfyvVpW1REilZJOpuePpJoGJmfoFS1tc+3bJBUKjqxo/6LUW8sr5rWztqAbrsYgNdPbFmGoY9J/natyPyfcegduY7p00T55wSJI/+hi8f+lRYAvTcJjswuHqliynRUQYvAoY6Ptx9xfVYcPt5lmuOW58O8wU4JEdEJySmaidQXktO3isxpNTBu7tJ0JWzKeM1bvf2APYj5v2LyRMVRMYcK8Mee8mrAs/GniRbUeUkVMR/A1I5jYOpjpqRazTKGTxXNnIm6wYSVOxaq4Rhgsw53kwNMpXTZAihm25ah5I= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(6133799003)(3023799007)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Qk1VRXppYUwrc1diMVVvdmgzK2laWXRPV0pFOGhyRVhvbnd4MzN0ZGZYQXRj?= =?utf-8?B?STJ2Mi92Y1BGZGZEWWZyTG5kempIV3U5ZGV4NnBQYXBDNVdDSkdzNHBONTVn?= =?utf-8?B?YlMzRjh3dUhETFBBZURXTnBtR1oydjJjTGdwbVNEWGwvemFvc2xmc2ZxVEox?= =?utf-8?B?M2xUZ1FlODRHc0dKcWh0a1lvRVJET1VQVUVKWVZGakNiQkRRRlBYSmJwbFZE?= =?utf-8?B?Ri83TFF1djNnanJMQzhKck5QTWRlSEF1M096YkNFZVpKSzVjcHZUdkV5eUpS?= =?utf-8?B?ZmwvcVFFN3pMdkxLcXU0c1NzUlVablBWZGN6bmdMS3pCdnJIQy9rdytibkw5?= =?utf-8?B?bitGWHhpWmhaL0ZsTE1qK1pyMXpRbFZRTlNvc2VoR08rMFBSWlBlTDgvOXBP?= =?utf-8?B?bGQ4VHUyNDNQb1d3Y0pvZlFLbTEyZ240TTBocldPbE1aTmgxejNDeGtTQ2sv?= =?utf-8?B?bDFXZytoa25mVmI5Q3IwbTZmaUlhUUxkbUd3M093bk44WHhPSGhOTC94K0RX?= =?utf-8?B?cm13blNUaVdaMzFTYjUzaXk1anNTVWFEY2tkNGlNcG5odlROUUllNUdBOWNP?= =?utf-8?B?b2RGbW5TbXM3NEJXUUF5SEh4cDA5VmRNWUtOV2VibEJMR1hyS1lDZE5UTWpR?= =?utf-8?B?elNwSStuNmVMK1d4ZEZ6VnNOU3Y5d3lyYzg1RCtFdHRJK3I0OExCdC9HekJl?= =?utf-8?B?VnBJS21NVEN1NjNCY0tWdUl5Nm12UCs2SUppc0xYRFpaaHZkTEdrUkkxTDBn?= =?utf-8?B?bVc2UnZQa2ZIKzhKOGtINmxqWjVUc092Ris3dGFLYXlKVWlZeFd3Y1g5UG93?= =?utf-8?B?OTlsMnErcXZpYTFoZ08ySUxyOVJUd0JZQzlISkVMN25vNStHeFo2alVNNVZY?= =?utf-8?B?MUdFK05EUGxNQ2NQU2RuQktKMmFMdjJWMU9leWtpZWpzLzUrT3J6bTNqdWo3?= =?utf-8?B?cXhNOFpyNzJncER0SmVZQUo4QWtpV1EwZU1ESzQwOWQzQ0x6TS9PVjRFRThq?= =?utf-8?B?b1E3SWN6TXBLRDM2Wnpud2tadDYvWmltbFcrYnJUK2g2Y0phMnNvQnZZY0ln?= =?utf-8?B?SW1NRHBuMndiZVI0TGtYZW5zanhtaXJ5ZUhva3lKSkhObVhqcFZaYWxoMWVP?= =?utf-8?B?enlmYkhhNWJtdmFwSkd5eXJUUjFMell3RDJWdXdNQStLWGlWaXA1QksrbXM3?= =?utf-8?B?UkxXV0dOTHROcVpVVHBqRzN6bk9Lb0sxVkhqK0NvQ2F2RUhOeXY0akVSSkRo?= =?utf-8?B?bXA1YkF4SzBFM09QdVp2UjV1UkhpbCtXWXovWmR1aXFLa1JBQnBQb2svMjRE?= =?utf-8?B?bTZHT0U3L1pScjBxaUdIRFNiUkpvZThNU0wxUG9sUm5uRGZqS1pjT0JseE9D?= =?utf-8?B?aDkxQWJHaWhKWmJqQTNIZi9WTnRWby9UdW9aalZPeDBMeGhJNjFtNXErSW5v?= =?utf-8?B?eHI0ZlE3Um9mUFFUWFY0RThJQldYRmQyMEc1RGJ4ZU1ZQUptOWJKRmZRVFRQ?= =?utf-8?B?ZnhUVlVaQWZ5Y3NqdEh0TGpWMnBvbFJTOVYyMEcrMmlXU0pzdWpXUHkzSmRB?= =?utf-8?B?WDBaZmhyRWNsRHZCSWkvUHVyY3dnNlF1bmVmY1orUWNKclRXam5rWVloZzZx?= =?utf-8?B?MHRkbHJLYTNWeHNGaUhlMG9Xdy9GbUs4MHc4QkdNTWFmMWlQMm1rMjkxbjR4?= =?utf-8?B?cDlyOFk3TTA0UEp6WTc5ZTB5Y2l4UnBrSWhHckMwL09YLzlieTc3VTF6MEhw?= =?utf-8?B?endPVEFwbkhCZ0E1TEMxai9zSW1ycDlwNlpuUEcrOHVhTjF0NUxJZWt6RzVH?= =?utf-8?B?c2NCS2h0bCtrcms4emErQ2dSQWt4VFJObk8xakh6OXk0elJBQ0VxaFpnNEpT?= =?utf-8?B?T3hKRnJ5cVpSSWEvb2pxV2xMQW1LNlozSDNQZWNmRXY3UzQ2N2lvMXBYUVhG?= =?utf-8?B?SUZpWkFVUnVrSzlSYkRGbmJmQlpnMkJiU0hyUE0vOGlTRlY5U3JDenZQSGpE?= =?utf-8?B?TTVjVGdYOFBYWllvRkxDVmUyTi9CWlVnL2xOSnJnYmlMSkFuY3VTSmU1NjAr?= =?utf-8?B?aVR3ZnpBa1hZbnNseDVpU2hPU3dZS3JybzJNUEwzT3lvSkhPNVlGVDBCL0hq?= =?utf-8?B?YzR0WlpDRUVsc25IeEdnbGI4VzU1YnAwdGxiWkdtN0hGN1ByT3FTWTREQVlI?= =?utf-8?B?SzQxTkhiZWlRaWlyZ25QTS9zYjFUVTFjb2RGc1pPVWtyeFNIQmxXcjE0cFZW?= =?utf-8?B?aHQwZXVWVnRRWDh6cWNoeG1XREhnN0s1R3pHY3N0VEZsS0w2ZStsM2Nxa21k?= =?utf-8?B?UjVFcmhvU1FqZ2ZKaGhCSWh2bTlnUnN4ZkdkdVp2WXZ5OWhBTzJabGpKSnUy?= =?utf-8?Q?44qy11PVMBo3JIHJaSTtlLeYuOkCXL9g48BoI+zpX1BUn?= X-MS-Exchange-AntiSpam-MessageData-1: VuwIoPYmroJyaQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b561070c-bd6d-4f9f-cf6b-08df0e26eadf X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:42.3054 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LJjX2JNJeuqV16wOs6Coll8FakZiJYJ2JyZGSBPiKtL1Sf1szagJ0gTUttve0u/Vr4KkTC43RaGy6nwNgeCanQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add the page table mapping and unmapping API to the Virtual Memory Manager, implementing a two-phase prepare/execute model suitable for use both inside and outside the DMA fence signalling critical path. Signed-off-by: Joel Fernandes [ecourtney: pass mutable GpuMm, replace window guards with scoped borrows] [ecourtney: use current raw address helpers, drop stale expects] [ecourtney: zero page table pages through one PRAMIN view] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/mm/pagetable.rs | 1 + drivers/gpu/nova-core/mm/pagetable/map.rs | 342 ++++++++++++++++++++++++++= ++++ drivers/gpu/nova-core/mm/vmm.rs | 260 +++++++++++++++++++++-- 3 files changed, 584 insertions(+), 19 deletions(-) diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index 99145846daed..ffc69fbdb067 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -8,6 +8,7 @@ =20 #![expect(dead_code)] =20 +pub(super) mod map; pub(super) mod ver2; pub(super) mod ver3; pub(super) mod walk; diff --git a/drivers/gpu/nova-core/mm/pagetable/map.rs b/drivers/gpu/nova-c= ore/mm/pagetable/map.rs new file mode 100644 index 000000000000..21fc66e7c62c --- /dev/null +++ b/drivers/gpu/nova-core/mm/pagetable/map.rs @@ -0,0 +1,342 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Page table mapping operations for NVIDIA GPUs. + +use core::marker::PhantomData; + +use kernel::{ + gpu::buddy::{ + AllocatedBlocks, + GpuBuddyAllocFlags, + GpuBuddyAllocMode, // + }, + io::io_write, + prelude::*, + ptr::Alignment, + rbtree::{RBTree, RBTreeNode}, + sizes::SZ_4K, // +}; + +use super::{ + walk::{ + PtWalkInner, + WalkPdeResult, + WalkResult, // + }, + AperturePde, + AperturePte, + DualPdeOps, + MmuConfig, + MmuV2, + MmuV3, + MmuVersion, + PageTableLevel, + PdeOps, + PteOps, // +}; +use crate::{ + mm::{ + GpuMm, + Pfn, + Vfn, + VramAddress, + PAGE_SIZE, // + }, + num::{ + IntoSafeCast, // + }, +}; + +/// A pre-allocated and zeroed page table page. +/// +/// Created during the mapping prepare phase and consumed during the execu= te phase. +/// Stored in an [`RBTree`] keyed by the PDE slot address (`install_addr`). +pub(in crate::mm) struct PreparedPtPage { + /// The allocated and zeroed page table page. + pub(in crate::mm) alloc: Pin>, + /// Page table level -- needed to determine if this PT page is for a d= ual PDE. + pub(in crate::mm) level: PageTableLevel, +} + +/// Page table mapper. +pub(in crate::mm) struct PtMapInner { + walker: PtWalkInner, + pdb_addr: VramAddress, + _phantom: PhantomData, +} + +impl PtMapInner { + /// Create a new [`PtMapInner`]. + pub(super) fn new(pdb_addr: VramAddress) -> Self { + Self { + walker: PtWalkInner::::new(pdb_addr), + pdb_addr, + _phantom: PhantomData, + } + } + + /// Allocate and zero a physical page table page. + fn alloc_and_zero_page(mm: &mut GpuMm<'_>, level: PageTableLevel) -> R= esult { + let blocks =3D KBox::pin_init( + mm.buddy().alloc_blocks( + GpuBuddyAllocMode::Simple, + SZ_4K.into_safe_cast(), + Alignment::new::(), + GpuBuddyAllocFlags::default(), + ), + GFP_KERNEL, + )?; + + let page_vram =3D VramAddress::from_raw(blocks.iter().next().ok_or= (ENOMEM)?.offset()); + + // Zero via PRAMIN. + let window =3D mm + .pramin_mut() + .window_at::<[u64; PAGE_SIZE / 8]>(page_vram)?; + for i in 0..PAGE_SIZE / 8 { + io_write!(window.view(), [build: i], 0); + } + + Ok(PreparedPtPage { + alloc: blocks, + level, + }) + } + + /// Ensure all intermediate page table pages exist for a single VFN. + /// + /// The mutable PRAMIN borrow ends before each allocation. + fn ensure_single_pte_path( + &self, + mm: &mut GpuMm<'_>, + vfn: Vfn, + pt_pages: &mut RBTree, + ) -> Result { + let max_iter =3D 2 * M::PDE_LEVELS.len(); + + for _ in 0..max_iter { + let result =3D self + .walker + .walk_pde_levels(mm.pramin_mut(), vfn, |install_addr| { + pt_pages.get(&install_addr).and_then(|p| { + p.alloc + .iter() + .next() + .map(|b| VramAddress::from_raw(b.offset())) + }) + })?; + + match result { + WalkPdeResult::Complete { .. } =3D> { + return Ok(()); + } + WalkPdeResult::Missing { + install_addr, + level, + } =3D> { + let page =3D Self::alloc_and_zero_page(mm, level)?; + let node =3D RBTreeNode::new(install_addr, page, GFP_K= ERNEL)?; + let old =3D pt_pages.insert(node); + if old.is_some() { + kernel::pr_warn_once!( + "VMM: duplicate install_addr in pt_pages (inte= rnal consistency error)\n" + ); + return Err(EIO); + } + } + } + } + + kernel::pr_warn!( + "VMM: ensure_pte_path: loop exhausted after {} iters (VFN {:?}= )\n", + max_iter, + vfn + ); + Err(EIO) + } + + /// Prepare page table resources for mapping `num_pages` pages startin= g at `vfn_start`. + /// + /// Reserves capacity in `page_table_allocs`, then walks the hierarchy + /// per-VFN to prepare pages for all missing PDEs. + pub(super) fn prepare_map( + &self, + mm: &mut GpuMm<'_>, + vfn_start: Vfn, + num_pages: usize, + page_table_allocs: &mut KVec>>, + pt_pages: &mut RBTree, + ) -> Result { + // Pre-reserve so install_mappings() can use push_within_capacity = (no alloc + // in fence signalling critical path). + let pt_upper_bound =3D M::pt_pages_upper_bound(num_pages); + page_table_allocs.reserve(pt_upper_bound, GFP_KERNEL)?; + + // Walk the hierarchy per-VFN to prepare pages for all missing PDE= s. + for i in 0..num_pages { + let i_u64: u64 =3D i.into_safe_cast(); + let vfn =3D Vfn::new(vfn_start.raw() + i_u64); + self.ensure_single_pte_path(mm, vfn, pt_pages)?; + } + Ok(()) + } + + /// Install prepared PDEs and write PTEs, then flush TLB. + /// + /// Drains `pt_pages` and moves allocations into `page_table_allocs`. + pub(super) fn install_mappings( + &self, + mm: &mut GpuMm<'_>, + pt_pages: &mut RBTree, + page_table_allocs: &mut KVec>>, + vfn_start: Vfn, + pfns: &[Pfn], + writable: bool, + ) -> Result { + { + let pramin =3D mm.pramin_mut(); + + // Drain prepared PT pages, install all pending PDEs. + let mut cursor =3D pt_pages.cursor_front_mut(); + while let Some(c) =3D cursor { + let (next, node) =3D c.remove_current(); + let (install_addr, page) =3D node.to_key_value(); + let page_vram =3D + VramAddress::from_raw(page.alloc.iter().next().ok_or(E= NOMEM)?.offset()); + + if page.level =3D=3D M::DUAL_PDE_LEVEL { + let new_dpde =3D M::DualPde::new_small(Pfn::from(page_= vram)); + new_dpde.write(pramin, install_addr)?; + } else { + let new_pde =3D M::Pde::new(AperturePde::VideoMemory, = Pfn::from(page_vram)); + new_pde.write(pramin, install_addr)?; + } + + page_table_allocs + .push_within_capacity(page.alloc) + .map_err(|_| ENOMEM)?; + + cursor =3D next; + } + + // Write PTEs (all PDEs now installed in HW). + for (i, &pfn) in pfns.iter().enumerate() { + let i_u64: u64 =3D i.into_safe_cast(); + let vfn =3D Vfn::new(vfn_start.raw() + i_u64); + let result =3D self.walker.walk_to_pte_lookup_with_window(= pramin, vfn)?; + + match result { + WalkResult::Unmapped { pte_addr } | WalkResult::Mapped= { pte_addr, .. } =3D> { + let pte =3D M::Pte::new(AperturePte::VideoMemory, = pfn, writable); + pte.write(pramin, pte_addr)?; + } + WalkResult::PageTableMissing =3D> { + kernel::pr_warn_once!("VMM: page table missing for= VFN {vfn:?}\n"); + return Err(EIO); + } + } + } + } + + // Flush TLB. + mm.tlb().flush(self.pdb_addr) + } + + /// Invalidate PTEs for a range and flush TLB. + pub(super) fn invalidate_ptes( + &self, + mm: &mut GpuMm<'_>, + vfn_start: Vfn, + num_pages: usize, + ) -> Result { + let invalid_pte =3D M::Pte::invalid(); + + { + let pramin =3D mm.pramin_mut(); + for i in 0..num_pages { + let i_u64: u64 =3D i.into_safe_cast(); + let vfn =3D Vfn::new(vfn_start.raw() + i_u64); + let result =3D self.walker.walk_to_pte_lookup_with_window(= pramin, vfn)?; + + match result { + WalkResult::Mapped { pte_addr, .. } | WalkResult::Unma= pped { pte_addr } =3D> { + invalid_pte.write(pramin, pte_addr)?; + } + WalkResult::PageTableMissing =3D> { + continue; + } + } + } + } + + mm.tlb().flush(self.pdb_addr) + } +} + +macro_rules! pt_map_dispatch { + ($self:expr, $method:ident ( $($arg:expr),* $(,)? )) =3D> { + match $self { + PtMap::V2(inner) =3D> inner.$method($($arg),*), + PtMap::V3(inner) =3D> inner.$method($($arg),*), + } + }; +} + +/// Page table mapper dispatch. +pub(in crate::mm) enum PtMap { + /// MMU v2 (Turing/Ampere/Ada). + V2(PtMapInner), + /// MMU v3 (Hopper+). + V3(PtMapInner), +} + +impl PtMap { + /// Create a new page table mapper for the given MMU version. + pub(in crate::mm) fn new(pdb_addr: VramAddress, version: MmuVersion) -= > Self { + match version { + MmuVersion::V2 =3D> Self::V2(PtMapInner::::new(pdb_addr= )), + MmuVersion::V3 =3D> Self::V3(PtMapInner::::new(pdb_addr= )), + } + } + + /// Prepare page table resources for a mapping. + pub(in crate::mm) fn prepare_map( + &self, + mm: &mut GpuMm<'_>, + vfn_start: Vfn, + num_pages: usize, + page_table_allocs: &mut KVec>>, + pt_pages: &mut RBTree, + ) -> Result { + pt_map_dispatch!( + self, + prepare_map(mm, vfn_start, num_pages, page_table_allocs, pt_pa= ges) + ) + } + + /// Install prepared PDEs and write PTEs, then flush TLB. + pub(in crate::mm) fn install_mappings( + &self, + mm: &mut GpuMm<'_>, + pt_pages: &mut RBTree, + page_table_allocs: &mut KVec>>, + vfn_start: Vfn, + pfns: &[Pfn], + writable: bool, + ) -> Result { + pt_map_dispatch!( + self, + install_mappings(mm, pt_pages, page_table_allocs, vfn_start, p= fns, writable) + ) + } + + /// Invalidate PTEs for a range and flush TLB. + pub(in crate::mm) fn invalidate_ptes( + &self, + mm: &mut GpuMm<'_>, + vfn_start: Vfn, + num_pages: usize, + ) -> Result { + pt_map_dispatch!(self, invalidate_ptes(mm, vfn_start, num_pages)) + } +} diff --git a/drivers/gpu/nova-core/mm/vmm.rs b/drivers/gpu/nova-core/mm/vmm= .rs index 0bcae29db4f2..411710d03f7a 100644 --- a/drivers/gpu/nova-core/mm/vmm.rs +++ b/drivers/gpu/nova-core/mm/vmm.rs @@ -3,21 +3,30 @@ //! Virtual Memory Manager for NVIDIA GPU page table management. //! //! The [`Vmm`] provides high-level page mapping and unmapping operations = for GPU -//! virtual address spaces (Channels, BAR1, BAR2). It wraps the page table= walker -//! and handles TLB flushing after modifications. +//! virtual address spaces (Channels, BAR1, BAR2). =20 use kernel::{ gpu::buddy::AllocatedBlocks, maple_tree::MapleTreeAlloc, - prelude::*, // + prelude::*, + rbtree::RBTree, // }; =20 -use core::ops::Range; +use core::{ + cell::Cell, + ops::Range, // +}; =20 use crate::{ mm::{ pagetable::{ - walk::{PtWalk, WalkResult}, + map::{ + PtMap, // + }, + walk::{ + PtWalk, + WalkResult, // + }, MmuVersion, // }, GpuMm, @@ -31,22 +40,108 @@ }, }; =20 +/// Multi-page prepared mapping -- VA range allocated, ready for execute. +/// +/// Produced by [`Vmm::prepare_map()`], consumed by [`Vmm::execute_map()`]. +/// The VA space allocation is tracked in the [`Vmm`]'s maple tree and fre= ed +/// on error or via [`Vmm::unmap_pages()`]. +/// +/// Dropping without calling [`Vmm::execute_map()`] logs a warning and lea= ks +/// the VA range in the maple tree. +pub(crate) struct PreparedMapping { + vfn_start: Vfn, + num_pages: usize, + /// Logs a warning if dropped without executing. + _drop_guard: MustExecuteGuard, +} + +/// Result of a mapping operation -- tracks the active mapped range. +/// +/// Returned by [`Vmm::execute_map()`] and [`Vmm::map_pages()`]. +/// Callers must call [`Vmm::unmap_pages()`] before dropping to invalidate +/// PTEs and free the VA range. Dropping without unmapping logs a warning +/// and leaks the VA range in the maple tree. +pub(crate) struct MappedRange { + pub(super) vfn_start: Vfn, + pub(super) num_pages: usize, + /// Logs a warning if dropped without unmapping. + _drop_guard: MustUnmapGuard, +} + +/// Guard that logs a warning if a [`PreparedMapping`] is dropped without +/// being consumed by [`Vmm::execute_map()`]. +struct MustExecuteGuard { + armed: Cell, +} + +impl MustExecuteGuard { + const fn new() -> Self { + Self { + armed: Cell::new(true), + } + } + + fn disarm(&self) { + self.armed.set(false); + } +} + +impl Drop for MustExecuteGuard { + fn drop(&mut self) { + if self.armed.get() { + kernel::pr_warn!("PreparedMapping dropped without calling exec= ute_map()\n"); + } + } +} + +/// Guard that logs a warning if a [`MappedRange`] is dropped without +/// calling [`Vmm::unmap_pages()`]. +struct MustUnmapGuard { + armed: Cell, +} + +impl MustUnmapGuard { + const fn new() -> Self { + Self { + armed: Cell::new(true), + } + } + + fn disarm(&self) { + self.armed.set(false); + } +} + +impl Drop for MustUnmapGuard { + fn drop(&mut self) { + if self.armed.get() { + kernel::pr_warn!("MappedRange dropped without calling unmap_pa= ges()\n"); + } + } +} + /// Virtual Memory Manager for a GPU address space. /// /// Each [`Vmm`] instance manages a single address space identified by its= Page -/// Directory Base (`PDB`) address. The [`Vmm`] is used for Channel, BAR1 = and -/// BAR2 mappings. +/// Directory Base (`PDB`) address. Used for Channel, BAR1 and BAR2 mappin= gs. pub(crate) struct Vmm { /// Page Directory Base address for this address space. pdb_addr: VramAddress, - /// MMU version used for page table layout. - mmu_version: MmuVersion, + /// Page table walker for reading existing mappings. + pt_walk: PtWalk, + /// Page table mapper for prepare/execute operations. + pt_map: PtMap, /// Page table allocations required for mappings. page_table_allocs: KVec>>, /// Maple tree allocator for virtual address range tracking. virt_alloc: Pin>>, /// Total number of pages in the virtual address space. va_pages: usize, + /// Prepared PT pages pending PDE installation, keyed by `install_addr= `. + /// + /// Populated during prepare phase and drained in execute phase. Share= d by all + /// pending maps, preventing races on the same PDE slot. + pt_pages: RBTree, } =20 impl Vmm { @@ -64,20 +159,16 @@ pub(crate) fn new( =20 Ok(Self { pdb_addr, - mmu_version, + pt_walk: PtWalk::new(pdb_addr, mmu_version), + pt_map: PtMap::new(pdb_addr, mmu_version), page_table_allocs: KVec::new(), virt_alloc, va_pages, + pt_pages: RBTree::new(), }) } =20 /// Allocate a contiguous virtual frame number range. - /// - /// # Arguments - /// - /// - `num_pages`: Number of pages to allocate. - /// - `va_range`: `None` =3D allocate anywhere, `Some(range)` =3D cons= train allocation to the given - /// range. fn alloc_vfn_range(&self, num_pages: usize, va_range: Option>) -> Result { let page_size: u64 =3D PAGE_SIZE.into_safe_cast(); =20 @@ -113,11 +204,142 @@ fn free_vfn(&self, vfn: Vfn) { =20 /// Read the [`Pfn`] for a mapped [`Vfn`] if one is mapped. pub(super) fn read_mapping(&self, mm: &mut GpuMm<'_>, vfn: Vfn) -> Res= ult> { - let walker =3D PtWalk::new(self.pdb_addr, self.mmu_version); - - match walker.walk_to_pte(mm, vfn)? { + match self.pt_walk.walk_to_pte(mm, vfn)? { WalkResult::Mapped { pfn, .. } =3D> Ok(Some(pfn)), WalkResult::Unmapped { .. } | WalkResult::PageTableMissing =3D= > Ok(None), } } + + /// Prepare resources for mapping `num_pages` pages. + /// + /// Allocates a contiguous VA range, then walks the hierarchy per-VFN = to prepare pages + /// for all missing PDEs. Returns a [`PreparedMapping`] with the VA al= location. + /// + /// If `va_range` is not `None`, the VA range is constrained to the gi= ven range. Safe + /// to call outside the fence signalling critical path. + pub(crate) fn prepare_map( + &mut self, + mm: &mut GpuMm<'_>, + num_pages: usize, + va_range: Option>, + ) -> Result { + if num_pages =3D=3D 0 { + return Err(EINVAL); + } + + // Allocate contiguous VA range. + let vfn_start =3D self.alloc_vfn_range(num_pages, va_range)?; + + if let Err(e) =3D self.pt_map.prepare_map( + mm, + vfn_start, + num_pages, + &mut self.page_table_allocs, + &mut self.pt_pages, + ) { + self.free_vfn(vfn_start); + return Err(e); + } + + Ok(PreparedMapping { + vfn_start, + num_pages, + _drop_guard: MustExecuteGuard::new(), + }) + } + + /// Execute a prepared multi-page mapping. + /// + /// Installs all prepared PDEs and writes PTEs into the page table, th= en flushes TLB. + pub(crate) fn execute_map( + &mut self, + mm: &mut GpuMm<'_>, + prepared: PreparedMapping, + pfns: &[Pfn], + writable: bool, + ) -> Result { + if pfns.len() !=3D prepared.num_pages { + self.free_vfn(prepared.vfn_start); + return Err(EINVAL); + } + + let PreparedMapping { + vfn_start, + num_pages, + _drop_guard, + } =3D prepared; + _drop_guard.disarm(); + + if let Err(e) =3D self.pt_map.install_mappings( + mm, + &mut self.pt_pages, + &mut self.page_table_allocs, + vfn_start, + pfns, + writable, + ) { + self.free_vfn(vfn_start); + return Err(e); + } + + Ok(MappedRange { + vfn_start, + num_pages, + _drop_guard: MustUnmapGuard::new(), + }) + } + + /// Map pages doing prepare and execute in the same call. + /// + /// This is a convenience wrapper for callers outside the fence signal= ling critical + /// path (e.g., BAR mappings). For DRM usecases, [`Vmm::prepare_map()`= ] and + /// [`Vmm::execute_map()`] will be called separately. + pub(crate) fn map_pages( + &mut self, + mm: &mut GpuMm<'_>, + pfns: &[Pfn], + va_range: Option>, + writable: bool, + ) -> Result { + if pfns.is_empty() { + return Err(EINVAL); + } + + // Check if provided VA range is sufficient (if provided). + if let Some(ref range) =3D va_range { + let required: u64 =3D pfns + .len() + .checked_mul(PAGE_SIZE) + .ok_or(EOVERFLOW)? + .into_safe_cast(); + let available =3D range.end.checked_sub(range.start).ok_or(EIN= VAL)?; + if available < required { + return Err(EINVAL); + } + } + + let prepared =3D self.prepare_map(mm, pfns.len(), va_range)?; + self.execute_map(mm, prepared, pfns, writable) + } + + /// Unmap all pages in a [`MappedRange`] with a single TLB flush. + pub(crate) fn unmap_pages(&mut self, mm: &mut GpuMm<'_>, range: Mapped= Range) -> Result { + let result =3D self + .pt_map + .invalidate_ptes(mm, range.vfn_start, range.num_pages); + + // TODO: Internal page table pages (PDE, PTE pages) are still kept= around. + // This is by design as repeated maps/unmaps will be fast. As a fu= ture TODO, + // we can add a reclaimer here to reclaim if VRAM is short. For no= w, the PT + // pages are dropped once the `Vmm` is dropped. + + // Free the VA range regardless of PTE invalidation success, so th= at the VA + // range is recovered even on failure (PTEs may be stale, but that= is better + // than leaking both PTEs and VA range). + self.free_vfn(range.vfn_start); + + // Unmap complete, safe to drop `MappedRange`. + range._drop_guard.disarm(); + result + } } --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010013.outbound.protection.outlook.com [52.101.56.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1E1F35674F for ; Wed, 9 Sep 2026 04:00:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.13 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926453; cv=fail; b=QR5ObkLly8hROaxkU08BRLe3SvIEVRjB3hRhJBYRM7JAhOzI+NoJQNVe2Qtg34IPgN5aRRYXmM3Z90FnIcBRKps4RPcchgX2M63NIj/0ygZLo+hle63tXCJpefxbMlDxzrSMI0/88319ulNqfvcQumJ+LtVrKgr8OQdo6CGPmzo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926453; c=relaxed/simple; bh=yY84pOwgi+uJQ0ezRhCNDQoArLVEpbVeAQrc2Z4L3dU=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=HTjWYEr9in1U/x0G1KUxpV+klXjTgcnc4lsTWKAGHFEGzwgG0ZaZ7MA1Fk+0EeNe1+2uvB0vkPb1c1RBWgPzW3p7wgw/G/Gmu1M/PQ3MyAtA+KcArUX/0K23tTRSNxg8uVdS27cSLctq8YgQl6lW8DF5GfoA6d1W/NG1L5Pai3Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=VIgfFW/W; arc=fail smtp.client-ip=52.101.56.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="VIgfFW/W" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KGbOgN0OgNwk/PI6LC1AgTJwtd+bmFXZqiXG0Jh0c9HxdNuSQzJCZsmLb+2Wjxz46DmhKyt0YV1RJjRxy0zaBoXCyuQRX5dsHivfzsTcDCFaA32bYlKxSZpYfrWoT8LTjBoxT2BO5BRjw2gTuktWR7z8r7RIpRjFmPADF+zgUK5W+GgKEwQLcX+T2oe2cbFT4l8zSxSK30tSPooyEPs344KIzynchyYhJvNqMSWCYqWT/ldfFCy7wjVJED07zFJ076hRxWXzNWNqE9n8KGTY2G6pym75B3xU1w/Q/x0apD6hoK/smArRDTZPUmKj4kyeix9bRHoF+ABxVs6KC2Re5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S0ihvBMjGbUPL+I3j6UFPDxiv+5njam0YLbVLrWc7Cw=; b=jyG7ZdU42+OedQ6AZddQ3wStAUArAIDLpi39czyrpHGu8dL1g6L1Mt5a6sgvuct7Z8DESUvTtEV35D77nzi1grxR0Rsqdc0vD4SXAifolXj/ODnpLHzP5VJYEXc+1RmvHfwsDDlQv8NxLPMilWxWhK15wszZqOVBHHjR2apJYJkwb4+IZxVUKpzaRwenPxUenJL45MRBCWdSdTHbszTxydG5Kxx/1pBClM8SNFcMist2PX7jvlOtCc4fCK5pgs3MSz0brTCeYpRQ/ciJ1WeNRzclKuG1E15ZPlc0A+M15/CPruMlCXpoS7CWFJRmJlM8C0Q1QtQfVFSLzX+g/NW6oQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S0ihvBMjGbUPL+I3j6UFPDxiv+5njam0YLbVLrWc7Cw=; b=VIgfFW/WE57o4UE8jgpscYkKisVvrhMnK574ESMAJutw4KfmVMhepiq0SUd5ypC10phL42RJ7jeXH4eZR/SwxIHQ0cf7LEy4hHLKAX+4pzENgqQJYuHI2JbeJe8pk8jqJEwjakZeb85GY+PktWBjD2LsHv6BXrJ2iMAFFJOZeMVJ6zFOmmzhzknp8m7UT+EpzOmBGBCgSB47C+4aCv0nw/hEzXq8UPAoVMoJKwDAw1nWrcAd1IM5U7ob/hS2SxJQ2PYlXFMJXKdyNU3UH4ebL3WTjBY2VkOj7g4qbQ7KEljiGkgk95O+nr2MmFUEriUigbeTbv5MBsicobvfKTtIig== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by SA3PR12MB699018.namprd12.prod.outlook.com (2603:10b6:806:510::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:46 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:46 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:52 +0900 Subject: [PATCH 14/16] gpu: nova-core: Add BAR1 aperture type and size constant Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-14-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes , Zhi Wang X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P301CA0042.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:2be::18) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|SA3PR12MB699018:EE_ X-MS-Office365-Filtering-Correlation-Id: 4df0453c-3c50-4838-cb4a-08df0e26ed1c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|23010399003|366016|10070799003|3023799007|10067099003|11063799006|56012099006|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: JWrzQpSRxsgG+EogE2CfV6FkfNMl4lWZwRxktvM1iPWrP/4e/XTHj1ZKMCkosqN1qoVoaqyq4ri5qtbWjWyDKqieZoUpQgmwqrDyVY5XRWlzVM8Yer2zDIXBt9kOrH3oEJE/5KPmjOEKi4HYbAz7XJ8REGGvlmCHP2c5aXPNK5F0GzwSsVfPRxubT2upCb08JPbtdy95QJI0NkHVFvShc8AvFN6yaEw0+Ksfgpk7PdJm/2av1dQdW3pYrEjzQ9IO/NNwFIG0W1uWhfL7GIo7B3VKi1w/b/Ro8Rww19JKrOTPwmRje/cgqU40z8c0YSPtP/yRlEOgTMWizwkxjc+dANu75WsOTFKwbqlBSgPbA4INp8DGvbGnWHckqSjFk8DcTSRNzFy0eXZXYO4saLqFZGev/aU8bpA3gRoHZ9eFNiapYAYn4oczFBDXnM2jZjRdBqqWWXQEh+wdhwIOhjzE8sm80Y6y+xw1IhPdt5XC9Tycior8TvqXaLfgTsolE0U9KMcOAXd/FkaQqDB1Ir2WLojs90TpNZNI3BCehUZYkHaCrzAfiKnmdOjJRHNNNr8Z1ZdhVS4IsoAbab6x/v0jDIwX7mvLbMqb2AfcfqUPqrDJnkj396+Ecu4Wt5mkv2hxc7THtTRFYDUZOPilAjVq8gcINYUJXtN+eFZ4PWGSpQ0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(23010399003)(366016)(10070799003)(3023799007)(10067099003)(11063799006)(56012099006)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZTQ5aStxaHpiNzBzQWFrRys4eDdXeTVVcjhWMXpodG8wY2pvbmlvcXJPU3VD?= =?utf-8?B?eWRwTWZQRFcxVWxXZmlLaHpHdXJXWS9TSFYvYTY1cTVZZ2NnblpoMmM2c1NE?= =?utf-8?B?dnV4ZlR2Ym5ZZGxGNTBlTUJndlZrUmhweENNSFhCOGsrRjkvNHozZzVUZDRa?= =?utf-8?B?OEEzVEg5UHlWVEwvSVlqTGdUZ3E5Rlo3Z0FENGVtbUcySEt1VERUVUVxWXRY?= =?utf-8?B?dHBIWHBIcW1rRFQ2MTYybGFWMmVwZkJzTDBnbFQ1WmdnMU9sRy9VTmcwQ2Jj?= =?utf-8?B?MnpmSXV4dVVRczhva3l1ZGc3eDBtUXNqdEhkOEtLWm80bXlKbHdEdXVtckta?= =?utf-8?B?RTJhNkpMM1JxNk9lYUdlRmVMWk9SU1hxQkg3djZGc3JGUW1EWWhIaHkvM3lI?= =?utf-8?B?Ykt3SFpTcS8wVGFhYW45NS9tMlRPSVdXSDhuQ1pmMGZzd1hlNDhVS2VRSFZC?= =?utf-8?B?eG9BTDhhV0lIcFU5MHNMZ1ZlMEdhOThubjhRdjJJWWJINnhhMWZERjViQ09r?= =?utf-8?B?WGZoUldEcEFLVkJGcVRKdzdKQ1RrdCtyQTMwWWN5NG5DUzRWSFNEWVNoVkIz?= =?utf-8?B?UzIzRm9UeXRnb2dnTzRLS2VLMWFzTzY5TFpLN2lCWVJUWmlaSjJjMXV6dG82?= =?utf-8?B?OFZFWVZ0enlMdGNJTFo0eXdCK3BMeW9JTzloNlJlNTVBc2xGNnAxb1dtS3E3?= =?utf-8?B?Skptam9oM2gybnUzZm04SVF0b3pZTW1QZnBWVTI2cFZ4K1dCemxnVnBIeHJl?= =?utf-8?B?eFUrVkxmZXNrTkpEM0g0VGVGanRtKzZzSmdIKzVmVlJjQlZjYXVnK0pva1ZS?= =?utf-8?B?dWQxcFZKMHVOR0wyWXhDY28zOHlwcVVJYVEveExqWmVZZ1M3K0Y4c2xmRFRi?= =?utf-8?B?eWNOOGFlTTFYdDVVcTlOeWgrNkJMdGJhRE1HUFMyVGpza1MxdlFLT2Q3UmNq?= =?utf-8?B?Y1B2QTRFSFdJZzBzWGZmOFVSU3BRVmJsdjUvbFhpLzdqaS9MTWthVUVZM2ZR?= =?utf-8?B?Rm56eG4yc0hDUGdFRnhnTDg2RUJ1Z2JtVUh2WmFxaytHNDlNU1ltaE00Ty9l?= =?utf-8?B?VHFXQk1YOVBMTm04ZTFDKzUvdjd1T3RnRXlBMXdEYnM3bEdNQmY3WXIzY01a?= =?utf-8?B?VytBcTFBSnpVRjNES1FnNVF0TzB5Ynczc0FzTGl0b3VSYlljMjZ4UjFYejVE?= =?utf-8?B?cWhua3drSXFXWVBPU1dXV3d2TE05d2RsbExoOUMzeTUxbTJJLzVuKzJGY0pn?= =?utf-8?B?ZWxUL1c0WHcvMFRKWE9DYmtUaDgwZFU2WmlleDlEWkFvRjIxODhLWlZUdUU2?= =?utf-8?B?elVsbzVhcS9vaUdBRVRVbjVyalJzVjFLa1pNMDFlTW5iejJ2aFN0bmFtbjhs?= =?utf-8?B?WkdpQ0FVZjF4cGdTZGNVZGEyMU5nRWNJbmxaUE14WUVZTDN1bjJPeU9LTXVl?= =?utf-8?B?Zk83TnRHYUI0RTNVRHduVUtEc3ZkbVRsOUxWdjVlMjBycUtaSFpRRStiYkRG?= =?utf-8?B?aHFDZXBTMm9OVWhKRmpQaEJ1L1cxazRSTUkyMkMyTFNjaWVuVWlyRTZFN1ZW?= =?utf-8?B?VWIxeXZFL2UwY1JZNnlxU0RqM2t4WVpON0lWd2JjMS91enArellEaG8yZUhW?= =?utf-8?B?OVVCekNaYXpOZFF3V0NyL3Yxd0I2MHhKZ3lmdk1yR2ZJVVhPK0xjZFJEUkZ6?= =?utf-8?B?K09EaTFnQVFMZjRSMlU3b0ExclVWc3dJa0tjOWlzQUZndC9xbGVWVFNoOFB1?= =?utf-8?B?Z3MzaWE0TGo1bjhQNUlraHM3cGRQT25maUdPcFIvVXBncFplV0c1a0ZMaWhs?= =?utf-8?B?RG5ZM2FBZ3VleGNHb3drZC9MOE9EZk03MDFNMy9TUmlCM0FFVTRUSUNMeXY4?= =?utf-8?B?dVdIV21WYXpxelFORTBTNjBqK3BqSm1JbUNnMVQrUnY3eldFVXVsK3FISzVX?= =?utf-8?B?VDZjTmRaL0c0aDVxQ1grODhFVDJzbVE4eHFKeWgyS2ZMRmh4bHVWT2J5cW9Z?= =?utf-8?B?REUzMWdxZUduaktibFdaMVVoMFg0emtzYlBBTnY2TGMrZHgwZXEvSzhDSUJa?= =?utf-8?B?UURCSmZodGsrUnljK3ZNbkxzV0JBQ3BOeXpScjJQRVBEZGhwd3RnOXBhNGhB?= =?utf-8?B?eXhXbk9kVS85eFZUdTVDK1Fqd0hJWi9TbGQ5UDF1d1NjeUlqeFlZTFpqK0h1?= =?utf-8?B?bjNPblRRTzY4K0pUZVc3SGR1YUs4WU96Z3djNFdJWEtSVlB6WjBwM0VGTDUy?= =?utf-8?B?L2I5QzdFU3lYNWZyQXM3bXdITVhpN2cvdFVML0piMFlWM05vb1Q1L1ptNlZr?= =?utf-8?B?enkvWUFIMTZUYmN4NDdHb3VwL2hkY0tNUy80eVhlRU00QWNhSlpLemhMcFBj?= =?utf-8?Q?C3N8G2UmBXW55dyCBbPGcXeEnJeiQL7uEZ4Fmvn5X8rS+?= X-MS-Exchange-AntiSpam-MessageData-1: DKkU/iUfEqSNUw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4df0453c-3c50-4838-cb4a-08df0e26ed1c X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:46.0832 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Moiqsnob4pbxYgWLBHgylaRnD7jkLMvVLrtrfuriHdh9GP3Z45tmk6AG9ehMhq5tbE/4Xtt9UBABoJRYlbxCaQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB699018 From: Joel Fernandes Add BAR1_SIZE constant and Bar1 type alias for the 256MB BAR1 aperture. These are prerequisites for BAR1 memory access functionality. Co-developed-by: Zhi Wang Signed-off-by: Zhi Wang Signed-off-by: Joel Fernandes [ecourtney: add the Bar1 lifetime for current PCI BAR ownership] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/driver.rs | 2 ++ drivers/gpu/nova-core/gsp/commands.rs | 4 ++++ drivers/gpu/nova-core/gsp/fw/commands.rs | 8 ++++++++ 3 files changed, 14 insertions(+) diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver= .rs index 8cef4c284277..5723ff8f71ea 100644 --- a/drivers/gpu/nova-core/driver.rs +++ b/drivers/gpu/nova-core/driver.rs @@ -38,6 +38,8 @@ pub(crate) struct NovaCore<'bound> { =20 pub(crate) type Bar0<'a> =3D &'a pci::Bar<'a, BAR0_SIZE>; pub(crate) type NovaRegisters =3D kernel::io::Region; +#[expect(dead_code)] +pub(crate) type Bar1<'a> =3D pci::Bar<'a>; =20 kernel::pci_device_table!( PCI_TABLE, diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/= gsp/commands.rs index eea1c9ed4684..1c467718c679 100644 --- a/drivers/gpu/nova-core/gsp/commands.rs +++ b/drivers/gpu/nova-core/gsp/commands.rs @@ -214,6 +214,9 @@ fn init(&self) -> impl Init { /// The reply from the GSP to the [`GetGspStaticInfo`] command. pub(crate) struct GetGspStaticInfoReply { gpu_name: [u8; 64], + /// BAR1 Page Directory Entry base address. + #[expect(dead_code)] + pub(crate) bar1_pde_base: u64, /// Usable FB (VRAM) regions for driver memory allocation. pub(crate) usable_fb_regions: KVec>, /// Exclusive end of the FB physical address space. @@ -237,6 +240,7 @@ fn read( =20 Ok(GetGspStaticInfoReply { gpu_name: msg.gpu_name_str(), + bar1_pde_base: msg.bar1_pde_base(), usable_fb_regions, total_fb_end, }) diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-co= re/gsp/fw/commands.rs index 6e85442a4b13..32856ff74183 100644 --- a/drivers/gpu/nova-core/gsp/fw/commands.rs +++ b/drivers/gpu/nova-core/gsp/fw/commands.rs @@ -131,6 +131,14 @@ impl GspStaticConfigInfo { self.0.gpuNameString } =20 + /// Returns the BAR1 Page Directory Entry base address. + /// + /// This is the root page table address for BAR1 virtual memory, + /// set up by GSP-RM firmware. + pub(crate) fn bar1_pde_base(&self) -> u64 { + self.0.bar1PdeBase + } + /// Returns an iterator over valid FB regions from GSP firmware data. fn fb_regions( &self, --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012044.outbound.protection.outlook.com [52.101.53.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A7062E040D for ; Wed, 9 Sep 2026 04:00:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.44 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926459; cv=fail; b=mHStG6XkmHXvUraxpPfr2HNUWh4IGaeK3A2YdjEo2HSwiusuJXWBMpYwenaH2qN+3WEHysJtkR0w/zKpRutKoATb/4dNe0YyuPT0DX3iX9Uloo1EOX4c65K5L0Cnk79AOiEvQlyFHBCR0+yx24CyZlwV5sIQbWaXazG2HcOR880= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926459; c=relaxed/simple; bh=5gYwsX6rXiyM8l6TdNoPznEEsPkp3R225B7/NHmHxwM=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=VocRkSjboRoeuYKTz8DM5h+rtamAHcNyW5l3ctbr31FXiNpCK2PBe3nhuuShj7Hjrs3m/3tKseq5ghBBAHoqCkR4C1APc/HthhKaa4EGNeARgEKfjcDVZZVJm4CPIISIENKdMKxb1OUaxd24Kgyfi8K0wcsTMsZSy6Qbi4MWbk0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=EjNpAgvE; arc=fail smtp.client-ip=52.101.53.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="EjNpAgvE" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aR4rdddh+KO+dXKw7MNtUnfK66ojJjz3DoGiS12/1dktdO1moq3cLL5/VnRPMM7BfputRcngw4MH0dlO8+eGswUx7aLcBFHnwfw4ygrsZaVw44728j1/AWyJS6kG8xA3fc19lL2Nu/zckuvb7UtUdZ6DhxUeSEIR0TYaT87CB5LrIp+KfSw0LXSNgTKojTwGlrATPy7vrrmMQcnMPqEEcGkrpe31Exq1+OSYLwWkzbPvL3NLYnffe5fCzV3BSpGQrDey+f2Q5Unzut4tycnJlToudxs0X8oZmGQCDPrKbMfoy0UoIZ7hEZ6kW3SgnCfKQ+NdOxzkJKbqImcm0iJzlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tMPtr8qviiLeGsjIENXD9ecKTxUwCkEynyB9oU+f+nI=; b=ijlrEBWWyENEVtmjscto+y9ohsIAtqOHAZiDr6wJfChi/YQcqCYlxZRcmzUOo/i2yJ+L0NTqFjguucDyNw/zvkoWNJZv6sBp8xBsBwCOiK0YD48YIfXa5NDvXq1LdfRp5P8A9Yike+rK9Xem8ceZGpiu6tjMRMT0NHz7YBG2VhCX3uZTohTl1fROfbWiDWgiWgiAuHtk2d8HibepszR5ykQzqW6doD0H4COWj6YlTPxT6Ad8mIhVFelxIDVOKmOFR+beqWZSQBCaSMTX1N3l2F5sMhL/4KtAoj7N7zR7c+aWr27V8raWOqI0P9TVTl4PJaPGzGhQqyxhCWdJI+uEeA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tMPtr8qviiLeGsjIENXD9ecKTxUwCkEynyB9oU+f+nI=; b=EjNpAgvEBoe7m+Phy3w2ogSRf4p1Vg24Xcy2CtR+0L5lsEtQtXnwQWsZl0/zjO5bPO6MXT8aflrhOHxX/iupEVJLb8rM5TGUJ49Q9V/lSrNfynSJvJflSJ/oeuvLjiarVVIgyJdVLXFCWEHIFFjK1hZekB9idkzH/btF1BiKTEkVJLCyiEYyzWDdW/7em7VGMWdkTa9n8/P8JytHrGlfajHcFj8VJPhT6mZ1Q0sKTfErG2rFA6YnQ4UmJoGGh2vwdW/NBYXEviWJYUMzz1rrw+x3CVUuJeDbFZVuuoDHj36zmul8X5QadsdIJG1ehmcAYFN8RO68wbKvSyAVZrMbEg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by DS0PR12MB8070.namprd12.prod.outlook.com (2603:10b6:8:dc::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:50 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:50 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:53 +0900 Subject: [PATCH 15/16] gpu: nova-core: mm: Add BAR1 user interface Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-15-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TYCP286CA0293.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:3c8::11) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|DS0PR12MB8070:EE_ X-MS-Office365-Filtering-Correlation-Id: 8e2d3f85-aaf9-44da-1f07-08df0e26ef5c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|23010399003|10070799003|376014|366016|10067099003|6133799003|3023799007|7136999003|22082099003|18002099003|11062099010|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: XJ6PnzWcNDIVIxHLEOTA8aVlfG1a2Dqmxi56+RazeMgrZrHyMjcn2usERj7XvE3I+gBsr/vfBhGU++qe9Lt00Wf9HUA39tKUGTJvS1IGWsLeJtMheh3S1C3LxgqNY3woEEzi+88duKX8OtbVl2gGjLM2qzfHVZfA0OnO6BBopyG7AHjhdkiM8cnI/9IiA3G8jifCDhA4XsV1fFlDJllXUDoqWWkNDFEKXblbsvXvaIi06pMdCfBTmEcq4dYTr8Rda938OHQ3AZzurpnOCBqcw1opFHiDvL9xBuGI0RH6/KVk5UnEok0uyntUbQkzlgePqZGCLavg+5obzVBWUNq1qJmqiFPpoZdBKNyQ4mtRNcbBVcHu8scqGc4BzGoPOy3d2879VtMqtDe0CWL3eoFmHCzz5TCboxVWcFUThWNsOQ5B+zhg6BmhFNtT0B/+YlRcFgFoi3e8Rq3Q92SIVJBiR9zPDUq2aZ47/ZYorhYzBytzRmWgWdpELDmKXZ1+aTvpPjMadGfp5O58p/Ibr2X9OUKn0gi+qZtg9JPCmFkyqIAPhwK0yClvseNszXGeIpnLIcgDx/cnxr1fNYIGURf+qXMGtyffHRF8TUi4vGDYwJ9Tk533VD1lLEsrQ+Hkoc6RLdMK4KOGOu4yo5g2NF8xBN80MP3/WXENOQp5MJl4yeY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(23010399003)(10070799003)(376014)(366016)(10067099003)(6133799003)(3023799007)(7136999003)(22082099003)(18002099003)(11062099010)(11063799006)(56012099006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZWtXa0N4b1RvNTBnbFhUb3JPSWtKeXIxU1MvSWxQWFcvdjJHLzY0RnQrWm9J?= =?utf-8?B?aFVoUkorSWpxcTltREZPUEZNWElqRkR4N0dPcStGdGlOOXBVNG5tT1ltejIv?= =?utf-8?B?K2FZbzl0YlhVUXNWT21yeU9ITFdNM3lvVWRUWUtrUC9ZUHg3T1FTY0tSSXhp?= =?utf-8?B?aHdyZWwwRjVrQ1lnOGNLSUdJcVdYRmtnMnRMWXNNcHlWUjBZMFFPbGZzMTVu?= =?utf-8?B?ZUV3K0x3Q0V6QnpKY0d6ODBaWEFCS1pUTHNLbSs5QjczVzNBQUx3TTYrd3A4?= =?utf-8?B?MEVVOVk0ZzY4c2F0c2xSbzRrc3o5c0l1cm5yTGpRR3NSTmtjVUwrVzkyQWY0?= =?utf-8?B?eTJQaGk2ZUJpdnpYcm5lUld0cEw3NjdyWnlJNDVackhjWUNsMTZKUU03SHdT?= =?utf-8?B?VWptL1ljU1FocHQxT0JqMm5kSGQ2aG5rSjI1cTF6SDg0bmxVbWhBenQ1a00x?= =?utf-8?B?TmQ2T1FMZzNoejd2dStiYStmaEhYdDlWc2hiUUxmdUQ1QkRRQXh1Sy9pY3Na?= =?utf-8?B?amtjQ0l2NkRScm4vQzNGZHpYUnJIMkJCclN3VFI5ZFRFcGYxY3Z5bzhEWXMr?= =?utf-8?B?QnY5eXdIei8wR1hjYTgra1V0NFRRbEV3bFhLOUFwQ1ZmYzgxZGcra0k1V2Vo?= =?utf-8?B?dUFIS3RlN0huaDU4ZVNRdm40eVpuQVE1aGthVHQ4dm1GUkIxOFN0ajA3QlFT?= =?utf-8?B?OFRXMngvMnJFN2Q2OVQrTC85V1ZMZG5FTCtsN01VSDRCZkZhaXV3cnBLNnRZ?= =?utf-8?B?R0Z2T1hvcGg3ZnBGQ3FiR0J3a3AvaHkwQkJTNFZRT21TZ210STZNZ2ZzOXBs?= =?utf-8?B?WEtPcXRZUHFSRDFGVDVmUS9xRDZXQy9VYlc2ZlVUUWRCM0cyMzBTVklURGdI?= =?utf-8?B?MEY4NHp0aW16R2FVNThUOHZ5VXJCb3RwUU9qQVBGMllINkZvbzNOOVl0OWxk?= =?utf-8?B?aW1CdmV0ZnM3Ri9kR1JiM3AxcmdvM0ZTbk1RN3IvZDRaVml6L0hHalJjdlZO?= =?utf-8?B?c3FYaWozdE1mRE9wdWtVbzZ0WVlhTmNTWm84aHdMVytrS2JWS2pFQlowcVRl?= =?utf-8?B?VnhWN2Z0OEROa01EUTd4enlnWktHZVltM1YzNUZnK1k3SkxoY01hK2Z3akEy?= =?utf-8?B?QWNYMHU3S0M4K0RmYzhGRi9PT0t4bnRjcVcyZEUwRFNjQ2dCWE1QRlYxTXB6?= =?utf-8?B?UWU5dzBxOEhBbXhMQkdVbFhaOHJXVWR0d2VTZVZOblpSRm1kazBJcUdoUHdy?= =?utf-8?B?SW5wSnlKYm56UGZJRFA1ekc5M0IvT3VzWnFoWi8zcUpsU09wVjlZUHQ5OU5u?= =?utf-8?B?SlZIY0VudDRQNUhPTWJKbkNLMzBiVmNzSTl3dzdPczh6U05yc0pIL1lzTytr?= =?utf-8?B?RU9ISlN4T3N4OTBDbmFqYm1BWXhQRlhHbkY5OWpwZll5M1pGUUNrdDZnNXd3?= =?utf-8?B?NGgrWXUveHA5cHp6OVhBSFphcjRqVEJ1QkJheTNuVW42NURFeDhEdnVTY2xl?= =?utf-8?B?N2tKcDJjTVNNcmtHeFFqVXh5WXJuZU9HZ0o1YjBpbktQUTFEMzVidnBacVA1?= =?utf-8?B?YXl1MWc3MlArMHJqRGJjUU82R0t2eHV3RGZCb1FVWUdYdjFJc1RlUzgwaHlM?= =?utf-8?B?TUJTNGY3cFc5U0NkZDNueHR2MFZVRGJBNGQ0VERDOTA0L0gyb0pDRmhPNk8z?= =?utf-8?B?NUJ5K280czFLTmxsaEluMWdubkt4V3dCMVRDYk5scWg3N1YwSlRxYXhLSnI0?= =?utf-8?B?MU9BRDdVTC9PcExLNEwxOEhvMFlGTGUyRnI2c1BSVTNxcUNwL1cxeFZ6RE9G?= =?utf-8?B?d0g0RXpQSEo1VlFFTzBWME51M2g1QWcxdElTZVZEaVRWUVVOY3J1eUtEMWlt?= =?utf-8?B?eW5RZ3B3N25JYTg5WDgrRCt6c2tuaTY3dlJZS1NWcmlJdlFlZVphNVlEb290?= =?utf-8?B?Z0N6ZnY2ekdETHhFRzhuelFxWktsVEtsQTl0TWVEdGlRcFd0amNVYkZrWjlN?= =?utf-8?B?YVA4Y0RBNDRXTkNlcWsvMGtyZlBibGdOU3k1NFpwOGZybXg0TC9jRGZsMlJv?= =?utf-8?B?NnZ2UXpWcFpzUjVnUjNLMXhBRUxLaEpZVHFZQVh2c1NFb21oZDZ5bHNlM0pn?= =?utf-8?B?YS9QeGxUdk5vbmFYKzhNU3VCYTBaYksxU3lpa0pzallRUi9mdzhsTXlESTJj?= =?utf-8?B?c1o4N1VNUkI2Z1JXcElFSGJuOS9VZTFrZnNiMk84QWxwaWtZUDl0TFVZT3Zx?= =?utf-8?B?UGRKbGRCS2oyRHpvbzVCUXp6ekhzNmRxdEJ4ZTVyRW5lUnA4ZHF2TzlkeHlW?= =?utf-8?B?UTRscVVUa3AxZEozanRaNisxWFFMaVRiNWt0Z1dKd1VSc1RJWndzRjRub1Rp?= =?utf-8?Q?yG7LNA4rYH77A1DLYQANowhtl8VAAR3oWTTONBKhNVf3c?= X-MS-Exchange-AntiSpam-MessageData-1: Lmb6KS0CuTGkBg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8e2d3f85-aaf9-44da-1f07-08df0e26ef5c X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:49.9850 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8C/PqwFJNNdByjnx4uu1EbkX10Z/G9oBIlwRgm2rYVv8rYGzfEs3m9dQST3zRn/+nqLlm46o1D66sYuwLsWaRA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8070 From: Joel Fernandes Add the BAR1 user interface for CPU access to GPU virtual memory through the BAR1 aperture. Signed-off-by: Joel Fernandes [ecourtney: map BAR1 in NovaCore, borrow it in Gpu and BarUser, no Devres] [ecourtney: update for the VramAddress raw API and gsp_resources chipset] [ecourtney: drop the owned GpuMm, pass it mutably through map and release] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/driver.rs | 45 +++++++-- drivers/gpu/nova-core/gpu.rs | 32 ++++++- drivers/gpu/nova-core/gsp/commands.rs | 1 - drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/bar_user.rs | 171 ++++++++++++++++++++++++++++++= ++++ 5 files changed, 240 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver= .rs index 5723ff8f71ea..0672a0707a71 100644 --- a/drivers/gpu/nova-core/driver.rs +++ b/drivers/gpu/nova-core/driver.rs @@ -2,7 +2,11 @@ =20 use kernel::{ auxiliary, - device::Core, + device::{ + Bound, + Core, // + }, + io::resource, pci, pci::{ Class, @@ -28,6 +32,7 @@ pub(crate) struct NovaCore<'bound> { #[pin] pub(crate) gpu: Gpu<'bound>, bar: pci::Bar<'bound, BAR0_SIZE>, + bar1: Bar1<'bound>, #[allow(clippy::type_complexity)] _reg: auxiliary::Registration<'bound, CovariantForLt!(())>, } @@ -38,9 +43,27 @@ pub(crate) struct NovaCore<'bound> { =20 pub(crate) type Bar0<'a> =3D &'a pci::Bar<'a, BAR0_SIZE>; pub(crate) type NovaRegisters =3D kernel::io::Region; -#[expect(dead_code)] pub(crate) type Bar1<'a> =3D pci::Bar<'a>; =20 +/// Returns the Linux PCI resource index that holds BAR1 for an NVIDIA GPU. +/// +/// On Maxwell through Ada, BAR0 is a 32-bit memory BAR occupying a single +/// Linux PCI resource slot, so BAR1 lives at index 1. Starting with Black= well +/// (and on some Ampere GA100 / Hopper SKUs) BAR0 is a 64-bit memory BAR t= hat +/// consumes two consecutive resource slots: index 0 holds the low 32 bits= and +/// index 1 holds the high 32 bits (with no `flags` / or size of its own), +/// shifting BAR1 to index 2. +pub(crate) fn bar1_resource_index(pdev: &pci::Device) -> Result { + // Probe the `IORESOURCE_MEM_64` flag of BAR0 as a robust way of expos= ing + // if BAR0 and hence BAR1 is 64-bit. + let flags0 =3D pdev.resource_flags(0)?; + if flags0.contains(resource::Flags::IORESOURCE_MEM_64) { + Ok(2) + } else { + Ok(1) + } +} + kernel::pci_device_table!( PCI_TABLE, ::IdInfo, @@ -82,12 +105,18 @@ fn probe<'bound>( =20 Ok(try_pin_init!(NovaCore { bar: pdev.iomap_region_sized::(0, c"nova-core/b= ar0")?, - // TODO: Use `&bar` self-referential pin-init syntax once = available. - // - // SAFETY: `bar` is initialized before this expression is = evaluated - // (`try_pin_init!()` initializes fields in declaration or= der), lives at a pinned - // stable address, and is dropped after `gpu` (struct fiel= d drop order). - gpu <- Gpu::new(pdev, unsafe { &*core::ptr::from_ref(bar) = }), + bar1: { + let bar1_idx =3D bar1_resource_index(pdev)?; + pdev.iomap_region(bar1_idx, c"nova-core/bar1")? + }, + // TODO: Use self-referential pin-init syntax once availab= le. + gpu <- Gpu::new( + pdev, + // SAFETY: `bar` is initialized above, pinned, and out= lives `gpu`. + unsafe { &*core::ptr::from_ref(bar) }, + // SAFETY: `bar1` is initialized above, pinned, and ou= tlives `gpu`. + unsafe { &*core::ptr::from_ref(bar1) }, + ), // Run optional GPU selftests. #[cfg(CONFIG_NOVA_CORE_SELFTESTS)] _: { gpu.run_selftests(pdev) }, diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs index b797472279d3..f72a92e045e5 100644 --- a/drivers/gpu/nova-core/gpu.rs +++ b/drivers/gpu/nova-core/gpu.rs @@ -16,11 +16,15 @@ SizeConstants, SZ_4K, // }, + sync::Arc, }; =20 use crate::{ bounded_enum, - driver::Bar0, + driver::{ + Bar0, + Bar1, // + }, falcon::{ gsp::Gsp as GspFalcon, sec2::Sec2 as Sec2Falcon, @@ -35,6 +39,8 @@ GspBootContext, // }, mm::{ + bar_user::BarUser, + pagetable::MmuVersion, GpuMm, VramAddress, // }, @@ -148,6 +154,11 @@ pub(crate) const fn arch(self) -> Architecture { pub(crate) fn pci_config_mirror_range(self) -> Range { hal::gpu_hal(self).pci_config_mirror_range() } + + /// Returns the MMU version for this chipset. + pub(crate) fn mmu_version(self) -> MmuVersion { + MmuVersion::from(self.arch()) + } } =20 // TODO @@ -297,6 +308,8 @@ pub(crate) struct Gpu<'gpu> { /// Must be kept declared *before* `gsp_resources`, so that its compon= ents are dropped while /// the GSP is still operational. mm: GpuMm<'gpu>, + /// BAR1 user interface for CPU access to GPU virtual memory. + bar_user: Arc>, /// GSP and its resources. #[pin] gsp_resources: GspResources<'gpu>, @@ -340,6 +353,7 @@ impl<'gpu> Gpu<'gpu> { pub(crate) fn new<'a>( pdev: &'gpu pci::Device>, bar: Bar0<'gpu>, + bar1: &'gpu Bar1<'gpu>, ) -> impl PinInit + use<'gpu, 'a> { let dev =3D pdev.as_ref(); =20 @@ -442,6 +456,22 @@ pub(crate) fn new<'a>( VramAddress::from_raw(gsp_static_info.total_fb_end), )? }, + + // Create BAR1 user interface for CPU access to GPU virtual me= mory. + bar_user: { + let pdb_addr =3D VramAddress::from_raw(gsp_static_info.bar= 1_pde_base); + let bar1_idx =3D crate::driver::bar1_resource_index(pdev)?; + let bar1_size =3D pdev.resource_len(bar1_idx)?; + Arc::pin_init( + BarUser::new( + pdb_addr, + gsp_resources.spec.chipset, + bar1_size, + bar1, + )?, + GFP_KERNEL, + )? + }, }) } =20 diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/= gsp/commands.rs index 1c467718c679..663e1d124781 100644 --- a/drivers/gpu/nova-core/gsp/commands.rs +++ b/drivers/gpu/nova-core/gsp/commands.rs @@ -215,7 +215,6 @@ fn init(&self) -> impl Init { pub(crate) struct GetGspStaticInfoReply { gpu_name: [u8; 64], /// BAR1 Page Directory Entry base address. - #[expect(dead_code)] pub(crate) bar1_pde_base: u64, /// Usable FB (VRAM) regions for driver memory allocation. pub(crate) usable_fb_regions: KVec>, diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs index 2cf37254fbb9..e04497256519 100644 --- a/drivers/gpu/nova-core/mm.rs +++ b/drivers/gpu/nova-core/mm.rs @@ -60,6 +60,7 @@ macro_rules! impl_pfn_bounded { =20 pub(crate) use tlb::Tlb; =20 +pub(crate) mod bar_user; mod hal; pub(super) mod pagetable; mod pramin; diff --git a/drivers/gpu/nova-core/mm/bar_user.rs b/drivers/gpu/nova-core/m= m/bar_user.rs new file mode 100644 index 000000000000..ef1d8e6f8c9c --- /dev/null +++ b/drivers/gpu/nova-core/mm/bar_user.rs @@ -0,0 +1,171 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! BAR1 user interface for CPU access to GPU virtual memory. Used for USE= RD +//! for GPU work submission, and applications to access GPU buffers via mm= ap(). + +use kernel::{ + io::Io, + new_mutex, + prelude::*, + sync::{ + Arc, + Mutex, // + }, +}; + +use crate::{ + driver::Bar1, + gpu::Chipset, + mm::{ + vmm::{ + MappedRange, + Vmm, // + }, + GpuMm, + Pfn, + Vfn, + VirtualAddress, + VramAddress, + PAGE_SIZE, // + }, + num::IntoSafeCast, +}; + +/// BAR1 user interface for virtual memory mappings. +/// +/// Owns the [`Vmm`] for the BAR1 address space. +#[pin_data] +pub(crate) struct BarUser<'gpu> { + #[pin] + vmm: Mutex, + bar1: &'gpu Bar1<'gpu>, +} + +impl<'gpu> BarUser<'gpu> { + /// Create a pin-initializer for [`BarUser`]. + pub(crate) fn new( + pdb_addr: VramAddress, + chipset: Chipset, + va_size: u64, + bar1: &'gpu Bar1<'gpu>, + ) -> Result + 'gpu> { + let vmm =3D Vmm::new(pdb_addr, chipset.mmu_version(), va_size)?; + Ok(pin_init!(Self { + vmm <- new_mutex!(vmm, "bar_user_vmm"), + bar1, + })) + } + + /// Map physical pages to a contiguous BAR1 virtual range. + pub(crate) fn map( + self: &Arc, + mm: &mut GpuMm<'_>, + pfns: &[Pfn], + writable: bool, + ) -> Result> { + if pfns.is_empty() { + return Err(EINVAL); + } + let mut vmm =3D self.vmm.lock(); + let mapped =3D vmm.map_pages(mm, pfns, None, writable)?; + + Ok(BarUserAccess { + bar_user: self.clone(), + mapped: Some(mapped), + }) + } +} + +/// Access object for a mapped BAR1 region. +pub(crate) struct BarUserAccess<'gpu> { + bar_user: Arc>, + /// [`BarUserAccess::release`] [`Option::take`]s this; `Some` at + /// drop time means `release()` was never called. + mapped: Option, +} + +impl BarUserAccess<'_> { + /// Tear down the BAR1 mapping. + pub(crate) fn release(mut self, mm: &mut GpuMm<'_>) -> Result { + let mapped =3D self.mapped.take().ok_or(EINVAL)?; + let mut vmm =3D self.bar_user.vmm.lock(); + vmm.unmap_pages(mm, mapped)?; + Ok(()) + } + + /// Returns the active mapping. + fn mapped(&self) -> &MappedRange { + // `mapped` is only `None` after `take()` in `release`; hence unwr= ap() + // cannot panic here. + self.mapped.as_ref().unwrap() + } + + /// Get the base virtual address of this mapping. + pub(crate) fn base(&self) -> VirtualAddress { + VirtualAddress::from(self.mapped().vfn_start) + } + + /// Get the total size of the mapped region in bytes. + pub(crate) fn size(&self) -> usize { + self.mapped().num_pages * PAGE_SIZE + } + + /// Get the starting virtual frame number. + pub(crate) fn vfn_start(&self) -> Vfn { + self.mapped().vfn_start + } + + /// Get the number of pages in this mapping. + pub(crate) fn num_pages(&self) -> usize { + self.mapped().num_pages + } + + /// Translate an offset within this mapping to a BAR1 aperture offset. + fn bar_offset(&self, offset: usize) -> Result { + if offset >=3D self.size() { + return Err(EINVAL); + } + + let base_vfn: usize =3D self.mapped().vfn_start.raw().into_safe_ca= st(); + let base =3D base_vfn.checked_mul(PAGE_SIZE).ok_or(EOVERFLOW)?; + base.checked_add(offset).ok_or(EOVERFLOW) + } + + // Fallible accessors with runtime bounds checking. + + /// Read a 32-bit value at the given offset. + pub(crate) fn try_read32(&self, offset: usize) -> Result { + let off =3D self.bar_offset(offset)?; + self.bar_user.bar1.try_read32(off) + } + + /// Write a 32-bit value at the given offset. + pub(crate) fn try_write32(&self, value: u32, offset: usize) -> Result { + let off =3D self.bar_offset(offset)?; + self.bar_user.bar1.try_write32(value, off) + } + + /// Read a 64-bit value at the given offset. + pub(crate) fn try_read64(&self, offset: usize) -> Result { + let off =3D self.bar_offset(offset)?; + self.bar_user.bar1.try_read64(off) + } + + /// Write a 64-bit value at the given offset. + pub(crate) fn try_write64(&self, value: u64, offset: usize) -> Result { + let off =3D self.bar_offset(offset)?; + self.bar_user.bar1.try_write64(value, off) + } +} + +impl Drop for BarUserAccess<'_> { + fn drop(&mut self) { + if self.mapped.is_some() { + kernel::pr_warn!( + "BarUserAccess dropped without calling release(). BarUser = address space will leak.\n" + ); + } + // The inner `MappedRange`'s own `MustUnmapGuard` will also fire, + // identifying the leaked VA range. + } +} --=20 2.55.0 From nobody Fri Sep 25 20:02:38 2026 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012010.outbound.protection.outlook.com [52.101.53.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C04A35674F for ; Wed, 9 Sep 2026 04:01:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.10 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926466; cv=fail; b=aWk0AYlIPiM6Ju5m9zVXiNGEmoKU7sX1W78ypfjrQNn483KDlYkwTJrc1K83NWedPmdIEMVndGj1DZZkhfyEe1iZTqjrZmvQMbtuMvtTU8DcezpF0K8WNgd9z0eoCIGFcFQ+DSBYRU8Fy/vBr4ap0hmKzciy+z2gVqz5AY+KmcE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788926466; c=relaxed/simple; bh=VObYjq2QSkzXkFu/wYqFSKFkVhry1yCj/LfOixjX+s0=; h=From:Date:Subject:Content-Type:Message-Id:References:In-Reply-To: To:Cc:MIME-Version; b=cZDczH2yM0x/ZfTs/f8V4wmqPkZ0ga7LuZZLcQHLz3qX15j5DkRF+ig1W1fSf/qsFAbkaNoufPi+519d6Yjh+mUxMWWph7+tt53rDF+XDARu59ZnunJC1uUVJWBKivY0vqamG2hamExpdSVCIJp1pxp6024f+e0mkImQZMtgNkY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=WMy4+7US; arc=fail smtp.client-ip=52.101.53.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="WMy4+7US" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JCqcc+G1lK/MEIZ9opPECl94QK9QxD3ambM9HgN6TEq+gRAtItMk8qn2/FC0n9dt6mHY6J3NLoRFcTZ30UcpCGAiC0bdq1xdeHRfXxnSYwaBFO7/W1HW3datqUzTEnum/uKB9F48SKX+HpPYwDM3uaPAbgboJ8WmxUixGQuvam64yrUl0dNu96xsgsnokj2dRV5MNTdTyB+gpFu8iB0ykLaL2rU/su67HoYhJ7fub8Qo53ospFYRGYlT1zMXoflnkY64ulMbVVQj0uN0vonOsdGZ5URgJ2LheS/BdaBjKVCFQCiE+4sZQq4usrqAvQupna5rriMSfEUbuKdYesrH4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wDSARerljZgu1Ih5Jw3k4TvWtnU+WAE2s2Akt6NMglE=; b=rEkLqXvotCu4y+yCCVJs+5ZudvttnZxIwWTLlmLGkR//gvTc1AxByE+euP711sKhyY3TDrzazO6CrmhHd7r2T17rACgdvVA2y7H6nuT9XnVF2IYNAOG6kpOla8htk9b+bYQK01h/GVM4gPXHRZ19rIQBitgBjUEL6SmahLPjfIVHsAe6xZs5LZGM0zNyKs4jpgSDuaC0dn7Ej0N4kqMb9gSGP5e+bo0TzsTsypl4qww3ON3IdZVoIOYOiX+RxBAW2ihqzbwNwowWc+diSSI4vArzSQay5MxBq2Fz0MaFQqpMwDCj76SYoAItf3CC63502dug6hRMZLb/Jj1lz+ay7Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wDSARerljZgu1Ih5Jw3k4TvWtnU+WAE2s2Akt6NMglE=; b=WMy4+7USY9jMix3B54HSp3HkUABF5bAPaTTmsJNusGUYTq/QOazo0/n8ETGn5vyGfY+05QHPIQCivb3wOf259LXHXIus6Gzw+zGsjFN6GSt1X6oxspkPWKx4jWh5Ob7ZxV0q6sNjXJtLT50fhDcbpbe2SQyyW6z6+6FLONCiIyMSGvmIIyqyUW1BSb5zy6cIbROArmjCO+FBwt4QpWPKTKk3mFXK/yiQdDJf4R9CP6d1RK4tXtxJePgelbuHvLv+SbEW9hwCAreM7mLQh/0RUo8m3bDwtRdmnVUPHePY1O0yMQkb0CnT95e3b2R3IG6hN2kzoHerGh4b/5wAF2bpTA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) by DS0PR12MB8070.namprd12.prod.outlook.com (2603:10b6:8:dc::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.406.7; Wed, 9 Sep 2026 04:00:53 +0000 Received: from DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa]) by DS0PR12MB6413.namprd12.prod.outlook.com ([fe80::e82a:6673:4142:37fa%5]) with mapi id 15.21.0406.005; Wed, 9 Sep 2026 04:00:53 +0000 From: Eliot Courtney Date: Wed, 09 Sep 2026 12:59:54 +0900 Subject: [PATCH 16/16] gpu: nova-core: mm: Add BAR1 memory management self-tests Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260909-mmrebase-v1-16-8dd5d4225d2e@nvidia.com> References: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> In-Reply-To: <20260909-mmrebase-v1-0-8dd5d4225d2e@nvidia.com> To: Danilo Krummrich , Alexandre Courbot Cc: Alice Ryhl , John Hubbard , Alistair Popple , Timur Tabi , nova-gpu@lists.linux.dev, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Eliot Courtney , Joel Fernandes X-Mailer: b4 0.15.2 X-ClientProxiedBy: TY4P301CA0111.JPNP301.PROD.OUTLOOK.COM (2603:1096:405:37b::10) To DS0PR12MB6413.namprd12.prod.outlook.com (2603:10b6:8:ce::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6413:EE_|DS0PR12MB8070:EE_ X-MS-Office365-Filtering-Correlation-Id: 3517869c-18c6-4a08-94e5-08df0e26f1a1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|23010399003|10070799003|376014|366016|10067099003|6133799003|22082099003|18002099003|11063799006|56012099006; X-Microsoft-Antispam-Message-Info: 44msYEh8j5g6IBxV0+3nxM3AwQT2kbj3He138hwh90/ZATarfIyP0kCO46t0KMZRLyWyld2Qv9vWSUgERerWSiINDH0uI3Rw6BMkkWPk4qyMBoRTQZKP/ArMONiVSdKOyuxWECya+3ryKo4IewnZJHsjIiL2NFb997cKbsMPQwMAwCHzcMkOhsmYa9wU1Qb+Dvo8Sy2j+rSaGUCe/9AjuRvLbfWj1DA0m1fELHnKQ19cEUfq0dBIITUEU0Mu3p5S+7l3mm32fRqRraY6LhOhqIjK4d1od5Lbuj/YrQiC/RLF+3wrQDUqYqKXL58rqKDtP+T5OyDce7gorSs6F91MSafpNWSG+L4rqM6jLRl2wCixHOyWkNAVtxUYzaUxThKpViSDrEFZdtAvDM4b3FXNrRvqclyuheT62Z7NUsi1Wqa+JGmc9Vb7ZecPZhCltFO5BAY8YmrffIVZM/5TBzm8dj3Ohz7dSzJPwQePspNx14AHET51Q+apBpSOqXBkYPlD/do2+X4fMyY0EJ0/qZB/nuB80nKo27qur/EvURXrAhsV6pAWvwm8dDUWW4lEd8tcaW/hyY3bTY0lnjkERnrCqOEH5Fl7TJICxr4AEilZWrahJEDZWEFQcDG3cGSgdxftiVkeByBtYdqXVdvqBMVmjedbQ/bpv9US5eAsBfumC7o= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6413.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(23010399003)(10070799003)(376014)(366016)(10067099003)(6133799003)(22082099003)(18002099003)(11063799006)(56012099006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NDdkUnhkMGRHbnBHdkE4SXhMR1M2VFloZ0RVaktCL0tRQ1lFOUtLVlQxYkNW?= =?utf-8?B?Q2lxeFd1QndNenNvemxQWVRDQmo0SmRVLzZIUS80TFYzNEZ2UjNid1FEN2V2?= =?utf-8?B?RHFqNmEzSnl0VGh1SWtwSzFXakIra1gxZUZDa05XM08veFJrNDl4L3c2N1Bp?= =?utf-8?B?MThickt3QVU3dC9XRStwbGVkV2ZFa05TQTkrVlN3S2krdmhTdWVaMGpFTEJk?= =?utf-8?B?a3hHbmtqNmllME15OEZRalBUTU1VL1o0eDFVQnk2NUUzcGpyaStnNFBlUmQv?= =?utf-8?B?UmdFMmE1RXExSmoxV1B1T1I5OTM5OXdiQ3VEQUJTVjJWdTZFcDE5UWRtSTNQ?= =?utf-8?B?R09SaEJEWloxclloWG1xZ21sL1IrMzJCYUVtL1NZNC8xQUIvaXZvaURoWStI?= =?utf-8?B?cGVQZXFIZXlZcWczQWk2M0RZZGtDYTQwWXVhWGloRy9INWVRbloyUmloVnFv?= =?utf-8?B?VXZvbTdFMXpuSWdLRkFpcVN6cXVxcFh2K1hzdEFvT3NRTktvS3NReWpUZ3dx?= =?utf-8?B?V054ODZyQ2ExTGY5L3h2eTM4OFEzeVo3RTBDVEVvS1o2WmJ6UFRFcisrRzR2?= =?utf-8?B?d2ljNmNkeFBzQTBvSzFLaGUrVUY5RW5HdldpZDlUanYzeTZEVE56N3pXQ0wy?= =?utf-8?B?bE9tTFkwaWRjMlgxNXhWR09MMDJPa3NEWlB2TjA0ZktmMWVqeDlWVUNYckRl?= =?utf-8?B?bTlLVFQ1dzFGM3J4ZU1ZUktCWC8xSXZLTm12bnhQam5XN1R0ejlZY01lUWxx?= =?utf-8?B?dzFBb1FHV1dKdkNYVzBma3FrL09BQk53a0xqRFZWTzk2VGl3Y2dpdHNzZCtS?= =?utf-8?B?WHFuTGJxOTllQlYzUlE3Vzc1ZVZta0J2b1R2R1hCbysyU3Z0d3pYVlJhajAw?= =?utf-8?B?Sy9tSzRsZ3RaNzBOb1ZpdFZJajFVRUpvWXZLVGZydWZuY2NJbGQxQjhvNGsy?= =?utf-8?B?L3pZVHhOZ0tKQzZTcEdMc2NObDlkOVZKbHdSalpxZVM3QkUzam1OTDVIbkVB?= =?utf-8?B?S2NBdHhpcEFDSDIwYmU0Y0xCWW96bnpmM0M1Y2lPc0YrMnZRWDNaKzQ1NTNJ?= =?utf-8?B?MitleDVrVytVYmRXc1FVUXBqcWNKVUxMcUNiaDg2YXVKS250RGgrbG9ySVFH?= =?utf-8?B?TUdCUTRPU3lQQ2R6RGdlQzVTeEh4OERldnFKSDh6VEZSWDRhUXR2Yk1BTlZi?= =?utf-8?B?ZkI4WDkxN2lpV1R6NEt5bmhMT3hCNytOV2Rva2oyRkJneEhLcnA1QkpyNmJ5?= =?utf-8?B?bmVoS1VDSGVZVllpWTEyVFM5N2syOEx4VTBWZUhjUjFPOGdsZ3krTG5yVkp5?= =?utf-8?B?c1V1MUN6YTM1Z0dOS2Y5b2Q2V0tBNThwZDlhaVczOGJBeWdLWTU3R3VRRlJH?= =?utf-8?B?SGNWR0RGc1dTUFRDU29tS2w4MllTOE5VMTFYVElGcGNhbnVtY3VKVjFMQ0Zj?= =?utf-8?B?T3kwVUJLc01ZM1dNaUM1N0xLVTIzNXpTYVhGVEVCV0lXUitGOGZNTlNPZVJ6?= =?utf-8?B?MlZ3SFFVK3BqRGo3YkpXNWt2ZE9VY2RBNForNURGZW1Rb0EzSlVwOXlQS0Jy?= =?utf-8?B?aC9kREhkVmVSdHQ2K1hUSDFxRFlDUC9lSVAvbVJwSmw0MUJWS0JCNnNYa2Va?= =?utf-8?B?N1cxZ3lqNXpBN0hUL01SNXBnWUFtTklxR1BNK2U0K05yaWwrNWNsYW1KME9i?= =?utf-8?B?aWRRTDNIQWNpb25lYnBNMjF4TTJVdzZEMXUyV2ZWRTBhR1h6U3FaSFlITXRE?= =?utf-8?B?T1JJVVlwTzRyL0FiS3J0dGVwM09hUlovejVtRDc4UUQ2TjYraFMxZUlIQTZQ?= =?utf-8?B?aE1SVE0wUUNyampJMXdhd3FTT200VEtzTWp0aEJuTXVSVDRvSkVmY3ZsSWZL?= =?utf-8?B?eldYUXdhM1ZscjRBNW11bm9na25QZERmWndPUVlTMFIxOW53U3ZEbWs0VkYr?= =?utf-8?B?ejVoWHVPL2dZV1Y3VlptVEJtZ1UvQWJ3RVdSOHhITlVCemlDclg1VFg3VURu?= =?utf-8?B?ZFZZZWg3dTI3VC9mcFdTU055TVcwbm44T3VhM2VXZmh4c3NtRVVYbDVjUUU2?= =?utf-8?B?QVJlWERyY2hidHIwYXh5UWtKNE0wZEhTZlFQaXc4eDdrTVRzM0pndG1LazN5?= =?utf-8?B?L2o1eGxEdFZvakdSdDFKM2RPYWR0TFRlT0p6dVYrWldiWm1SeEU4NXJqOG1n?= =?utf-8?B?ZzNiNUtWd0p5akNqN0xqRm9hTGFGbkFjRXI3OVZBVkM4UytRSGdyR3JRWHhx?= =?utf-8?B?QzU5NUdWZUI5b01sQ3NXOTY5SUU0SndVcE9ZdmtaZ2E3Ym8yYlFGSm1vd1h5?= =?utf-8?B?WHVhSFVtVm45QmFsL29aY1huK0g5UWtIRCtZMm5rT3FOc2w3akVENjRMWWV1?= =?utf-8?Q?vEgOKZwoBYzseqMm2Az6EH3h+rpdHQyPDXMTcqhBnLV1S?= X-MS-Exchange-AntiSpam-MessageData-1: xM2IBFIuKQwXqg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3517869c-18c6-4a08-94e5-08df0e26f1a1 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6413.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2026 04:00:53.6493 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: X+mX+zZ/uY3cEm0eggrJBWCBpEE2cY5E1ut2swDWXXPfSkSjKv6TpKqVgQhiAhUq1JZfP4m2aAGYPkCeRCnUqg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8070 From: Joel Fernandes Add self-tests for BAR1 access during driver probe when CONFIG_NOVA_CORE_SELFTESTS is enabled (default disabled). This results in testing the Vmm, GPU buddy allocator and BAR1 region all of which should function correctly for the tests to pass. Signed-off-by: Joel Fernandes [ecourtney: use existing self-test runner and CONFIG_NOVA_CORE_SELFTESTS] [ecourtney: update for mutable GpuMm, PRAMIN views and VramAddress raw API] [ecourtney: restore the conditional dead_code expect] Signed-off-by: Eliot Courtney Acked-by: Danilo Krummrich --- drivers/gpu/nova-core/gpu.rs | 9 +- drivers/gpu/nova-core/mm.rs | 12 +- drivers/gpu/nova-core/mm/bar_user.rs | 251 ++++++++++++++++++++++++++++++= ++++ drivers/gpu/nova-core/mm/pagetable.rs | 24 ++++ drivers/gpu/nova-core/mm/vmm.rs | 1 + 5 files changed, 293 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs index f72a92e045e5..f5bdfdea803a 100644 --- a/drivers/gpu/nova-core/gpu.rs +++ b/drivers/gpu/nova-core/gpu.rs @@ -482,7 +482,14 @@ pub(crate) fn run_selftests(self: Pin<&mut Self>, pdev= : &pci::Device`. /// @@ -202,6 +202,7 @@ pub(crate) struct VirtualAddress(u64) { =20 impl VirtualAddress { /// Create a new virtual address from a raw value. + #[expect(dead_code)] pub(crate) const fn new(addr: u64) -> Self { Self::from_raw(addr) } @@ -297,7 +298,8 @@ pub(crate) mod selftest { =20 use kernel::{ device, - sizes::SizeConstants, // + sizes::SizeConstants, + sync::Arc, // }; =20 use super::*; @@ -307,6 +309,9 @@ pub(crate) fn run( dev: &device::Device, mm: &mut GpuMm<'_>, usable_fb_regions: &[Range], + bar_user: &Arc>, + bar1_pdb: u64, + chipset: Chipset, ) -> Result { // VRAM span the self-tests are free to overwrite, from the chosen= test base. const SELFTEST_SPAN: u64 =3D u64::SZ_64M; @@ -325,6 +330,7 @@ pub(crate) fn run( return Ok(()); }; =20 - pramin::selftest::run(dev, mm.pramin_mut(), VramAddress::from_raw(= base)) + pramin::selftest::run(dev, mm.pramin_mut(), VramAddress::from_raw(= base))?; + bar_user::run_self_test(dev, mm, bar_user, bar1_pdb, chipset) } } diff --git a/drivers/gpu/nova-core/mm/bar_user.rs b/drivers/gpu/nova-core/m= m/bar_user.rs index ef1d8e6f8c9c..1bef01d147ad 100644 --- a/drivers/gpu/nova-core/mm/bar_user.rs +++ b/drivers/gpu/nova-core/mm/bar_user.rs @@ -31,6 +31,9 @@ num::IntoSafeCast, }; =20 +#[cfg(CONFIG_NOVA_CORE_SELFTESTS)] +use kernel::device; + /// BAR1 user interface for virtual memory mappings. /// /// Owns the [`Vmm`] for the BAR1 address space. @@ -84,6 +87,7 @@ pub(crate) struct BarUserAccess<'gpu> { mapped: Option, } =20 +#[expect(dead_code)] impl BarUserAccess<'_> { /// Tear down the BAR1 mapping. pub(crate) fn release(mut self, mm: &mut GpuMm<'_>) -> Result { @@ -169,3 +173,250 @@ fn drop(&mut self) { // identifying the leaked VA range. } } + +/// Run MM subsystem self-tests during probe. +/// +/// Tests page table infrastructure and `BAR1` MMIO access using the `BAR1` +/// address space. Uses the `GpuMm`'s buddy allocator to allocate page tab= les +/// and test pages as needed. +#[cfg(CONFIG_NOVA_CORE_SELFTESTS)] +pub(crate) fn run_self_test( + dev: &device::Device, + mm: &mut GpuMm<'_>, + bar_user: &Arc>, + bar1_pdb: u64, + chipset: Chipset, +) -> Result { + use kernel::gpu::buddy::{ + GpuBuddyAllocFlags, + GpuBuddyAllocMode, // + }; + use kernel::ptr::Alignment; + use kernel::sizes::{ + SZ_16K, + SZ_32K, + SZ_4K, + SZ_64K, // + }; + + // Test patterns. + const PATTERN_PRAMIN: u32 =3D 0xDEAD_BEEF; + const PATTERN_BAR1: u32 =3D 0xCAFE_BABE; + + let bar1 =3D bar_user.bar1; + dev_info!(dev, "MM: Starting self-test...\n"); + + let pdb_addr =3D VramAddress::from_raw(bar1_pdb); + + // Check if initial page tables are in VRAM. + if crate::mm::pagetable::check_pdb_valid(mm.pramin_mut(), pdb_addr, ch= ipset).is_err() { + dev_info!(dev, "MM: Self-test SKIPPED - no valid VRAM page tables\= n"); + return Ok(()); + } + + // Set up a test page from the buddy allocator. + let test_page_blocks =3D KBox::pin_init( + mm.buddy().alloc_blocks( + GpuBuddyAllocMode::Simple, + SZ_4K.into_safe_cast(), + Alignment::new::(), + GpuBuddyAllocFlags::default(), + ), + GFP_KERNEL, + )?; + let test_vram_offset =3D test_page_blocks.iter().next().ok_or(ENOMEM)?= .offset(); + let test_vram =3D VramAddress::from_raw(test_vram_offset); + let test_pfn =3D Pfn::from(test_vram); + + // Create a VMM of size 64K to track virtual memory mappings. + let mut vmm =3D Vmm::new(pdb_addr, chipset.mmu_version(), SZ_64K.into_= safe_cast())?; + + // Create a test mapping. + let mapped =3D vmm.map_pages(mm, &[test_pfn], None, true)?; + let test_vfn =3D mapped.vfn_start; + + // Pre-compute test addresses for the PRAMIN to BAR1 read test. + let vfn_offset: usize =3D test_vfn.raw().into_safe_cast(); + let bar1_base_offset =3D vfn_offset.checked_mul(PAGE_SIZE).ok_or(EOVER= FLOW)?; + let bar1_read_offset: usize =3D bar1_base_offset + 0x100; + let vram_read_addr =3D test_vram + 0x100; + + // Test 1: Write via PRAMIN, read via BAR1. + mm.pramin_mut() + .window_at::(vram_read_addr)? + .view() + .write_val(PATTERN_PRAMIN); + + // Read back via BAR1 aperture. + let bar1_value =3D bar1.try_read32(bar1_read_offset)?; + + let test1_passed =3D if bar1_value =3D=3D PATTERN_PRAMIN { + true + } else { + dev_err!( + dev, + "MM: Test 1 FAILED - Expected {:#010x}, got {:#010x}\n", + PATTERN_PRAMIN, + bar1_value + ); + false + }; + + // Cleanup - invalidate PTE. + vmm.unmap_pages(mm, mapped)?; + + // Test 2: Two-phase prepare/execute API. + let prepared =3D vmm.prepare_map(mm, 1, None)?; + let mapped2 =3D vmm.execute_map(mm, prepared, &[test_pfn], true)?; + let readback =3D vmm.read_mapping(mm, mapped2.vfn_start)?; + let test2_passed =3D if readback =3D=3D Some(test_pfn) { + true + } else { + dev_err!(dev, "MM: Test 2 FAILED - Two-phase map readback mismatch= \n"); + false + }; + vmm.unmap_pages(mm, mapped2)?; + + // Test 3: Range-constrained allocation with a hole =E2=80=94 exercise= s block.size()-driven + // BAR1 mapping. A 4K hole is punched at base+16K, then a single 32K a= llocation + // is requested within [base, base+36K). The buddy allocator must spli= t around the + // hole, returning multiple blocks (expected: {16K, 4K, 8K, 4K} =3D 32= K total). + // Each block is mapped into BAR1 and verified via PRAMIN read-back. + // + // Address layout (base =3D 0x10000): + // [ 16K ] [HOLE 4K] [4K] [ 8K ] [4K] + // 0x10000 0x14000 0x15000 0x16000 0x18000 0x19000 + let range_base: u64 =3D SZ_64K.into_safe_cast(); + let sz_4k: u64 =3D SZ_4K.into_safe_cast(); + let sz_16k: u64 =3D SZ_16K.into_safe_cast(); + let sz_32k_4k: u64 =3D (SZ_32K + SZ_4K).into_safe_cast(); + + // Punch a 4K hole at base+16K so the subsequent 32K allocation must s= plit. + let _hole =3D KBox::pin_init( + mm.buddy().alloc_blocks( + GpuBuddyAllocMode::Range(range_base + sz_16k..range_base + sz_= 16k + sz_4k), + SZ_4K.into_safe_cast(), + Alignment::new::(), + GpuBuddyAllocFlags::default(), + ), + GFP_KERNEL, + )?; + + // Allocate 32K within [base, base+36K). The hole forces the allocator= to return + // split blocks whose sizes are determined by buddy alignment. + let blocks =3D KBox::pin_init( + mm.buddy().alloc_blocks( + GpuBuddyAllocMode::Range(range_base..range_base + sz_32k_4k), + SZ_32K.into_safe_cast(), + Alignment::new::(), + GpuBuddyAllocFlags::default(), + ), + GFP_KERNEL, + )?; + + let mut test3_passed =3D true; + let mut total_size =3D 0usize; + + for block in blocks.iter() { + total_size +=3D IntoSafeCast::::into_safe_cast(block.size()= ); + + // Map all pages of this block. + let page_size: u64 =3D PAGE_SIZE.into_safe_cast(); + let num_pages: usize =3D (block.size() / page_size).into_safe_cast= (); + + let mut pfns =3D KVec::new(); + for j in 0..num_pages { + let j_u64: u64 =3D j.into_safe_cast(); + pfns.push( + Pfn::from(VramAddress::from_raw( + block.offset() + j_u64.checked_mul(page_size).ok_or(EO= VERFLOW)?, + )), + GFP_KERNEL, + )?; + } + + let mapped =3D vmm.map_pages(mm, &pfns, None, true)?; + let bar1_base_vfn: usize =3D mapped.vfn_start.raw().into_safe_cast= (); + let bar1_base =3D bar1_base_vfn.checked_mul(PAGE_SIZE).ok_or(EOVER= FLOW)?; + + for j in 0..num_pages { + let page_bar1_off =3D bar1_base + j * PAGE_SIZE; + let j_u64: u64 =3D j.into_safe_cast(); + let page_phys =3D block.offset() + + j_u64 + .checked_mul(PAGE_SIZE.into_safe_cast()) + .ok_or(EOVERFLOW)?; + + bar1.try_write32(PATTERN_BAR1, page_bar1_off)?; + + let pramin_val =3D mm + .pramin_mut() + .window_at::(VramAddress::from_raw(page_phys))? + .view() + .read_val(); + + if pramin_val !=3D PATTERN_BAR1 { + dev_err!( + dev, + "MM: Test 3 FAILED block offset {:#x} page {} (val=3D{= :#x})\n", + block.offset(), + j, + pramin_val + ); + test3_passed =3D false; + } + } + + vmm.unmap_pages(mm, mapped)?; + } + + // Verify aggregate: all returned block sizes must sum to allocation s= ize. + if total_size !=3D SZ_32K { + dev_err!( + dev, + "MM: Test 3 FAILED - total size {} !=3D expected {}\n", + total_size, + SZ_32K + ); + test3_passed =3D false; + } + + // Release Tests 1-3's Vmm before Test 4 constructs a fresh BarUser on + // the same PDB. + drop(vmm); + + // Test 4: Exercise `BarUser::map()` end-to-end. + let bar_user =3D Arc::pin_init( + BarUser::new(pdb_addr, chipset, SZ_64K.into_safe_cast(), bar1)?, + GFP_KERNEL, + )?; + let access =3D bar_user.map(mm, &[test_pfn], true)?; + + // Write pattern via PRAMIN, read via BarUserAccess. + mm.pramin_mut() + .window_at::(test_vram)? + .view() + .write_val(PATTERN_BAR1); + + let readback =3D access.try_read32(0)?; + let test4_passed =3D if readback =3D=3D PATTERN_BAR1 { + true + } else { + dev_err!( + dev, + "MM: Test 4 FAILED - Expected {:#010x}, got {:#010x}\n", + PATTERN_BAR1, + readback + ); + false + }; + access.release(mm)?; + + if test1_passed && test2_passed && test3_passed && test4_passed { + dev_info!(dev, "MM: All self-tests PASSED\n"); + Ok(()) + } else { + dev_err!(dev, "MM: Self-tests FAILED\n"); + Err(EIO) + } +} diff --git a/drivers/gpu/nova-core/mm/pagetable.rs b/drivers/gpu/nova-core/= mm/pagetable.rs index ffc69fbdb067..de2f8ea5aeef 100644 --- a/drivers/gpu/nova-core/mm/pagetable.rs +++ b/drivers/gpu/nova-core/mm/pagetable.rs @@ -396,3 +396,27 @@ fn from(val: AperturePde) -> Self { Bounded::from_expr(val as u64 & 0x3) } } + +/// Check if the PDB has valid, VRAM-backed page tables. +#[cfg(CONFIG_NOVA_CORE_SELFTESTS)] +fn check_pdb_inner(pramin: &mut pramin::Pramin<'_>, pdb_addr= : VramAddress) -> Result { + let raw =3D pramin.window_at::(pdb_addr)?.view().read_val(); + + if !M::Pde::from_raw(raw).is_valid_vram() { + return Err(ENOENT); + } + Ok(()) +} + +/// Check if the PDB has valid, VRAM-backed page tables, dispatching by MM= U version. +#[cfg(CONFIG_NOVA_CORE_SELFTESTS)] +pub(super) fn check_pdb_valid( + pramin: &mut pramin::Pramin<'_>, + pdb_addr: VramAddress, + chipset: crate::gpu::Chipset, +) -> Result { + match MmuVersion::from(chipset.arch()) { + MmuVersion::V2 =3D> check_pdb_inner::(pramin, pdb_addr), + MmuVersion::V3 =3D> check_pdb_inner::(pramin, pdb_addr), + } +} diff --git a/drivers/gpu/nova-core/mm/vmm.rs b/drivers/gpu/nova-core/mm/vmm= .rs index 411710d03f7a..51b500a27233 100644 --- a/drivers/gpu/nova-core/mm/vmm.rs +++ b/drivers/gpu/nova-core/mm/vmm.rs @@ -126,6 +126,7 @@ fn drop(&mut self) { /// Directory Base (`PDB`) address. Used for Channel, BAR1 and BAR2 mappin= gs. pub(crate) struct Vmm { /// Page Directory Base address for this address space. + #[expect(dead_code)] pdb_addr: VramAddress, /// Page table walker for reading existing mappings. pt_walk: PtWalk, --=20 2.55.0