From nobody Thu Nov 28 11:39:02 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=RiversideResearch.org); dmarc=pass(p=none dis=none) header.from=RiversideResearch.org ARC-Seal: i=2; a=rsa-sha256; t=1670961361; cv=pass; d=zohomail.com; s=zohoarc; b=RBLGSJJgb9L5MrRS4Lha4qFdDCpSNHVHUp8oPsUudEND+HpmhDLWIea/1FMmt1m0PTuGqCrHO+hTe0uFfi5ryow14UCnnxTIdetLC9TpiQnTrWaJHKgWjOitcw8Dy6eHRO1WtWdzoJt/cRiuEbkoAbj5RAVS8xZQvYBebfW3ZP4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1670961361; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8IMI+mmkmPVmMi766DubIF+lD8dUSlj9ua78sytnAyM=; b=WuE0hdpNCXmgQKLleCi0/gpK6pYV1170UteZuDM0efPxJ80e40ginoBQlwWZs7QIe49O+X69lxU3/wb51jS5Wch2m9jdx4JxyhV+vEBLmS4RiQdi7KFsN+dw3rczfBHx3UQjlExFijuZf9TcopW8RN/oqtb7vywSbgJP7YwHRxk= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=RiversideResearch.org); dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1670961360975870.73950104223; Tue, 13 Dec 2022 11:56:00 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.461144.719187 (Exim 4.92) (envelope-from ) id 1p5BNM-0004DQ-2j; Tue, 13 Dec 2022 19:55:40 +0000 Received: by outflank-mailman (output) from mailman id 461144.719187; Tue, 13 Dec 2022 19:55:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p5BNL-0004DJ-Vl; Tue, 13 Dec 2022 19:55:39 +0000 Received: by outflank-mailman (input) for mailman id 461144; Tue, 13 Dec 2022 19:55:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p5BNL-0004DD-81 for xen-devel@lists.xenproject.org; Tue, 13 Dec 2022 19:55:39 +0000 Received: from USG02-BN3-obe.outbound.protection.office365.us (mail-bn3usg02on0102.outbound.protection.office365.us [23.103.208.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1c7be157-7b20-11ed-8fd2-01056ac49cbb; Tue, 13 Dec 2022 20:55:37 +0100 (CET) Received: from BN0P110MB1642.NAMP110.PROD.OUTLOOK.COM (2001:489a:200:185::22) by BN0P110MB1723.NAMP110.PROD.OUTLOOK.COM (2001:489a:200:16f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5880.19; Tue, 13 Dec 2022 19:55:34 +0000 Received: from BN0P110MB1642.NAMP110.PROD.OUTLOOK.COM ([fe80::81df:6431:7a2d:4610]) by BN0P110MB1642.NAMP110.PROD.OUTLOOK.COM ([fe80::81df:6431:7a2d:4610%5]) with mapi id 15.20.5880.019; Tue, 13 Dec 2022 19:55:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1c7be157-7b20-11ed-8fd2-01056ac49cbb ARC-Seal: i=1; a=rsa-sha256; s=arcselector5401; d=microsoft.com; cv=none; b=imL7yCjurKa27rC7dgfE9HCqhq2TiwWcoSEFvXDBAG5kMKLiPyAVNqOfs7lECKpHAkrzyNSBJsfXX/2PMt7m0tv2gQdZVwbYC3cMFzX9bJSj1XV5ChmXeqpLzrwQItDfyRAKFdQrfqvpipV74TwigptuzKK0Xn2huMKgbmXYA4egtvLudr2UTenrIq9ilQMqj90wjvnSZ4r1y4sgSGOi1lH9zd1x+P+E9VD9MpNr61EvkxFMwcTCGEo7TxMpnYxrloRwjs6ESpI8lMDUR8WEE1jBRmOr6aIug3eAcWzyPYlIBAxI5+HE+wIM8o9OVW9SeDuhnAKY48OSm7VvPhORzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector5401; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8IMI+mmkmPVmMi766DubIF+lD8dUSlj9ua78sytnAyM=; b=wC6IgpssASedNx2cvzJHW+WDfM/Tka3DNL46sE0PVLvl7FzPrtK78gMTQjd6CBz8pLueKdX3XoCaGKtWPx1Dm3j3Rc5RS4WSGrf5Jy0kCYCc/uHNl7VWvYcku+p6XuLqhq1x05ZOWXlDx0WTs2xZ6Bu8y5Fl5lCRO3Qa6v0s0UFxqqnyVBgwnOGBNgojcRslKIna7QZIxDqSPAHZrBeTOGYPvTf0xx43gJAGHGrmv7SMGz0WxuBENAVZQtX2wDshM7p76cWT19S+P5RqFxcg27qKNOqO60HfR3N0hpGbd81UzlJnkBBteBdbWsgCr+Hgv4BtLYuOEhxj4o8YbS567A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=riversideresearch.org; dmarc=pass action=none header.from=riversideresearch.org; dkim=pass header.d=riversideresearch.org; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=riversideresearch.org; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8IMI+mmkmPVmMi766DubIF+lD8dUSlj9ua78sytnAyM=; b=ZoCWXtXVTv1GMonMKnUjoZWIlVDNS3C/tcHIdJ8GMB+eChENikRlbn1cypGgT3dP6KWEb90yaOLN68qgtCpjt2t7jWzubcWgUAvF4epzuHcHOyfgI15QG+6re12NZDTdT+JiSTv1L8mQFl0me/vB6Z+2In+95+cmO+QchriWbptYrSi+XGxd+nC141IFQx1/fD+JLf06kE7lxd/3oncbnESZpZsg/cWYYK/n2jkw4NDZt8LiJJa5r5dAB7I+BrwVBaelwB45bOgS9rY1r/mmghqEOm3UmlDUgnMxyg/Phs0m2E0oQHrmdC1j4sp2g5Uc+PFO+18d8TqOg4QF3Ekfew== From: "Smith, Jackson" To: "Smith, Jackson" CC: "Brookes, Scott" , Xen-devel , Stefano Stabellini , Julien Grall , "bertrand.marquis@arm.com" , "jbeulich@suse.com" , Andrew Cooper , =?iso-8859-1?Q?Roger_Pau_Monn=E9?= , George Dunlap , "demi@invisiblethingslab.com" , "Daniel P. Smith" , "christopher.w.clark@gmail.com" Subject: [RFC 4/4] Implement VMF for arm64 Thread-Topic: [RFC 4/4] Implement VMF for arm64 Thread-Index: AQHZDyzdNLqpW42xnEGVL+R/wWY6pg== Date: Tue, 13 Dec 2022 19:55:34 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: suggested_attachment_session_id: c8fc6a6e-64fc-8b95-9f14-261ab31ebef3 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=RiversideResearch.org; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BN0P110MB1642:EE_|BN0P110MB1723:EE_ x-ms-office365-filtering-correlation-id: d191301d-682b-4a07-86f5-08dadd43ffa8 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4CPAHUiMcyua0BIinpi+oOCBLmY/iVlBxOOEIKZrdIJscUMnXvhgzJXp75JK5prO3KqIHWJQhTGQ6obOaNBIFdLXPdS/YYUJX7tzophdFDkeRp+SIfMKd0QvpQfhOgcaKbA2mde3b7y8MkEMHBmRbzzUlO6HttTlOX9GRbDfHOjrc5FEkPKgwCr2k2yX/bzol4Q0WKwVvXCAEAh0HMHuZACn6AnSdQoCfq3C5i2U58TTQ/CkKhVkz3EGK3s7MW/NgBDDflztsVmS6FEaQvF3o8znCeBQM4haRQA/A6flceQXV/lDAj4ePerP+9qzsjy9uitH7n6535JdaX1IA+0aAyBdCI0VbmhdPsO+1xnjpa5hQBydUmp+UxKxoYNFbT6nRowUtRa7dt2dYw+S68yAn+4rwovl54DdIxLY3lTmS9XpxRo/9aMIHjjNj2vZ7Rz7G9L934BTxRg3Ejnmtvb1BEDdPPQ6oTsiitqi5sxNtSeQH826N/jMwSmjVzohWIX+3NXs7uC+PIhNOQQHB7daP7bNfONLU29Mbv/6BxP1rNvsulAW+X82udHvUffpG/4BozbyFvhNmJwKZjuNo2s3gj/smMS1fdKgMG1c/Md3wr1gEpFl4iQu7W3R0s3TQq8CdRmmiQyv31QBpTrDUadZT61SrJOtCVwrdwTV85a0IrCrXYbLFo6F4Jh5f98fW5eM x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN0P110MB1642.NAMP110.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230022)(366004)(451199015)(7696005)(2906002)(33656002)(66946007)(86362001)(6506007)(186003)(508600001)(6200100001)(9686003)(2940100002)(83380400001)(4326008)(66556008)(64756008)(66476007)(76116006)(6862004)(26005)(52536014)(5660300002)(122000001)(7416002)(30864003)(82960400001)(8936002)(8676002)(54906003)(71200400001)(38070700005)(38100700002)(66446008)(55016003)(2004002)(21314003)(85282002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: 9oz6PTr2lz0cxAWTwBFXqzjvmZj1AYqsuiYvpd7O2r3l8Mkg0wwu+CsT8lB0FSLxkLtuEeEizXSfvB+SXEyB7gOZEIOU/6Gw3v6hD+3RA5JOoPQaonVPPuqG+aurEbhyCvrZfuF6ij/Q1wQEQz9ATESYiT6QYLUZsD2gc2hq0DgUYhH/Tw0mhyIGPLMzqV0oKC0W+iSzD3Flevg/jk2QwQ57CCHeLZvFROBTM5Gzes19SP+6PvuErWTZsbqqVyGn9DAVgZ1Ga/lBjsPRo6IK2SVyLykYLkQE/7y5KudKfxQztxuOSYwdGhQOIzI20EGygGTMriwfGMbrhxRTziLESGhRjTBY2Q1uBNftvU1I14w/d6KUjX6bW3kiF2AvoNXl1+E4siv14xdZjhRU8GZBInWfmojDMJqgKHY45wpRFcw= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: riversideresearch.org X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN0P110MB1642.NAMP110.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: d191301d-682b-4a07-86f5-08dadd43ffa8 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Dec 2022 19:55:34.7816 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: bfc64a8d-9064-4c64-91c3-9d10b44c1cb6 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN0P110MB1723 X-ZohoMail-DKIM: pass (identity @riversideresearch.org) X-ZM-MESSAGEID: 1670961362231100001 Content-Type: text/plain; charset="utf-8" Implements the functions from xen/vmf.h for arm64. Introduces an xen/arch/arm/mm-walk.c helper file for walking an entire page table structure. --- xen/arch/arm/Makefile | 1 + xen/arch/arm/include/asm/mm-walk.h | 53 ++++++++++ xen/arch/arm/include/asm/mm.h | 11 +++ xen/arch/arm/mm-walk.c | 181 +++++++++++++++++++++++++++++++++ xen/arch/arm/mm.c | 198 +++++++++++++++++++++++++++++++++= +++- xen/common/Kconfig | 2 + 6 files changed, 445 insertions(+), 1 deletion(-) create mode 100644 xen/arch/arm/include/asm/mm-walk.h create mode 100644 xen/arch/arm/mm-walk.c diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 4d076b2..e358452 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -37,6 +37,7 @@ obj-y +=3D kernel.init.o obj-$(CONFIG_LIVEPATCH) +=3D livepatch.o obj-y +=3D mem_access.o obj-y +=3D mm.o +obj-y +=3D mm-walk.o obj-y +=3D monitor.o obj-y +=3D p2m.o obj-y +=3D percpu.o diff --git a/xen/arch/arm/include/asm/mm-walk.h b/xen/arch/arm/include/asm/= mm-walk.h new file mode 100644 index 0000000..770cc89 --- /dev/null +++ b/xen/arch/arm/include/asm/mm-walk.h @@ -0,0 +1,53 @@ +#ifndef __ARM_MM_WALK_H__ +#define __ARM_MM_WALK_H__ + +#include + +#define RECURSIVE_IDX ((unsigned long)(XEN_PT_LPAE_ENTRIES-1)) +#define RECURSIVE_VA (RECURSIVE_IDX << ZEROETH_SHIFT) + +/* + * Remove all mappings in these tables from Xen's address space + * Only makes sense if walking a guest's tables + */ +#define WALK_HIDE_GUEST_MAPPING (1U << 0) +/* + * Remove all mappings to these tables from Xen's address space + * Makes sense if walking a guest's table (hide guest tables from Xen) + * Or if walking Xen's tables (lock Xen's virtual memory configuration) + */ +#define WALK_HIDE_GUEST_TABLE (1U << 1) + +/* + * Before we can hide individual table entires, + * we need to split the directmap superpages + */ +#define WALK_SPLIT_DIRECTMAP_TABLE (1U << 2) +/* + * Like walk table hide, but using recursive mapping + * to bypass walking directmap when table is in the directmap + */ +#define WALK_HIDE_DIRECTMAP_TABLE (1U << 3) + +/* These are useful for development/debug */ +/* Show all pte's for a given address space */ +#define WALK_DUMP_ENTRIES (1U << 4) +/* Show all mappings for a given address space */ +#define WALK_DUMP_MAPPINGS (1U << 5) + +/* + * Given the value of a ttbr register, this function walks every valid ent= ry in the trie + * (As opposed to dump_pt_walk, which follows a single address from root t= o leaf) + */ +void do_walk_tables(paddr_t ttbr, int root_level, int nr_root_tables, int = flags); + +#endif /* __ARM_MM_WALK_H__ */ +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 68adcac..2e85885 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -209,6 +209,17 @@ extern void mmu_init_secondary_cpu(void); * For Arm64, map the region in the directmap area. */ extern void setup_directmap_mappings(unsigned long base_mfn, unsigned long= nr_mfns); +/* Shatter superpages for these mfns if needed */ +extern int split_directmap_mapping(unsigned long mfn, unsigned long nr_mfn= s); +/* Remove these mfns from the directmap */ +extern int destroy_directmap_mapping(unsigned long mfn, unsigned long nr_m= fns); +/* + * Remove this mfn from the directmap (bypassing normal update code) + * This is a workaround for current pgtable update code, which cannot be u= sed + * to remove directmap table entries from the directmap (because they are + * needed to walk the directmap) + */ +extern void destroy_directmap_table(unsigned long mfn); /* Map a frame table to cover physical addresses ps through pe */ extern void setup_frametable_mappings(paddr_t ps, paddr_t pe); /* map a physical range in virtual memory */ diff --git a/xen/arch/arm/mm-walk.c b/xen/arch/arm/mm-walk.c new file mode 100644 index 0000000..48f9b2d --- /dev/null +++ b/xen/arch/arm/mm-walk.c @@ -0,0 +1,181 @@ +/* + * xen/arch/arm/mm-walk.c + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include + +#include +#include + +typedef struct { + /* Keeps track of all the table offsets so we can reconstruct the VA i= f we need to */ + int off[4]; + + /* Keeps track of root level so we can make sense of the table offsets= */ + int root_level; + int root_table_idx; /* only meaningful when nr_root_tables > 1 */ +} walk_info_t; + +/* + * Turn a walk_info_t into a virtual address + * + * XXX: This only applies to the lower VA range + * Ie. if you are looking at a table in ttbr1, this is different + * XXX: doesn't work for concat tables right now either + */ +static unsigned long walk_to_va(int level, walk_info_t *walk) +{ +/* #define off_valid(x) (((x) <=3D level) && ((x) >=3D walk->root_level)) = */ +#define off_valid(x) ((x) <=3D level) +#define off_val(x) ((u64)(off_valid(x) ? walk->off[x] : 0)) + + return (off_val(0) << ZEROETH_SHIFT) \ + | (off_val(1) << FIRST_SHIFT) \ + | (off_val(2) << SECOND_SHIFT) \ + | (off_val(3) << THIRD_SHIFT); +} + +/* Prints each entry in the form "\t @XTH TABLE:0.0.0.0 =3D 0xENTRY" */ +static void dump_entry(int level, lpae_t pte, walk_info_t *walk) +{ + int i; + static const char *level_strs[4] =3D { "0TH", "1ST", "2ND", "3RD" }; + ASSERT(level <=3D 3); + + for (i =3D 0; i < level; i++) + printk(" "); + + printk("@%s %i:", level_strs[level], walk->root_table_idx); + + for (i =3D walk->root_level; i < level; i++) + printk("%d.", walk->off[i]); + + printk("%d =3D 0x%lx\n", walk->off[level], pte.bits); +} + +/* Prints each mapping in the form IA:0xIA -> OFN:0xOFN XG,M,K */ +static void dump_mapping(int level, lpae_t pte, walk_info_t *walk) +{ + unsigned long va; + unsigned long ofn =3D pte.walk.base; + const char *size[4] =3D {"??", "1G", "2M", "4K"}; + + ASSERT(level >=3D 1); + ASSERT(level <=3D 3); + + va =3D walk_to_va(level, walk); + + /* ofn stands for output frame number.. I just made it up. */ + printk("0x%lx -> 0x%lx %s\n", va, ofn, size[level]); +} + +/* Recursive walk function */ +static void walk_table(mfn_t mfn, int level, walk_info_t *walk, int flags) +{ + lpae_t *table; + + #define i (walk->off[level]) + + BUG_ON(level > 3); + + table =3D map_domain_page(mfn); + for ( i =3D 0; i < XEN_PT_LPAE_ENTRIES; i++ ) + { + lpae_t pte =3D table[i]; + if ( !lpae_is_valid(pte) ) + continue; + + /* Skip recursive mapping */ + if ( level =3D=3D 0 && i =3D=3D RECURSIVE_IDX ) + continue; + + if ( flags & WALK_DUMP_ENTRIES ) + dump_entry(level, pte, walk); + + if ( lpae_is_mapping(pte, level) ) + { + /* Do mapping related things */ + if ( flags & WALK_DUMP_MAPPINGS ) + dump_mapping(level, pte, walk); + if ( flags & WALK_HIDE_GUEST_MAPPING ) + /* Destroy all of Xen's mappings to the physical frames co= vered by this entry */ + destroy_directmap_mapping(pte.walk.base, 1 << XEN_PT_LEVEL= _ORDER(level)); + } + else if ( lpae_is_table(pte, level) ) + { + /* else, pte is a table: recurse! */ + walk_table(lpae_get_mfn(pte), level + 1, walk, flags); + + /* Note that the entry is a normal entry in xen's page tables = */ + if ( flags & WALK_HIDE_GUEST_TABLE ) + /* + * This call will look up the table pointed to by this ent= ry in the directmap + * and remove it in the typical way + * This leaves the table intact, but removes the directmap= mapping to it, hiding it from xen + */ + destroy_directmap_mapping(pte.walk.base, 1); + if ( flags & WALK_SPLIT_DIRECTMAP_TABLE ) + /* + * This call will look up the table pointed to by this ent= ry in the directmap + * and make sure that it has it's own l3 entry, splitting = superpages if needed + */ + split_directmap_mapping(pte.walk.base, 1); + if ( flags & WALK_HIDE_DIRECTMAP_TABLE ) + /* + * This call will look up the table pointed to by this ent= ry in the directmap + * and (now that it has it's own l3 entry) overwrite that = entry with 0's + * This leaves the table intact, but removes the directmap= mapping to it, hiding it from xen + */ + destroy_directmap_table(pte.walk.base); + } + /* else, invalid pte, level =3D=3D 3, vaild =3D=3D true, table =3D= false */ + } + unmap_domain_page(table); + + #undef i +} + +void do_walk_tables(paddr_t ttbr, int root_level, int nr_root_tables, int = flags) +{ + int i; + mfn_t root =3D maddr_to_mfn(ttbr & PADDR_MASK); + walk_info_t walk =3D { + .off =3D {0}, + .root_level =3D root_level, + }; + + BUG_ON( !mfn_x(root) || !mfn_valid(root) ); + + for ( i =3D 0; i < nr_root_tables; i++, root =3D mfn_add(root, 1) ) { + walk.root_table_idx =3D i; + walk_table(root, root_level, &walk, flags); + + /* Our walk doesn't consider the root table, so do that here */ + if ( flags & WALK_SPLIT_DIRECTMAP_TABLE ) + split_directmap_mapping(mfn_x(root), 1); + if ( flags & WALK_HIDE_GUEST_TABLE ) + destroy_directmap_mapping(mfn_x(root), 1); + if ( flags & WALK_HIDE_DIRECTMAP_TABLE ) + destroy_directmap_table(mfn_x(root)); + } +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 91b9c2b..64e9efd 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -21,11 +21,13 @@ #include #include #include +#include =20 #include =20 #include #include +#include =20 #include =20 @@ -1164,7 +1166,8 @@ static int xen_pt_update(unsigned long virt, * * XXX: Add a check. */ - const mfn_t root =3D virt_to_mfn(THIS_CPU_PGTABLE); + /* TODO: does this change have a negative performance impact? */ + const mfn_t root =3D maddr_to_mfn(READ_SYSREG64(TTBR0_EL2)); =20 /* * The hardware was configured to forbid mapping both writeable and @@ -1273,6 +1276,199 @@ int modify_xen_mappings(unsigned long s, unsigned l= ong e, unsigned int flags) return xen_pt_update(s, INVALID_MFN, (e - s) >> PAGE_SHIFT, flags); } =20 +static void insert_recursive_mapping(void) +{ + uint64_t ttbr =3D READ_SYSREG64(TTBR0_EL2); + const mfn_t root_mfn =3D maddr_to_mfn(ttbr & PADDR_MASK); + lpae_t *pgtable =3D map_domain_page(root_mfn); + + lpae_t pte =3D mfn_to_xen_entry(root_mfn, MT_NORMAL); + pte.pt.table =3D 1; + + spin_lock(&xen_pt_lock); + + write_pte(&pgtable[RECURSIVE_IDX], pte); + clean_dcache(pgtable[RECURSIVE_IDX]); + + unmap_domain_page(pgtable); + spin_unlock(&xen_pt_lock); +} + +/* + * Converts va to a table pointer through the recursive mapping + * Only valid for the current address space obviously + */ +static lpae_t *va_to_table(int level, unsigned long va) +{ + /* Shift everything by 9 for each walk we skip */ + /* Last off shifted out becomes becomes offset into page */ + for ( ;level <=3D 3; level++ ) { + va >>=3D XEN_PT_LPAE_SHIFT; + va |=3D RECURSIVE_VA; + } + + /* Mask out any offset, in case caller is asking about a misalligned v= a */ + va &=3D ~0x7; + return (lpae_t *)va; +} + +/* + * Zero out the table at level when walking to virt + * Do this through the recursive mapping, in case we have already + * removed part of the directmap and can't walk to that entry + */ +static void clear_pte_directly(int level, void *virt) +{ + unsigned long va =3D (unsigned long)virt; + lpae_t empty =3D {.pt =3D {0x0}}; + lpae_t *table; + + spin_lock(&xen_pt_lock); + + /* We're assuming we can safely remove an entry at `level` */ + /* This depends on va not living in a superpage */ + BUG_ON(level > 1 && !va_to_table(1, va)->pt.table); + BUG_ON(level > 2 && !va_to_table(2, va)->pt.table); + + table =3D va_to_table(level, va); + write_pte(table, empty); + clean_dcache(*table); + flush_xen_tlb_range_va((vaddr_t)table, sizeof(*table)); + + spin_unlock(&xen_pt_lock); +} + +static void remove_recursive_mapping(void) +{ + clear_pte_directly(0, (void *)RECURSIVE_VA); +} + +static int modify_virt_mapping(void *virt, int nr_pages, int flags) +{ + unsigned long va =3D (unsigned long)virt; + return modify_xen_mappings(va, va + (PAGE_SIZE * nr_pages), flags); +} + +static int destroy_virt_mapping(void *virt, int nr_pages) +{ + return modify_virt_mapping(virt, nr_pages, 0); +} + +static int modify_directmap_mapping(unsigned long mfn, unsigned long nr_mf= ns, int flags) +{ + if ( mfn & pfn_hole_mask ) + { + printk("** Skipping mfn 0x%lx because it lives in the pfn hole **\= n", mfn); + return 0; + } + + return modify_virt_mapping(__mfn_to_virt(mfn), nr_mfns, flags); +} + +int split_directmap_mapping(unsigned long mfn, unsigned long nr_mfns) +{ + return modify_directmap_mapping(mfn, nr_mfns, PAGE_HYPERVISOR); +} + +int destroy_directmap_mapping(unsigned long mfn, unsigned long nr_mfns) +{ + return modify_directmap_mapping(mfn, nr_mfns, 0); +} + +void destroy_directmap_table(unsigned long mfn) +{ + BUG_ON(mfn & pfn_hole_mask); + clear_pte_directly(3, __mfn_to_virt(mfn)); +} + +static void unmap_xen_root_tables(void) +{ + destroy_virt_mapping(xen_xenmap, 1); + destroy_virt_mapping(xen_fixmap, 1); + destroy_virt_mapping(xen_second, 1); +#if defined(CONFIG_ARM_64) + destroy_virt_mapping(xen_first, 1); + destroy_virt_mapping(xen_pgtable, 1); +#endif +} + +static void walk_hyp_tables(int flags) +{ + uint64_t httbr =3D READ_SYSREG64(TTBR0_EL2); + do_walk_tables(httbr, HYP_PT_ROOT_LEVEL, 1, flags); +} + +static void walk_guest_tables(struct domain *d, int flags) +{ + uint64_t vttbr =3D d->arch.p2m.vttbr; + do_walk_tables(vttbr, P2M_ROOT_LEVEL, 1<arch.p2m.vttbr; + uint64_t httbr =3D READ_SYSREG64(TTBR0_EL2); + + printk("Dump domain info...\n"); + printk("guest mfn =3D 0x%lx\n", paddr_to_pfn(vttbr & PADDR_MASK)); + printk("xen mfn =3D 0x%lx\n", paddr_to_pfn(httbr & PADDR_MASK)); +} + +void vmf_dump_xen_tables() +{ + walk_hyp_tables(WALK_DUMP_MAPPINGS | WALK_DUMP_ENTRIES); +} + +void vmf_dump_domain_tables(struct domain *d) +{ + walk_guest_tables(d, WALK_DUMP_MAPPINGS | WALK_DUMP_ENTRIES); +} + /* Release all __init and __initdata ranges to be reused */ void free_init_memory(void) { diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 3bf92b8..c087371 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -94,6 +94,8 @@ config STATIC_MEMORY =20 config VMF bool "Virtual Memory Fuse Support" + depends on ARM_64 + default y =20 menu "Speculative hardening" =20 --=20 2.7.4