From nobody Sat Oct 4 08:13:30 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EF653469E3 for ; Tue, 19 Aug 2025 21:52:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755640351; cv=none; b=AqhoG7EKjNnam9COI52mwHqy1RCXJZKfMbtd3KMGfuN1sIYT2kLGFrEvo1+4xSFb+ak89K2voJ6KA66SoqgEhcZCCKwP6EaGVzy0fDCmZwtQuR6QbfbKs8mJhp/b+DzbGlBXO0qk1BzccLR0RiYkxIcb4PnVl0zKzD7La7kLBkg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755640351; c=relaxed/simple; bh=hXDOvZIpu6yyJHFw/mqRtW1+koiyAV4owG5DPpyR3nE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ljOyWGfTnGi+ACN0w8xM2tS6qHuWaNhzzymD62lZOWOdru5o+tRTnOW9AgS2mNfiE6qiUgM9ojllwOWBTvuW1JFe+S3gnI5KgBgM4/16OgxTpEsgLteqORTJhy+6FJ5uRgEyxXPGg39+9iKuDkbMkmMZyRciBtlXY9xXwwA/+Xg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WhZehyrU; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WhZehyrU" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-45a1b0045a0so31009725e9.0 for ; Tue, 19 Aug 2025 14:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1755640347; x=1756245147; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UN5Lx0TpQ9JBjLBscgxu6am7pN7RbulKEGMCKcFuzuI=; b=WhZehyrUxCJYJUKE0j13tgPTOGQiDzv/p217ykpQOkF/J5umJyvtVg49Hukrw/B8B6 +rerd4QDR/6Ld53/dJBgaoVnxVnse5rqRXkiUrZbjvsA3ITqGhaInFAxNNEgL1Ian15G 6XQJFEs8mmsIqsEEFgkOS/wJY4hEz8/32C2EnjktRbxHDryk04i3nISpaLWOend7tWTS ZLE3Pmf2wZWxwQjmeNuHOZvA30T3Fx6DFo557j3RhaY0QpnoJ5jDCzU0j5RGQb21B1oF lhlNZVFhZK/+vGRyUQohLZSN16XFo0f73ILTzsVO+EZYzKGYBQsFgXvrHH8l+F6tiTgA IQ5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755640347; x=1756245147; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UN5Lx0TpQ9JBjLBscgxu6am7pN7RbulKEGMCKcFuzuI=; b=A70CKMG0nBMKT+vUN2U0bY8XKGgAQmH2UvEfCX03J+L+kUKn66xkMNobycILkwtSF9 MCxNBwoov9gExcDs4OnYf6ODeK6P/V6+WajoaE2wOE/pXd3VUe/5mLBP2rTeu17JGte2 sXxDaAMnyRRkikUzGq3j1JxeQkF1j+rQyJUMOkHO1fEIiLvjOM6Qx2Eg62wmVuiOOoxm HzjJz7slIbTSKCs7HgNdhXUI7NfXDty23Zi47aTq9Gynj4gCGUfCGkVjhq5AOjnyp57x WguQyIKDKS5wCWeA7E+uB+e/xCC88lyFArdCFzp/e5qzXckbg/FUHTozi3e5mfu+MLEh aEMQ== X-Gm-Message-State: AOJu0Yzw+++tLYcuZYvh4TAkb67Gg/lECn9Q83SWinN0JZLlNp7Niq8g O58wTDAD4SIam3tSgvslN3nbbuRC61uOXpg7M1qYF/sMjBceXObRkNe4TiQfMNHQc3f9pW4NYEV GksM9lNNv8EGfeegFsy4P0aeuTuaeizHQynOWPvJTJ3q9cJmBgljdnI2jzITcYKj4/zRX7rZvaO jQ7NxF0xUnY7qsxnuwsKxRojYJGZ7+pePFRDQH2Q4wDEP44AZJDBqynX0= X-Google-Smtp-Source: AGHT+IFfGGhQp/LMSHGNHUFDTsK3KwMtLRsoFmkddiawwjZDD2qQVNA+kxHOOymS+x1LtUNJ/bavT/Nny02iqw== X-Received: from wmbeb11.prod.google.com ([2002:a05:600c:678b:b0:459:d5f6:df30]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:47c9:b0:458:bfe1:4a81 with SMTP id 5b1f17b1804b1-45b47a2042dmr2041815e9.17.1755640346929; Tue, 19 Aug 2025 14:52:26 -0700 (PDT) Date: Tue, 19 Aug 2025 21:51:32 +0000 In-Reply-To: <20250819215156.2494305-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250819215156.2494305-1-smostafa@google.com> X-Mailer: git-send-email 2.51.0.rc1.167.g924127e9c0-goog Message-ID: <20250819215156.2494305-5-smostafa@google.com> Subject: [PATCH v4 04/28] iommu/io-pgtable-arm: Move selftests to a separate file From: Mostafa Saleh To: linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev Cc: maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, robin.murphy@arm.com, jean-philippe@linaro.org, qperret@google.com, tabba@google.com, jgg@ziepe.ca, mark.rutland@arm.com, praan@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Soon, io-pgtable-arm.c will be compiled as part of the KVM/arm64 in the hypervisor object, which doesn't have many of the kernel APIs, as faux devices, printk... We would need to factor this things outside of this file, this patch moves the selftests outside, which remove many of the kernel dependencies, which also is not needed by the hypervisor. Create io-pgtable-arm-kernel.c for that, and in the next patch the rest of the code is factored out. Signed-off-by: Mostafa Saleh Reviewed-by: Pranjal Shrivastava --- drivers/iommu/Makefile | 2 +- drivers/iommu/io-pgtable-arm-kernel.c | 216 +++++++++++++++++++++++ drivers/iommu/io-pgtable-arm.c | 245 -------------------------- drivers/iommu/io-pgtable-arm.h | 41 +++++ 4 files changed, 258 insertions(+), 246 deletions(-) create mode 100644 drivers/iommu/io-pgtable-arm-kernel.c diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 355294fa9033..d601b0e25ef5 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -11,7 +11,7 @@ obj-$(CONFIG_IOMMU_DEBUGFS) +=3D iommu-debugfs.o obj-$(CONFIG_IOMMU_DMA) +=3D dma-iommu.o obj-$(CONFIG_IOMMU_IO_PGTABLE) +=3D io-pgtable.o obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) +=3D io-pgtable-arm-v7s.o -obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) +=3D io-pgtable-arm.o +obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) +=3D io-pgtable-arm.o io-pgtable-arm-k= ernel.o obj-$(CONFIG_IOMMU_IO_PGTABLE_DART) +=3D io-pgtable-dart.o obj-$(CONFIG_IOMMU_IOVA) +=3D iova.o obj-$(CONFIG_OF_IOMMU) +=3D of_iommu.o diff --git a/drivers/iommu/io-pgtable-arm-kernel.c b/drivers/iommu/io-pgtab= le-arm-kernel.c new file mode 100644 index 000000000000..f3b869310964 --- /dev/null +++ b/drivers/iommu/io-pgtable-arm-kernel.c @@ -0,0 +1,216 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * CPU-agnostic ARM page table allocator. + * + * Copyright (C) 2014 ARM Limited + * + * Author: Will Deacon + */ +#define pr_fmt(fmt) "arm-lpae io-pgtable: " fmt + +#include +#include +#include + +#include "io-pgtable-arm.h" + +#ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST + +static struct io_pgtable_cfg *cfg_cookie __initdata; + +static void __init dummy_tlb_flush_all(void *cookie) +{ + WARN_ON(cookie !=3D cfg_cookie); +} + +static void __init dummy_tlb_flush(unsigned long iova, size_t size, + size_t granule, void *cookie) +{ + WARN_ON(cookie !=3D cfg_cookie); + WARN_ON(!(size & cfg_cookie->pgsize_bitmap)); +} + +static void __init dummy_tlb_add_page(struct iommu_iotlb_gather *gather, + unsigned long iova, size_t granule, + void *cookie) +{ + dummy_tlb_flush(iova, granule, granule, cookie); +} + +static const struct iommu_flush_ops dummy_tlb_ops __initconst =3D { + .tlb_flush_all =3D dummy_tlb_flush_all, + .tlb_flush_walk =3D dummy_tlb_flush, + .tlb_add_page =3D dummy_tlb_add_page, +}; + +static void __init arm_lpae_dump_ops(struct io_pgtable_ops *ops) +{ + struct arm_lpae_io_pgtable *data =3D io_pgtable_ops_to_data(ops); + struct io_pgtable_cfg *cfg =3D &data->iop.cfg; + + pr_err("cfg: pgsize_bitmap 0x%lx, ias %u-bit\n", + cfg->pgsize_bitmap, cfg->ias); + pr_err("data: %d levels, 0x%zx pgd_size, %u pg_shift, %u bits_per_level, = pgd @ %p\n", + ARM_LPAE_MAX_LEVELS - data->start_level, ARM_LPAE_PGD_SIZE(data), + ilog2(ARM_LPAE_GRANULE(data)), data->bits_per_level, data->pgd); +} + +#define __FAIL(ops, i) ({ \ + WARN(1, "selftest: test failed for fmt idx %d\n", (i)); \ + arm_lpae_dump_ops(ops); \ + -EFAULT; \ +}) + +static int __init arm_lpae_run_tests(struct io_pgtable_cfg *cfg) +{ + static const enum io_pgtable_fmt fmts[] __initconst =3D { + ARM_64_LPAE_S1, + ARM_64_LPAE_S2, + }; + + int i, j; + unsigned long iova; + size_t size, mapped; + struct io_pgtable_ops *ops; + + for (i =3D 0; i < ARRAY_SIZE(fmts); ++i) { + cfg_cookie =3D cfg; + ops =3D alloc_io_pgtable_ops(fmts[i], cfg, cfg); + if (!ops) { + pr_err("selftest: failed to allocate io pgtable ops\n"); + return -ENOMEM; + } + + /* + * Initial sanity checks. + * Empty page tables shouldn't provide any translations. + */ + if (ops->iova_to_phys(ops, 42)) + return __FAIL(ops, i); + + if (ops->iova_to_phys(ops, SZ_1G + 42)) + return __FAIL(ops, i); + + if (ops->iova_to_phys(ops, SZ_2G + 42)) + return __FAIL(ops, i); + + /* + * Distinct mappings of different granule sizes. + */ + iova =3D 0; + for_each_set_bit(j, &cfg->pgsize_bitmap, BITS_PER_LONG) { + size =3D 1UL << j; + + if (ops->map_pages(ops, iova, iova, size, 1, + IOMMU_READ | IOMMU_WRITE | + IOMMU_NOEXEC | IOMMU_CACHE, + GFP_KERNEL, &mapped)) + return __FAIL(ops, i); + + /* Overlapping mappings */ + if (!ops->map_pages(ops, iova, iova + size, size, 1, + IOMMU_READ | IOMMU_NOEXEC, + GFP_KERNEL, &mapped)) + return __FAIL(ops, i); + + if (ops->iova_to_phys(ops, iova + 42) !=3D (iova + 42)) + return __FAIL(ops, i); + + iova +=3D SZ_1G; + } + + /* Full unmap */ + iova =3D 0; + for_each_set_bit(j, &cfg->pgsize_bitmap, BITS_PER_LONG) { + size =3D 1UL << j; + + if (ops->unmap_pages(ops, iova, size, 1, NULL) !=3D size) + return __FAIL(ops, i); + + if (ops->iova_to_phys(ops, iova + 42)) + return __FAIL(ops, i); + + /* Remap full block */ + if (ops->map_pages(ops, iova, iova, size, 1, + IOMMU_WRITE, GFP_KERNEL, &mapped)) + return __FAIL(ops, i); + + if (ops->iova_to_phys(ops, iova + 42) !=3D (iova + 42)) + return __FAIL(ops, i); + + iova +=3D SZ_1G; + } + + /* + * Map/unmap the last largest supported page of the IAS, this can + * trigger corner cases in the concatednated page tables. + */ + mapped =3D 0; + size =3D 1UL << __fls(cfg->pgsize_bitmap); + iova =3D (1UL << cfg->ias) - size; + if (ops->map_pages(ops, iova, iova, size, 1, + IOMMU_READ | IOMMU_WRITE | + IOMMU_NOEXEC | IOMMU_CACHE, + GFP_KERNEL, &mapped)) + return __FAIL(ops, i); + if (mapped !=3D size) + return __FAIL(ops, i); + if (ops->unmap_pages(ops, iova, size, 1, NULL) !=3D size) + return __FAIL(ops, i); + + free_io_pgtable_ops(ops); + } + + return 0; +} + +static int __init arm_lpae_do_selftests(void) +{ + static const unsigned long pgsize[] __initconst =3D { + SZ_4K | SZ_2M | SZ_1G, + SZ_16K | SZ_32M, + SZ_64K | SZ_512M, + }; + + static const unsigned int address_size[] __initconst =3D { + 32, 36, 40, 42, 44, 48, + }; + + int i, j, k, pass =3D 0, fail =3D 0; + struct faux_device *dev; + struct io_pgtable_cfg cfg =3D { + .tlb =3D &dummy_tlb_ops, + .coherent_walk =3D true, + .quirks =3D IO_PGTABLE_QUIRK_NO_WARN, + }; + + dev =3D faux_device_create("io-pgtable-test", NULL, 0); + if (!dev) + return -ENOMEM; + + cfg.iommu_dev =3D &dev->dev; + + for (i =3D 0; i < ARRAY_SIZE(pgsize); ++i) { + for (j =3D 0; j < ARRAY_SIZE(address_size); ++j) { + /* Don't use ias > oas as it is not valid for stage-2. */ + for (k =3D 0; k <=3D j; ++k) { + cfg.pgsize_bitmap =3D pgsize[i]; + cfg.ias =3D address_size[k]; + cfg.oas =3D address_size[j]; + pr_info("selftest: pgsize_bitmap 0x%08lx, IAS %u OAS %u\n", + pgsize[i], cfg.ias, cfg.oas); + if (arm_lpae_run_tests(&cfg)) + fail++; + else + pass++; + } + } + } + + pr_info("selftest: completed with %d PASS %d FAIL\n", pass, fail); + faux_device_destroy(dev); + + return fail ? -EFAULT : 0; +} +subsys_initcall(arm_lpae_do_selftests); +#endif diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 96425e92f313..791a2c4ecb83 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -7,15 +7,10 @@ * Author: Will Deacon */ =20 -#define pr_fmt(fmt) "arm-lpae io-pgtable: " fmt - #include #include #include -#include -#include #include -#include #include #include =20 @@ -24,33 +19,6 @@ #include "io-pgtable-arm.h" #include "iommu-pages.h" =20 -#define ARM_LPAE_MAX_ADDR_BITS 52 -#define ARM_LPAE_S2_MAX_CONCAT_PAGES 16 -#define ARM_LPAE_MAX_LEVELS 4 - -/* Struct accessors */ -#define io_pgtable_to_data(x) \ - container_of((x), struct arm_lpae_io_pgtable, iop) - -#define io_pgtable_ops_to_data(x) \ - io_pgtable_to_data(io_pgtable_ops_to_pgtable(x)) - -/* - * Calculate the right shift amount to get to the portion describing level= l - * in a virtual address mapped by the pagetable in d. - */ -#define ARM_LPAE_LVL_SHIFT(l,d) \ - (((ARM_LPAE_MAX_LEVELS - (l)) * (d)->bits_per_level) + \ - ilog2(sizeof(arm_lpae_iopte))) - -#define ARM_LPAE_GRANULE(d) \ - (sizeof(arm_lpae_iopte) << (d)->bits_per_level) -#define ARM_LPAE_PGD_SIZE(d) \ - (sizeof(arm_lpae_iopte) << (d)->pgd_bits) - -#define ARM_LPAE_PTES_PER_TABLE(d) \ - (ARM_LPAE_GRANULE(d) >> ilog2(sizeof(arm_lpae_iopte))) - /* * Calculate the index at level l used to map virtual address a using the * pagetable in d. @@ -163,18 +131,6 @@ #define iopte_set_writeable_clean(ptep) \ set_bit(ARM_LPAE_PTE_AP_RDONLY_BIT, (unsigned long *)(ptep)) =20 -struct arm_lpae_io_pgtable { - struct io_pgtable iop; - - int pgd_bits; - int start_level; - int bits_per_level; - - void *pgd; -}; - -typedef u64 arm_lpae_iopte; - static inline bool iopte_leaf(arm_lpae_iopte pte, int lvl, enum io_pgtable_fmt fmt) { @@ -1274,204 +1230,3 @@ struct io_pgtable_init_fns io_pgtable_arm_mali_lpae= _init_fns =3D { .alloc =3D arm_mali_lpae_alloc_pgtable, .free =3D arm_lpae_free_pgtable, }; - -#ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST - -static struct io_pgtable_cfg *cfg_cookie __initdata; - -static void __init dummy_tlb_flush_all(void *cookie) -{ - WARN_ON(cookie !=3D cfg_cookie); -} - -static void __init dummy_tlb_flush(unsigned long iova, size_t size, - size_t granule, void *cookie) -{ - WARN_ON(cookie !=3D cfg_cookie); - WARN_ON(!(size & cfg_cookie->pgsize_bitmap)); -} - -static void __init dummy_tlb_add_page(struct iommu_iotlb_gather *gather, - unsigned long iova, size_t granule, - void *cookie) -{ - dummy_tlb_flush(iova, granule, granule, cookie); -} - -static const struct iommu_flush_ops dummy_tlb_ops __initconst =3D { - .tlb_flush_all =3D dummy_tlb_flush_all, - .tlb_flush_walk =3D dummy_tlb_flush, - .tlb_add_page =3D dummy_tlb_add_page, -}; - -static void __init arm_lpae_dump_ops(struct io_pgtable_ops *ops) -{ - struct arm_lpae_io_pgtable *data =3D io_pgtable_ops_to_data(ops); - struct io_pgtable_cfg *cfg =3D &data->iop.cfg; - - pr_err("cfg: pgsize_bitmap 0x%lx, ias %u-bit\n", - cfg->pgsize_bitmap, cfg->ias); - pr_err("data: %d levels, 0x%zx pgd_size, %u pg_shift, %u bits_per_level, = pgd @ %p\n", - ARM_LPAE_MAX_LEVELS - data->start_level, ARM_LPAE_PGD_SIZE(data), - ilog2(ARM_LPAE_GRANULE(data)), data->bits_per_level, data->pgd); -} - -#define __FAIL(ops, i) ({ \ - WARN(1, "selftest: test failed for fmt idx %d\n", (i)); \ - arm_lpae_dump_ops(ops); \ - -EFAULT; \ -}) - -static int __init arm_lpae_run_tests(struct io_pgtable_cfg *cfg) -{ - static const enum io_pgtable_fmt fmts[] __initconst =3D { - ARM_64_LPAE_S1, - ARM_64_LPAE_S2, - }; - - int i, j; - unsigned long iova; - size_t size, mapped; - struct io_pgtable_ops *ops; - - for (i =3D 0; i < ARRAY_SIZE(fmts); ++i) { - cfg_cookie =3D cfg; - ops =3D alloc_io_pgtable_ops(fmts[i], cfg, cfg); - if (!ops) { - pr_err("selftest: failed to allocate io pgtable ops\n"); - return -ENOMEM; - } - - /* - * Initial sanity checks. - * Empty page tables shouldn't provide any translations. - */ - if (ops->iova_to_phys(ops, 42)) - return __FAIL(ops, i); - - if (ops->iova_to_phys(ops, SZ_1G + 42)) - return __FAIL(ops, i); - - if (ops->iova_to_phys(ops, SZ_2G + 42)) - return __FAIL(ops, i); - - /* - * Distinct mappings of different granule sizes. - */ - iova =3D 0; - for_each_set_bit(j, &cfg->pgsize_bitmap, BITS_PER_LONG) { - size =3D 1UL << j; - - if (ops->map_pages(ops, iova, iova, size, 1, - IOMMU_READ | IOMMU_WRITE | - IOMMU_NOEXEC | IOMMU_CACHE, - GFP_KERNEL, &mapped)) - return __FAIL(ops, i); - - /* Overlapping mappings */ - if (!ops->map_pages(ops, iova, iova + size, size, 1, - IOMMU_READ | IOMMU_NOEXEC, - GFP_KERNEL, &mapped)) - return __FAIL(ops, i); - - if (ops->iova_to_phys(ops, iova + 42) !=3D (iova + 42)) - return __FAIL(ops, i); - - iova +=3D SZ_1G; - } - - /* Full unmap */ - iova =3D 0; - for_each_set_bit(j, &cfg->pgsize_bitmap, BITS_PER_LONG) { - size =3D 1UL << j; - - if (ops->unmap_pages(ops, iova, size, 1, NULL) !=3D size) - return __FAIL(ops, i); - - if (ops->iova_to_phys(ops, iova + 42)) - return __FAIL(ops, i); - - /* Remap full block */ - if (ops->map_pages(ops, iova, iova, size, 1, - IOMMU_WRITE, GFP_KERNEL, &mapped)) - return __FAIL(ops, i); - - if (ops->iova_to_phys(ops, iova + 42) !=3D (iova + 42)) - return __FAIL(ops, i); - - iova +=3D SZ_1G; - } - - /* - * Map/unmap the last largest supported page of the IAS, this can - * trigger corner cases in the concatednated page tables. - */ - mapped =3D 0; - size =3D 1UL << __fls(cfg->pgsize_bitmap); - iova =3D (1UL << cfg->ias) - size; - if (ops->map_pages(ops, iova, iova, size, 1, - IOMMU_READ | IOMMU_WRITE | - IOMMU_NOEXEC | IOMMU_CACHE, - GFP_KERNEL, &mapped)) - return __FAIL(ops, i); - if (mapped !=3D size) - return __FAIL(ops, i); - if (ops->unmap_pages(ops, iova, size, 1, NULL) !=3D size) - return __FAIL(ops, i); - - free_io_pgtable_ops(ops); - } - - return 0; -} - -static int __init arm_lpae_do_selftests(void) -{ - static const unsigned long pgsize[] __initconst =3D { - SZ_4K | SZ_2M | SZ_1G, - SZ_16K | SZ_32M, - SZ_64K | SZ_512M, - }; - - static const unsigned int address_size[] __initconst =3D { - 32, 36, 40, 42, 44, 48, - }; - - int i, j, k, pass =3D 0, fail =3D 0; - struct faux_device *dev; - struct io_pgtable_cfg cfg =3D { - .tlb =3D &dummy_tlb_ops, - .coherent_walk =3D true, - .quirks =3D IO_PGTABLE_QUIRK_NO_WARN, - }; - - dev =3D faux_device_create("io-pgtable-test", NULL, 0); - if (!dev) - return -ENOMEM; - - cfg.iommu_dev =3D &dev->dev; - - for (i =3D 0; i < ARRAY_SIZE(pgsize); ++i) { - for (j =3D 0; j < ARRAY_SIZE(address_size); ++j) { - /* Don't use ias > oas as it is not valid for stage-2. */ - for (k =3D 0; k <=3D j; ++k) { - cfg.pgsize_bitmap =3D pgsize[i]; - cfg.ias =3D address_size[k]; - cfg.oas =3D address_size[j]; - pr_info("selftest: pgsize_bitmap 0x%08lx, IAS %u OAS %u\n", - pgsize[i], cfg.ias, cfg.oas); - if (arm_lpae_run_tests(&cfg)) - fail++; - else - pass++; - } - } - } - - pr_info("selftest: completed with %d PASS %d FAIL\n", pass, fail); - faux_device_destroy(dev); - - return fail ? -EFAULT : 0; -} -subsys_initcall(arm_lpae_do_selftests); -#endif diff --git a/drivers/iommu/io-pgtable-arm.h b/drivers/iommu/io-pgtable-arm.h index ba7cfdf7afa0..a06a23543cff 100644 --- a/drivers/iommu/io-pgtable-arm.h +++ b/drivers/iommu/io-pgtable-arm.h @@ -2,6 +2,8 @@ #ifndef IO_PGTABLE_ARM_H_ #define IO_PGTABLE_ARM_H_ =20 +#include + #define ARM_LPAE_TCR_TG0_4K 0 #define ARM_LPAE_TCR_TG0_64K 1 #define ARM_LPAE_TCR_TG0_16K 2 @@ -27,4 +29,43 @@ #define ARM_LPAE_TCR_PS_48_BIT 0x5ULL #define ARM_LPAE_TCR_PS_52_BIT 0x6ULL =20 +/* Struct accessors */ +#define io_pgtable_to_data(x) \ + container_of((x), struct arm_lpae_io_pgtable, iop) + +#define io_pgtable_ops_to_data(x) \ + io_pgtable_to_data(io_pgtable_ops_to_pgtable(x)) + +struct arm_lpae_io_pgtable { + struct io_pgtable iop; + + int pgd_bits; + int start_level; + int bits_per_level; + + void *pgd; +}; + +#define ARM_LPAE_MAX_ADDR_BITS 52 +#define ARM_LPAE_S2_MAX_CONCAT_PAGES 16 +#define ARM_LPAE_MAX_LEVELS 4 + +/* + * Calculate the right shift amount to get to the portion describing level= l + * in a virtual address mapped by the pagetable in d. + */ +#define ARM_LPAE_LVL_SHIFT(l,d) \ + (((ARM_LPAE_MAX_LEVELS - (l)) * (d)->bits_per_level) + \ + ilog2(sizeof(arm_lpae_iopte))) + +#define ARM_LPAE_GRANULE(d) \ + (sizeof(arm_lpae_iopte) << (d)->bits_per_level) +#define ARM_LPAE_PGD_SIZE(d) \ + (sizeof(arm_lpae_iopte) << (d)->pgd_bits) + +#define ARM_LPAE_PTES_PER_TABLE(d) \ + (ARM_LPAE_GRANULE(d) >> ilog2(sizeof(arm_lpae_iopte))) + +typedef u64 arm_lpae_iopte; + #endif /* IO_PGTABLE_ARM_H_ */ --=20 2.51.0.rc1.167.g924127e9c0-goog