From nobody Thu Nov 28 08:48:34 2024 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B28C71D0E03 for ; Wed, 2 Oct 2024 14:17:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727878648; cv=none; b=L/6vFA9cWmEzIwgRQvZfTNyMsRwcxEApjDGjkn0E0TVfEQJreChAqtvBHUQH094ILLepCgdThnmBOQI7FYoCrllyH7cn5qyVIH2wKWW2+7GV0pK0bjqcXe6GBUahXqAASsvPcbkIZy1Ub5S4hHRpC7NtfZNeeVQEMQd3bOC8GvU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727878648; c=relaxed/simple; bh=sxJgMQ9yKRA5DPM4sF3wUbDqO4aCtx14ZPCI7ryj4Q4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mhbINkI+ezewAjH6gwkAXhTGzkAkOgimv6VqGUvo1bnjkCHmKpImHUmYTrk6SnBN2Coer3lWF/QI//rzBIo498pM/iRHG/qg2RjE7KCyUFtAeE0ls5mbP76QpZKShM1FxZrdKAe+OWLHh6V7X5deHEpyrTloO9g0uRo+gEMG0xg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AFAB5339; Wed, 2 Oct 2024 07:17:55 -0700 (PDT) Received: from e122027.arm.com (unknown [10.57.64.205]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E93293F58B; Wed, 2 Oct 2024 07:17:23 -0700 (PDT) From: Steven Price To: Marc Zyngier , Thomas Gleixner Cc: Steven Price , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Will Deacon , Suzuki K Poulose , Michael Kelley Subject: [PATCH v2 1/2] irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor Date: Wed, 2 Oct 2024 15:16:29 +0100 Message-Id: <20241002141630.433502-2-steven.price@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241002141630.433502-1-steven.price@arm.com> References: <20241002141630.433502-1-steven.price@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Within a realm guest the ITS is emulated by the host. This means the allocations must have been made available to the host by a call to set_memory_decrypted(). Introduce an allocation function which performs this extra call. For the ITT use a custom genpool-based allocator that calls set_memory_decrypted() for each page allocated, but then suballocates the size needed for each ITT. Note that there is no mechanism implemented to return pages from the genpool, but it is unlikely the peak number of devices will so much larger than the normal level - so this isn't expected to be an issue. Co-developed-by: Suzuki K Poulose Signed-off-by: Suzuki K Poulose Tested-by: Will Deacon Reviewed-by: Marc Zyngier Signed-off-by: Steven Price Reported-by: Shanker Donthineni Reviewed-by: Catalin Marinas --- Changes since v1: * Drop WARN_ONs and add a comment explaining that, if they fail, set_memory_{en,de}crypted() will already have WARNed and that we are purposefully leaking memory in those cases. --- drivers/irqchip/irq-gic-v3-its.c | 151 ++++++++++++++++++++++++++----- 1 file changed, 128 insertions(+), 23 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index fdec478ba5e7..7a62fd3a8673 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -12,12 +12,14 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include #include #include @@ -27,6 +29,7 @@ #include #include #include +#include #include #include =20 @@ -164,6 +167,7 @@ struct its_device { struct its_node *its; struct event_lpi_map event_map; void *itt; + u32 itt_sz; u32 nr_ites; u32 device_id; bool shared; @@ -199,6 +203,93 @@ static DEFINE_IDA(its_vpeid_ida); #define gic_data_rdist_rd_base() (gic_data_rdist()->rd_base) #define gic_data_rdist_vlpi_base() (gic_data_rdist_rd_base() + SZ_128K) =20 +static struct page *its_alloc_pages_node(int node, gfp_t gfp, + unsigned int order) +{ + struct page *page; + int ret =3D 0; + + page =3D alloc_pages_node(node, gfp, order); + + if (!page) + return NULL; + + ret =3D set_memory_decrypted((unsigned long)page_address(page), + 1 << order); + /* + * If set_memory_decrypted() fails then we don't know what state the + * page is in, so we can't free it. Instead we leak it. + * set_memory_decrypted() will already have WARNed. + */ + if (ret) + return NULL; + + return page; +} + +static struct page *its_alloc_pages(gfp_t gfp, unsigned int order) +{ + return its_alloc_pages_node(NUMA_NO_NODE, gfp, order); +} + +static void its_free_pages(void *addr, unsigned int order) +{ + /* + * If the memory cannot be encrypted again then we must leak the pages. + * set_memory_encrypted will already have WARNed. + */ + if (set_memory_encrypted((unsigned long)addr, 1 << order)) + return; + free_pages((unsigned long)addr, order); +} + +static struct gen_pool *itt_pool; + +static void *itt_alloc_pool(int node, int size) +{ + unsigned long addr; + struct page *page; + + if (size >=3D PAGE_SIZE) { + page =3D its_alloc_pages_node(node, + GFP_KERNEL | __GFP_ZERO, + get_order(size)); + + if (!page) + return NULL; + + return page_address(page); + } + + do { + addr =3D gen_pool_alloc(itt_pool, size); + if (addr) + break; + + page =3D its_alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 1); + if (!page) + break; + + gen_pool_add(itt_pool, (unsigned long)page_address(page), + PAGE_SIZE, node); + } while (!addr); + + return (void *)addr; +} + +static void itt_free_pool(void *addr, int size) +{ + if (!addr) + return; + + if (size >=3D PAGE_SIZE) { + its_free_pages(addr, get_order(size)); + return; + } + + gen_pool_free(itt_pool, (unsigned long)addr, size); +} + /* * Skip ITSs that have no vLPIs mapped, unless we're on GICv4.1, as we * always have vSGIs mapped. @@ -2181,7 +2272,8 @@ static struct page *its_allocate_prop_table(gfp_t gfp= _flags) { struct page *prop_page; =20 - prop_page =3D alloc_pages(gfp_flags, get_order(LPI_PROPBASE_SZ)); + prop_page =3D its_alloc_pages(gfp_flags, + get_order(LPI_PROPBASE_SZ)); if (!prop_page) return NULL; =20 @@ -2192,8 +2284,8 @@ static struct page *its_allocate_prop_table(gfp_t gfp= _flags) =20 static void its_free_prop_table(struct page *prop_page) { - free_pages((unsigned long)page_address(prop_page), - get_order(LPI_PROPBASE_SZ)); + its_free_pages(page_address(prop_page), + get_order(LPI_PROPBASE_SZ)); } =20 static bool gic_check_reserved_range(phys_addr_t addr, unsigned long size) @@ -2315,7 +2407,8 @@ static int its_setup_baser(struct its_node *its, stru= ct its_baser *baser, order =3D get_order(GITS_BASER_PAGES_MAX * psz); } =20 - page =3D alloc_pages_node(its->numa_node, GFP_KERNEL | __GFP_ZERO, order); + page =3D its_alloc_pages_node(its->numa_node, + GFP_KERNEL | __GFP_ZERO, order); if (!page) return -ENOMEM; =20 @@ -2328,7 +2421,7 @@ static int its_setup_baser(struct its_node *its, stru= ct its_baser *baser, /* 52bit PA is supported only when PageSize=3D64K */ if (psz !=3D SZ_64K) { pr_err("ITS: no 52bit PA support when psz=3D%d\n", psz); - free_pages((unsigned long)base, order); + its_free_pages(base, order); return -ENXIO; } =20 @@ -2384,7 +2477,7 @@ static int its_setup_baser(struct its_node *its, stru= ct its_baser *baser, pr_err("ITS@%pa: %s doesn't stick: %llx %llx\n", &its->phys_base, its_base_type_string[type], val, tmp); - free_pages((unsigned long)base, order); + its_free_pages(base, order); return -ENXIO; } =20 @@ -2523,8 +2616,8 @@ static void its_free_tables(struct its_node *its) =20 for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { if (its->tables[i].base) { - free_pages((unsigned long)its->tables[i].base, - its->tables[i].order); + its_free_pages(its->tables[i].base, + its->tables[i].order); its->tables[i].base =3D NULL; } } @@ -2790,7 +2883,8 @@ static bool allocate_vpe_l2_table(int cpu, u32 id) =20 /* Allocate memory for 2nd level table */ if (!table[idx]) { - page =3D alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(psz)); + page =3D its_alloc_pages(GFP_KERNEL | __GFP_ZERO, + get_order(psz)); if (!page) return false; =20 @@ -2909,7 +3003,8 @@ static int allocate_vpe_l1_table(void) =20 pr_debug("np =3D %d, npg =3D %lld, psz =3D %d, epp =3D %d, esz =3D %d\n", np, npg, psz, epp, esz); - page =3D alloc_pages(GFP_ATOMIC | __GFP_ZERO, get_order(np * PAGE_SIZE)); + page =3D its_alloc_pages(GFP_ATOMIC | __GFP_ZERO, + get_order(np * PAGE_SIZE)); if (!page) return -ENOMEM; =20 @@ -2955,8 +3050,8 @@ static struct page *its_allocate_pending_table(gfp_t = gfp_flags) { struct page *pend_page; =20 - pend_page =3D alloc_pages(gfp_flags | __GFP_ZERO, - get_order(LPI_PENDBASE_SZ)); + pend_page =3D its_alloc_pages(gfp_flags | __GFP_ZERO, + get_order(LPI_PENDBASE_SZ)); if (!pend_page) return NULL; =20 @@ -2968,7 +3063,7 @@ static struct page *its_allocate_pending_table(gfp_t = gfp_flags) =20 static void its_free_pending_table(struct page *pt) { - free_pages((unsigned long)page_address(pt), get_order(LPI_PENDBASE_SZ)); + its_free_pages(page_address(pt), get_order(LPI_PENDBASE_SZ)); } =20 /* @@ -3303,8 +3398,9 @@ static bool its_alloc_table_entry(struct its_node *it= s, =20 /* Allocate memory for 2nd level table */ if (!table[idx]) { - page =3D alloc_pages_node(its->numa_node, GFP_KERNEL | __GFP_ZERO, - get_order(baser->psz)); + page =3D its_alloc_pages_node(its->numa_node, + GFP_KERNEL | __GFP_ZERO, + get_order(baser->psz)); if (!page) return false; =20 @@ -3399,7 +3495,6 @@ static struct its_device *its_create_device(struct it= s_node *its, u32 dev_id, if (WARN_ON(!is_power_of_2(nvecs))) nvecs =3D roundup_pow_of_two(nvecs); =20 - dev =3D kzalloc(sizeof(*dev), GFP_KERNEL); /* * Even if the device wants a single LPI, the ITT must be * sized as a power of two (and you need at least one bit...). @@ -3407,7 +3502,11 @@ static struct its_device *its_create_device(struct i= ts_node *its, u32 dev_id, nr_ites =3D max(2, nvecs); sz =3D nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1); sz =3D max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1; - itt =3D kzalloc_node(sz, GFP_KERNEL, its->numa_node); + + itt =3D itt_alloc_pool(its->numa_node, sz); + + dev =3D kzalloc(sizeof(*dev), GFP_KERNEL); + if (alloc_lpis) { lpi_map =3D its_lpi_alloc(nvecs, &lpi_base, &nr_lpis); if (lpi_map) @@ -3419,9 +3518,9 @@ static struct its_device *its_create_device(struct it= s_node *its, u32 dev_id, lpi_base =3D 0; } =20 - if (!dev || !itt || !col_map || (!lpi_map && alloc_lpis)) { + if (!dev || !itt || !col_map || (!lpi_map && alloc_lpis)) { kfree(dev); - kfree(itt); + itt_free_pool(itt, sz); bitmap_free(lpi_map); kfree(col_map); return NULL; @@ -3431,6 +3530,7 @@ static struct its_device *its_create_device(struct it= s_node *its, u32 dev_id, =20 dev->its =3D its; dev->itt =3D itt; + dev->itt_sz =3D sz; dev->nr_ites =3D nr_ites; dev->event_map.lpi_map =3D lpi_map; dev->event_map.col_map =3D col_map; @@ -3458,7 +3558,7 @@ static void its_free_device(struct its_device *its_de= v) list_del(&its_dev->entry); raw_spin_unlock_irqrestore(&its_dev->its->lock, flags); kfree(its_dev->event_map.col_map); - kfree(its_dev->itt); + itt_free_pool(its_dev->itt, its_dev->itt_sz); kfree(its_dev); } =20 @@ -5116,8 +5216,9 @@ static int __init its_probe_one(struct its_node *its) } } =20 - page =3D alloc_pages_node(its->numa_node, GFP_KERNEL | __GFP_ZERO, - get_order(ITS_CMD_QUEUE_SZ)); + page =3D its_alloc_pages_node(its->numa_node, + GFP_KERNEL | __GFP_ZERO, + get_order(ITS_CMD_QUEUE_SZ)); if (!page) { err =3D -ENOMEM; goto out_unmap_sgir; @@ -5181,7 +5282,7 @@ static int __init its_probe_one(struct its_node *its) out_free_tables: its_free_tables(its); out_free_cmd: - free_pages((unsigned long)its->cmd_base, get_order(ITS_CMD_QUEUE_SZ)); + its_free_pages(its->cmd_base, get_order(ITS_CMD_QUEUE_SZ)); out_unmap_sgir: if (its->sgir_base) iounmap(its->sgir_base); @@ -5667,6 +5768,10 @@ int __init its_init(struct fwnode_handle *handle, st= ruct rdists *rdists, bool has_v4_1 =3D false; int err; =20 + itt_pool =3D gen_pool_create(get_order(ITS_ITT_ALIGN), -1); + if (!itt_pool) + return -ENOMEM; + gic_rdists =3D rdists; =20 lpi_prop_prio =3D irq_prio; --=20 2.34.1 From nobody Thu Nov 28 08:48:34 2024 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4BC871D0DF7 for ; Wed, 2 Oct 2024 14:17:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727878650; cv=none; b=rI1Yqz39mfBRcgsi8kNZuVtX9WOIdmsgNISdnwNKWlR0LhNwLENt6eKi75GKOEHVDANFbHGKQ0vYl3bCZ64t09IcDVLC9a+7ORg9po9LRjLd89MXvoytsgHPoeYOub13iVXRztUQcDY4gyfYRZx+f05aYYA0di7yjlxwap5G2X4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727878650; c=relaxed/simple; bh=FwyocT23P2ZBQV7I2j1YtltD7RvOJSqqLqEZaqt92fI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=M2Vx40RoxrSRvnnvaPIFRe8N9E+ZQ1IXXuRsuXULLWij50sb32sHoP6Tk7KiYFaLvnfDli3WV1Y6PniU1wG6lY6TZKNIOxC0y2ciKYpy32K/G6sRLndWGN0G8x6FC8pYntIY+l15NV9snPDyHh+K3rvuYL8d1A7Cu/B+zQYE3ZI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2B60A497; Wed, 2 Oct 2024 07:17:58 -0700 (PDT) Received: from e122027.arm.com (unknown [10.57.64.205]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 96CA23F58B; Wed, 2 Oct 2024 07:17:26 -0700 (PDT) From: Steven Price To: Marc Zyngier , Thomas Gleixner Cc: Steven Price , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Will Deacon , Suzuki K Poulose , Michael Kelley Subject: [PATCH v2 2/2] irqchip/gic-v3-its: Rely on genpool alignment Date: Wed, 2 Oct 2024 15:16:30 +0100 Message-Id: <20241002141630.433502-3-steven.price@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241002141630.433502-1-steven.price@arm.com> References: <20241002141630.433502-1-steven.price@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" its_create_device() over-allocated by ITS_ITT_ALIGN - 1 bytes to ensure that an aligned area was available within the allocation. The new genpool allocator has its min_alloc_order set to get_order(ITS_ITT_ALIGN) so all allocations from it should be appropriately aligned. Remove the over-allocation from its_create_device() and alignment from its_build_mapd_cmd(). Tested-by: Will Deacon Reviewed-by: Marc Zyngier Signed-off-by: Steven Price Reviewed-by: Catalin Marinas --- drivers/irqchip/irq-gic-v3-its.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index 7a62fd3a8673..1d2a952e342d 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -712,7 +712,6 @@ static struct its_collection *its_build_mapd_cmd(struct= its_node *its, u8 size =3D ilog2(desc->its_mapd_cmd.dev->nr_ites); =20 itt_addr =3D virt_to_phys(desc->its_mapd_cmd.dev->itt); - itt_addr =3D ALIGN(itt_addr, ITS_ITT_ALIGN); =20 its_encode_cmd(cmd, GITS_CMD_MAPD); its_encode_devid(cmd, desc->its_mapd_cmd.dev->device_id); @@ -3501,7 +3500,7 @@ static struct its_device *its_create_device(struct it= s_node *its, u32 dev_id, */ nr_ites =3D max(2, nvecs); sz =3D nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1); - sz =3D max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1; + sz =3D max(sz, ITS_ITT_ALIGN); =20 itt =3D itt_alloc_pool(its->numa_node, sz); =20 --=20 2.34.1