From nobody Mon Jun 15 07:32:45 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70BE62ED154 for ; Wed, 8 Apr 2026 19:47:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677678; cv=none; b=eMEE52Pj3fJ0R1e+DVxWctwZrT/r8HVDKaUVGGY4MyC3SdSrnkjnTPaPCzeMTHotbYB8cm7mbdz7L7fNHm9Vl3S+VpQCjkpNSckwIdzEP/x3rjGO8DnKSl9PIpV4OZg9+huA2iUZMzh3otxeNckgSFLe16oGOiTMgzabTcA6CDI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677678; c=relaxed/simple; bh=IQ7dKN8L4UZaeFiDQv8QKTj4LSuRG6pNboyXJnO/vcU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sz+fpGTY2tZL1o55xXzBVL67KrZJdk7ZvDKON1MLOjdY1D23f8x/mkB4/cQzMvp/EIsSOWqPnUGgLkpoFX1xdLejD1tFxF9rYTqZh82Ezx7TkvDDmziu2FqW5XaDVDYLxIBC36oVFAaUUS8xbn9Exjhu2RqWMoWfOuwilyGX95Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pZOyQJPO; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pZOyQJPO" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-48887ff8b73so454295e9.2 for ; Wed, 08 Apr 2026 12:47:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1775677675; x=1776282475; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gIZkNljvc8euLjmnwxUEEJNh/0dVrZPblskV0P24de4=; b=pZOyQJPODmWDwuP6WwHq8bQfGfZTlUipCEB9p2/tVpM0Wotd18xqFlCi0hzaWEbiR7 +bEHFrTMrF1AOy//OnKC/u9KonxWKm/HY/AyqwnQYD5pUPnM69bdFvPnCpr/dSs3E9As 90H2amB6vz6lLxb688AuZ6LIYZYe58E7RmMIZF0lovP00BAw1OczSdUnbpWPVkjCcwhC SmI32wa+VUgTDjyWWyV+KvrsdJ4K4V9rD4Zxmgoo9i4NGe/huAKWPF9E+BH/78rh+kXk gvM54mjznKqq09ttUSEqgIJ/VYFDTXJkIbullmkyTwBvl6O2I3MIEfc0CzOMMf5/BQ3E UCEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775677675; x=1776282475; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gIZkNljvc8euLjmnwxUEEJNh/0dVrZPblskV0P24de4=; b=eTVIv8D2h0HjGcn9ALv6eaJwQ5Y8XmaPkAh0cVzqozudx4JvQLK9x0m6iC+rURhrrI 9n4r+y4iUwhHwz1s/wIo/SwupczUfLwWFMU0YBXehh94acgwkGCjdZBn2+C0KloHq9xq gfFFmpzOoj8qltR7SRLtAlLyb+CaYGjYq59TrPqVV2ycoGTa2iVfXJOGgrw+KQS3J8fX OwaMDlSpBAv57jMldmWCSRYo83VyNSTJBThINWZluBL5Pkon5ZpOUZbCSYz9jyowd8d0 l/Y53XGR8IcUAKZwy6r+FmmhGXK+jQ+2+IjaBCdtW6r0pRjCJoCgYma3HX88gU4EIOUt UJVQ== X-Forwarded-Encrypted: i=1; AJvYcCUWyyegYrYLT7ErULQ0baxXWp6qjXbeE2L/HwKojekY/+/k+a4nO0qOr4tsTscWYOpOfRxaZKqe5UapS34=@vger.kernel.org X-Gm-Message-State: AOJu0YxgrhHsmGabC+d7nj99bGlSQ9D5CPazDkePy2C+A+SivwK4uIoy w7oSa40yEx43xi5CykhrfaECwqJpkh1ojkc1s99f5CrepmI9cMJOISyJ3qNaZ4kRW2NimCpUoMm k0alSPXibCps7cw== X-Received: from wmqy6.prod.google.com ([2002:a05:600c:3646:b0:488:7f5c:7693]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4ecf:b0:488:aa33:dcbd with SMTP id 5b1f17b1804b1-488aa33e02emr202999135e9.26.1775677674735; Wed, 08 Apr 2026 12:47:54 -0700 (PDT) Date: Wed, 8 Apr 2026 19:47:38 +0000 In-Reply-To: <20260408194750.2280873-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260408194750.2280873-1-smostafa@google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog Message-ID: <20260408194750.2280873-2-smostafa@google.com> Subject: [RFC PATCH v3 1/5] swiotlb: Return state of memory from swiotlb_alloc() From: Mostafa Saleh To: iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: robin.murphy@arm.com, m.szyprowski@samsung.com, will@kernel.org, maz@kernel.org, suzuki.poulose@arm.com, catalin.marinas@arm.com, jiri@resnulli.us, jgg@ziepe.ca, aneesh.kumar@kernel.org, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make swiotlb_alloc() return the state of the allocated memory, at the moment all the pools are decrypted but that would change soon. In the next patches dma-direct will use the returned state to determine whether to decrypt the memory and use the proper memory decryption/encryption related functions. Also, add swiotlb_is_decrypted(), that will be used before calling swiotlb_free() to check whether the memory needs to be encrypted by the caller. Signed-off-by: Mostafa Saleh --- include/linux/swiotlb.h | 25 +++++++++++++++++++++++-- kernel/dma/direct.c | 2 +- kernel/dma/swiotlb.c | 23 ++++++++++++++++++++++- 3 files changed, 46 insertions(+), 4 deletions(-) diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h index 3dae0f592063..24be65494ce8 100644 --- a/include/linux/swiotlb.h +++ b/include/linux/swiotlb.h @@ -63,6 +63,7 @@ extern void __init swiotlb_update_mem_attributes(void); * @area_nslabs: Number of slots in each area. * @areas: Array of memory area descriptors. * @slots: Array of slot descriptors. + * @decrypted: Whether the pool was decrypted or left in default state. * @node: Member of the IO TLB memory pool list. * @rcu: RCU head for swiotlb_dyn_free(). * @transient: %true if transient memory pool. @@ -77,6 +78,7 @@ struct io_tlb_pool { unsigned int area_nslabs; struct io_tlb_area *areas; struct io_tlb_slot *slots; + bool decrypted; #ifdef CONFIG_SWIOTLB_DYNAMIC struct list_head node; struct rcu_head rcu; @@ -281,16 +283,31 @@ static inline void swiotlb_sync_single_for_cpu(struct= device *dev, =20 extern void swiotlb_print_info(void); =20 +/* + * This contains the state of pages returned by swiotlb_alloc() + * A page can either be: + * SWIOTLB_PAGE_DEFAULT: The page was not decrypted by the pool. + * SWIOTLB_PAGE_DECRYPTED: The page was decrypted by the pool. + */ +enum swiotlb_page_state { + SWIOTLB_PAGE_DEFAULT, + SWIOTLB_PAGE_DECRYPTED, +}; + #ifdef CONFIG_DMA_RESTRICTED_POOL -struct page *swiotlb_alloc(struct device *dev, size_t size); +struct page *swiotlb_alloc(struct device *dev, size_t size, + enum swiotlb_page_state *state); bool swiotlb_free(struct device *dev, struct page *page, size_t size); =20 +bool swiotlb_is_decrypted(struct device *dev, struct page *page, size_t si= ze); + static inline bool is_swiotlb_for_alloc(struct device *dev) { return dev->dma_io_tlb_mem->for_alloc; } #else -static inline struct page *swiotlb_alloc(struct device *dev, size_t size) +static inline struct page *swiotlb_alloc(struct device *dev, size_t size, + enum swiotlb_page_state *state) { return NULL; } @@ -299,6 +316,10 @@ static inline bool swiotlb_free(struct device *dev, st= ruct page *page, { return false; } +static inline bool swiotlb_is_decrypted(struct device *dev, struct page *p= age, size_t size) +{ + return false; +} static inline bool is_swiotlb_for_alloc(struct device *dev) { return false; diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 8f43a930716d..6efb5973fbd3 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -106,7 +106,7 @@ static void __dma_direct_free_pages(struct device *dev,= struct page *page, =20 static struct page *dma_direct_alloc_swiotlb(struct device *dev, size_t si= ze) { - struct page *page =3D swiotlb_alloc(dev, size); + struct page *page =3D swiotlb_alloc(dev, size, NULL); =20 if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { swiotlb_free(dev, page, size); diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 9fd73700ddcf..8468ee5d3ff2 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -1763,7 +1763,8 @@ static inline void swiotlb_create_debugfs_files(struc= t io_tlb_mem *mem, =20 #ifdef CONFIG_DMA_RESTRICTED_POOL =20 -struct page *swiotlb_alloc(struct device *dev, size_t size) +struct page *swiotlb_alloc(struct device *dev, size_t size, + enum swiotlb_page_state *state) { struct io_tlb_mem *mem =3D dev->dma_io_tlb_mem; struct io_tlb_pool *pool; @@ -1787,6 +1788,8 @@ struct page *swiotlb_alloc(struct device *dev, size_t= size) return NULL; } =20 + if (state) + *state =3D pool->decrypted ? SWIOTLB_PAGE_DECRYPTED : SWIOTLB_PAGE_DEFAU= LT; return pfn_to_page(PFN_DOWN(tlb_addr)); } =20 @@ -1804,6 +1807,18 @@ bool swiotlb_free(struct device *dev, struct page *p= age, size_t size) return true; } =20 +bool swiotlb_is_decrypted(struct device *dev, struct page *page, size_t si= ze) +{ + phys_addr_t tlb_addr =3D page_to_phys(page); + struct io_tlb_pool *pool; + + pool =3D swiotlb_find_pool(dev, tlb_addr); + if (!pool) + return false; + + return pool->decrypted; +} + static int rmem_swiotlb_device_init(struct reserved_mem *rmem, struct device *dev) { @@ -1844,6 +1859,12 @@ static int rmem_swiotlb_device_init(struct reserved_= mem *rmem, return -ENOMEM; } =20 + /* + * At the moment all restricted dma pools are always decrypted, + * although that should change soon with CCA solutions introducing + * device passthrough. + */ + pool->decrypted =3D true; set_memory_decrypted((unsigned long)phys_to_virt(rmem->base), rmem->size >> PAGE_SHIFT); swiotlb_init_io_tlb_pool(pool, rmem->base, nslabs, --=20 2.53.0.1213.gd9a14994de-goog From nobody Mon Jun 15 07:32:45 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B7D737F8DF for ; Wed, 8 Apr 2026 19:47:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677678; cv=none; b=VDl7YwLaxTDiGdk3THQ5TmhtXSiAcb26VA91YAT6NGkp6vTa9U9H5tbYaFn9F1iflg7LBoYrYxWGdFFew+Z5H/AoHCIAUnSEXsuuoiSQF7B+HH+rrZQp9CRjzfv0ofDwQRza1MlQg/oDKiCQIWA+ROLr9B4BUfE87Hu/8uRleXU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677678; c=relaxed/simple; bh=MTYNkJlUHc2hk2wVY4COdGRvfk5v8kr43GQAGEytGr8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jcC+j/caqZ+uQWNQ5ZQFRtPFdqP7Wvh3Niuxpd74DFFQZLBYTglvBrFkBNWt6Wbu9Edy7jnDyJBtTiTbFW5sWmvsyvi9wDwFpILjKFSYmYpldyGlliXt7Dssk93D03PyaFXbD7vbpCXFMJ/Bjm1CdmV+FjNf9WGgFcNshlFs3Gw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FDncD0vv; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FDncD0vv" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4887d32788bso679025e9.0 for ; Wed, 08 Apr 2026 12:47:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1775677676; x=1776282476; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=R9wr9J2jjZi9zDgPEShl28f6yfD5q9/Iu+uj378SHds=; b=FDncD0vvaR17++Jk1YvsAiw0DkrxTtalpULtU5DFrTH9VI/0NS7b3y15U8ELFxylbV dIWtA2KgpYCQCvmKhZomH0i3UFJQXXjTeNWphri8Qz6vOVQqDHZFI+uP0PldaaX9xWhS MH9B5rdYK1C65ghuBMVjvU7XSA5HVBgd1DdLhO47ljVNANhxLvOOGlUymMzaLLcMbw5i S84AYM8H7UoEhC5WplhBz+es8kn88sHTDQOhsJGG7Y5ovs6rU3PnVgjXMaAUX/rsCBNY yF+6Q2NFGYc8IqSemcsAHirzguGqwHhZ+CZeMb9RrZqU/xgxO2j13EQfumdKrTf8VnGF DPMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775677676; x=1776282476; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=R9wr9J2jjZi9zDgPEShl28f6yfD5q9/Iu+uj378SHds=; b=DffEmcXlLOQy91oR7qBY4t01HlsCpAvTq8cEYhMbkfbnRUTR3Mcjyx43mR0jFgX2DT NuU43KdA0pdGSy8GSRFHFmqt3SkQ/x+Rc1eFoOEGEKmfvBRln7MKp1YtQnAbJGfhM+a1 ack46p6NXahn9W099rJ1XNhpBIwYYd56SWB+wpPHMGx2ncNA3bn5/t2aNAousk1j8olx XCRhCaisgSXYWG6xXD6C+paTkEZdkPTl1hlEvd/myWt3Aclvk3FyAk+9eOnYyhqxYoxd U8kp5erXcBtNhua1PVNhwaktr4D36d54sWRanF1vJunEE7JA4biF2W7Uzrr3m/dAWclO iRmw== X-Forwarded-Encrypted: i=1; AJvYcCXTZW7PSSppp9AsQMKJeog4Wk+KCwB0cZOM6tMpPoxhknuDgvy1pSQqWZ2BDfti/4QLZ82tnVe6msKhqA0=@vger.kernel.org X-Gm-Message-State: AOJu0YwW0GfV3pEU2JH3QwOpALyiN/gN2NouhRRczolkWf4ZwPG5yqYU XEOb7oWpFrgNuKDa/kdRSa3fkZzEAGetRCGmuX6hrNcJkw0dYlgBAJudZllNRnj1VxetwSzIPV9 RKzPbYkF4yI38fg== X-Received: from wmcn9.prod.google.com ([2002:a05:600c:c0c9:b0:485:3ee6:2d4]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8b75:b0:488:7ff6:1f75 with SMTP id 5b1f17b1804b1-488997bcd86mr326807635e9.21.1775677675905; Wed, 08 Apr 2026 12:47:55 -0700 (PDT) Date: Wed, 8 Apr 2026 19:47:39 +0000 In-Reply-To: <20260408194750.2280873-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260408194750.2280873-1-smostafa@google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog Message-ID: <20260408194750.2280873-3-smostafa@google.com> Subject: [RFC PATCH v3 2/5] dma-mapping: Move encryption in __dma_direct_free_pages() From: Mostafa Saleh To: iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: robin.murphy@arm.com, m.szyprowski@samsung.com, will@kernel.org, maz@kernel.org, suzuki.poulose@arm.com, catalin.marinas@arm.com, jiri@resnulli.us, jgg@ziepe.ca, aneesh.kumar@kernel.org, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the next patches, we will need to avoid encrypting memory allocated from SWIOTLB, so instead of calling dma_set_encrypted() before __dma_direct_free_pages(), call it inside, conditional on the memory state passed to the function. Signed-off-by: Mostafa Saleh --- kernel/dma/direct.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 6efb5973fbd3..ce74f213ec40 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -97,8 +97,11 @@ static int dma_set_encrypted(struct device *dev, void *v= addr, size_t size) } =20 static void __dma_direct_free_pages(struct device *dev, struct page *page, - size_t size) + size_t size, bool encrypt) { + if (encrypt && dma_set_encrypted(dev, page_address(page), size)) + return; + if (swiotlb_free(dev, page, size)) return; dma_free_contiguous(dev, page, size); @@ -203,7 +206,7 @@ static void *dma_direct_alloc_no_mapping(struct device = *dev, size_t size, void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) { - bool remap =3D false, set_uncached =3D false; + bool remap =3D false, set_uncached =3D false, encrypt =3D false; struct page *page; void *ret; =20 @@ -298,10 +301,9 @@ void *dma_direct_alloc(struct device *dev, size_t size, return ret; =20 out_encrypt_pages: - if (dma_set_encrypted(dev, page_address(page), size)) - return NULL; + encrypt =3D true; out_free_pages: - __dma_direct_free_pages(dev, page, size); + __dma_direct_free_pages(dev, page, size, encrypt); return NULL; out_leak_pages: return NULL; @@ -311,6 +313,7 @@ void dma_direct_free(struct device *dev, size_t size, void *cpu_addr, dma_addr_t dma_addr, unsigned long attrs) { unsigned int page_order =3D get_order(size); + bool encrypt =3D false; =20 if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) && !force_dma_unencrypted(dev) && !is_swiotlb_for_alloc(dev)) { @@ -343,11 +346,10 @@ void dma_direct_free(struct device *dev, size_t size, } else { if (IS_ENABLED(CONFIG_ARCH_HAS_DMA_CLEAR_UNCACHED)) arch_dma_clear_uncached(cpu_addr, size); - if (dma_set_encrypted(dev, cpu_addr, size)) - return; + encrypt =3D true; } =20 - __dma_direct_free_pages(dev, dma_direct_to_page(dev, dma_addr), size); + __dma_direct_free_pages(dev, dma_direct_to_page(dev, dma_addr), size, enc= rypt); } =20 struct page *dma_direct_alloc_pages(struct device *dev, size_t size, @@ -384,9 +386,7 @@ void dma_direct_free_pages(struct device *dev, size_t s= ize, dma_free_from_pool(dev, vaddr, size)) return; =20 - if (dma_set_encrypted(dev, vaddr, size)) - return; - __dma_direct_free_pages(dev, page, size); + __dma_direct_free_pages(dev, page, size, true); } =20 #if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \ --=20 2.53.0.1213.gd9a14994de-goog From nobody Mon Jun 15 07:32:45 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83D2D3815E8 for ; Wed, 8 Apr 2026 19:47:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677682; cv=none; b=TmBNkJOVJUnKxuFGjuE5IhNPZSfXqNysxZ/tYxGsb3QFh0z5lMyAYUt2Xl/L0Zg83HI7ZW4u49nHHNSnU7wox5ypaPcOPHKJHDyfK5nA9/mUF0N8Qqd6uIxmIujZQy3zXGlrh9gk32u9V16lBEvVjX0zcxeDfwZc7oPNyy4eLq8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677682; c=relaxed/simple; bh=Wap6lbG/SCEnn+rOeCGCnUqunIqCe5Tn1Ow/jjzxAEc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sOHdo2aHwHRxwUPIMb75qmq0qGY50X+RMd3Vd7nXHN0S9Z6d/81GeVdruoB9StG8DP2bW6DGqJRNncT2cWgMRy7tMVV48DsbCG9tDWGo6+7swN059UVNK2yPX+nO4X5sTuo9+HxydtRqr4XX6WmlBIFCZeBZlqJ/VzJe3zSimVU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lFmlPkJA; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lFmlPkJA" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-486fa07f2bbso343315e9.2 for ; Wed, 08 Apr 2026 12:47:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1775677677; x=1776282477; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FmQc+pnXBAjV16L6lXzAX5g89yFxEhVx/QorzEaCsdc=; b=lFmlPkJAxW549St55Sol7mqnyfphwXgVQ7n5q7suNy8otl92ZKA8KOqs/d+OHr1+oC nQt99O7SCSZ4QgdmpXmZfeua2cHZXi2+BEaKqV3+Rci5i+dFb/iv/NSJQs4GJ5v+DSCO +RXZm5bsbKUdMfDwaZ/s+PMtU3/Yb/lwY0jhcGX2DRD7Q/Zw/e3SWDASXXrXVCkwwSRk h054L1RTBmYbTsPHD1KE3I+PILYfqb9qYFHlF9uVTlvOIQ1GdD0JWzrCC3lTDWN1sbPO O6f13UAIQG45J4dkt1rw4ld/nQip9L+lgaZMGy3RO2UyO/rpZqECzZKqxBwQzf2g3LkJ saKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775677677; x=1776282477; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FmQc+pnXBAjV16L6lXzAX5g89yFxEhVx/QorzEaCsdc=; b=TOj1S/18ZVUBrroWBnTopUz6aUpP8T/wZhDxcKBTZACm0GGe2S3BgqSrN51qhzQ6QU pbg7ubDC4qdc7bxsyWociYQoBnvpAEIznX7vRg+vVIjE9474cpK+mfOdGxHWuA3CnFZc VfUsETfhXTh8NDhJ6ACixzUyRrbvdIXHGDF6IAukhMM0kduBgm0Hto+z6abc4dfzoxcN G6dyGPNztsnJyY9IF4Kx6R5ZIskJ3gv6ph2zChbGrZ2tT0EonNUsa+C39jFGO1L97mEO cS7TS85IhG0KJr8dx9Cz12qZuMtyvaA79Wq7RpW09Xlw8g6hxUTJtaGAR8M4Mc2jJtgm /rvQ== X-Forwarded-Encrypted: i=1; AJvYcCWd3cqeCUPC79Axn1iRZ9DAKATiV8JiQDEnjBAGPDkIdQombDThDLxbMvBsAtN0GzqPsRUXnAmxKFaBh+E=@vger.kernel.org X-Gm-Message-State: AOJu0YwqUmvmESzIXmAiilAh5QADUA2JoCDArixqc2UR3Cp82TGkeK+Y kh2U7kA1Yn2+THEIUxXYTVztOZQRZDWKG7rY9Cx4DTLzSji0r7TiKP3wWa0wKs68Zg/GM/of68Z m+kBhsvUVScibNA== X-Received: from wmbgx14.prod.google.com ([2002:a05:600c:858e:b0:488:a474:87b6]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8597:b0:487:575:5e1 with SMTP id 5b1f17b1804b1-488cd008ec4mr10978775e9.24.1775677676908; Wed, 08 Apr 2026 12:47:56 -0700 (PDT) Date: Wed, 8 Apr 2026 19:47:40 +0000 In-Reply-To: <20260408194750.2280873-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260408194750.2280873-1-smostafa@google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog Message-ID: <20260408194750.2280873-4-smostafa@google.com> Subject: [RFC PATCH v3 3/5] dma-mapping: Decrypt memory on remap From: Mostafa Saleh To: iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: robin.murphy@arm.com, m.szyprowski@samsung.com, will@kernel.org, maz@kernel.org, suzuki.poulose@arm.com, catalin.marinas@arm.com, jiri@resnulli.us, jgg@ziepe.ca, aneesh.kumar@kernel.org, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In case memory needs to be remapped on systems with force_dma_unencrypted(), where this memory is not allocated from a restricted-dma pool, this was currently ignored, while only setting the decrypted pgprot in the remapped alias. The memory still needs to be decrypted in that case. With memory decryption, don't allow highmem allocations, but that shouldn't be a problem on such modern systems. Also, move force_dma_unencrypted() outside of dma_set_* to make it clear to be able to use more generic logic to decided memory state. Reported-by: Catalin Marinas Fixes: f3c962226dbe ("dma-direct: clean up the remapping checks in dma_dire= ct_alloc") Signed-off-by: Mostafa Saleh --- kernel/dma/direct.c | 31 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index ce74f213ec40..de63e0449700 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -79,8 +79,6 @@ bool dma_coherent_ok(struct device *dev, phys_addr_t phys= , size_t size) =20 static int dma_set_decrypted(struct device *dev, void *vaddr, size_t size) { - if (!force_dma_unencrypted(dev)) - return 0; return set_memory_decrypted((unsigned long)vaddr, PFN_UP(size)); } =20 @@ -88,8 +86,6 @@ static int dma_set_encrypted(struct device *dev, void *va= ddr, size_t size) { int ret; =20 - if (!force_dma_unencrypted(dev)) - return 0; ret =3D set_memory_encrypted((unsigned long)vaddr, PFN_UP(size)); if (ret) pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n"); @@ -206,7 +202,7 @@ static void *dma_direct_alloc_no_mapping(struct device = *dev, size_t size, void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) { - bool remap =3D false, set_uncached =3D false, encrypt =3D false; + bool remap =3D false, set_uncached =3D false, decrypt =3D force_dma_unenc= rypted(dev); struct page *page; void *ret; =20 @@ -215,7 +211,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, gfp |=3D __GFP_NOWARN; =20 if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) && - !force_dma_unencrypted(dev) && !is_swiotlb_for_alloc(dev)) + !decrypt && !is_swiotlb_for_alloc(dev)) return dma_direct_alloc_no_mapping(dev, size, dma_handle, gfp); =20 if (!dev_is_dma_coherent(dev)) { @@ -249,12 +245,15 @@ void *dma_direct_alloc(struct device *dev, size_t siz= e, * Remapping or decrypting memory may block, allocate the memory from * the atomic pools instead if we aren't allowed block. */ - if ((remap || force_dma_unencrypted(dev)) && + if ((remap || decrypt) && dma_direct_use_pool(dev, gfp)) return dma_direct_alloc_from_pool(dev, size, dma_handle, gfp); =20 - /* we always manually zero the memory once we are done */ - page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true); + /* + * we always manually zero the memory once we are done, and only allow + * high mem if pages doesn't need decryption. + */ + page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, !decrypt); if (!page) return NULL; =20 @@ -268,10 +267,12 @@ void *dma_direct_alloc(struct device *dev, size_t siz= e, set_uncached =3D false; } =20 + if (decrypt && dma_set_decrypted(dev, page_address(page), size)) + goto out_leak_pages; if (remap) { pgprot_t prot =3D dma_pgprot(dev, PAGE_KERNEL, attrs); =20 - if (force_dma_unencrypted(dev)) + if (decrypt) prot =3D pgprot_decrypted(prot); =20 /* remove any dirty cache lines on the kernel alias */ @@ -281,11 +282,9 @@ void *dma_direct_alloc(struct device *dev, size_t size, ret =3D dma_common_contiguous_remap(page, size, prot, __builtin_return_address(0)); if (!ret) - goto out_free_pages; + goto out_encrypt_pages; } else { ret =3D page_address(page); - if (dma_set_decrypted(dev, ret, size)) - goto out_leak_pages; } =20 memset(ret, 0, size); @@ -301,9 +300,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, return ret; =20 out_encrypt_pages: - encrypt =3D true; -out_free_pages: - __dma_direct_free_pages(dev, page, size, encrypt); + __dma_direct_free_pages(dev, page, size, decrypt); return NULL; out_leak_pages: return NULL; @@ -366,7 +363,7 @@ struct page *dma_direct_alloc_pages(struct device *dev,= size_t size, return NULL; =20 ret =3D page_address(page); - if (dma_set_decrypted(dev, ret, size)) + if (force_dma_unencrypted(dev) && dma_set_decrypted(dev, ret, size)) goto out_leak_pages; memset(ret, 0, size); *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page)); --=20 2.53.0.1213.gd9a14994de-goog From nobody Mon Jun 15 07:32:45 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7B0A3815DB for ; Wed, 8 Apr 2026 19:47:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677682; cv=none; b=mHaWoiwMVzjOv+8sP2hqxPhClPLCzqXNzuZaJ9XCG1mbF8hO7am4D7SXqa5x76SE3RTWQPLNufCD/2DBjeOlzCXXVhaIFILIHjZzc6drngvdq/SmILeEteUtHBJiDlZUZjj+Vn8FRAmlImIeiqMUOTYm5tj8kAEOekaJre3O1Ns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677682; c=relaxed/simple; bh=ljnywAXYuD9yVasMcE6vMsdXOxIf5S7KW0HfwwIoGA4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bsozyqTfMTVjIO2p5nh2MOsmNjVP5zfpBagTeEd2HfEVcLHNWv3xivsWdequ18/0dmvIFHvIi5VC7RmAaGnMYDdF1/3t+zVFdrdeBCK76LDNoMgzOJfQ4BehaVsD1nfkW0AdHSGNmZS+i7PlS/RiIna8cAqXXsoHYVF0sLRt+Kg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=onnRlBxs; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="onnRlBxs" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-488c768a9a9so474895e9.1 for ; Wed, 08 Apr 2026 12:47:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1775677678; x=1776282478; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0HFQ1jpsIj20Rmmvy6mz/JJ67LDNNDb4EoQDQuxQbic=; b=onnRlBxsbFoEIWm8HT6Nw4FwKGH5AChdOqih70InQvbmNzHt6mshStRn2Hma9uIpiG V18LDZSWud2PNOUrct+NdC61zRp0WMjde7Ik1Gt1hwh/1f9X8tNsQm7E3VZdLk58v9IN a6AaoFbXAyyqyzqJBXCVPQ1tMYGnT1MgXMbauI9V9gjArwnZyHGCGiJu5sH/S58eERcr HWq4WPBszWUDS52KZcDRwMuoDy4CoffN6Hd6x/1MBn+ZcdiMjID1i8KF6FXzMAn0Upv1 qe0Bt3iDf6qT6w+ANYy191OdqfheO14LBogX09s7Tk2Xhjy9Tvvc7MUwVQDjOti8lk3l FJTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775677678; x=1776282478; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0HFQ1jpsIj20Rmmvy6mz/JJ67LDNNDb4EoQDQuxQbic=; b=PZA6wLwnF/ngcb/nDGzB1XEoDGcY3/b3D0ywMz3wMklx/Ulv/+Xl2G2lvcFq37CR9o fsybC/kaOmV23iYy+LKJtDV3OegG014lEzsO2BKl6/IYaBvEoJfetOdrd5aWx1D+Bb2b 2233kluRy9y/eAdNRTS6bNTHVQ6AcRPvws9nUshzJFLI21Kl9p4i68KPhfaXyjK4Ssry T+in7NYf+N/W4cQchzE6C57UinaryHS+iwxq1Y4JZyLgyvrch8dSyyF+mz9OdAvYh3Xg SUoXKBj9f5K5s0s6DYJgFeLRXAG1nRrW8oxaqbRqtNzURWt6jtVJOzjgxCFDwUq/lI7C hd/A== X-Forwarded-Encrypted: i=1; AJvYcCWbwcgCf762KdYAaReFr4rovk54zlg+Z9KsmB4oLNll9RD1uErvYx4kDSwRtlY80e0Bng29d3jdEXy8TUU=@vger.kernel.org X-Gm-Message-State: AOJu0YyxraVpWXhhn2xkMZg3RR7VbjFrrLgygKL6R1Oa5IA5CNPo8k0s tbdVACK7tmJNXjGNJxi2GUhzFDlwtNESYJJ78pK0uqTB1069Y5K5QcLH7y8/fjiztLk70ot/+1S vyJHOSOZQFLu/RA== X-Received: from wrbgl27-n2.prod.google.com ([2002:a05:6000:299b:20b0:43c:f8a2:96a5]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8710:b0:488:c40b:c8a4 with SMTP id 5b1f17b1804b1-488ccf3a129mr15045395e9.1.1775677678033; Wed, 08 Apr 2026 12:47:58 -0700 (PDT) Date: Wed, 8 Apr 2026 19:47:41 +0000 In-Reply-To: <20260408194750.2280873-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260408194750.2280873-1-smostafa@google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog Message-ID: <20260408194750.2280873-5-smostafa@google.com> Subject: [RFC PATCH v3 4/5] dma-mapping: Encapsulate memory state during allocation From: Mostafa Saleh To: iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: robin.murphy@arm.com, m.szyprowski@samsung.com, will@kernel.org, maz@kernel.org, suzuki.poulose@arm.com, catalin.marinas@arm.com, jiri@resnulli.us, jgg@ziepe.ca, aneesh.kumar@kernel.org, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce a new dma-direct internal type dma_page which is "struct page" and a bit indicate whether the memory has been decrypted or not. This is useful to pass such information encapsulated through allocation functions, which is currently set from swiotlb_alloc(). No functional changes. Signed-off-by: Mostafa Saleh --- kernel/dma/direct.c | 58 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 46 insertions(+), 12 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index de63e0449700..204bc566480c 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -16,6 +16,33 @@ #include #include "direct.h" =20 +/* + * Represent DMA allocation and 1 bit flag for it's state + */ +struct dma_page { + unsigned long val; +}; + +#define DMA_PAGE_DECRYPTED_FLAG BIT(0) + +#define DMA_PAGE_NULL ((struct dma_page){ .val =3D 0 }) + +static inline struct dma_page page_to_dma_page(struct page *page, bool dec= rypted) +{ + struct dma_page dma_page; + + dma_page.val =3D (unsigned long)page; + if (decrypted) + dma_page.val |=3D DMA_PAGE_DECRYPTED_FLAG; + + return dma_page; +} + +static inline struct page *dma_page_to_page(struct dma_page dma_page) +{ + return (struct page *)(dma_page.val & ~DMA_PAGE_DECRYPTED_FLAG); +} + /* * Most architectures use ZONE_DMA for the first 16 Megabytes, but some use * it for entirely different regions. In that case the arch code needs to @@ -103,20 +130,21 @@ static void __dma_direct_free_pages(struct device *de= v, struct page *page, dma_free_contiguous(dev, page, size); } =20 -static struct page *dma_direct_alloc_swiotlb(struct device *dev, size_t si= ze) +static struct dma_page dma_direct_alloc_swiotlb(struct device *dev, size_t= size) { - struct page *page =3D swiotlb_alloc(dev, size, NULL); + enum swiotlb_page_state state; + struct page *page =3D swiotlb_alloc(dev, size, &state); =20 if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) { swiotlb_free(dev, page, size); - return NULL; + return DMA_PAGE_NULL; } =20 - return page; + return page_to_dma_page(page, state =3D=3D SWIOTLB_PAGE_DECRYPTED); } =20 -static struct page *__dma_direct_alloc_pages(struct device *dev, size_t si= ze, - gfp_t gfp, bool allow_highmem) +static struct dma_page __dma_direct_alloc_pages(struct device *dev, size_t= size, + gfp_t gfp, bool allow_highmem) { int node =3D dev_to_node(dev); struct page *page; @@ -132,7 +160,7 @@ static struct page *__dma_direct_alloc_pages(struct dev= ice *dev, size_t size, if (page) { if (dma_coherent_ok(dev, page_to_phys(page), size) && (allow_highmem || !PageHighMem(page))) - return page; + return page_to_dma_page(page, false); =20 dma_free_contiguous(dev, page, size); } @@ -148,10 +176,10 @@ static struct page *__dma_direct_alloc_pages(struct d= evice *dev, size_t size, else if (IS_ENABLED(CONFIG_ZONE_DMA) && !(gfp & GFP_DMA)) gfp =3D (gfp & ~GFP_DMA32) | GFP_DMA; else - return NULL; + return DMA_PAGE_NULL; } =20 - return page; + return page_to_dma_page(page, false); } =20 /* @@ -184,9 +212,11 @@ static void *dma_direct_alloc_from_pool(struct device = *dev, size_t size, static void *dma_direct_alloc_no_mapping(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp) { + struct dma_page dma_page; struct page *page; =20 - page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true); + dma_page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true); + page =3D dma_page_to_page(dma_page); if (!page) return NULL; =20 @@ -203,6 +233,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) { bool remap =3D false, set_uncached =3D false, decrypt =3D force_dma_unenc= rypted(dev); + struct dma_page dma_page; struct page *page; void *ret; =20 @@ -253,7 +284,8 @@ void *dma_direct_alloc(struct device *dev, size_t size, * we always manually zero the memory once we are done, and only allow * high mem if pages doesn't need decryption. */ - page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, !decrypt); + dma_page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, !decr= ypt); + page =3D dma_page_to_page(dma_page); if (!page) return NULL; =20 @@ -352,13 +384,15 @@ void dma_direct_free(struct device *dev, size_t size, struct page *dma_direct_alloc_pages(struct device *dev, size_t size, dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp) { + struct dma_page dma_page; struct page *page; void *ret; =20 if (force_dma_unencrypted(dev) && dma_direct_use_pool(dev, gfp)) return dma_direct_alloc_from_pool(dev, size, dma_handle, gfp); =20 - page =3D __dma_direct_alloc_pages(dev, size, gfp, false); + dma_page =3D __dma_direct_alloc_pages(dev, size, gfp, false); + page =3D dma_page_to_page(dma_page); if (!page) return NULL; =20 --=20 2.53.0.1213.gd9a14994de-goog From nobody Mon Jun 15 07:32:45 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1863A381B07 for ; Wed, 8 Apr 2026 19:48:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677685; cv=none; b=r+d0CIae8MOoYLFJK3OuY5NwGGFh80o/NVYQK+prJhBxVTERemCXLtEwj13Is+/cUoq5c/EKiX4Dq5AbIYaCOxnSYTACaioLchmd95HW9nASen0IZ00KTiSKVAjQ6gtBSjB/fhZEyil0Tk2Uju77bofrSVlTlAfyjC3Tv4gGuy4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775677685; c=relaxed/simple; bh=c5G7I4UDifLrETkTh7TKGEdYvvLFqIwAGOJBOKM1n6c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZOizB5bl5Wy+y1p/GwohNTHZeaIxGyeF99s6ea+1msVsnVX/QlJJX+FDxahLETS5x+UwlDs/X7Ep2l6OCZKDIXV6fMAWdu61dXpmPjQuZoZ9B2MEh6MwR5+uuQQ+e5MnIb7uGln4UFL59Y3Juicx8RhcXITf8nxIu1UuYgwjMyk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bsIcz200; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bsIcz200" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-43cfedb10a8so43492f8f.1 for ; Wed, 08 Apr 2026 12:48:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1775677679; x=1776282479; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=I8Ii68uqF49RZqdN3es99368zeVz3d5ax63VAcg1ZH4=; b=bsIcz200P+eSgMcyE4XwfOrHaglUD3KwVg8K+cBipf+pAgB+iSXs4CqqRngC/fUw+b xDBKV8IiLvyLO/ZclS0HYu74R4DeIbnaejJfVXdY/KnK3+/3tS52VgFDaTj2MI5WHS1P DYuWILJHucirAQ4jle4UzxrlfoaIX6Dv5aGK/5moU8vo/T25sbB6mQP+VINrPWcPm4Na 9H/mNEVLjiV7iGhlPpWX8WXUJP9AC9BRj9XIVuH3Y2P6Nmobbu8bGWV5afBDq8q0Xj+b 9S2Ebr0TFFXeVG7kL/ddoFIXnEvBsjxxsPuf9z3F7H0IkbArX4eA+NHmIuV4h2qBkBXi Q9VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775677679; x=1776282479; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=I8Ii68uqF49RZqdN3es99368zeVz3d5ax63VAcg1ZH4=; b=stTrbihFfFPRTYvneCoBmXyuIDWV/unj3dImWKdujb8J+hAnhcYooRrur4K82b4Oph kmzYj8wcR/xLn2GS2MdRKXnbVNHpF+i1a824By3p5Y1kHeB9vrekjoiflNR0gSGOdW06 SUPJ0Fo2PKhuk++8BExdnXNsrw+8jM7Sndsprjo8gTX6LR5u94i3O4Rkda1OZl7PGbyd TVoPVPhURl5GupYxKknmfhPwNHdLjkUxFMDkmyJG2bCdE1Ho9S+M8EDXXZAfb8aWJjPN oT7Yq6UF5ecPaUH5bent21xZX2A1G5VBbMwqfxiXtUe9ECpL6WIbo/x6x9EZ4qmQM+ye K9pA== X-Forwarded-Encrypted: i=1; AJvYcCX6dr9AMY7JfFjsica/XSfcIfOyPvqDEriU1Mq7Ryz+C4NKWK6p+qJ3VNJ8DgyRVjz0l6loXS3AZfKsj2g=@vger.kernel.org X-Gm-Message-State: AOJu0Yy/99Z2M4k6T70UBuw05KcFrS949jgsBrL1vrR1z6vK7exBvZeR efq5h9PrUY+FJH2EdUemmqKrlV3Bg5tNKMuochielCd5vPgcqtHKxZzV4g5dsw0ZvXAzDRd3Qi+ hHQ2xMlWrqUqjoQ== X-Received: from wmdd5.prod.google.com ([2002:a05:600c:a205:b0:488:9a47:9aa0]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3549:b0:485:41c4:e2e4 with SMTP id 5b1f17b1804b1-488997d2ccemr289069375e9.23.1775677679080; Wed, 08 Apr 2026 12:47:59 -0700 (PDT) Date: Wed, 8 Apr 2026 19:47:42 +0000 In-Reply-To: <20260408194750.2280873-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260408194750.2280873-1-smostafa@google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog Message-ID: <20260408194750.2280873-6-smostafa@google.com> Subject: [RFC PATCH v3 5/5] dma-mapping: Fix memory decryption issues From: Mostafa Saleh To: iommu@lists.linux.dev, linux-kernel@vger.kernel.org Cc: robin.murphy@arm.com, m.szyprowski@samsung.com, will@kernel.org, maz@kernel.org, suzuki.poulose@arm.com, catalin.marinas@arm.com, jiri@resnulli.us, jgg@ziepe.ca, aneesh.kumar@kernel.org, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fix 2 existing issues: 1) In case a device have a restricted DMA pool, memory will be decrypted (which is now returned in the state from swiotlb_alloc(). Later the main function will attempt to decrypt the memory if force_dma_unencrypted() is true. Which results in the memory being decrypted twice. Change that to only encrypt/decrypt memory that is not already decrypted as indicated in the new dma_page struct. 2) Using phys_to_dma_unencrypted() is not enlighted about already decrypted memory and will use the wrong functions for that. Fixes: f4111e39a52a ("swiotlb: Add restricted DMA alloc/free support") Signed-off-by: Mostafa Saleh --- kernel/dma/direct.c | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 204bc566480c..26611d5e5757 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -43,6 +43,11 @@ static inline struct page *dma_page_to_page(struct dma_p= age dma_page) return (struct page *)(dma_page.val & ~DMA_PAGE_DECRYPTED_FLAG); } =20 +static inline bool is_dma_page_decrypted(struct dma_page dma_page) +{ + return dma_page.val & DMA_PAGE_DECRYPTED_FLAG; +} + /* * Most architectures use ZONE_DMA for the first 16 Megabytes, but some use * it for entirely different regions. In that case the arch code needs to @@ -51,9 +56,9 @@ static inline struct page *dma_page_to_page(struct dma_pa= ge dma_page) u64 zone_dma_limit __ro_after_init =3D DMA_BIT_MASK(24); =20 static inline dma_addr_t phys_to_dma_direct(struct device *dev, - phys_addr_t phys) + phys_addr_t phys, bool already_decrypted) { - if (force_dma_unencrypted(dev)) + if (already_decrypted || force_dma_unencrypted(dev)) return phys_to_dma_unencrypted(dev, phys); return phys_to_dma(dev, phys); } @@ -67,7 +72,7 @@ static inline struct page *dma_direct_to_page(struct devi= ce *dev, u64 dma_direct_get_required_mask(struct device *dev) { phys_addr_t phys =3D (phys_addr_t)(max_pfn - 1) << PAGE_SHIFT; - u64 max_dma =3D phys_to_dma_direct(dev, phys); + u64 max_dma =3D phys_to_dma_direct(dev, phys, false); =20 return (1ULL << (fls64(max_dma) - 1)) * 2 - 1; } @@ -96,7 +101,7 @@ static gfp_t dma_direct_optimal_gfp_mask(struct device *= dev, u64 *phys_limit) =20 bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size) { - dma_addr_t dma_addr =3D phys_to_dma_direct(dev, phys); + dma_addr_t dma_addr =3D phys_to_dma_direct(dev, phys, false); =20 if (dma_addr =3D=3D DMA_MAPPING_ERROR) return false; @@ -122,11 +127,14 @@ static int dma_set_encrypted(struct device *dev, void= *vaddr, size_t size) static void __dma_direct_free_pages(struct device *dev, struct page *page, size_t size, bool encrypt) { - if (encrypt && dma_set_encrypted(dev, page_address(page), size)) + bool keep_encrypted =3D swiotlb_is_decrypted(dev, page, size); + + if (!keep_encrypted && encrypt && dma_set_encrypted(dev, page_address(pag= e), size)) return; =20 if (swiotlb_free(dev, page, size)) return; + dma_free_contiguous(dev, page, size); } =20 @@ -205,7 +213,7 @@ static void *dma_direct_alloc_from_pool(struct device *= dev, size_t size, page =3D dma_alloc_from_pool(dev, size, &ret, gfp, dma_coherent_ok); if (!page) return NULL; - *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page)); + *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page), false); return ret; } =20 @@ -225,7 +233,8 @@ static void *dma_direct_alloc_no_mapping(struct device = *dev, size_t size, arch_dma_prep_coherent(page, size); =20 /* return the page pointer as the opaque cookie */ - *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page)); + *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page), + is_dma_page_decrypted(dma_page)); return page; } =20 @@ -234,6 +243,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, { bool remap =3D false, set_uncached =3D false, decrypt =3D force_dma_unenc= rypted(dev); struct dma_page dma_page; + bool already_decrypted; struct page *page; void *ret; =20 @@ -289,6 +299,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, if (!page) return NULL; =20 + already_decrypted =3D is_dma_page_decrypted(dma_page); /* * dma_alloc_contiguous can return highmem pages depending on a * combination the cma=3D arguments and per-arch setup. These need to be @@ -299,12 +310,13 @@ void *dma_direct_alloc(struct device *dev, size_t siz= e, set_uncached =3D false; } =20 - if (decrypt && dma_set_decrypted(dev, page_address(page), size)) + if (!already_decrypted && decrypt && + dma_set_decrypted(dev, page_address(page), size)) goto out_leak_pages; if (remap) { pgprot_t prot =3D dma_pgprot(dev, PAGE_KERNEL, attrs); =20 - if (decrypt) + if (decrypt || already_decrypted) prot =3D pgprot_decrypted(prot); =20 /* remove any dirty cache lines on the kernel alias */ @@ -328,11 +340,11 @@ void *dma_direct_alloc(struct device *dev, size_t siz= e, goto out_encrypt_pages; } =20 - *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page)); + *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page), already_decry= pted); return ret; =20 out_encrypt_pages: - __dma_direct_free_pages(dev, page, size, decrypt); + __dma_direct_free_pages(dev, page, size, decrypt && !already_decrypted); return NULL; out_leak_pages: return NULL; @@ -385,6 +397,7 @@ struct page *dma_direct_alloc_pages(struct device *dev,= size_t size, dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp) { struct dma_page dma_page; + bool already_decrypted; struct page *page; void *ret; =20 @@ -396,11 +409,13 @@ struct page *dma_direct_alloc_pages(struct device *de= v, size_t size, if (!page) return NULL; =20 + already_decrypted =3D is_dma_page_decrypted(dma_page); ret =3D page_address(page); - if (force_dma_unencrypted(dev) && dma_set_decrypted(dev, ret, size)) + if (!already_decrypted && force_dma_unencrypted(dev) && + dma_set_decrypted(dev, ret, size)) goto out_leak_pages; memset(ret, 0, size); - *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page)); + *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page), already_decry= pted); return page; out_leak_pages: return NULL; --=20 2.53.0.1213.gd9a14994de-goog