From nobody Fri Dec 19 17:36:14 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28C40330D32 for ; Thu, 6 Nov 2025 16:40:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447213; cv=none; b=irhTPjfv1y06ZApanTxZXg/a/f+ccQiQSm1wQ2pDdWV44GEpo0wvKxJL5/ABWdZQM5BvFX584hD/tZteKHdZAutx5WiBhBTrxzVJdNU08R8EWLSkCK4Vjodn6AbLz4LRhYFHXk414MyLuO5vBRQfO4eLX6QczD9dR4WbRzcAniw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447213; c=relaxed/simple; bh=FoAxF7xHsJFWFTyb3pXeNpXfEV9K4D055QRfjPUOidg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KbrF3LxyjgnSwXoKJK64vZEqxUYsTIWW6V9EkoK8ArdARzEBfaJDPF9MGqp6MZffmn/6ibH+0p8czu7wmE7u3Zj41SAXWD5e3TAHJaVupLY0hHa0hx4o1aVLWfjYgpABrs3pEbSmco1VlZYU38SDmuheCQY7l5EQjzxHv+Mz/vk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uBgyH+ez; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uBgyH+ez" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4775e00b16fso9247785e9.2 for ; Thu, 06 Nov 2025 08:40:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1762447209; x=1763052009; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yGn9S9TG2oeGoFuSE1bVv5EDFMyzjo3ZeltmQ55DAp0=; b=uBgyH+ezP8B4qsSnVT4g7CoEH8o9hi7D2EZk+6WhZt24qMXiKpaTcDv0ZIJlmSvXLn JgnlAkTp5AbLjF7a6FieK98coQlsxY2jWAyOURYvxR3UZ82E+dvQLO8wfIlPsYAPN4v1 NsHEoPTEeYqfIugvGbyhAyEb6JSTSeGFGQpZo8h2JdJ5RwqwHvdKnVHv6zqk2jgx2TpR aEgVvxxSVXw7d3/X/PqErAKlTRAFsNAEUxAovsKl/FtincMwbSAjeKu/Qej6CyLOFBpu rsAECR9UnbVJmOEUsj7to9K5mVmwBnB37pgNkX1O1bmrzY6M9fzgiOSVOY0wNqFUuY/o IHvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762447209; x=1763052009; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yGn9S9TG2oeGoFuSE1bVv5EDFMyzjo3ZeltmQ55DAp0=; b=in52G5FeJHwsynw95e1KgzKkU8dclKOS7+dC3lA1zOImVhCmfNwpaydwBh42xrPKkG LC8vk8XFp5tPjiL14Bbb+qduCDLIRCRTN/Mb7IEsbstYHpKvklfFOdcGHbaeHv1iwjLC p8rxnKRYp6V08GyxhosDKm81vcPkX/Q+X+eEMy2APk1g5aw1lI4W+v7UHc0GLoMpolvU rZEiSV3Z5BV2KrErQFzs1bTne8oc4kGAacCjZZ1irKqG8es1H9MzwhzaEJ15EBn9/T4W jd4Pr9dRp2T/bhWhKcfSTTIuwfdgT16+QCaslzG6Sw+/S8RKKwEGGkDELwecQ6/CEzlP Je+Q== X-Forwarded-Encrypted: i=1; AJvYcCWIdOkP/q7Ry9WrcezE2Z3HWUe4KWPwllR6gTijqPT7PCaeV6DNktKDHEKUKK4xjh2FS1shdVvyjZfps84=@vger.kernel.org X-Gm-Message-State: AOJu0YyXv/bgTf/PEaLeFDYSPTq0ZyupRo9YjwkXCtNTg6mQrrlgcUvW vIzdKysLza9TMrjBd2+206L+PzMUkfV4m4QJUpjUfsTopk2f5qhFr8tzQ1/KbRE1gCpzr2JGfup r4BZ5W8ouMpNE6g== X-Google-Smtp-Source: AGHT+IGin0gdaVVLFPqJJbxATM/GoLhvdHY35FLM6SzMry8X5z6uv1NEwI1PHMOviV8zQ1+HDeBhnFureuyQHA== X-Received: from wmcn19.prod.google.com ([2002:a05:600c:c0d3:b0:45f:29fc:83d]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e88:b0:46f:d897:516f with SMTP id 5b1f17b1804b1-4775ce24d3dmr93382575e9.34.1762447209641; Thu, 06 Nov 2025 08:40:09 -0800 (PST) Date: Thu, 6 Nov 2025 16:39:50 +0000 In-Reply-To: <20251106163953.1971067-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251106163953.1971067-1-smostafa@google.com> X-Mailer: git-send-email 2.51.2.1026.g39e6a42477-goog Message-ID: <20251106163953.1971067-2-smostafa@google.com> Subject: [PATCH v2 1/4] drivers/iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC From: Mostafa Saleh To: linux-mm@kvack.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Cc: corbet@lwn.net, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, akpm@linux-foundation.org, vbabka@suse.cz, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, Mostafa Saleh , Qinxin Xia Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a new config IOMMU_DEBUG_PAGEALLOC, which registers new data to page_ext. This config will be used by the IOMMU API to track pages mapped in the IOMMU to catch drivers trying to free kernel memory that they still map in their domains, causing all types of memory corruption. This behaviour is disabled by default and can be enabled using kernel cmdline iommu.debug_pagealloc. Signed-off-by: Mostafa Saleh Tested-by: Qinxin Xia --- .../admin-guide/kernel-parameters.txt | 6 ++++ drivers/iommu/Kconfig | 15 +++++++++ drivers/iommu/Makefile | 1 + drivers/iommu/iommu-debug-pagealloc.c | 32 +++++++++++++++++++ include/linux/iommu-debug-pagealloc.h | 17 ++++++++++ mm/page_ext.c | 4 +++ 6 files changed, 75 insertions(+) create mode 100644 drivers/iommu/iommu-debug-pagealloc.c create mode 100644 include/linux/iommu-debug-pagealloc.h diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 6c42061ca20e..9a1c4ac8ba96 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2557,6 +2557,12 @@ 1 - Bypass the IOMMU for DMA. unset - Use value of CONFIG_IOMMU_DEFAULT_PASSTHROUGH. =20 + iommu.debug_pagealloc=3D + [KNL,EARLY] When CONFIG_IOMMU_DEBUG_PAGEALLOC is set, this + parameter enables the feature at boot time. By default, it + is disabled and the system will work mostly the same as a + kernel built without CONFIG_IOMMU_DEBUG_PAGEALLOC. + io7=3D [HW] IO7 for Marvel-based Alpha systems See comment before marvel_specify_io7 in arch/alpha/kernel/core_marvel.c. diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index 70d29b14d851..6b5e9a2d936a 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -383,4 +383,19 @@ config SPRD_IOMMU =20 Say Y here if you want to use the multimedia devices listed above. =20 +config IOMMU_DEBUG_PAGEALLOC + bool "Debug page memory allocations against IOMMU" + depends on DEBUG_PAGEALLOC && IOMMU_API && PAGE_EXTENSION + help + This config checks that a page is freed(unmapped) or mapped by the + kernel is not mapped in any IOMMU domain. It can help with debugging + use-after-free or out-of-bound maps from drivers doing DMA through + the IOMMU API. + This santaizer can have false-negative cases where some problems + won't be detected. + Expect overhead when enabling this and enabling the kernel command + line iommu.debug_pagealloc. + + If unsure, say N here. + endif # IOMMU_SUPPORT diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 355294fa9033..8f5130b6a671 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -34,3 +34,4 @@ obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva.o obj-$(CONFIG_IOMMU_IOPF) +=3D io-pgfault.o obj-$(CONFIG_SPRD_IOMMU) +=3D sprd-iommu.o obj-$(CONFIG_APPLE_DART) +=3D apple-dart.o +obj-$(CONFIG_IOMMU_DEBUG_PAGEALLOC) +=3D iommu-debug-pagealloc.o diff --git a/drivers/iommu/iommu-debug-pagealloc.c b/drivers/iommu/iommu-de= bug-pagealloc.c new file mode 100644 index 000000000000..385c8bfae02b --- /dev/null +++ b/drivers/iommu/iommu-debug-pagealloc.c @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2025 - Google Inc + * Author: Mostafa Saleh + * IOMMU API debug page alloc sanitizer + */ +#include +#include +#include +#include + +static bool needed; + +struct iommu_debug_metadate { + atomic_t ref; +}; + +static __init bool need_iommu_debug(void) +{ + return needed; +} + +struct page_ext_operations page_iommu_debug_ops =3D { + .size =3D sizeof(struct iommu_debug_metadate), + .need =3D need_iommu_debug, +}; + +static int __init iommu_debug_pagealloc(char *str) +{ + return kstrtobool(str, &needed); +} +early_param("iommu.debug_pagealloc", iommu_debug_pagealloc); diff --git a/include/linux/iommu-debug-pagealloc.h b/include/linux/iommu-de= bug-pagealloc.h new file mode 100644 index 000000000000..83e64d70bf6c --- /dev/null +++ b/include/linux/iommu-debug-pagealloc.h @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2025 - Google Inc + * Author: Mostafa Saleh + * IOMMU API debug page alloc sanitizer + */ + +#ifndef __LINUX_IOMMU_DEBUG_PAGEALLOC_H +#define __LINUX_IOMMU_DEBUG_PAGEALLOC_H + +#ifdef CONFIG_IOMMU_DEBUG_PAGEALLOC + +extern struct page_ext_operations page_iommu_debug_ops; + +#endif /* CONFIG_IOMMU_DEBUG_PAGEALLOC */ + +#endif /* __LINUX_IOMMU_DEBUG_PAGEALLOC_H */ diff --git a/mm/page_ext.c b/mm/page_ext.c index d7396a8970e5..297e4cd8ce90 100644 --- a/mm/page_ext.c +++ b/mm/page_ext.c @@ -11,6 +11,7 @@ #include #include #include +#include =20 /* * struct page extension @@ -89,6 +90,9 @@ static struct page_ext_operations *page_ext_ops[] __initd= ata =3D { #ifdef CONFIG_PAGE_TABLE_CHECK &page_table_check_ops, #endif +#ifdef CONFIG_IOMMU_DEBUG_PAGEALLOC + &page_iommu_debug_ops, +#endif }; =20 unsigned long page_ext_size; --=20 2.51.2.1026.g39e6a42477-goog From nobody Fri Dec 19 17:36:14 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F5C13451C8 for ; Thu, 6 Nov 2025 16:40:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447214; cv=none; b=TUKxAaXkNizTpHfWvsAczZGF1WGS/cBfiByI+RzV+QjRYWTK1BAT+hB+oeiic9hTwl2qnkUYPGdsCD05roGgQ2D617RkbFKKYMk67K14dvmV3Xk3VkJZDJVJy5GfkewP9Xw63/i9yClz61tFOvoUc/qH30ujoROjK5CG2c/2PNg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447214; c=relaxed/simple; bh=NQFUpooTxwel5bKTK83SG1NYi4wOT9pFGjG9pCWdTI4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aQK0kaeoGOl3VpJwf7KV67Sif1BO9CQuImimi05N9ksnAhpanCLA0gJ9kV01g9wUM9thi6WrFvzdbFD00lwcvcngT/hdllynOkFISBkNfAuYOToHGzXfVtYytKyEFsvzOEA/lQp1kOFvqlhxDYCJ0dmnHP5NDagDVx0U3jGXHUI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VQIPSKLL; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VQIPSKLL" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-477632c240bso8580615e9.2 for ; Thu, 06 Nov 2025 08:40:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1762447211; x=1763052011; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tSiZ7a1SAJCcKkBVDBEHEQqlSOfVk/u+1v4U8UB7ILg=; b=VQIPSKLLg+Q8mRixsje8+SgimBsTX8D5SsMV8WY9y+iwvFEMITB7nMCuWpCUvWcu75 7U6VJbXmXzv81lPBMuhWoL84g+qqtWiW+Yk8f2wsH+qnwwyxH+WianBGjb8L2v6lGl7m GHyyl+JREmVBBamvXBp59E5VZDKtDXco+xL6ZpmcS8K5vucY5yJufJQtxUmR375uFwQo CcbJ2YSmZTpc4ExBqtKBokwU4HFS6hC1iroa5PfXP91epWbiUwmA8E0OdK4gM4zBq0eW mQSWkU/++Shq/8iDTUCZdtSOsdcIBqRc57LiJlvvtJ/Avv0/2PIgpQE0o3CAbBB04X3v 9Q1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762447211; x=1763052011; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tSiZ7a1SAJCcKkBVDBEHEQqlSOfVk/u+1v4U8UB7ILg=; b=sFcdQPRZFqQ8tK/Vm8sKoeNnDnfmhqzG56W2rJXOtEcq6QGIAIO+vFRuwwlsrzaEbk lvbpVl2ruMDzL4GsK7czTIADL+RvV6K91jStSgyVzQ+2s4W+6jmW+oACWNc6wyZdiDyh w/OOPqdQJCyD1d6G7fKpNZesESXUtc/QVDBSiryWyXghu7QlpanYP+OTDo5Ws8KuWrWv /A+KfTg5na3YFZrfG/NJB1BlH90vF7czPqJpV+dnQW2wygJ7TJbK+03p4g3lcrbr4KxI LOPQAHB201F3aUsZxzL5M611Q5fbvOIkp2efJ0atr+JUfCeZh0+Eb//BB0Z64lbvE/Fe T/dg== X-Forwarded-Encrypted: i=1; AJvYcCXXACM/A8FZLeL5Xh8Re+KFyMf/mFdF8re837wkOVlu2/Cmxfl4qt70jx7hnrwoTr7ce39nlv/C8TIBf70=@vger.kernel.org X-Gm-Message-State: AOJu0YwQ2DwHVeRTZvnbry+C5i8KgP07GQkRBNoc7CgOhiqdjRcaQu3B rnx5HBQl+B7FmklMnu2eN23AufJMI9ZfLONVtM3ThTNsB8NBrMNh9yIK9dAuKUGjJgAoM34TPmv SBKawP3JcTbho2w== X-Google-Smtp-Source: AGHT+IEulRepKpg83hX8wGoEp36I0gmjB5yebzIOWO3BeWZ0yZNLjb0D/EtrwnuCx8tGevmN/RGY3y+bKKt1iw== X-Received: from wmnc9.prod.google.com ([2002:a05:600c:1709:b0:477:31ea:6473]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1987:b0:471:56:6f79 with SMTP id 5b1f17b1804b1-4775ce3c5b5mr66270665e9.41.1762447210611; Thu, 06 Nov 2025 08:40:10 -0800 (PST) Date: Thu, 6 Nov 2025 16:39:51 +0000 In-Reply-To: <20251106163953.1971067-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251106163953.1971067-1-smostafa@google.com> X-Mailer: git-send-email 2.51.2.1026.g39e6a42477-goog Message-ID: <20251106163953.1971067-3-smostafa@google.com> Subject: [PATCH v2 2/4] drivers/iommu: Add calls for IOMMU_DEBUG_PAGEALLOC From: Mostafa Saleh To: linux-mm@kvack.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Cc: corbet@lwn.net, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, akpm@linux-foundation.org, vbabka@suse.cz, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, Mostafa Saleh , Qinxin Xia Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap: Track iommu pages unmapped, using IO virtual address. - iommu_debug_remap: Track iommu pages, already mapped using IOVA. We have to do the unmap/remap as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Signed-off-by: Mostafa Saleh Tested-by: Qinxin Xia --- drivers/iommu/iommu-debug-pagealloc.c | 23 ++++++++++++ drivers/iommu/iommu.c | 14 ++++++- include/linux/iommu-debug-pagealloc.h | 54 +++++++++++++++++++++++++++ 3 files changed, 89 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommu-debug-pagealloc.c b/drivers/iommu/iommu-de= bug-pagealloc.c index 385c8bfae02b..a6a2f844b09d 100644 --- a/drivers/iommu/iommu-debug-pagealloc.c +++ b/drivers/iommu/iommu-debug-pagealloc.c @@ -5,11 +5,13 @@ * IOMMU API debug page alloc sanitizer */ #include +#include #include #include #include =20 static bool needed; +DEFINE_STATIC_KEY_FALSE(iommu_debug_initialized); =20 struct iommu_debug_metadate { atomic_t ref; @@ -25,6 +27,27 @@ struct page_ext_operations page_iommu_debug_ops =3D { .need =3D need_iommu_debug, }; =20 +void __iommu_debug_map(struct iommu_domain *domain, phys_addr_t phys, size= _t size) +{ +} + +void __iommu_debug_unmap(struct iommu_domain *domain, unsigned long iova, = size_t size) +{ +} + +void __iommu_debug_remap(struct iommu_domain *domain, unsigned long iova, = size_t size) +{ +} + +void iommu_debug_init(void) +{ + if (!needed) + return; + + pr_info("iommu: Debugging page allocations, expect overhead or disable io= mmu.debug_pagealloc"); + static_branch_enable(&iommu_debug_initialized); +} + static int __init iommu_debug_pagealloc(char *str) { return kstrtobool(str, &needed); diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 59244c744eab..f374ac0fdf95 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -231,6 +232,8 @@ static int __init iommu_subsys_init(void) if (!nb) return -ENOMEM; =20 + iommu_debug_init(); + for (int i =3D 0; i < ARRAY_SIZE(iommu_buses); i++) { nb[i].notifier_call =3D iommu_bus_notifier; bus_register_notifier(iommu_buses[i], &nb[i]); @@ -2544,10 +2547,12 @@ int iommu_map_nosync(struct iommu_domain *domain, u= nsigned long iova, } =20 /* unroll mapping in case something went wrong */ - if (ret) + if (ret) { iommu_unmap(domain, orig_iova, orig_size - size); - else + } else { trace_map(orig_iova, orig_paddr, orig_size); + iommu_debug_map(domain, orig_paddr, orig_size); + } =20 return ret; } @@ -2609,6 +2614,8 @@ static size_t __iommu_unmap(struct iommu_domain *doma= in, =20 pr_debug("unmap this: iova 0x%lx size 0x%zx\n", iova, size); =20 + iommu_debug_unmap(domain, iova, size); + /* * Keep iterating until we either unmap 'size' bytes (or more) * or we hit an area that isn't mapped. @@ -2628,6 +2635,9 @@ static size_t __iommu_unmap(struct iommu_domain *doma= in, unmapped +=3D unmapped_page; } =20 + if (unmapped < size) + iommu_debug_remap(domain, iova, size - unmapped); + trace_unmap(orig_iova, size, unmapped); return unmapped; } diff --git a/include/linux/iommu-debug-pagealloc.h b/include/linux/iommu-de= bug-pagealloc.h index 83e64d70bf6c..180446d6d86f 100644 --- a/include/linux/iommu-debug-pagealloc.h +++ b/include/linux/iommu-debug-pagealloc.h @@ -8,10 +8,64 @@ #ifndef __LINUX_IOMMU_DEBUG_PAGEALLOC_H #define __LINUX_IOMMU_DEBUG_PAGEALLOC_H =20 +struct iommu_domain; + #ifdef CONFIG_IOMMU_DEBUG_PAGEALLOC =20 +DECLARE_STATIC_KEY_FALSE(iommu_debug_initialized); + extern struct page_ext_operations page_iommu_debug_ops; =20 +void __iommu_debug_map(struct iommu_domain *domain, phys_addr_t phys, + size_t size); +void __iommu_debug_unmap(struct iommu_domain *domain, unsigned long iova, + size_t size); +void __iommu_debug_remap(struct iommu_domain *domain, unsigned long iova, + size_t size); + +static inline void iommu_debug_map(struct iommu_domain *domain, + phys_addr_t phys, size_t size) +{ + if (static_branch_unlikely(&iommu_debug_initialized)) + __iommu_debug_map(domain, phys, size); +} + +static inline void iommu_debug_unmap(struct iommu_domain *domain, + unsigned long iova, size_t size) +{ + if (static_branch_unlikely(&iommu_debug_initialized)) + __iommu_debug_unmap(domain, iova, size); +} + +static inline void iommu_debug_remap(struct iommu_domain *domain, + unsigned long iova, size_t size) +{ + if (static_branch_unlikely(&iommu_debug_initialized)) + __iommu_debug_remap(domain, iova, size); +} + +void iommu_debug_init(void); + +#else +static inline void iommu_debug_map(struct iommu_domain *domain, + phys_addr_t phys, size_t size) +{ +} + +static inline void iommu_debug_unmap(struct iommu_domain *domain, + unsigned long iova, size_t size) +{ +} + +static inline void iommu_debug_remap(struct iommu_domain *domain, + unsigned long iova, size_t size) +{ +} + +static inline void iommu_debug_init(void) +{ +} + #endif /* CONFIG_IOMMU_DEBUG_PAGEALLOC */ =20 #endif /* __LINUX_IOMMU_DEBUG_PAGEALLOC_H */ --=20 2.51.2.1026.g39e6a42477-goog From nobody Fri Dec 19 17:36:14 2025 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30184346799 for ; Thu, 6 Nov 2025 16:40:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447215; cv=none; b=k2h6v/omXRUu0Ucd6N8e4PTaeWGYtpd3Mf7gEYlwQal5Yw6CRjR7HTg4pfyKZxA0a8v/mdwKSqUkJOpC8B7xytpdNdGCGa+62uG5EZWHOGOiYgRJ19pm/lhx8v1013+9c2XUc73WKy5j0SMFUWcqx2yp6flVHeENln6nskd9iaY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447215; c=relaxed/simple; bh=YwTj3xPUl5UlBDRODGhVqQxa9CgJ+NfuerK8zGcP/W8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Y9yC23egop8uwtYmhQOtNcYoM4Jkr6ZWgjwaZBvuYT6qzZ/n9lMrm87SDS2B3WWJrlHoisaPSDYcyuCLh96+S6izLrUk9Wy52njq+9zgkqaXwtPbh99haH8TPg0HJ0oIED0ctXvImwT+e27poVuDJuYX0DJ5TggaV9O5Ykppfm0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=l21vwygq; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="l21vwygq" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-429c76c8a1bso868257f8f.0 for ; Thu, 06 Nov 2025 08:40:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1762447211; x=1763052011; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1MuGWRmebEyeJKEBuBok+7/p5tCLW5lhor0NHCb2aik=; b=l21vwygqxJCC3adJzkhXiBTTp7wrmjQyT4WPNlh9SFWiXY0gogHsn+CGo1EhtgorCc D6NIlaitgGg0rCTwKDwzFNgdfB7/LZMZv++Q20q3uTx65v3O+U/FNbxtNG4mVM7WnDic ZPIZavbN1sEYzH08N/ot6rBWfmVXibWkVXeliWzTH7C34zYtLOlH84y5dBDS5UQr5fdO Z02Y3s5xAgsUhIH+4LFoOw0usTKk6Tdj/p95KUcP4jLMO6zBpr2yVlV/S928iPMiGJZj 34DiSMqDbuBpyjdbIrjFELmId+W4+mrnj3a2MP7c9yuje1q7Q8ZoWkpErgwa4aG6hkj4 41Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762447211; x=1763052011; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1MuGWRmebEyeJKEBuBok+7/p5tCLW5lhor0NHCb2aik=; b=VL37zYrBNTKSw3RGyfQzoJUWaQ1nDjWY08slf7ntRtIlxruSGTsgAtcL0G9IY+Jegk E12OSKTL5jBjWrwAkiS5z8TLNSNWcujdQQ6qrnhfOMxkM46NGkupa6lJbTiKEFo4GTGe dryqdHPWh+lAHrf2jJX5fx68m305BEOpoKtpfdOpwjTo7SJlKjw2EhNetazIW+kPo521 xXS630lVPSyKTzu/n/TxpwA46i0xq1aUK/qsWCq9g2R3tw6zs1wqzYbVAwzL+3U4PhFb sdAEYTtyb6QPA0qk/hsdMnTlKzytU4/GOpwlsaH43O3TkPbC0F934ns1QOkNBxLrD5KR da6g== X-Forwarded-Encrypted: i=1; AJvYcCUosH03v1RxCLy3k2m3Gwz/UQCeWCg4Rm3t7o8JwCOtmrHf40aPO/jIDwgMnTbS3u8wQLJ5P1DuRvCDirg=@vger.kernel.org X-Gm-Message-State: AOJu0YziJ5qK+o5K4uiP6humOTqsb7AawCQj7jhKhSTnHOn1DAoVoy0M ICwMooXscJQSqk1IT7RsHm6WCYjUbc5EI9FvhJTEqqSbnlcl4p26uCnysSZpjUqbYuEZ+YmVJbl IBqMQXsjorHMRWw== X-Google-Smtp-Source: AGHT+IHysWr4kZOr+q4Pq/IsxPJWwhd0Upf8H/2toPB1V7pns+GJ5i8QYBGBp3tQwp2KTNC96A5o74tpeTcSWw== X-Received: from wrf23.prod.google.com ([2002:a05:6000:43d7:b0:429:eeb3:9047]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:220c:b0:429:d4e1:cb9d with SMTP id ffacd0b85a97d-429e32c879dmr5734101f8f.8.1762447211597; Thu, 06 Nov 2025 08:40:11 -0800 (PST) Date: Thu, 6 Nov 2025 16:39:52 +0000 In-Reply-To: <20251106163953.1971067-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251106163953.1971067-1-smostafa@google.com> X-Mailer: git-send-email 2.51.2.1026.g39e6a42477-goog Message-ID: <20251106163953.1971067-4-smostafa@google.com> Subject: [PATCH v2 3/4] drivers/iommu-debug-pagealloc: Track IOMMU pages From: Mostafa Saleh To: linux-mm@kvack.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Cc: corbet@lwn.net, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, akpm@linux-foundation.org, vbabka@suse.cz, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, Mostafa Saleh , Qinxin Xia Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Signed-off-by: Mostafa Saleh Tested-by: Qinxin Xia --- drivers/iommu/iommu-debug-pagealloc.c | 74 +++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/drivers/iommu/iommu-debug-pagealloc.c b/drivers/iommu/iommu-de= bug-pagealloc.c index a6a2f844b09d..0e14104b971c 100644 --- a/drivers/iommu/iommu-debug-pagealloc.c +++ b/drivers/iommu/iommu-debug-pagealloc.c @@ -27,16 +27,90 @@ struct page_ext_operations page_iommu_debug_ops =3D { .need =3D need_iommu_debug, }; =20 +static struct page_ext *get_iommu_page_ext(phys_addr_t phys) +{ + struct page *page =3D phys_to_page(phys); + struct page_ext *page_ext =3D page_ext_get(page); + + return page_ext; +} + +static struct iommu_debug_metadate *get_iommu_data(struct page_ext *page_e= xt) +{ + return page_ext_data(page_ext, &page_iommu_debug_ops); +} + +static void iommu_debug_inc_page(phys_addr_t phys) +{ + struct page_ext *page_ext =3D get_iommu_page_ext(phys); + struct iommu_debug_metadate *d =3D get_iommu_data(page_ext); + + WARN_ON(atomic_inc_return(&d->ref) <=3D 0); + page_ext_put(page_ext); +} + +static void iommu_debug_dec_page(phys_addr_t phys) +{ + struct page_ext *page_ext =3D get_iommu_page_ext(phys); + struct iommu_debug_metadate *d =3D get_iommu_data(page_ext); + + WARN_ON(atomic_dec_return(&d->ref) < 0); + page_ext_put(page_ext); +} + +/* + * IOMMU page size might not match the CPU page size, in that case, we use + * the smallest IOMMU page size to refcount the pages in the vmemmap. + * That is important as both map and unmap has to use the same page size + * to update the refcount to avoid double counting the same page. + * And as we can't know from iommu_unmap() what was the original page size + * used for map, we just use the minimum supported one for both. + */ +static size_t iommu_debug_page_size(struct iommu_domain *domain) +{ + return 1UL << __ffs(domain->pgsize_bitmap); +} + void __iommu_debug_map(struct iommu_domain *domain, phys_addr_t phys, size= _t size) { + size_t off; + size_t page_size =3D iommu_debug_page_size(domain); + + for (off =3D 0 ; off < size ; off +=3D page_size) { + if (!pfn_valid(__phys_to_pfn(phys + off))) + continue; + iommu_debug_inc_page(phys + off); + } } =20 void __iommu_debug_unmap(struct iommu_domain *domain, unsigned long iova, = size_t size) { + size_t off; + size_t page_size =3D iommu_debug_page_size(domain); + + for (off =3D 0 ; off < size ; off +=3D page_size) { + phys_addr_t phys =3D iommu_iova_to_phys(domain, iova + off); + + if (!phys || !pfn_valid(__phys_to_pfn(phys + off))) + continue; + + iommu_debug_dec_page(phys); + } } =20 void __iommu_debug_remap(struct iommu_domain *domain, unsigned long iova, = size_t size) { + size_t off; + size_t page_size =3D iommu_debug_page_size(domain); + + for (off =3D 0 ; off < size ; off +=3D page_size) { + phys_addr_t phys =3D iommu_iova_to_phys(domain, iova + off); + + if (!phys || !pfn_valid(__phys_to_pfn(phys + off))) + continue; + + iommu_debug_inc_page(phys); + } } =20 void iommu_debug_init(void) --=20 2.51.2.1026.g39e6a42477-goog From nobody Fri Dec 19 17:36:14 2025 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38E6E34BA53 for ; Thu, 6 Nov 2025 16:40:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447217; cv=none; b=h6js9MffG+U8pCF8n1tUEdDLLvJwCFHNdmrJIQAT8qhJ/niMl55rK9S4utzyoc9LZg9RvHoHXG6OSbvwA6J8GuzodVmtFalGZZso5HwX/3MUU6WoFxzoVaY1e40Vm4q59Sc+yCRByIvNDFQ4Fgy3Ri+U/8s3gGF5bM3yc+MdLbY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762447217; c=relaxed/simple; bh=QPnuTeoHkJAdsy3GLHh3OinUFPR46uTzV5UBwbwi06c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GVXCpeMKPXBSma3D0UPGuhxNwKbL4X1kc2HhQJ38FMWWI5VW/l52DA7eJiVvDzvEXIz7ianbfGjhC+uEg66isqvtP6dqXPleA+B22aVq/52711F2bzeqgna0YYrm6Xn6GIs9RDj5tIKFoHECJlBJU19s9V6FTVngKE481KENt7w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=k3WylvHt; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="k3WylvHt" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-429f0907e13so517475f8f.1 for ; Thu, 06 Nov 2025 08:40:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1762447212; x=1763052012; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=NM/GZbyZiL8bh5z49/8PTeWfbgNSwSL55H6jI2E+lmQ=; b=k3WylvHth50gK6qApAeHkMJsGWajNky8dUpWySaoNpFfHI1c87KNlb6NqJqH/qO22q fEH2hziWn1rIzGcSi37TvguDB0dp4dVQdo/VjlnK+xHDKT7hRJhjgG6uq3+L1iSSTDT3 L1yy5UksmzIIgMxXPirJvMZkPTEVwqZQIYTyGAd0ryveWarhmdOUxQ8vUZwO7qXWMbzS 7UNKMEdJ+IefjJSh+UiQuyT962HOePlcP12VKOsn0jigriqeW8zQ32oSRxGlJNLu+QSO /511l4eQNRGKr/eT0UcML+jV2OP8k3dOOUiT5dELd/1cDN9ki5ZN4JFne6yJOG8t2xD5 PRpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762447212; x=1763052012; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=NM/GZbyZiL8bh5z49/8PTeWfbgNSwSL55H6jI2E+lmQ=; b=vIRjgBr9vt7MX4sOOilXr1uiqhbihezQPZYl8k22d36NLjuynV4w2+CWZSQ4U3WeY+ We39Y+AemNI6yDM8CMqinkZfFDW02W6lgewm/HNi1wCanR3zysZ+NseqklOsF5q6Wyz2 XvO4OkF8Tfp1H6IUlBz5e+IEIXJ2mcDTD7Scv5Y/7f099sVjczXMcl5orJ6ymVYixSFP xpyPiBtzrLAdR+f4StXKj0IWiYpekjDRK9KQVcDUB/ZDD9P5g/KuRSMxRchWQfFPqGQe pv0jJPuEfvOaZNGkzF4Lo1LDPl9iXnRgd3rjtlh4+pkVRi0VynohRBLSN/N39NEC6v4M jr+w== X-Forwarded-Encrypted: i=1; AJvYcCUWSzfPVJmNh74B9fR5Rr/mLbPqVAMnYiZ0IyCSqU4Do6dBCUCa356MwX9IXNFpXA7g42hKVmU2nL4uB9Q=@vger.kernel.org X-Gm-Message-State: AOJu0YwEj/D8SYW01HdaRSCeLocbLIXefTykVV5elB8mEOAJInnrANe7 u2LaYrHZUZOpnOS0/9jdmfDlytlI9nGRw57vr9IDg0Lh50hJ25pRX4F/9KBZdskv5hNrCJe4+aJ oEmC+7v916cc8yA== X-Google-Smtp-Source: AGHT+IFXE6Qsd8iIgYNiQDlI0sqI8FtMmoIupsqXvZU2YQ+mpshBBiH57cinC3fDVEftkTGmawxtksFJp1JKIw== X-Received: from wrsy17.prod.google.com ([2002:a5d:4ad1:0:b0:429:dc18:dd64]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:5f87:0:b0:429:b7cd:47ff with SMTP id ffacd0b85a97d-429e33083c8mr6867222f8f.40.1762447212631; Thu, 06 Nov 2025 08:40:12 -0800 (PST) Date: Thu, 6 Nov 2025 16:39:53 +0000 In-Reply-To: <20251106163953.1971067-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251106163953.1971067-1-smostafa@google.com> X-Mailer: git-send-email 2.51.2.1026.g39e6a42477-goog Message-ID: <20251106163953.1971067-5-smostafa@google.com> Subject: [PATCH v2 4/4] drivers/iommu-debug-pagealloc: Check mapped/unmapped kernel memory From: Mostafa Saleh To: linux-mm@kvack.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Cc: corbet@lwn.net, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, akpm@linux-foundation.org, vbabka@suse.cz, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, david@redhat.com, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, rppt@kernel.org, Mostafa Saleh , Qinxin Xia Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now, as the page_ext holds count of IOMMU mappings, we can use it to assert that any page allocated/freed is indeed not in the IOMMU. The sanitizer doesn=E2=80=99t protect against mapping/unmapping during this period. However, that=E2=80=99s less harmful as the page is not used by the kernel. Signed-off-by: Mostafa Saleh Tested-by: Qinxin Xia --- drivers/iommu/iommu-debug-pagealloc.c | 19 +++++++++++++++++++ include/linux/iommu-debug-pagealloc.h | 12 ++++++++++++ include/linux/mm.h | 5 +++++ 3 files changed, 36 insertions(+) diff --git a/drivers/iommu/iommu-debug-pagealloc.c b/drivers/iommu/iommu-de= bug-pagealloc.c index 0e14104b971c..5b26c84d3a0e 100644 --- a/drivers/iommu/iommu-debug-pagealloc.c +++ b/drivers/iommu/iommu-debug-pagealloc.c @@ -71,6 +71,25 @@ static size_t iommu_debug_page_size(struct iommu_domain = *domain) return 1UL << __ffs(domain->pgsize_bitmap); } =20 +static unsigned int iommu_debug_page_count(unsigned long phys) +{ + unsigned int ref; + struct page_ext *page_ext =3D get_iommu_page_ext(phys); + struct iommu_debug_metadate *d =3D get_iommu_data(page_ext); + + ref =3D atomic_read(&d->ref); + page_ext_put(page_ext); + return ref; +} + +void __iommu_debug_check_unmapped(const struct page *page, int numpages) +{ + while (numpages--) { + WARN_ON(iommu_debug_page_count(page_to_phys(page))); + page++; + } +} + void __iommu_debug_map(struct iommu_domain *domain, phys_addr_t phys, size= _t size) { size_t off; diff --git a/include/linux/iommu-debug-pagealloc.h b/include/linux/iommu-de= bug-pagealloc.h index 180446d6d86f..84110e4ecfaa 100644 --- a/include/linux/iommu-debug-pagealloc.h +++ b/include/linux/iommu-debug-pagealloc.h @@ -22,6 +22,7 @@ void __iommu_debug_unmap(struct iommu_domain *domain, uns= igned long iova, size_t size); void __iommu_debug_remap(struct iommu_domain *domain, unsigned long iova, size_t size); +void __iommu_debug_check_unmapped(const struct page *page, int numpages); =20 static inline void iommu_debug_map(struct iommu_domain *domain, phys_addr_t phys, size_t size) @@ -44,6 +45,12 @@ static inline void iommu_debug_remap(struct iommu_domain= *domain, __iommu_debug_remap(domain, iova, size); } =20 +static inline void iommu_debug_check_unmapped(const struct page *page, int= numpages) +{ + if (static_branch_unlikely(&iommu_debug_initialized)) + __iommu_debug_check_unmapped(page, numpages); +} + void iommu_debug_init(void); =20 #else @@ -66,6 +73,11 @@ static inline void iommu_debug_init(void) { } =20 +static inline void iommu_debug_check_unmapped(const struct page *page, + int numpages) +{ +} + #endif /* CONFIG_IOMMU_DEBUG_PAGEALLOC */ =20 #endif /* __LINUX_IOMMU_DEBUG_PAGEALLOC_H */ diff --git a/include/linux/mm.h b/include/linux/mm.h index d16b33bacc32..895a60a49120 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -36,6 +36,7 @@ #include #include #include +#include =20 struct mempolicy; struct anon_vma; @@ -3811,12 +3812,16 @@ extern void __kernel_map_pages(struct page *page, i= nt numpages, int enable); #ifdef CONFIG_DEBUG_PAGEALLOC static inline void debug_pagealloc_map_pages(struct page *page, int numpag= es) { + iommu_debug_check_unmapped(page, numpages); + if (debug_pagealloc_enabled_static()) __kernel_map_pages(page, numpages, 1); } =20 static inline void debug_pagealloc_unmap_pages(struct page *page, int nump= ages) { + iommu_debug_check_unmapped(page, numpages); + if (debug_pagealloc_enabled_static()) __kernel_map_pages(page, numpages, 0); } --=20 2.51.2.1026.g39e6a42477-goog