From nobody Tue Dec 2 03:00:10 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C4903321BC for ; Mon, 17 Nov 2025 18:48:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763405319; cv=none; b=X5dyaZBiKAGjhbnOZMaZHeQTAHZtfmUgXnAMkzjTXkqIQPejHl9m+8X+s8aaWQkHGTFbdhHhBBqDHkRabLMeKCR9uiM4jpLSk00ntWI2NhoFDBqUhRpEeodLsKC01K4CjvO923R+VF5ZyNJdk3pH3dOdJkFQUHQt3+V8MFSvDak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763405319; c=relaxed/simple; bh=pOVN4YIln6H9/psIOPh7ht/smthij5U8tZ78Nxr8DpI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OyX/R5Gh0xGvitzAe1q0pFWeK3vFMPvZw1zU+nF84aN2NKwOr5Wzgrc4ihub9D7xbpSIr8UD3KnIrECflTJp8Vav5eBYHLopth89hBN8FKx4P3FqZmTLD8vnekI6UuNHiqobCFPtlsNNuhGY286TWd2w2j6GfF6DM/n3voRddu8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=I+jVEAcx; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="I+jVEAcx" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-477a60a23adso6833395e9.2 for ; Mon, 17 Nov 2025 10:48:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1763405315; x=1764010115; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=86k/mgvI8zrVMpw0/0P0jY4VXuRBlWrQXsQcBMGtbfo=; b=I+jVEAcxv0kkgWr0CZEto/CzKnINcPD+npakUWCSHfCyM9sHQeYsLOWV5fcHTM1o9Y Hi/KwtKlHPfHfkXV7FGCN47Rh1CvzhzzM5a3p6Vhj+rvRWVMCmyaqwfYhAbJoztFQnpG 4cZqtykb/wiuETuAaRB/Fg+qzGQjyk4gpKHOKNv2ASKHa4UQnoTmyG4wQGnMQVLJQDiG Q313M1XY7eAiDYfHKVWGp8SOXttkWheL5e+t4AblKVb06SU2io5jnDaX7AGj1DzTM3Pe IUmS4uBJzqpWthB0Hd8gPLX0j4dA69tSwXZnX8tabJjTBF2RiDMu08C3QxoLENaZHbIY oFOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763405315; x=1764010115; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=86k/mgvI8zrVMpw0/0P0jY4VXuRBlWrQXsQcBMGtbfo=; b=o5rtgy3IcRNnZMQjST6mCUdbziMzRT+ZZGyh5ww7DIHmPmvrGIuNZaXEuIw9W7r0MN aAcrlWvPLeJbLqi3cUZDgl3/wSyuA1kpF+tf99tdv5cOWzRLVInHJS/5MUXTC471MtVw LNaeB3GGBv4fYIM07iJV3uKG0AjqS6Os1BZspZuECIxcADf9V60RGpvBNQG/DKvxwiye tly+F7pmCKHdOpUNXL969whFt9MS031ZzLd7u8gQ8s4drseQdGRvrjySySnpX1koLUhi t5s0bF8MkhPGdPTJMN04HJLSOEzjiCWjmzehVLePM81pNLSTEHt04+Uby4ztr0saaF9D TF5Q== X-Forwarded-Encrypted: i=1; AJvYcCXLuB08yx454hBZ3xoSKwPjlAifcOcTna/38/BpiVQ+rqkCd34k8BGntAWie1aRs7yiPZ9w7EOho7oOnjY=@vger.kernel.org X-Gm-Message-State: AOJu0YyN1d5GZgKWR7WIrYP4URSSpmrLSvssxmzGUkX8ONCumePUV5q9 ujoys6/tT0ZUDc4E0YMs3R7EizpcpycFpnnzGvaKBHHB86b5lIczNNjxQpKpw0/7Q2VmqOrWVqN 7M7cpFlkhnLsTFw== X-Google-Smtp-Source: AGHT+IEj4z2+t+yDF5un/88u8JO0bKDq8uupoiOB9c833JgKVOtSjewqPTAdRZkxIX1uyr5HoS4xTfM/25eAYQ== X-Received: from wmbz8.prod.google.com ([2002:a05:600c:c088:b0:475:df4b:95a9]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4753:b0:477:333a:f71f with SMTP id 5b1f17b1804b1-4778fe60658mr132994865e9.17.1763405315623; Mon, 17 Nov 2025 10:48:35 -0800 (PST) Date: Mon, 17 Nov 2025 18:47:56 +0000 In-Reply-To: <20251117184815.1027271-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251117184815.1027271-1-smostafa@google.com> X-Mailer: git-send-email 2.52.0.rc1.455.g30608eb744-goog Message-ID: <20251117184815.1027271-10-smostafa@google.com> Subject: [PATCH v5 09/27] KVM: arm64: iommu: Shadow host stage-2 page table From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jean-philippe@linaro.org, jgg@ziepe.ca, praan@google.com, danielmentz@google.com, mark.rutland@arm.com, qperret@google.com, tabba@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Create a shadow page table for the IOMMU that shadows the host CPU stage-2 into the IOMMUs to establish DMA isolation. An initial snapshot is created after the driver init, then on every permission change a callback would be called for the IOMMU driver to update the page table. For some cases, an SMMUv3 may be able to share the same page table used with the host CPU stage-2 directly. However, this is too strict and requires changes to the core hypervisor page table code, plus it would require the hypervisor to handle IOMMU page faults. This can be added later as an optimization for SMMUV3. Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/hyp/include/nvhe/iommu.h | 4 ++ arch/arm64/kvm/hyp/nvhe/iommu/iommu.c | 83 ++++++++++++++++++++++++- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 5 ++ 3 files changed, 90 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/iommu.h b/arch/arm64/kvm/hyp/i= nclude/nvhe/iommu.h index 1ac70cc28a9e..219363045b1c 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/iommu.h +++ b/arch/arm64/kvm/hyp/include/nvhe/iommu.h @@ -3,11 +3,15 @@ #define __ARM64_KVM_NVHE_IOMMU_H__ =20 #include +#include =20 struct kvm_iommu_ops { int (*init)(void); + void (*host_stage2_idmap)(phys_addr_t start, phys_addr_t end, int prot); }; =20 int kvm_iommu_init(void); =20 +void kvm_iommu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, + enum kvm_pgtable_prot prot); #endif /* __ARM64_KVM_NVHE_IOMMU_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/iommu/iommu.c b/arch/arm64/kvm/hyp/nvh= e/iommu/iommu.c index a01c036c55be..414bd4c97690 100644 --- a/arch/arm64/kvm/hyp/nvhe/iommu/iommu.c +++ b/arch/arm64/kvm/hyp/nvhe/iommu/iommu.c @@ -4,15 +4,94 @@ * * Copyright (C) 2022 Linaro Ltd. */ +#include + #include +#include +#include =20 /* Only one set of ops supported */ struct kvm_iommu_ops *kvm_iommu_ops; =20 +/* Protected by host_mmu.lock */ +static bool kvm_idmap_initialized; + +static inline int pkvm_to_iommu_prot(enum kvm_pgtable_prot prot) +{ + int iommu_prot =3D 0; + + if (prot & KVM_PGTABLE_PROT_R) + iommu_prot |=3D IOMMU_READ; + if (prot & KVM_PGTABLE_PROT_W) + iommu_prot |=3D IOMMU_WRITE; + if (prot =3D=3D PKVM_HOST_MMIO_PROT) + iommu_prot |=3D IOMMU_MMIO; + + /* We don't understand that, might be dangerous. */ + WARN_ON(prot & ~PKVM_HOST_MEM_PROT); + return iommu_prot; +} + +static int __snapshot_host_stage2(const struct kvm_pgtable_visit_ctx *ctx, + enum kvm_pgtable_walk_flags visit) +{ + u64 start =3D ctx->addr; + kvm_pte_t pte =3D *ctx->ptep; + u32 level =3D ctx->level; + u64 end =3D start + kvm_granule_size(level); + int prot =3D IOMMU_READ | IOMMU_WRITE; + + /* Keep unmapped. */ + if (pte && !kvm_pte_valid(pte)) + return 0; + + if (kvm_pte_valid(pte)) + prot =3D pkvm_to_iommu_prot(kvm_pgtable_stage2_pte_prot(pte)); + else if (!addr_is_memory(start)) + prot |=3D IOMMU_MMIO; + + kvm_iommu_ops->host_stage2_idmap(start, end, prot); + return 0; +} + +static int kvm_iommu_snapshot_host_stage2(void) +{ + int ret; + struct kvm_pgtable_walker walker =3D { + .cb =3D __snapshot_host_stage2, + .flags =3D KVM_PGTABLE_WALK_LEAF, + }; + struct kvm_pgtable *pgt =3D &host_mmu.pgt; + + hyp_spin_lock(&host_mmu.lock); + ret =3D kvm_pgtable_walk(pgt, 0, BIT(pgt->ia_bits), &walker); + /* Start receiving calls to host_stage2_idmap. */ + kvm_idmap_initialized =3D !ret; + hyp_spin_unlock(&host_mmu.lock); + + return ret; +} + int kvm_iommu_init(void) { - if (!kvm_iommu_ops || !kvm_iommu_ops->init) + int ret; + + if (!kvm_iommu_ops || !kvm_iommu_ops->init || + !kvm_iommu_ops->host_stage2_idmap) return -ENODEV; =20 - return kvm_iommu_ops->init(); + ret =3D kvm_iommu_ops->init(); + if (ret) + return ret; + return kvm_iommu_snapshot_host_stage2(); +} + +void kvm_iommu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, + enum kvm_pgtable_prot prot) +{ + hyp_assert_lock_held(&host_mmu.lock); + + if (!kvm_idmap_initialized) + return; + kvm_iommu_ops->host_stage2_idmap(start, end, pkvm_to_iommu_prot(prot)); } diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index c3eac0da7cbe..f60acfb868d0 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -15,6 +15,7 @@ #include =20 #include +#include #include #include #include @@ -529,6 +530,7 @@ static void __host_update_page_state(phys_addr_t addr, = u64 size, enum pkvm_page_ int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id) { int ret; + enum kvm_pgtable_prot prot; =20 if (!range_is_memory(addr, addr + size)) return -EPERM; @@ -538,6 +540,9 @@ int host_stage2_set_owner_locked(phys_addr_t addr, u64 = size, u8 owner_id) if (ret) return ret; =20 + prot =3D owner_id =3D=3D PKVM_ID_HOST ? PKVM_HOST_MEM_PROT : 0; + kvm_iommu_host_stage2_idmap(addr, addr + size, prot); + /* Don't forget to update the vmemmap tracking for the host */ if (owner_id =3D=3D PKVM_ID_HOST) __host_update_page_state(addr, size, PKVM_PAGE_OWNED); --=20 2.52.0.rc1.455.g30608eb744-goog