From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C125944A71E for ; Wed, 15 Jul 2026 11:59:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116757; cv=none; b=DkXH4KDYQAMdiIsCRU1UfOMzOJIqbb/Mr4cl9kQG1xQnTyizHZeO8aWgowvpeXVcpPLwKS9OTkrYqjqgVmCsLCQ/+S+du0dK3KiH5pWtIEfIKSkT8y7Fi95BmkkvuMuub8pA6gwUqkLZP7B+bDyQl8PzRZPp7t9fFM25tXWlRUU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116757; c=relaxed/simple; bh=l/hpWbBATRgNusqw4IkGxPNUk54fHjitd3cZTYOU0Us=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gT3Dn9Mv4RDfPGdjio6ACSaz7KCgetssvuG7e4F7DzWbDOBh5PVC+yZnzoS9gE61fF3gEW+QSbjkRRsDFx480B4EM6nz3K0AUvNdocodRoSt9dHMLOw1x3TLuCiUZJV4OSBablnfIkcq3K0gJfneFZ9aiNWdCpTs+nfP95T32MY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SraMscVO; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SraMscVO" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493ba771d1bso20790815e9.2 for ; Wed, 15 Jul 2026 04:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116754; x=1784721554; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ol5Kv9IuMOC5gLvUo+ftrDGNdodUvogtRoPdfqHnaTA=; b=SraMscVOoAd1w8aKHKa2jtiScu0h502C/ylEe2qsMEbmamfiaJt4T86/2pN0MxAIYX V2XsIrpebCuiLOo7WIMGOGx/rcX67i5GbGzDRXErGKsyGnRiOLB4ECmKBJfoc3uxH8A3 f/cQ0f7k9VF3NrVdkAxFQUuScb07toAbhjchPEQJq9T4Dx0mptGBegb02+eDd9gcO3FO aHBa4NlOB6893W06EqL7ZsbGhhhFz+hqeHmCj9u0JHGCBoNP+WpbZ/l+BMomz6178lre yZO1ygQKudPw3tLvmzALYflGN0ZZIkEgiDHekEj9ck60VVpp/ooVISBXrKp4V0NYyuBp rg+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116754; x=1784721554; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ol5Kv9IuMOC5gLvUo+ftrDGNdodUvogtRoPdfqHnaTA=; b=a5wNKMdonYU/7kgsUvl3nQSbXs36eoNkZgDrxBMwTBryOO9tBDIsrvrhZhmbPXC4jg iz/Uh0nvo8RIJcDegUom2IXJpRTo3OgdzDAhTC6V+hs38JxCo3zMyFRtRlFcuTBKJOeu R8ZgF2PfbuuGV5teJmIWTwW28/opH7Zk68kAiAg/5OyKxDXTSWCMLQ0CZYTo+QsesUX8 z2KZwddp0n4t3LYYOz/CIQ3dvd8oeFZgvVRSLmN3HizjbRzSE/Emt73yidN3IbAOJ4jH GTSwFXb3RJI5OyjIuUzlpac31erFu1vfCpfWF9AyjDI1f/N02MPVxr8naS1GuBm2TBbR Gelg== X-Forwarded-Encrypted: i=1; AHgh+RqCYDXHFanJPzqyS45ytxyIcjfcFhU8uGAfjlWcWjTG7CCGiFB9UajcEwrAu5TjXXQC+lTlxK+G1ZuYlsQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yzs38ns8m4PrdVVPpEYYrs0rf3W3OjDzrFdNwPyhzVzv06yeQEi ldgaSnBwD++HrF/2xLZUgpbttb0u1No19P7qRWcp/IvbUTa7HcxM3PuGtDzDAf9zY3DyKsmbkYF MrYskj6s0JqXXmg== X-Received: from wmbe13.prod.google.com ([2002:a05:600c:590d:b0:493:bb14:ecfa]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3f06:b0:495:3c8c:c16a with SMTP id 5b1f17b1804b1-4953c8cc25bmr27537525e9.0.1784116753865; Wed, 15 Jul 2026 04:59:13 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:42 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-2-smostafa@google.com> Subject: [PATCH v7 01/24] KVM: arm64: Add a generic clock From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" IOMMU drivers need to track time, mainly for timeouts. Add 2 new functions to nvhe/clock.c: - hyp_clock_init(): Get the system timer frequency at boot - hyp_clock_ns(): Get current time in nano seconds. This is mainly used for timeouts, so a malicious host can DoS the system or cause premature timeouts which likely end up in hyp panic, that should be acceptable as neither of those would undermine the security guarantees. Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/hyp/include/nvhe/clock.h | 3 +++ arch/arm64/kvm/hyp/nvhe/Makefile | 4 ++-- arch/arm64/kvm/hyp/nvhe/clock.c | 29 +++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/setup.c | 5 +++++ 4 files changed, 39 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/clock.h b/arch/arm64/kvm/hyp/i= nclude/nvhe/clock.h index ae03ec6965af..7ef982939bf9 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/clock.h +++ b/arch/arm64/kvm/hyp/include/nvhe/clock.h @@ -13,4 +13,7 @@ static inline void trace_hyp_clock_update(u32 mult, u32 shift, u64 epoch_ns, u64 epoch_cyc) {= } static inline u64 trace_hyp_clock(void) { return 0; } #endif + +u64 hyp_clock_ns(void); +int hyp_clock_init(void); #endif diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Mak= efile index f57450ebcb49..7c879292974d 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -26,10 +26,10 @@ hyp-obj-y :=3D timer-sr.o sysreg-sr.o debug-sr.o switch= .o tlb.o hyp-init.o host.o hyp-main.o hyp-smp.o psci-relay.o early_alloc.o page_alloc.o \ cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o hyp-obj-y +=3D ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../en= try.o \ - ../hyp-entry.o ../exception.o ../pgtable.o ../vgic-v5-sr.o + ../hyp-entry.o ../exception.o ../pgtable.o ../vgic-v5-sr.o clock.o hyp-obj-y +=3D ../../../kernel/smccc-call.o hyp-obj-$(CONFIG_LIST_HARDENED) +=3D list_debug.o -hyp-obj-$(CONFIG_NVHE_EL2_TRACING) +=3D clock.o trace.o events.o +hyp-obj-$(CONFIG_NVHE_EL2_TRACING) +=3D trace.o events.o hyp-obj-y +=3D $(lib-objs) =20 # Path to simple_ring_buffer.c diff --git a/arch/arm64/kvm/hyp/nvhe/clock.c b/arch/arm64/kvm/hyp/nvhe/cloc= k.c index f3e2619db4e4..43d2cba4f810 100644 --- a/arch/arm64/kvm/hyp/nvhe/clock.c +++ b/arch/arm64/kvm/hyp/nvhe/clock.c @@ -8,7 +8,12 @@ =20 #include #include +#include +#include =20 +static u32 timer_freq; + +#ifdef CONFIG_NVHE_EL2_TRACING static struct clock_data { struct { u32 mult; @@ -66,3 +71,27 @@ u64 trace_hyp_clock(void) =20 return (u64)ns + clock->data[bank].epoch_ns; } +#endif /* CONFIG_NVHE_EL2_TRACING */ + +int hyp_clock_init(void) +{ + timer_freq =3D read_sysreg(cntfrq_el0); + /* + * KVM will not initialize if FW didn't set cntfrq_el0, that is already + * part of the boot protocol. + */ + if (!timer_freq) + return -ENODEV; + + /* Timer freq can't be larger than 1Ghz by spec. */ + if (timer_freq > NSEC_PER_SEC) + return -EINVAL; + + return 0; +} + +/* Return time in ns. */ +u64 hyp_clock_ns(void) +{ + return mul_u64_u32_div(__arch_counter_get_cntvct(), NSEC_PER_SEC, timer_f= req); +} diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index 75b00c323310..970c5cf342f5 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -10,6 +10,7 @@ #include #include =20 +#include #include #include #include @@ -320,6 +321,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; =20 + ret =3D hyp_clock_init(); + if (ret) + goto out; + ret =3D fix_host_ownership(); if (ret) goto out; --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E4D944C645 for ; Wed, 15 Jul 2026 11:59:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116759; cv=none; b=ShFjToL4zbepE7v4HQboAZfUPH3f5X9xR/0n8to90hryawUvY7bKzce/IismYr1XvEYJ+aInkBlBqfqOc6KmMi2LE8DCiBSyLU4VpGREf677QLAP5XXeQZ7OSd4mSojQS2/di0jE3UuZTC+5qfPc+MJGg4f4bCKxuRxdZCcUbwc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116759; c=relaxed/simple; bh=Ep2a0G/52eGsYrTUgu5RVlmketd92FEfjFvjwS2XdcA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OEmO+IUVvVvrPOTjQhhGgONVIztzEdREWNXlIUWuULKpyVllJSsYqqp7hMbJpSnjPPEXSCP9krBNfc6xVAoUiE54+NKsOufgT+uGLlQztiXjktchvViWn4W5fzy/gLv18MvZz8EpaVwsdYeAa+HzrnjhFRgVvGPYkXB4DwJaOCU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uvB7Ph2K; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uvB7Ph2K" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-472a798fc7cso1433118f8f.1 for ; Wed, 15 Jul 2026 04:59:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116755; x=1784721555; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:from:to:cc :subject:date:message-id:reply-to:content-type; bh=M3eZs1fgfcJqE0urG21tDoGKs0yo1icKieB+aaaZBxk=; b=uvB7Ph2KylqRAdX4NxDpeZh1LgkxyWVVNfhEW73iggiMp6x+W8FuaZbYiejaV+VSkx y1umWWf7eMw+vrReBwYHKaIyd+MnyiHZREdKJOT8hWVuYG1DKlQWTMQq+ShDLybawyUm HDsSoF/8ziJtm5a9jwcqBpRfsq1AhRhZm0e9MoiXuSa0dXjP3853YrS47zquUyOV5v+5 3rltwgAgITouNRipEdvVUBelcMhdhwvu0AqdVxCQwXccwsgJBht+G+FyVT+FSEl1rUBv vHurUmdkLsCGQugDgZI0X47uOm+Ws2w9M1ihB/kmX9+hbIdakAG/pWEYw2EZbkp+mj02 4w5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116755; x=1784721555; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=M3eZs1fgfcJqE0urG21tDoGKs0yo1icKieB+aaaZBxk=; b=G6FNvEOa9A4VVtEmHjU6EXZM69Vaz5WWrjjFAwbGw+rlHyS3YuOHV2gzCotRlHJGC7 m5eJ07IaXTpr5jw1TOghIipNLq9vQLATFofCPOH0epiTNbE7p0N4IIfDA8l39ZONqbGX SpaI4TbRojWoOF02OGmXtuAhihXkAudsl+RBmNFt+FLicJrxdZAbHgOQIbwCLBZ5biDj D1Sj5Ls9TWgJab5SpnSzLg8LmcqftnZtDGXXiZFy35XU9feVv664BWiJY8j5eMdpn1Lh 2vF5CubTrxCjIeLlUIyZI2oThx1JVrKDt5kz9TMrs2ggOLB4tBK9fR9mDCs23/BDs1Z1 aNfA== X-Forwarded-Encrypted: i=1; AHgh+Rrop/ZSpow782lktMEcR4TBenbVcYDAksv6C4ATIEkxi97FOR7PV5Cy+JSkZjwormDMg2ZUDkwLqXYBAtA=@vger.kernel.org X-Gm-Message-State: AOJu0YxH83BykUXeV/0xPN+EqyI4B+WhZ2Fu97SlBVMLT1sfEP+ErjnJ cZXGz0J0igCQybEW43qwmfvwvB5EGhWAG/H0vpcuJuZgCfa3u5FmgG0BXjWZpxpOcRKYgUWY2ih 4rUQbW9KJilPg/g== X-Received: from wmdn11.prod.google.com ([2002:a05:600c:294b:b0:494:415:3ecd]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:621b:b0:493:faf3:3ead with SMTP id 5b1f17b1804b1-4952cc3c566mr74288995e9.4.1784116755156; Wed, 15 Jul 2026 04:59:15 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:43 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-3-smostafa@google.com> Subject: [PATCH v7 02/24] KVM: arm64: Donate MMIO to the hypervisor From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a function to donate MMIO to the hypervisor so IOMMU hypervisor drivers can protect and access the MMIO of IOMMUs. As donating MMIO is very rare, and we don=E2=80=99t need to encode the full state, it=E2=80=99s reasonable to have a separate function to do this. It will init the host s2 page table with an invalid leaf with the owner ID to prevent the host from mapping the page on faults. Also, prevent kvm_pgtable_stage2_unmap() from removing owner ID from stage-2 PTEs, as this can be triggered from recycle logic under memory pressure. There is no code relying on this, as all ownership changes is done via kvm_pgtable_stage2_set_owner() For the error path in IOMMU drivers, add a function to donate MMIO back from hyp to host. However, that leaks the hypervisor virtual address range which should be acceptable as this is quite rare and it matches the behaviour of fix_map/block. Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 7 ++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 91 ++++++++++++++++++- arch/arm64/kvm/hyp/pgtable.c | 11 +-- 3 files changed, 102 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm= /hyp/include/nvhe/mem_protect.h index 29935c7da1de..51b0eb3844a9 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -36,6 +36,13 @@ int __pkvm_guest_share_host(struct pkvm_hyp_vcpu *vcpu, = u64 gfn); int __pkvm_guest_unshare_host(struct pkvm_hyp_vcpu *vcpu, u64 gfn); int __pkvm_host_unshare_hyp(u64 pfn); int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages); +/* + * Donate MMIO range to the hypervisor, it will be mapped in the hyperviso= r's + * private range and unmapped from the host stage-2. + */ +int __pkvm_host_donate_hyp_mmio(phys_addr_t addr, size_t size, unsigned lo= ng *haddr); +/* Remaps MMIO range in the host, typically used in error path. */ +int __pkvm_hyp_donate_host_mmio(phys_addr_t addr, size_t size); int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages); int __pkvm_host_share_ffa(u64 pfn, u64 nr_pages); int __pkvm_host_unshare_ffa(u64 pfn, u64 nr_pages); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 4e329e39a695..d803b3dd4cb4 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -378,7 +378,11 @@ static int host_stage2_unmap_dev_all(void) u64 addr =3D 0; int i, ret; =20 - /* Unmap all non-memory regions to recycle the pages */ + /* + * Unmap all non-memory regions to recycle the pages. + * That relies on kvm_pgtable_stage2_unmap() not clearing + * counted PTEs which include hypervisor MMIO. + */ for (i =3D 0; i < hyp_memblock_nr; i++, addr =3D reg->base + reg->size) { reg =3D &hyp_memory[i]; ret =3D kvm_pgtable_stage2_unmap(pgt, addr, reg->base - addr); @@ -1119,6 +1123,91 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) return ret; } =20 +int __pkvm_host_donate_hyp_mmio(phys_addr_t addr, size_t size, unsigned lo= ng *haddr) +{ + kvm_pte_t pte; + u64 offset; + int ret; + + /* Only before de-privilege. */ + if (static_branch_unlikely(&kvm_protected_mode_initialized)) + return -EPERM; + + if (!PAGE_ALIGNED(addr | size) || + !pfn_range_is_valid(hyp_phys_to_pfn(addr), size >> PAGE_SHIFT)) + return -EINVAL; + + ret =3D __pkvm_create_private_mapping(addr, size, PAGE_HYP_DEVICE, haddr); + if (ret) + return ret; + + host_lock_component(); + for (offset =3D 0; offset < size; offset +=3D PAGE_SIZE) { + if (addr_is_memory(addr + offset)) { + ret =3D -EINVAL; + goto unlock; + } + ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, addr + offset, &pte, NULL); + if (ret) + goto unlock; + if (pte && !kvm_pte_valid(pte)) { + ret =3D -EPERM; + goto unlock; + } + } + /* + * We set HYP as the owner of the MMIO pages in the host stage-2, for: + * - host aborts: host_stage2_adjust_range() would fail for invalid non z= ero PTEs. + * - recycle under memory pressure: host_stage2_unmap_dev_all() would call + * kvm_pgtable_stage2_unmap() which will not clear non zero invalid pte= s (counted). + * - other MMIO donation: Would fail as we check that the PTE is valid or= empty. + */ + ret =3D host_stage2_try(kvm_pgtable_stage2_annotate, &host_mmu.pgt, + addr, size, &host_s2_pool, + KVM_HOST_INVALID_PTE_TYPE_DONATION, + FIELD_PREP(KVM_HOST_DONATION_PTE_OWNER_MASK, PKVM_ID_HYP)); +unlock: + host_unlock_component(); + return ret; +} + +int __pkvm_hyp_donate_host_mmio(phys_addr_t addr, size_t size) +{ + kvm_pte_t pte; + u64 offset; + int ret =3D 0; + + if (static_branch_unlikely(&kvm_protected_mode_initialized)) + return -EPERM; + + if (!PAGE_ALIGNED(addr | size) || + !pfn_range_is_valid(hyp_phys_to_pfn(addr), size >> PAGE_SHIFT)) + return -EINVAL; + + host_lock_component(); + for (offset =3D 0; offset < size; offset +=3D PAGE_SIZE) { + if (addr_is_memory(addr + offset)) { + ret =3D -EINVAL; + goto unlock; + } + ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, addr + offset, &pte, NULL); + if (ret) + goto unlock; + if (!pte || kvm_pte_valid(pte)) { + ret =3D -EINVAL; + goto unlock; + } + if (FIELD_GET(KVM_HOST_DONATION_PTE_OWNER_MASK, pte) !=3D PKVM_ID_HYP) { + ret =3D -EPERM; + goto unlock; + } + } + WARN_ON(host_stage2_idmap_locked(addr, size, PKVM_HOST_MMIO_PROT)); +unlock: + host_unlock_component(); + return ret; +} + int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) { u64 phys =3D hyp_pfn_to_phys(pfn); diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 91a7dfad6686..3073184cf6ad 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1161,13 +1161,12 @@ static int stage2_unmap_walker(const struct kvm_pgt= able_visit_ctx *ctx, kvm_pte_t *childp =3D NULL; bool need_flush =3D false; =20 - if (!kvm_pte_valid(ctx->old)) { - if (stage2_pte_is_counted(ctx->old)) { - kvm_clear_pte(ctx->ptep); - mm_ops->put_page(ctx->ptep); - } + /* + * That also ignores stage2_pte_is_counted() instead of clearing + * the PTE as the MMIO can be owned by the hypervisor. + */ + if (!kvm_pte_valid(ctx->old)) return 0; - } =20 if (kvm_pte_table(ctx->old, ctx->level)) { childp =3D kvm_pte_follow(ctx->old, mm_ops); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8624E44C66F for ; Wed, 15 Jul 2026 11:59:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116761; cv=none; b=DoW7CPVzxBVKnpUDbKUcDjiKqa/8eV+ecqZG2KdLwZyw1rMm00TiWBHT3X9avPx5m5hvpGEXduU0DPO/6Rix+TuvDE1W1Z3I5HGXfykX1b0FbrjVjSEHzZIUMt8DXQTvVirD/1Hxf+o0CehHsvuRg2KjFGpXdQMx4MucPtB2rZg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116761; c=relaxed/simple; bh=6CKFHoNklLN4PCvtNpy9VDBy22n6jqUSWu7UMjm+Lpo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ahVLNGcjA9H3CCLA8w3Ow8Zwq4eLsm1bR572QcdiClmArc0eZH4W8T8I6J4TAUm8AEdEa1qf0YYFSlU6gXxZpNQ5Tsvrwrb7uaRTaPvsIcwikoNYA+uovPo9DaQ8Lli8mZfJnaTweVziuLzhO+r7sKhUfVpb3JOPvGM0oVaJRMk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=G+MmfX+r; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="G+MmfX+r" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493dc8408fdso16995215e9.1 for ; Wed, 15 Jul 2026 04:59:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116757; x=1784721557; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=KxyxCQ/BZUglxoL2L1s1Xxp7lH/IsK4WtW57NYwUHLs=; b=G+MmfX+rSJR5fh9h5huD8DlQP+XnOtYTpkN5gkmIGW9T1uwtx9G5LR/mMTbccTEMQq C9ZJAD/IK1GCiLJgozTA1bxzL46s9uh7+FqAzeZvLphl0yIBSIDoEeAZxukjmb5q0DyZ kLCCcbKaPfjDd44Jl1tJvkVTJY0V3Ogz2N84mKMU8FlVqv0h8Dv9X3KDHgBnagbXwUBc Rag+zS3WURYehlYLXM4JYQc1INvDK30NVgsId8EIj7/0BayPHo7A0DZO1fIE9cKKjYsw z1aKrcBA8X0AJkR3a5ocm+vDt1vzbWQtUJAMreiS0jmfuIXt0wuSE0rVDKWTPijZqMDS RqXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116757; x=1784721557; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=KxyxCQ/BZUglxoL2L1s1Xxp7lH/IsK4WtW57NYwUHLs=; b=ETPIUqP3axw/pYlWYPSPVIBJuOPGdmuXJnUklRtJnSXLoYdwVLrxNirfTDNogl7mLY ywDHIxB5aSzY9sn+7033d2DDIY9LrQOo1ugaT8BVmOVIamhs33sW9ZijRSPgt11yGUQb cCamYXflqpvn6o/lOP417OeH5mrv6n6sUwwaRoCAVq90CumtFoRvuX8T05IKAAcMebM1 H0uJwgiy8ucctA7Fqoceyh9osB4FGo3NjqKgl2papBUQjl2mPDVbsUmN/JcK78tx/iW4 l+GREl2S/gBzvra1S4v9f3aziImurPB/zpRqvvJtS8QDZRz3PmDAHDj37vM42uxp+7gE tkdg== X-Forwarded-Encrypted: i=1; AHgh+RrwkIUVGfhNfPbOnDrTxjR9GCPu3aVTGHH6rSWZ8Mqvk6rQqFxJxMji6e7T/HDmGu6gXJt0g5LC32OPw/g=@vger.kernel.org X-Gm-Message-State: AOJu0Yxz1qa+noU2YDiG+VXubu59r3BM1FAdMEo+yY/+k1eSbDDyZ3B9 Yi9lBMeW/lHWzfsxQCGWv5vzAMbU7jezRedRvWaxA/Vi7/dddhXsuuNvMdU7bLwE47w+nie/Hvb 8WyYAbMPREWZYvA== X-Received: from wmbez3.prod.google.com ([2002:a05:600c:83c3:b0:493:c274:1b7b]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600d:8496:10b0:492:5bb6:6d4b with SMTP id 5b1f17b1804b1-493f8833b6bmr131609825e9.34.1784116756444; Wed, 15 Jul 2026 04:59:16 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:44 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-4-smostafa@google.com> Subject: [PATCH v7 03/24] iommu/arm-smmu-v3: Split code with hyp From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The KVM SMMUv3 driver would re-use some of the cmdq and ste code inside the hypervisor, move these functions to the header file that is shared between the host kernel and the hypervisor. Signed-off-by: Mostafa Saleh --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 59 -------------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 60 +++++++++++++++++++++ 2 files changed, 60 insertions(+), 59 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index a10affb483a4..2a2289d4a3f3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -125,33 +125,6 @@ static void parse_driver_options(struct arm_smmu_devic= e *smmu) } =20 /* Low-level queue manipulation functions */ -static bool queue_has_space(struct arm_smmu_ll_queue *q, u32 n) -{ - u32 space, prod, cons; - - prod =3D Q_IDX(q, q->prod); - cons =3D Q_IDX(q, q->cons); - - if (Q_WRP(q, q->prod) =3D=3D Q_WRP(q, q->cons)) - space =3D (1 << q->max_n_shift) - (prod - cons); - else - space =3D cons - prod; - - return space >=3D n; -} - -static bool queue_full(struct arm_smmu_ll_queue *q) -{ - return Q_IDX(q, q->prod) =3D=3D Q_IDX(q, q->cons) && - Q_WRP(q, q->prod) !=3D Q_WRP(q, q->cons); -} - -static bool queue_empty(struct arm_smmu_ll_queue *q) -{ - return Q_IDX(q, q->prod) =3D=3D Q_IDX(q, q->cons) && - Q_WRP(q, q->prod) =3D=3D Q_WRP(q, q->cons); -} - static bool queue_consumed(struct arm_smmu_ll_queue *q, u32 prod) { return ((Q_WRP(q, q->cons) =3D=3D Q_WRP(q, prod)) && @@ -170,12 +143,6 @@ static void queue_sync_cons_out(struct arm_smmu_queue = *q) writel_relaxed(q->llq.cons, q->cons_reg); } =20 -static void queue_inc_cons(struct arm_smmu_ll_queue *q) -{ - u32 cons =3D (Q_WRP(q, q->cons) | Q_IDX(q, q->cons)) + 1; - q->cons =3D Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons); -} - static void queue_sync_cons_ovf(struct arm_smmu_queue *q) { struct arm_smmu_ll_queue *llq =3D &q->llq; @@ -207,12 +174,6 @@ static int queue_sync_prod_in(struct arm_smmu_queue *q) return ret; } =20 -static u32 queue_inc_prod_n(struct arm_smmu_ll_queue *q, int n) -{ - u32 prod =3D (Q_WRP(q, q->prod) | Q_IDX(q, q->prod)) + n; - return Q_OVF(q->prod) | Q_WRP(q, prod) | Q_IDX(q, prod); -} - static void queue_poll_init(struct arm_smmu_device *smmu, struct arm_smmu_queue_poll *qp) { @@ -240,14 +201,6 @@ static int queue_poll(struct arm_smmu_queue_poll *qp) return 0; } =20 -static void queue_write(__le64 *dst, u64 *src, size_t n_dwords) -{ - int i; - - for (i =3D 0; i < n_dwords; ++i) - *dst++ =3D cpu_to_le64(*src++); -} - static void queue_read(u64 *dst, __le64 *src, size_t n_dwords) { int i; @@ -1731,18 +1684,6 @@ static void arm_smmu_free_cd_tables(struct arm_smmu_= master *master) } =20 /* Stream table manipulation functions */ -static void arm_smmu_write_strtab_l1_desc(struct arm_smmu_strtab_l1 *dst, - dma_addr_t l2ptr_dma) -{ - u64 val =3D 0; - - val |=3D FIELD_PREP(STRTAB_L1_DESC_SPAN, STRTAB_SPLIT + 1); - val |=3D l2ptr_dma & STRTAB_L1_DESC_L2PTR_MASK; - - /* The HW has 64 bit atomicity with stores to the L2 STE table */ - WRITE_ONCE(dst->l2ptr, cpu_to_le64(val)); -} - struct arm_smmu_ste_writer { struct arm_smmu_entry_writer writer; u32 sid; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index c909c9a88538..e93489841ec7 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -1212,6 +1212,66 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_devi= ce *smmu, struct arm_smmu_cmd *cmds, int n, bool sync); =20 +/* Queue functions shared between kernel and hyp. */ +static inline bool queue_has_space(struct arm_smmu_ll_queue *q, u32 n) +{ + u32 space, prod, cons; + + prod =3D Q_IDX(q, q->prod); + cons =3D Q_IDX(q, q->cons); + + if (Q_WRP(q, q->prod) =3D=3D Q_WRP(q, q->cons)) + space =3D (1 << q->max_n_shift) - (prod - cons); + else + space =3D cons - prod; + + return space >=3D n; +} + +static inline bool queue_full(struct arm_smmu_ll_queue *q) +{ + return Q_IDX(q, q->prod) =3D=3D Q_IDX(q, q->cons) && + Q_WRP(q, q->prod) !=3D Q_WRP(q, q->cons); +} + +static inline bool queue_empty(struct arm_smmu_ll_queue *q) +{ + return Q_IDX(q, q->prod) =3D=3D Q_IDX(q, q->cons) && + Q_WRP(q, q->prod) =3D=3D Q_WRP(q, q->cons); +} + +static inline u32 queue_inc_prod_n(struct arm_smmu_ll_queue *q, int n) +{ + u32 prod =3D (Q_WRP(q, q->prod) | Q_IDX(q, q->prod)) + n; + return Q_OVF(q->prod) | Q_WRP(q, prod) | Q_IDX(q, prod); +} + +static inline void queue_inc_cons(struct arm_smmu_ll_queue *q) +{ + u32 cons =3D (Q_WRP(q, q->cons) | Q_IDX(q, q->cons)) + 1; + q->cons =3D Q_OVF(q->cons) | Q_WRP(q, cons) | Q_IDX(q, cons); +} + +static inline void queue_write(__le64 *dst, u64 *src, size_t n_dwords) +{ + int i; + + for (i =3D 0; i < n_dwords; ++i) + *dst++ =3D cpu_to_le64(*src++); +} + +static inline void arm_smmu_write_strtab_l1_desc(struct arm_smmu_strtab_l1= *dst, + dma_addr_t l2ptr_dma) +{ + u64 val =3D 0; + + val |=3D FIELD_PREP(STRTAB_L1_DESC_SPAN, STRTAB_SPLIT + 1); + val |=3D l2ptr_dma & STRTAB_L1_DESC_L2PTR_MASK; + + /* The HW has 64 bit atomicity with stores to the L2 STE table */ + WRITE_ONCE(dst->l2ptr, cpu_to_le64(val)); +} + #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); void arm_smmu_sva_notifier_synchronize(void); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F26DB442124 for ; Wed, 15 Jul 2026 11:59:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116761; cv=none; b=TQVfle4ca317WvwS6NZUk5HoSAC4NlyMDaY9efup5iin2Yx8JO1c3+hTyhhaL8hjB5BxNGK2ICQT4wLhexIN6wu5REaBy6JH3ewUH1vuMwDrjjFvFZyMBLEKDwrLFTnanKFuZYKcrOI2EPbKv+cCskxAvMduZtXrW5ryeIikTMg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116761; c=relaxed/simple; bh=1nwreBOIoo8Nq3x/9cqsrE39IlPY4nM+9QYDUvVAOnY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=e7v7aLVOktgRIwFMlWaobEIJwANfLTtX+qJghfnd4LKgfy/TgaA+VaH0SqcEQV5kv91hYl3ZfcQw/ZeMOG7Nvu4jk36L/qKa1Y7kcVaU1/DF+31Zee5ScomDeTMjH/ZFFjvX4H35Zwsc4YRGO1EKpgia66hfTZODzZf5eTWnuOw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pPAtGtfv; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pPAtGtfv" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-475e540a0ffso3108037f8f.3 for ; Wed, 15 Jul 2026 04:59:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116758; x=1784721558; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=116XGPIsH3uc1TP/YR0VtloeUIWM18cD/kEPtzKIUwA=; b=pPAtGtfvcebcKX8QpKkYdZs/AEd+Vmm+gyLXJ/pe8e1rGzsn1wnRPuchS2CwiJlVle a98t+0jsnp/xiRPBgWPlTrN5id1bgjATS3/Secr2BiIc39Ibx6SRSgDd/ukvnkzSI76W ZaKiqmEXXwWPhNJljG5yFmOmrdJrEGRDMZcyLog3+NyJOXgaOXTmJ46LgiJK9dZ1Qdmd 2rxXvjy+rVIKmBVjsWeFmoLRRsnD4+dXq8JjVGImYpwVMhnj61TmnEY4RC9iyfs7YI4z yb5ClI82Cw8q2VcfP6m6R5Lwn5fRmRFJUO8WwPSj4GucRdVjRdu8NYgFKZekST15hdps Urpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116758; x=1784721558; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=116XGPIsH3uc1TP/YR0VtloeUIWM18cD/kEPtzKIUwA=; b=s5xax0dFPgGCnO1yeD/XaWMQ4XlSyYc21nK7RmJ7pge2QHYEDxjCULfYfcKyATjqlw FfiPujeI3EVdhB/IFTSIgBBMhKGq9nJzpeGNsYD2DOGZI+zdhzwtJmcSA/CtD3qPnlHW A5zsg88QTUi1vO7BUKjVfyYHWznEk/5QEdL87FSWyxKtD+k9NvhrRXWDn+/GmBqi9ahQ T8LPALA9C7VQNHl6VJG7BOGVnjE9h4k6wN08y9DAdX8OL1yeaM1ToFlHoJRbIhELNxw/ X9WBzYgc9iJ0r7PctmKC21Jj05imv6czU67Bd4bxUJ69CkijA3XH1mXmy9/2D5JxM0pF ABKA== X-Forwarded-Encrypted: i=1; AHgh+Ro6NV/DJIfzQF4GrJv7fZcvqznpkU8RfcoI+n6g0HETYGMR59NAHzV4DJuqsfWIauVXvX9lUnw4YkkJnUA=@vger.kernel.org X-Gm-Message-State: AOJu0YzvBdYHCoAbfMxpGCvTW2SFF2PNPVkDocINvnVWwmYLoZrYtpds A3uXCE0UIZJ2HevaniQFXAOwQM7WZuG3bD01Tn+HZQMpKQN7TI70fUpssrHO+r2RqhEyIRBfeyF K99INZcqQNqsIoQ== X-Received: from wmap7.prod.google.com ([2002:a7b:cc87:0:b0:487:3739:c5c4]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600d:108:20b0:495:401d:9f4f with SMTP id 5b1f17b1804b1-495401da1a5mr2255395e9.25.1784116757928; Wed, 15 Jul 2026 04:59:17 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:45 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-5-smostafa@google.com> Subject: [PATCH v7 04/24] iommu/arm-smmu-v3: Move TLB range invalidation into common code From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Range TLB invalidation has a very specific algorithm. Instead of re-writing it for the hypervisor, move it to a function that can be re-used. Signed-off-by: Mostafa Saleh --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 79 +++--------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 89 +++++++++++++++++++++ 2 files changed, 102 insertions(+), 66 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 2a2289d4a3f3..cd17a3255312 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2336,81 +2336,28 @@ static void arm_smmu_tlb_inv_context(void *cookie) arm_smmu_domain_inv(smmu_domain); } =20 +static void __arm_smmu_cmdq_batch_add(void *__opaque, + struct arm_smmu_cmdq_batch *cmds, + struct arm_smmu_cmd *cmd) +{ + struct arm_smmu_device *smmu =3D (struct arm_smmu_device *)__opaque; + + arm_smmu_cmdq_batch_add_cmd_p(smmu, cmds, cmd); +} + static void arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu, struct arm_smmu_cmdq_batch *cmds, struct arm_smmu_cmd *cmd, bool leaf, unsigned long iova, size_t size, size_t granule, size_t pgsize) { - unsigned long end =3D iova + size, num_pages =3D 0, tg =3D pgsize; - u64 orig_data0 =3D cmd->data[0]; - size_t inv_range =3D granule; - u8 ttl =3D 0, tg_enc =3D 0; - if (WARN_ON_ONCE(!size)) return; =20 - if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) { - num_pages =3D size >> tg; - - /* Convert page size of 12,14,16 (log2) to 1,2,3 */ - tg_enc =3D (tg - 10) / 2; - - /* - * Determine what level the granule is at. For non-leaf, both - * io-pgtable and SVA pass a nominal last-level granule because - * they don't know what level(s) actually apply, so ignore that - * and leave TTL=3D0. However for various errata reasons we still - * want to use a range command, so avoid the SVA corner case - * where both scale and num could be 0 as well. - */ - if (leaf) - ttl =3D 4 - ((ilog2(granule) - 3) / (tg - 3)); - else if ((num_pages & CMDQ_TLBI_RANGE_NUM_MAX) =3D=3D 1) - num_pages++; - } - - while (iova < end) { - if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) { - /* - * On each iteration of the loop, the range is 5 bits - * worth of the aligned size remaining. - * The range in pages is: - * - * range =3D (num_pages & (0x1f << __ffs(num_pages))) - */ - unsigned long scale, num; - - /* Determine the power of 2 multiple number of pages */ - scale =3D __ffs(num_pages); - - /* Determine how many chunks of 2^scale size we have */ - num =3D (num_pages >> scale) & CMDQ_TLBI_RANGE_NUM_MAX; - - cmd->data[0] =3D orig_data0 | - FIELD_PREP(CMDQ_TLBI_0_NUM, num - 1) | - FIELD_PREP(CMDQ_TLBI_0_SCALE, scale); - - /* range is num * 2^scale * pgsize */ - inv_range =3D num << (scale + tg); - - /* Clear out the lower order bits for the next iteration */ - num_pages -=3D num << scale; - } - - /* - * IPA has fewer bits than VA, but they are reserved in the - * command and something would be very broken if iova had them - * set. - */ - cmd->data[1] =3D FIELD_PREP(CMDQ_TLBI_1_LEAF, leaf) | - FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) | - FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) | - (iova & ~GENMASK_U64(11, 0)); - - arm_smmu_cmdq_batch_add_cmd_p(smmu, cmds, cmd); - iova +=3D inv_range; - } + arm_smmu_tlb_inv_build(cmd, iova, size, granule, pgsize, + smmu->features & ARM_SMMU_FEAT_RANGE_INV, + smmu, leaf, __arm_smmu_cmdq_batch_add, + cmds); } =20 static bool arm_smmu_inv_size_too_big(struct arm_smmu_device *smmu, size_t= size, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index e93489841ec7..494e55a2dc34 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -1272,6 +1272,95 @@ static inline void arm_smmu_write_strtab_l1_desc(str= uct arm_smmu_strtab_l1 *dst, WRITE_ONCE(dst->l2ptr, cpu_to_le64(val)); } =20 +/** + * arm_smmu_tlb_inv_build - Create a range invalidation command + * @cmd: Base command initialized with OPCODE (S1, S2..), vmid and asid + * @iova: Start IOVA to invalidate + * @size: Size of range + * @granule: Granule of invalidation + * @pgsize: Page size of the invalidation + * @is_range: Use range invalidation commands + * @opaque: Pointer to pass to add_cmd + * @leaf: Is leaf invalidation + * @add_cmd: Function to send/batch the invalidation command + * @cmds: Incase of batching, it includes the pointer to the batch + */ +static inline void arm_smmu_tlb_inv_build(struct arm_smmu_cmd *cmd, + unsigned long iova, size_t size, + size_t granule, unsigned long pgsize, + bool is_range, void *opaque, bool leaf, + void (*add_cmd)(void *_opaque, + struct arm_smmu_cmdq_batch *cmds, + struct arm_smmu_cmd *cmd), + struct arm_smmu_cmdq_batch *cmds) +{ + unsigned long end =3D iova + size, num_pages =3D 0, tg =3D pgsize; + u64 orig_data0 =3D cmd->data[0]; + size_t inv_range =3D granule; + u8 ttl =3D 0, tg_enc =3D 0; + + if (is_range) { + num_pages =3D size >> tg; + + /* Convert page size of 12,14,16 (log2) to 1,2,3 */ + tg_enc =3D (tg - 10) / 2; + + /* + * Determine what level the granule is at. For non-leaf, both + * io-pgtable and SVA pass a nominal last-level granule because + * they don't know what level(s) actually apply, so ignore that + * and leave TTL=3D0. However for various errata reasons we still + * want to use a range command, so avoid the SVA corner case + * where both scale and num could be 0 as well. + */ + if (leaf) + ttl =3D 4 - ((ilog2(granule) - 3) / (tg - 3)); + else if ((num_pages & CMDQ_TLBI_RANGE_NUM_MAX) =3D=3D 1) + num_pages++; + } + + while (iova < end) { + if (is_range) { + /* + * On each iteration of the loop, the range is 5 bits + * worth of the aligned size remaining. + * The range in pages is: + * + * range =3D (num_pages & (0x1f << __ffs(num_pages))) + */ + unsigned long scale, num; + + /* Determine the power of 2 multiple number of pages */ + scale =3D __ffs(num_pages); + + /* Determine how many chunks of 2^scale size we have */ + num =3D (num_pages >> scale) & CMDQ_TLBI_RANGE_NUM_MAX; + + cmd->data[0] =3D orig_data0 | + FIELD_PREP(CMDQ_TLBI_0_NUM, num - 1) | + FIELD_PREP(CMDQ_TLBI_0_SCALE, scale); + + /* range is num * 2^scale * pgsize */ + inv_range =3D num << (scale + tg); + + /* Clear out the lower order bits for the next iteration */ + num_pages -=3D num << scale; + } + /* + * IPA has fewer bits than VA, but they are reserved in the + * command and something would be very broken if iova had them + * set. + */ + cmd->data[1] =3D FIELD_PREP(CMDQ_TLBI_1_LEAF, leaf) | + FIELD_PREP(CMDQ_TLBI_1_TTL, ttl) | + FIELD_PREP(CMDQ_TLBI_1_TG, tg_enc) | + (iova & ~GENMASK_U64(11, 0)); + + add_cmd(opaque, cmds, cmd); + iova +=3D inv_range; + } +} + #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); void arm_smmu_sva_notifier_synchronize(void); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7E0C44D68A for ; Wed, 15 Jul 2026 11:59:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116763; cv=none; b=dNR6Qpwr/IBngstFMssp2yPdogoL+Q5AiGO7O3OUDlRC/Xqnz+/zzIjRSahyXaLkU+YcmUtNZ0zDLKimgFaDlsAHxoUGRQDWM7FNg79ADhu+INVhhQV3KIuDrfO4PG5W/iRsKUQWns9YD1mxYaqDSRuZoIExFyewXsYuuL6tTbk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116763; c=relaxed/simple; bh=STqKZu1+JRPT/I5ZbjlCOBueLZbvaon11+KOo0bmUWc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YGrJUOFPr7JP6v9/LknIxk9BwnIvOIbqMh1+h/QhzIej+LhcnAi2qYqoP6EV/26ulbbogExK4m7SO+PKdRp2x4Qr7/hbDgzFfB3V/myJRuZ9az0JPdIRmCQ1wFRBFfPpUehSzRO5C5BNDY3J0/1tvEVlqlxjHx6guR9x6F9fiLU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OMdswSZ3; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OMdswSZ3" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493b21b1fe8so18018515e9.0 for ; Wed, 15 Jul 2026 04:59:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116759; x=1784721559; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=2KYF2oL0lGPGcMYNCJmZKWaj3D1T4zoKMpkf+eN5NFY=; b=OMdswSZ3K+k2Co7Kc9zWiK60Dmta0RWkv1p+ht2EBNwaVr96SepCks4PZKcwplVSs7 NVxSl2hmaUDz5UtnSFpzuCwO++IwBX/ryofVTdY0sPR0mTWBFcxEa1darmxcOZ9qJ+GY UncxzPKUgsKSAvbnHHrpYyCTjFeudB9MMC+1fMk/Xu/DP84j+ybKsz/l3qwDVaMjHIU9 WgF9k/JLHNNJ4PSFRQ6kAThaIslVNltpoOi6650OtfJ4hWAVg+sL4RHBjfa8LEbU9j6V FhV/18CIk33iST9icmogUd/OXfKe3vot6NbdQ+SjFmgMO9+ueOGR0k8dQbi9qXbUFsPn /xFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116759; x=1784721559; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=2KYF2oL0lGPGcMYNCJmZKWaj3D1T4zoKMpkf+eN5NFY=; b=kaSa5pXK7aAvpT0A3/48zmzLBf333Zq/r01twYb2g4/0YD2+36caVoFdmMuj5Qz4se w8a6W0S8a/ipSoQlr+FYM3SnC9jHjaFYqCkLocwVIBpjM1B1A1TMv6aVkhRqFuvx5Q7R ZIhzlB0ezpnacEdGPe5xSSfn/Xw39Nq4GoccNG2qKp8uV3Mq+Lyf54suk+BIJIB4sBUS B132/Tx6+SalEjhAYWUlkqtJHNfJGH8dVTKEGX7OiHPhdIkXWk3T/FN84+mVDhlJ5ZTW JQdPU1sT6jdPNlA3P2UZxvdBIL37/8iZMeEsHKJ2JpuGAwEwWtwQcV0XmzPpOMENL5NY oO/Q== X-Forwarded-Encrypted: i=1; AHgh+RqJfgwI3/1Gp1yoFgLR2tCLguuECc6q7rAki/gqowHLSYQYYQQ+nWRZqKLCC6Frib1RNwX2djxzz/k0JVA=@vger.kernel.org X-Gm-Message-State: AOJu0YwJP1oUmjQkDxYLKsPdDSwdKYK9hcMj112hQajqIhxgEMxo30ua d5RqqViEI5ZNq6I2hPwvTfLymY7eB8raojQGo5jmM1d+8TGxEdKCLNUQPzZTCTGnE320jA51AKf Ax6Z/dgipaMhE4Q== X-Received: from wmap7.prod.google.com ([2002:a7b:cc87:0:b0:487:3739:c5c4]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3f06:b0:495:3c8c:c16a with SMTP id 5b1f17b1804b1-4953c8cc25bmr27539775e9.0.1784116759041; Wed, 15 Jul 2026 04:59:19 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:46 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-6-smostafa@google.com> Subject: [PATCH v7 05/24] iommu/arm-smmu-v3: Move IDR parsing to common functions From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move parsing of IDRs to functions so that it can be re-used from the hypervisor. Signed-off-by: Mostafa Saleh --- drivers/iommu/arm/arm-smmu-v3/Makefile | 2 +- .../arm/arm-smmu-v3/arm-smmu-v3-common-lib.c | 184 ++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 162 ++------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 13 ++ 4 files changed, 215 insertions(+), 146 deletions(-) create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-common-lib.c diff --git a/drivers/iommu/arm/arm-smmu-v3/Makefile b/drivers/iommu/arm/arm= -smmu-v3/Makefile index 493a659cc66b..c9ce392e6d31 100644 --- a/drivers/iommu/arm/arm-smmu-v3/Makefile +++ b/drivers/iommu/arm/arm-smmu-v3/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ARM_SMMU_V3) +=3D arm_smmu_v3.o -arm_smmu_v3-y :=3D arm-smmu-v3.o +arm_smmu_v3-y :=3D arm-smmu-v3.o arm-smmu-v3-common-lib.o arm_smmu_v3-$(CONFIG_ARM_SMMU_V3_IOMMUFD) +=3D arm-smmu-v3-iommufd.o arm_smmu_v3-$(CONFIG_ARM_SMMU_V3_SVA) +=3D arm-smmu-v3-sva.o arm_smmu_v3-$(CONFIG_TEGRA241_CMDQV) +=3D tegra241-cmdqv.o diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-common-lib.c b/drive= rs/iommu/arm/arm-smmu-v3/arm-smmu-v3-common-lib.c new file mode 100644 index 000000000000..ce0ad88b81e3 --- /dev/null +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-common-lib.c @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2015 ARM Limited + * + * Author: Will Deacon + * Arm SMMUv3 driver functions shared with hypervisor. + */ + +#include "arm-smmu-v3.h" +#include + +#include + +u32 smmu_idr0_features(u32 reg) +{ + u32 features =3D 0; + + /* 2-level structures */ + if (FIELD_GET(IDR0_ST_LVL, reg) =3D=3D IDR0_ST_LVL_2LVL) + features |=3D ARM_SMMU_FEAT_2_LVL_STRTAB; + + if (reg & IDR0_CD2L) + features |=3D ARM_SMMU_FEAT_2_LVL_CDTAB; + + /* + * Translation table endianness. + * We currently require the same endianness as the CPU, but this + * could be changed later by adding a new IO_PGTABLE_QUIRK. + */ + switch (FIELD_GET(IDR0_TTENDIAN, reg)) { + case IDR0_TTENDIAN_MIXED: + features |=3D ARM_SMMU_FEAT_TT_LE | ARM_SMMU_FEAT_TT_BE; + break; +#ifdef __BIG_ENDIAN + case IDR0_TTENDIAN_BE: + features |=3D ARM_SMMU_FEAT_TT_BE; + break; +#else + case IDR0_TTENDIAN_LE: + features |=3D ARM_SMMU_FEAT_TT_LE; + break; +#endif + } + + /* Boolean feature flags */ + if (IS_ENABLED(CONFIG_PCI_PRI) && reg & IDR0_PRI) + features |=3D ARM_SMMU_FEAT_PRI; + + if (IS_ENABLED(CONFIG_PCI_ATS) && reg & IDR0_ATS) + features |=3D ARM_SMMU_FEAT_ATS; + + if (reg & IDR0_SEV) + features |=3D ARM_SMMU_FEAT_SEV; + + if (reg & IDR0_MSI) + features |=3D ARM_SMMU_FEAT_MSI; + + if (reg & IDR0_HYP) + features |=3D ARM_SMMU_FEAT_HYP; + + switch (FIELD_GET(IDR0_STALL_MODEL, reg)) { + case IDR0_STALL_MODEL_FORCE: + features |=3D ARM_SMMU_FEAT_STALL_FORCE; + fallthrough; + case IDR0_STALL_MODEL_STALL: + features |=3D ARM_SMMU_FEAT_STALLS; + } + + if (reg & IDR0_S1P) + features |=3D ARM_SMMU_FEAT_TRANS_S1; + + if (reg & IDR0_S2P) + features |=3D ARM_SMMU_FEAT_TRANS_S2; + + if ((features & ARM_SMMU_FEAT_TRANS_S1) && + (features & ARM_SMMU_FEAT_TRANS_S2)) + features |=3D ARM_SMMU_FEAT_NESTING; + + return features; +} + +u32 smmu_idr3_features(u32 reg) +{ + u32 features =3D 0; + + if (FIELD_GET(IDR3_RIL, reg)) + features |=3D ARM_SMMU_FEAT_RANGE_INV; + if (FIELD_GET(IDR3_FWB, reg)) + features |=3D ARM_SMMU_FEAT_S2FWB; + if (FIELD_GET(IDR3_BBM, reg) =3D=3D 2) + features |=3D ARM_SMMU_FEAT_BBML2; + return features; +} + +u32 smmu_idr5_to_oas(u32 reg) +{ + switch (FIELD_GET(IDR5_OAS, reg)) { + case IDR5_OAS_32_BIT: + return 32; + case IDR5_OAS_36_BIT: + return 36; + case IDR5_OAS_40_BIT: + return 40; + case IDR5_OAS_42_BIT: + return 42; + case IDR5_OAS_44_BIT: + return 44; + case IDR5_OAS_48_BIT: + return 48; + case IDR5_OAS_52_BIT: + return 52; + } + return 0; +} + +unsigned long smmu_idr5_to_pgsize(u32 reg) +{ + unsigned long pgsize_bitmap =3D 0; + + if (reg & IDR5_GRAN64K) + pgsize_bitmap |=3D SZ_64K | SZ_512M; + if (reg & IDR5_GRAN16K) + pgsize_bitmap |=3D SZ_16K | SZ_32M; + if (reg & IDR5_GRAN4K) + pgsize_bitmap |=3D SZ_4K | SZ_2M | SZ_1G; + return pgsize_bitmap; +} + +unsigned long smmu_iidr_features(u32 reg, unsigned long features) +{ + unsigned int implementer, productid, variant, revision; + + implementer =3D FIELD_GET(IIDR_IMPLEMENTER, reg); + productid =3D FIELD_GET(IIDR_PRODUCTID, reg); + variant =3D FIELD_GET(IIDR_VARIANT, reg); + revision =3D FIELD_GET(IIDR_REVISION, reg); + + switch (implementer) { + case IIDR_IMPLEMENTER_ARM: + switch (productid) { + case IIDR_PRODUCTID_ARM_MMU_600: + /* Arm erratum 1076982 */ + if (variant =3D=3D 0 && revision <=3D 2) + features &=3D ~ARM_SMMU_FEAT_SEV; + /* Arm erratum 1209401 */ + if (variant < 2) + features &=3D ~ARM_SMMU_FEAT_NESTING; + break; + case IIDR_PRODUCTID_ARM_MMU_700: + /* Many errata... */ + features &=3D ~ARM_SMMU_FEAT_BTM; + if (variant < 1 || revision < 1) { + /* Arm errata 2268618, 2812531 */ + features &=3D ~ARM_SMMU_FEAT_NESTING; + } + break; + case IIDR_PRODUCTID_ARM_MMU_L1: + case IIDR_PRODUCTID_ARM_MMU_S3: + /* Arm errata 3878312/3995052 */ + features &=3D ~ARM_SMMU_FEAT_BTM; + break; + } + break; + } + return features; +} + +unsigned long smmu_iidr_options(u32 reg, unsigned long options) +{ + unsigned int implementer, productid, variant, revision; + + implementer =3D FIELD_GET(IIDR_IMPLEMENTER, reg); + productid =3D FIELD_GET(IIDR_PRODUCTID, reg); + variant =3D FIELD_GET(IIDR_VARIANT, reg); + revision =3D FIELD_GET(IIDR_REVISION, reg); + + if ((implementer =3D=3D IIDR_IMPLEMENTER_ARM) && + (productid =3D=3D IIDR_PRODUCTID_ARM_MMU_700) && + (variant < 1 || revision < 1)) { + /* Arm erratum 2812531 */ + options |=3D ARM_SMMU_OPT_CMDQ_FORCE_SYNC; + } + return options; +} diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index cd17a3255312..674ff98706f6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -4759,52 +4759,13 @@ static int arm_smmu_device_reset(struct arm_smmu_de= vice *smmu) return 0; } =20 -#define IIDR_IMPLEMENTER_ARM 0x43b -#define IIDR_PRODUCTID_ARM_MMU_600 0x483 -#define IIDR_PRODUCTID_ARM_MMU_700 0x487 -#define IIDR_PRODUCTID_ARM_MMU_L1 0x48a -#define IIDR_PRODUCTID_ARM_MMU_S3 0x498 - static void arm_smmu_device_iidr_probe(struct arm_smmu_device *smmu) { u32 reg; - unsigned int implementer, productid, variant, revision; =20 reg =3D readl_relaxed(smmu->base + ARM_SMMU_IIDR); - implementer =3D FIELD_GET(IIDR_IMPLEMENTER, reg); - productid =3D FIELD_GET(IIDR_PRODUCTID, reg); - variant =3D FIELD_GET(IIDR_VARIANT, reg); - revision =3D FIELD_GET(IIDR_REVISION, reg); - - switch (implementer) { - case IIDR_IMPLEMENTER_ARM: - switch (productid) { - case IIDR_PRODUCTID_ARM_MMU_600: - /* Arm erratum 1076982 */ - if (variant =3D=3D 0 && revision <=3D 2) - smmu->features &=3D ~ARM_SMMU_FEAT_SEV; - /* Arm erratum 1209401 */ - if (variant < 2) - smmu->features &=3D ~ARM_SMMU_FEAT_NESTING; - break; - case IIDR_PRODUCTID_ARM_MMU_700: - /* Many errata... */ - smmu->features &=3D ~ARM_SMMU_FEAT_BTM; - if (variant < 1 || revision < 1) { - /* Arm erratum 2812531 */ - smmu->options |=3D ARM_SMMU_OPT_CMDQ_FORCE_SYNC; - /* Arm errata 2268618, 2812531 */ - smmu->features &=3D ~ARM_SMMU_FEAT_NESTING; - } - break; - case IIDR_PRODUCTID_ARM_MMU_L1: - case IIDR_PRODUCTID_ARM_MMU_S3: - /* Arm errata 3878312/3995052 */ - smmu->features &=3D ~ARM_SMMU_FEAT_BTM; - break; - } - break; - } + smmu->features =3D smmu_iidr_features(reg, smmu->features); + smmu->options =3D smmu_iidr_options(reg, smmu->options); } =20 static void arm_smmu_get_httu(struct arm_smmu_device *smmu, u32 reg) @@ -4837,57 +4798,17 @@ static int arm_smmu_device_hw_probe(struct arm_smmu= _device *smmu) /* IDR0 */ reg =3D readl_relaxed(smmu->base + ARM_SMMU_IDR0); =20 - /* 2-level structures */ - if (FIELD_GET(IDR0_ST_LVL, reg) =3D=3D IDR0_ST_LVL_2LVL) - smmu->features |=3D ARM_SMMU_FEAT_2_LVL_STRTAB; - - if (reg & IDR0_CD2L) - smmu->features |=3D ARM_SMMU_FEAT_2_LVL_CDTAB; - - /* - * Translation table endianness. - * We currently require the same endianness as the CPU, but this - * could be changed later by adding a new IO_PGTABLE_QUIRK. - */ - switch (FIELD_GET(IDR0_TTENDIAN, reg)) { - case IDR0_TTENDIAN_MIXED: - smmu->features |=3D ARM_SMMU_FEAT_TT_LE | ARM_SMMU_FEAT_TT_BE; - break; -#ifdef __BIG_ENDIAN - case IDR0_TTENDIAN_BE: - smmu->features |=3D ARM_SMMU_FEAT_TT_BE; - break; -#else - case IDR0_TTENDIAN_LE: - smmu->features |=3D ARM_SMMU_FEAT_TT_LE; - break; -#endif - default: + smmu->features |=3D smmu_idr0_features(reg); + if (!(smmu->features & (ARM_SMMU_FEAT_TT_LE | ARM_SMMU_FEAT_TT_BE))) { dev_err(smmu->dev, "unknown/unsupported TT endianness!\n"); return -ENXIO; } - - /* Boolean feature flags */ - if (IS_ENABLED(CONFIG_PCI_PRI) && reg & IDR0_PRI) - smmu->features |=3D ARM_SMMU_FEAT_PRI; - - if (IS_ENABLED(CONFIG_PCI_ATS) && reg & IDR0_ATS) - smmu->features |=3D ARM_SMMU_FEAT_ATS; - - if (reg & IDR0_SEV) - smmu->features |=3D ARM_SMMU_FEAT_SEV; - - if (reg & IDR0_MSI) { - smmu->features |=3D ARM_SMMU_FEAT_MSI; - if (coherent && !disable_msipolling) - smmu->options |=3D ARM_SMMU_OPT_MSIPOLL; - } - - if (reg & IDR0_HYP) { - smmu->features |=3D ARM_SMMU_FEAT_HYP; - if (cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN)) - smmu->features |=3D ARM_SMMU_FEAT_E2H; - } + if (coherent && !disable_msipolling && + smmu->features & ARM_SMMU_FEAT_MSI) + smmu->options |=3D ARM_SMMU_OPT_MSIPOLL; + if (smmu->features & ARM_SMMU_FEAT_HYP && + cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN)) + smmu->features |=3D ARM_SMMU_FEAT_E2H; =20 arm_smmu_get_httu(smmu, reg); =20 @@ -4899,21 +4820,7 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_= device *smmu) dev_warn(smmu->dev, "IDR0.COHACC overridden by FW configuration (%s)\n", str_true_false(coherent)); =20 - switch (FIELD_GET(IDR0_STALL_MODEL, reg)) { - case IDR0_STALL_MODEL_FORCE: - smmu->features |=3D ARM_SMMU_FEAT_STALL_FORCE; - fallthrough; - case IDR0_STALL_MODEL_STALL: - smmu->features |=3D ARM_SMMU_FEAT_STALLS; - } - - if (reg & IDR0_S1P) - smmu->features |=3D ARM_SMMU_FEAT_TRANS_S1; - - if (reg & IDR0_S2P) - smmu->features |=3D ARM_SMMU_FEAT_TRANS_S2; - - if (!(reg & (IDR0_S1P | IDR0_S2P))) { + if (!(smmu->features & (ARM_SMMU_FEAT_TRANS_S1 | ARM_SMMU_FEAT_TRANS_S2))= ) { dev_err(smmu->dev, "no translation support!\n"); return -ENXIO; } @@ -4972,13 +4879,7 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_= device *smmu) =20 /* IDR3 */ reg =3D readl_relaxed(smmu->base + ARM_SMMU_IDR3); - if (FIELD_GET(IDR3_RIL, reg)) - smmu->features |=3D ARM_SMMU_FEAT_RANGE_INV; - if (FIELD_GET(IDR3_FWB, reg)) - smmu->features |=3D ARM_SMMU_FEAT_S2FWB; - - if (FIELD_GET(IDR3_BBM, reg) =3D=3D 2) - smmu->features |=3D ARM_SMMU_FEAT_BBML2; + smmu->features |=3D smmu_idr3_features(reg); =20 /* IDR5 */ reg =3D readl_relaxed(smmu->base + ARM_SMMU_IDR5); @@ -4987,43 +4888,18 @@ static int arm_smmu_device_hw_probe(struct arm_smmu= _device *smmu) smmu->evtq.max_stalls =3D FIELD_GET(IDR5_STALL_MAX, reg); =20 /* Page sizes */ - if (reg & IDR5_GRAN64K) - smmu->pgsize_bitmap |=3D SZ_64K | SZ_512M; - if (reg & IDR5_GRAN16K) - smmu->pgsize_bitmap |=3D SZ_16K | SZ_32M; - if (reg & IDR5_GRAN4K) - smmu->pgsize_bitmap |=3D SZ_4K | SZ_2M | SZ_1G; + smmu->pgsize_bitmap =3D smmu_idr5_to_pgsize(reg); =20 /* Input address size */ if (FIELD_GET(IDR5_VAX, reg) =3D=3D IDR5_VAX_52_BIT) smmu->features |=3D ARM_SMMU_FEAT_VAX; =20 - /* Output address size */ - switch (FIELD_GET(IDR5_OAS, reg)) { - case IDR5_OAS_32_BIT: - smmu->oas =3D 32; - break; - case IDR5_OAS_36_BIT: - smmu->oas =3D 36; - break; - case IDR5_OAS_40_BIT: - smmu->oas =3D 40; - break; - case IDR5_OAS_42_BIT: - smmu->oas =3D 42; - break; - case IDR5_OAS_44_BIT: - smmu->oas =3D 44; - break; - case IDR5_OAS_52_BIT: - smmu->oas =3D 52; + smmu->oas =3D smmu_idr5_to_oas(reg); + if (smmu->oas =3D=3D 52) smmu->pgsize_bitmap |=3D 1ULL << 42; /* 4TB */ - break; - default: + else if (!smmu->oas) { dev_info(smmu->dev, - "unknown output address size. Truncating to 48-bit\n"); - fallthrough; - case IDR5_OAS_48_BIT: + "unknown output address size. Truncating to 48-bit\n"); smmu->oas =3D 48; } =20 @@ -5032,10 +4908,6 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_= device *smmu) dev_warn(smmu->dev, "failed to set DMA mask for table walker\n"); =20 - if ((smmu->features & ARM_SMMU_FEAT_TRANS_S1) && - (smmu->features & ARM_SMMU_FEAT_TRANS_S2)) - smmu->features |=3D ARM_SMMU_FEAT_NESTING; - arm_smmu_device_iidr_probe(smmu); =20 if (arm_smmu_sva_supported(smmu)) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 494e55a2dc34..842d0c9b883c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -84,6 +84,12 @@ struct arm_vsmmu; #define IIDR_REVISION GENMASK(15, 12) #define IIDR_IMPLEMENTER GENMASK(11, 0) =20 +#define IIDR_IMPLEMENTER_ARM 0x43b +#define IIDR_PRODUCTID_ARM_MMU_600 0x483 +#define IIDR_PRODUCTID_ARM_MMU_700 0x487 +#define IIDR_PRODUCTID_ARM_MMU_L1 0x48a +#define IIDR_PRODUCTID_ARM_MMU_S3 0x498 + #define ARM_SMMU_AIDR 0x1C =20 #define ARM_SMMU_CR0 0x20 @@ -1212,6 +1218,13 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_devi= ce *smmu, struct arm_smmu_cmd *cmds, int n, bool sync); =20 +u32 smmu_idr0_features(u32 reg); +u32 smmu_idr3_features(u32 reg); +u32 smmu_idr5_to_oas(u32 reg); +unsigned long smmu_idr5_to_pgsize(u32 reg); +unsigned long smmu_iidr_features(u32 reg, unsigned long features); +unsigned long smmu_iidr_options(u32 reg, unsigned long options); + /* Queue functions shared between kernel and hyp. */ static inline bool queue_has_space(struct arm_smmu_ll_queue *q, u32 n) { --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E3B14508F8 for ; Wed, 15 Jul 2026 11:59:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116764; cv=none; b=UM5Vw24brkUAbKzh4qI6CAn4QXbQgHesZj0tPu3duJKIkWA4bUoslKI14XVyKh1ukND3vZUZV8uI06Fxg/wNw96a4Yc7jEaVCEPin+DVY8/kt+9yIVauCKqKDP270l0YthwbpBQWgRyAlxzWwjb5w5/LIA8LMkv32TikVOJNI+Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116764; c=relaxed/simple; bh=l7xNH1sdHkX9F6VuAunb+LNEI8y2Y/WCTnyY4t5In2c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dOF0kheGfAjfns5SsoKTIge+fa1eGijkyUKqOKxYsMKtakceROTjIx11LtgHJERcNjtBmNORkMMRg80WS9LhVnGRH1IkHZRu0Oll6X8MqvNWpNjRqLnbD9IVdJyRoBiSwRNZxoHgw2dhrkUiREeOAWCJqsxYpomw0lKwCun9W2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OjkKHQiy; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OjkKHQiy" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493ae2a6a72so21472515e9.0 for ; Wed, 15 Jul 2026 04:59:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116760; x=1784721560; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=gLKA5ZqrCoYNFtJDAb4FxTuJKcFNr0bWkCPJJJzFwTU=; b=OjkKHQiyz+913az9rxIRVr1m50XKjN9miXUgnHaxj9RitowHt3LkbG4SdyD/aiskG8 yBP+8OGvYCCUfYBrNXjVitntj9QGU4kfAx1SRF5uuS2sF4d1pDy+EAK369CER/bdpiz/ PuQa+z0EquvJ9wxb1ndnZCcxudLlPIBmnnuZOFAudafF14+ib7hN02l/bpPGNndmHmA3 yvZ4j+cv/lHJ9Uq0rdRd5/7Vt8V2bvywdBd3xZkNuOh305AyyJ1La8I36AP8hrh1vb1s xynEXqPAmF5ZSnYTnyX2RiIK8pXsfYVpYUaP+1QloSSVVzPNOqP2efeNevJAZSa0Q4/6 pVNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116760; x=1784721560; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=gLKA5ZqrCoYNFtJDAb4FxTuJKcFNr0bWkCPJJJzFwTU=; b=IuIsCabAZUXOvEuA/Q4hy/m9RvvQTB8FeNxzj0Vr5FJttQzYhlBZS1lxYbzK+w6CIe Y1fqyhq8Rkf66cerIJrLIbBG2q9OUn4gZOhO9q8hpIfjjCE0Ze+5fTP9OWsrjbqAUYhh aUrROu7rclABlB32SjHjueW+EJTTXP7CO+ITb3EXvDX9IU5E7r9a247wQZ3wOqC1MMt1 9/92QRNNRAdifmCeMDHlI4cih6/DZzpmq6NmMd1sl9GebwXEa5qmfdeZxORXhwCufBlA DvxFK5pvOxo3lQXPyFZJZfPQEVdwI8miXaiJZeMX0Lp4ilAduyGHDhq+QzuZxfck0HVa 51og== X-Forwarded-Encrypted: i=1; AHgh+RpC8bZDEFz6ZO0bv8zoNYDhL8/bgpiXnjrI07/NnSWxM4Q4fiayUZmvlwHZwqNmprIbZNQULwIubCzfk0o=@vger.kernel.org X-Gm-Message-State: AOJu0Ywsi1TNBmME+k+q3MJXlac9MO9NdB38V0vYmgTXWaDLltP2L0NF c8/93Ffl0INzBc0/IFPzh8mZ4hwsZu1TyDvnb6D/PR2gMCq9Uwq/cI5QdDnRNHoyxXQ6MWZ5Sr0 t2O3D7rtrqiThXQ== X-Received: from wmrc3.prod.google.com ([2002:a05:600c:ac3:b0:493:b91c:52cb]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4205:b0:493:f7bc:26e4 with SMTP id 5b1f17b1804b1-493f8829970mr106991555e9.31.1784116760105; Wed, 15 Jul 2026 04:59:20 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:47 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-7-smostafa@google.com> Subject: [PATCH v7 06/24] KVM: arm64: iommu: Introduce IOMMU driver infrastructure From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh , Jean-Philippe Brucker Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To establish DMA isolation, KVM needs an IOMMU driver which provides ops implemented at EL2. Only one driver can be used and is registered with pkvm_iommu_register_driver() by passing pointer to the ops. This must be called before module_init() which is the point KVM initializes. Signed-off-by: Jean-Philippe Brucker Signed-off-by: Mostafa Saleh --- arch/arm64/include/asm/kvm_host.h | 5 +++++ arch/arm64/kvm/Makefile | 2 +- arch/arm64/kvm/hyp/include/nvhe/iommu.h | 13 +++++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 3 ++- arch/arm64/kvm/hyp/nvhe/iommu.c | 19 +++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/setup.c | 10 ++++++++++ arch/arm64/kvm/iommu.c | 25 +++++++++++++++++++++++++ 7 files changed, 75 insertions(+), 2 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/nvhe/iommu.h create mode 100644 arch/arm64/kvm/hyp/nvhe/iommu.c create mode 100644 arch/arm64/kvm/iommu.c diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index bae2c4f92ef5..43821767ccab 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1711,4 +1711,9 @@ static __always_inline enum fgt_group_id __fgt_reg_to= _group_id(enum vcpu_sysreg =20 long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext); =20 +#ifndef __KVM_NVHE_HYPERVISOR__ +struct pkvm_iommu_ops; +int pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops); +#endif + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 59612d2f277c..0ddef54f7434 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -24,7 +24,7 @@ kvm-y +=3D arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.= o \ vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o \ - vgic/vgic-v5.o + vgic/vgic-v5.o iommu.o =20 kvm-$(CONFIG_HW_PERF_EVENTS) +=3D pmu-emul.o pmu.o kvm-$(CONFIG_ARM64_PTR_AUTH) +=3D pauth.o diff --git a/arch/arm64/kvm/hyp/include/nvhe/iommu.h b/arch/arm64/kvm/hyp/i= nclude/nvhe/iommu.h new file mode 100644 index 000000000000..df3d0cc5d4db --- /dev/null +++ b/arch/arm64/kvm/hyp/include/nvhe/iommu.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ARM64_KVM_NVHE_IOMMU_H__ +#define __ARM64_KVM_NVHE_IOMMU_H__ + +#include + +struct pkvm_iommu_ops { + int (*init)(void); +}; + +int pkvm_iommu_init(void); + +#endif /* __ARM64_KVM_NVHE_IOMMU_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Mak= efile index 7c879292974d..b3e7473f18d2 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -24,7 +24,8 @@ CFLAGS_switch.nvhe.o +=3D -Wno-override-init =20 hyp-obj-y :=3D timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o= host.o \ hyp-main.o hyp-smp.o psci-relay.o early_alloc.o page_alloc.o \ - cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o + cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o \ + iommu.o hyp-obj-y +=3D ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../en= try.o \ ../hyp-entry.o ../exception.o ../pgtable.o ../vgic-v5-sr.o clock.o hyp-obj-y +=3D ../../../kernel/smccc-call.o diff --git a/arch/arm64/kvm/hyp/nvhe/iommu.c b/arch/arm64/kvm/hyp/nvhe/iomm= u.c new file mode 100644 index 000000000000..ef456eff42d2 --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/iommu.c @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * IOMMU operations for pKVM + * + * Copyright (C) 2022 Linaro Ltd. + */ +#include + +/* Only one set of ops supported */ +struct pkvm_iommu_ops *pkvm_iommu_ops; + +int pkvm_iommu_init(void) +{ + /* Keep DMA isolation optional. */ + if (!pkvm_iommu_ops || !pkvm_iommu_ops->init) + return 0; + + return pkvm_iommu_ops->init(); +} diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index 970c5cf342f5..c6698ecea4a2 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -333,6 +334,15 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; =20 + /* + * IOMMU init is done at the the end as it setups and initializes many re= sources + * that would be leaked if the hypervisor fails after as there is no + * remove_iommu_driver() ops at the moment, that can be added later if ne= cessary. + */ + ret =3D pkvm_iommu_init(); + if (ret) + goto out; + pkvm_hyp_vm_table_init(vm_table_base); =20 pkvm_ownership_selftest(selftest_base); diff --git a/arch/arm64/kvm/iommu.c b/arch/arm64/kvm/iommu.c new file mode 100644 index 000000000000..6492285012a4 --- /dev/null +++ b/arch/arm64/kvm/iommu.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2025 Google LLC + * Author: Mostafa Saleh + */ + +#include + +extern struct pkvm_iommu_ops *kvm_nvhe_sym(pkvm_iommu_ops); + +static DEFINE_MUTEX(pkvm_iommu_reg_lock); + +int __init pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops) +{ + guard(mutex)(&pkvm_iommu_reg_lock); + + if (!is_protected_kvm_enabled() || is_kvm_arm_initialised()) + return -EPERM; + + if (kvm_nvhe_sym(pkvm_iommu_ops)) + return -EBUSY; + + kvm_nvhe_sym(pkvm_iommu_ops) =3D hyp_ops; + return 0; +} --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B38D4534A2 for ; Wed, 15 Jul 2026 11:59:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116765; cv=none; b=dgPV8wZfQx6omw0IWAZyNEcA+PvcuAoxdP1qxK7woDO6BbiQQ9385H5MbCSkngQRlXNqKAjn0MEOHix+hzsoAkzN8PGGY8CG9M03X6EWsF40TlCb13U4V/eevAIHz0jCqAzQ0CxpmRsTNj3WU+EURmqv5aaghtB9yNf+IECOpx8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116765; c=relaxed/simple; bh=qCxK+JJaa38XcTX5702v5OnV9lqkbVBWQHIBYkDbADs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=CXDe1bQLjOV3lKwFQzGFkfoCgp87he/NWKqPdX+CakUTiC3q4Q/L/0hktFn1o0w0Fyu3IAsGJKbOSH4BKkdNeu9ZaIL6cfJV/y48wsLjSGxZk5HZAAr4WgAIzA/gEgfXuvvO0t6f3td9FchB+FtGtAajDWjHsnKrcEaEjK0T0yY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=l9Mp1jgm; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="l9Mp1jgm" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493dc8408fdso16995915e9.1 for ; Wed, 15 Jul 2026 04:59:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116762; x=1784721562; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=DulTGTw97uMRHQPlm6xWX2ICmo+Sh9dZV8TFaDf+QVU=; b=l9Mp1jgmo0Sa+AoqNYXWVhb8Gsar+QYYT9L71Dhtb1nSJVLhSNVnrfaitCuqa9jncJ 7K7JPMwud43dSvfE9rlCozT3qvwIT/5QhSk7RqUyyw2aV36juMSI0KeSP8LsuqtbjWsf U9nRGhmDLSxs5BEBP8c6WYDRO4Kgw444loM5iHYSYx8/tiAae5HkbspC8ztzAkMpySGS iCLv4MFcz2H/+W3F9ukFMgaB8psDn1z2QPtXH/UtV667G94K22n2fG8mT4inog+rRYz0 NzhkfrA0lTYOsi13jZHyzymg7bI4UswFEF6rueRM3muacjl1XTvOeNi5sdhy0kEDRb4/ uobg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116762; x=1784721562; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=DulTGTw97uMRHQPlm6xWX2ICmo+Sh9dZV8TFaDf+QVU=; b=c8En6QWBKxF3hlzV5eZKUvap8Wey1zRyoQWTZxTmmkYb36JVPimRaowchrzssHeFOV kBmR7eZ3SE4wYZDA8Et6wSlmw0Ul2LAjlJT2qKGMEJSQ9Lipj3mpQvVEp3CFNHcn0Vj6 q+8k39si/+MuLeNBVzZo5KnW/mwLh5o3317vQKbPK724Jw5vX1VAgLPgMUxThLEQZTOK Ao1exCJee08/KvLKzn0E5p2ft+sce8AolQWSlCUmcra9De4Lspuycjs9th7ReEpKwaDQ SjcoDC1rp0r7eIkZN4c5CFuWuInWTYJnr+ioKXLeWf+xIdoOTB1hm2zIVrkV9sayi78Q 5THQ== X-Forwarded-Encrypted: i=1; AHgh+Rp4rdlzf9sokt546H5tAgQxLZ6eebTzSVlv1R/z/Rjbn+U3tyqgu17ANNL2/mML9txBeTl5KCLVwMUFNzM=@vger.kernel.org X-Gm-Message-State: AOJu0YzXYPm7/Dy8eriPYF1+ALabX0LgvYxR0VrqxOfq0PLpz00FPNVa hDX1neI5MvkHlYoYgEvCzm3LsU9+sJBPh4PejCQ28Zza1NVNYjYGW8qWk8+4o8rGcWW1hIAGlp8 XeqGQN4fpQ0RVRA== X-Received: from wmby15.prod.google.com ([2002:a05:600c:c04f:b0:493:bdba:620b]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:e557:20b0:493:f822:ac5e with SMTP id 5b1f17b1804b1-493f8818052mr132724075e9.23.1784116761502; Wed, 15 Jul 2026 04:59:21 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:48 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-8-smostafa@google.com> Subject: [PATCH v7 07/24] KVM: arm64: iommu: Shadow host stage-2 page table From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Create a page-table for the IOMMU that shadows the host CPU stage-2 to establish DMA isolation. An initial snapshot is created after the driver init, then on every permission change a callback would be called for the IOMMU driver to update the page table. There are 3 different ways to add the callback: 1) In the high level memory transitions: (__pkvm_host_donate_hyp(), __pkvm_host_donate_guest()... 2) In Lower level functions covering all transitions - host_stage2_set_owner_metadata_locked() which covers: - __pkvm_host_donate_hyp() - __pkvm_host_donate_guest() - __pkvm_host_donate_hyp() - __pkvm_guest_unshare_host() - host_stage2_set_owner_locked() only for ID_HOST which covers: - __pkvm_hyp_donate_host() - __pkvm_host_force_reclaim_page_guest() - __pkvm_host_reclaim_page_guest() - __pkvm_guest_share_host() 3) In the lowest level function __host_update_page_state(), which requires only one callback. However, in that case the page state is not enough as we might need to know the old state also. Option #2 was implemented here. For some cases, an SMMUv3 may be able to share the same page-table used with the host CPU stage-2 directly. However, this is too strict and requires changes to the core hypervisor page-table code, plus it would require the hypervisor to handle IOMMU page-faults. This can be added later as an optimization for SMMUV3. Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/hyp/include/nvhe/iommu.h | 4 + arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/iommu.c | 129 +++++++++++++++++- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 27 ++-- 4 files changed, 145 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/iommu.h b/arch/arm64/kvm/hyp/i= nclude/nvhe/iommu.h index df3d0cc5d4db..857d7dd2ebc3 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/iommu.h +++ b/arch/arm64/kvm/hyp/include/nvhe/iommu.h @@ -3,11 +3,15 @@ #define __ARM64_KVM_NVHE_IOMMU_H__ =20 #include +#include =20 struct pkvm_iommu_ops { int (*init)(void); + int (*host_stage2_idmap)(phys_addr_t start, phys_addr_t end, int prot); }; =20 int pkvm_iommu_init(void); =20 +int pkvm_iommu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, + enum kvm_pgtable_prot prot); #endif /* __ARM64_KVM_NVHE_IOMMU_H__ */ diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm= /hyp/include/nvhe/mem_protect.h index 51b0eb3844a9..99b821b3cf65 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -59,6 +59,7 @@ int __pkvm_host_test_clear_young_guest(u64 gfn, u64 nr_pa= ges, bool mkold, struct int __pkvm_host_mkyoung_guest(u64 gfn, struct pkvm_hyp_vcpu *vcpu); =20 bool addr_is_memory(phys_addr_t phys); + int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_= prot prot); int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id); int kvm_host_prepare_stage2(void *pgt_pool_base); diff --git a/arch/arm64/kvm/hyp/nvhe/iommu.c b/arch/arm64/kvm/hyp/nvhe/iomm= u.c index ef456eff42d2..08009609ec59 100644 --- a/arch/arm64/kvm/hyp/nvhe/iommu.c +++ b/arch/arm64/kvm/hyp/nvhe/iommu.c @@ -4,16 +4,137 @@ * * Copyright (C) 2022 Linaro Ltd. */ +#include +#include + #include +#include +#include =20 /* Only one set of ops supported */ struct pkvm_iommu_ops *pkvm_iommu_ops; =20 -int pkvm_iommu_init(void) +/* Protected by host_mmu.lock */ +static bool pkvm_idmap_initialized; + +static inline int pkvm_to_iommu_prot(enum kvm_pgtable_prot prot) { - /* Keep DMA isolation optional. */ - if (!pkvm_iommu_ops || !pkvm_iommu_ops->init) + int iommu_prot =3D 0; + + if (prot & KVM_PGTABLE_PROT_R) + iommu_prot |=3D IOMMU_READ; + if (prot & KVM_PGTABLE_PROT_W) + iommu_prot |=3D IOMMU_WRITE; + + /* We don't understand that, might be dangerous. */ + WARN_ON(prot & ~PKVM_HOST_MEM_PROT); + return iommu_prot; +} + +/* + * IOMMU page tables are shadowed and not shared, that is mainly because: + * - Possible inconsistency between IOMMU and CPU features or format. + * - KVM relies on handling in page faults (BBM, lazy mapping). + */ +static int __snapshot_host_stage2(const struct kvm_pgtable_visit_ctx *ctx, + enum kvm_pgtable_walk_flags visit) +{ + u64 start =3D ctx->addr; + u64 block_end =3D ALIGN_DOWN(ctx->addr, kvm_granule_size(ctx->level)) + + kvm_granule_size(ctx->level); + u64 end =3D min(ctx->end, block_end); + kvm_pte_t pte =3D *ctx->ptep; + bool is_memory =3D *(bool *)ctx->arg; + int prot; + + /* + * Keep annotated PTEs unmapped, and map everything else even lazily + * mapped PTEs(0), as the IOMMU can't handle page faults. + * That maps the whole address space which can be large, but that doesn't + * use a lot of memory as it will be mostly large block (1 GB with 4kb pa= ges) + */ + if (pte && !kvm_pte_valid(pte)) return 0; =20 - return pkvm_iommu_ops->init(); + if (kvm_pte_valid(pte)) + prot =3D pkvm_to_iommu_prot(kvm_pgtable_stage2_pte_prot(pte)); + else + prot =3D IOMMU_READ | IOMMU_WRITE; + + if (!is_memory) + prot |=3D IOMMU_MMIO; + + return pkvm_iommu_ops->host_stage2_idmap(start, end, prot); +} + +static int pkvm_iommu_snapshot_host_stage2(void) +{ + struct kvm_pgtable *pgt =3D &host_mmu.pgt; + bool is_memory; + struct kvm_pgtable_walker walker =3D { + .cb =3D __snapshot_host_stage2, + .flags =3D KVM_PGTABLE_WALK_LEAF, + .arg =3D &is_memory, + }; + int ret =3D 0, i; + u64 start =3D 0; + + hyp_spin_lock(&host_mmu.lock); + for (i =3D 0; i < hyp_memblock_nr; i++) { + struct memblock_region *reg =3D &hyp_memory[i]; + + if (start < reg->base) { + is_memory =3D false; + ret =3D kvm_pgtable_walk(pgt, start, reg->base - start, &walker); + if (ret) + goto out_unlock; + } + + is_memory =3D true; + ret =3D kvm_pgtable_walk(pgt, reg->base, reg->size, &walker); + if (ret) + goto out_unlock; + + start =3D reg->base + reg->size; + } + + if (start < BIT(pgt->ia_bits)) { + is_memory =3D false; + ret =3D kvm_pgtable_walk(pgt, start, BIT(pgt->ia_bits) - start, &walker); + if (ret) + goto out_unlock; + } + + pkvm_idmap_initialized =3D true; + +out_unlock: + hyp_spin_unlock(&host_mmu.lock); + return ret; +} + +int pkvm_iommu_init(void) +{ + int ret; + + /* Keep DMA isolation optional. */ + if (!pkvm_iommu_ops || !pkvm_iommu_ops->init || + !pkvm_iommu_ops->host_stage2_idmap) + return 0; + + ret =3D pkvm_iommu_ops->init(); + if (ret) + return ret; + + return pkvm_iommu_snapshot_host_stage2(); +} + +int pkvm_iommu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, + enum kvm_pgtable_prot prot) +{ + hyp_assert_lock_held(&host_mmu.lock); + + if (!pkvm_idmap_initialized) + return 0; + + return pkvm_iommu_ops->host_stage2_idmap(start, end, pkvm_to_iommu_prot(p= rot)); } diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index d803b3dd4cb4..ce610274bda0 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -17,6 +17,7 @@ =20 #include #include +#include #include #include #include @@ -596,16 +597,15 @@ static int host_stage2_set_owner_metadata_locked(phys= _addr_t addr, u64 size, ret =3D host_stage2_try(kvm_pgtable_stage2_annotate, &host_mmu.pgt, addr, size, &host_s2_pool, KVM_HOST_INVALID_PTE_TYPE_DONATION, annotation); - if (!ret) { - /* - * After stage2 maintenance has happened, but before the page - * owner has changed. - */ - pkvm_sme_dvmsync_fw_call(); - __host_update_page_state(addr, size, PKVM_NOPAGE); - } - - return ret; + if (ret) + return ret; + /* + * After stage2 maintenance has happened, but before the page + * owner has changed. + */ + pkvm_sme_dvmsync_fw_call(); + __host_update_page_state(addr, size, PKVM_NOPAGE); + return pkvm_iommu_host_stage2_idmap(addr, addr + size, 0); } =20 int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id) @@ -618,8 +618,10 @@ int host_stage2_set_owner_locked(phys_addr_t addr, u64= size, u8 owner_id) return -EPERM; =20 ret =3D host_stage2_idmap_locked(addr, size, PKVM_HOST_MEM_PROT); - if (!ret) - __host_update_page_state(addr, size, PKVM_PAGE_OWNED); + if (ret) + break; + __host_update_page_state(addr, size, PKVM_PAGE_OWNED); + ret =3D pkvm_iommu_host_stage2_idmap(addr, addr + size, PKVM_HOST_MEM_PR= OT); break; case PKVM_ID_HYP: ret =3D host_stage2_set_owner_metadata_locked(addr, size, @@ -1022,6 +1024,7 @@ int __pkvm_guest_share_host(struct pkvm_hyp_vcpu *vcp= u, u64 gfn) pkvm_mkstate(KVM_PGTABLE_PROT_RWX, PKVM_PAGE_SHARED_OWNED), &vcpu->vcpu.arch.pkvm_memcache, 0)); WARN_ON(__host_set_page_state_range(phys, PAGE_SIZE, PKVM_PAGE_SHARED_BOR= ROWED)); + WARN_ON(pkvm_iommu_host_stage2_idmap(phys, phys + PAGE_SIZE, PKVM_HOST_ME= M_PROT)); unlock: guest_unlock_component(vm); host_unlock_component(); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C13C94657C6 for ; Wed, 15 Jul 2026 11:59:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116767; cv=none; b=DB4tVM4rU+BS32wl7PLDVgTcRrdXL+0u8kKvzKYOZbOlApqiuDrL0mYCnzkt1tNMRMf4tFEK6gNYzgJKSBEwwb0qQee+Z6cN5hqfXVHL1YubRPMFAvN1jKreu9fDz8LKKuc+PgwTp93PtD4SNAIi+6EMR6n4m1NatIbA9UCfSkU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116767; c=relaxed/simple; bh=BKXOjPm4HibB0zKKPDg8mVqvFcn5okB02q1ktl64G/I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Wdk7kzm2DvEViTuyOU1o1rnN1upUCKW7C/NUKvH5Q1BPHfg+2wPBhyxF38ZMULivBDAfxpErNFj6yaDT+LEcbrEorR4LnBFhoNW8z9jMbDbj14vHMauERrPf3R9EUBvCWhqolzZQpVp4b0Z7f9/p6pV433DxRaTUYAPfSqxwmBM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=n8BFiR7o; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n8BFiR7o" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-492488f8583so77518285e9.2 for ; Wed, 15 Jul 2026 04:59:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116763; x=1784721563; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=4P0vhJSxO+G+Vc4nk48G6MnzTMo1FQyjg7u/hgjKJK4=; b=n8BFiR7ofZgzoDS3nrCZVyLoOVzGGOctj1n4SfZ75Vc7z/CHFf/wnMuNkPxO79VtHm imxwvK/IeSYea7FzBODlhG2lRosGCUnXYaNEb5drfYMLHzY8c6WIdSbkd1qFtGalk3b7 5hRn9wntdKb3Y0MNH9QJYPGPvHrNKVi+A9+e8H17ErDSZlwGf7VJoJN0l65tfG3Xk7Ti J1MXK/ghHmj/TYLXS3oOlRVVRT5WMpsgVyq4IjmQqoxI+UHYytwRI+/KiEhWwyXmSjGC ROTth7iTkqmmKYYgtT0fFUHks8LjOamFaHt7axktAU1KQp+fTnbrc0IDbrKe6uaWmzeJ 39lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116763; x=1784721563; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=4P0vhJSxO+G+Vc4nk48G6MnzTMo1FQyjg7u/hgjKJK4=; b=QfJWCGBhYURYGLgUp/HGxqDzpj003O11qIM5Vn6tfv8/9AxlOTAPvPoRe0KJa0CN09 55NGv1pkGHse/9vRbuikpzpEu+91UXI0lFO0DysT0mN4Ib5VDug9//eGHykomlziJM97 gqzq7bp3/KJc0NQWX9FTAdkX9u28E55QY5SNF/ClFeKyaEeUdr6KD9liy/c+tdfC6Ych zi7mbQ+wTJTGtNqp2tV7TtXOe7eo4VEnCByL6Ppmoa8g+uXUmjUrlbMflqjlvRXXXB2A dZXXSOKnTPFzomk8kQ3frh90Vvl6mWj5io+MCOhsMtpVupuKI9rcnProKCuBd7IiZ18E HqeQ== X-Forwarded-Encrypted: i=1; AHgh+RpnAa/oCJQPjmYAhwbpt8e3w2XRJRPRcXfsyxAY9/1CStStgVcG44pktXxGnuYe6IBOgVXiNzeCN5d4V4c=@vger.kernel.org X-Gm-Message-State: AOJu0YziaRXqHijMj6UdIJqmkLePXtwqS1GuDmHnuHVh7ZMhM6QGUqVu AUFEiE8G/QBGOLGFGwSgsdH3HpkVwfaHFcNroFuKwotZ5auL7WH3iUyPyniiYF1XSd0b8HbSLcH xZIwJhduaJy1kIg== X-Received: from wmga19.prod.google.com ([2002:a05:600c:2d53:b0:495:3dd3:e994]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c16a:b0:493:b89b:9a27 with SMTP id 5b1f17b1804b1-495389d2a9cmr64618175e9.9.1784116762784; Wed, 15 Jul 2026 04:59:22 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:49 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-9-smostafa@google.com> Subject: [PATCH v7 08/24] KVM: arm64: iommu: Add memory pool From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" IOMMU drivers need to allocate memory for the shadow page table. Similar to the host stage-2 CPU page table, the IOMMU pool is allocated early from the carveout and it's memory is added to a pool which the IOMMU driver can allocate from and reclaim to at run time. As this is too early for drivers to use initcalls, the number of pages allocated is set from command line "kvm-arm.iommu_pgt_mem". Later when the driver registers, it will pass how many pages it needs, and if it was less than what was allocated, it will fail to register. Signed-off-by: Mostafa Saleh --- .../admin-guide/kernel-parameters.txt | 4 ++ arch/arm64/include/asm/kvm_host.h | 3 +- arch/arm64/kvm/hyp/include/nvhe/iommu.h | 8 +++- arch/arm64/kvm/hyp/nvhe/iommu.c | 21 +++++++++- arch/arm64/kvm/hyp/nvhe/setup.c | 11 ++++- arch/arm64/kvm/iommu.c | 42 ++++++++++++++++++- arch/arm64/kvm/pkvm.c | 1 + 7 files changed, 85 insertions(+), 5 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index b5493a7f8f22..5cbac46bc941 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3253,6 +3253,10 @@ Kernel parameters trap: set WFI instruction trap =20 notrap: clear WFI instruction trap + kvm-arm.iommu_pgt_mem=3Dnn[KMG] + [KVM, ARM, EARLY] + Memory allocated for the IOMMU pool from the KVM carveout + when running in protected mode. =20 kvm_cma_resv_ratio=3Dn [PPC,EARLY] Reserves given percentage from system memory area for diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 43821767ccab..8d3b4500b360 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1713,7 +1713,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, lo= ng *ext); =20 #ifndef __KVM_NVHE_HYPERVISOR__ struct pkvm_iommu_ops; -int pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops); +int pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops, unsigned in= t nr_pages); +unsigned int pkvm_iommu_pages(void); #endif =20 #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/hyp/include/nvhe/iommu.h b/arch/arm64/kvm/hyp/i= nclude/nvhe/iommu.h index 857d7dd2ebc3..028e89a3448d 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/iommu.h +++ b/arch/arm64/kvm/hyp/include/nvhe/iommu.h @@ -10,8 +10,14 @@ struct pkvm_iommu_ops { int (*host_stage2_idmap)(phys_addr_t start, phys_addr_t end, int prot); }; =20 -int pkvm_iommu_init(void); +int pkvm_iommu_init(void *pool_base, unsigned int nr_pages); =20 int pkvm_iommu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, enum kvm_pgtable_prot prot); + +/* Allocate pages from the IOMMU carveout, returns zeroed memory. */ +void *pkvm_iommu_donate_pages(u8 order); +/* Free pages from pkvm_iommu_donate_pages(). Must use same size */ +void pkvm_iommu_reclaim_pages(void *ptr); + #endif /* __ARM64_KVM_NVHE_IOMMU_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/iommu.c b/arch/arm64/kvm/hyp/nvhe/iomm= u.c index 08009609ec59..8891cb7920c4 100644 --- a/arch/arm64/kvm/hyp/nvhe/iommu.c +++ b/arch/arm64/kvm/hyp/nvhe/iommu.c @@ -16,6 +16,7 @@ struct pkvm_iommu_ops *pkvm_iommu_ops; =20 /* Protected by host_mmu.lock */ static bool pkvm_idmap_initialized; +static struct hyp_pool iommu_pages_pool; =20 static inline int pkvm_to_iommu_prot(enum kvm_pgtable_prot prot) { @@ -112,7 +113,7 @@ static int pkvm_iommu_snapshot_host_stage2(void) return ret; } =20 -int pkvm_iommu_init(void) +int pkvm_iommu_init(void *pool_base, unsigned int nr_pages) { int ret; =20 @@ -121,6 +122,14 @@ int pkvm_iommu_init(void) !pkvm_iommu_ops->host_stage2_idmap) return 0; =20 + if (!nr_pages) + return -ENOMEM; + + ret =3D hyp_pool_init(&iommu_pages_pool, hyp_virt_to_pfn(pool_base), + nr_pages, 0); + if (ret) + return ret; + ret =3D pkvm_iommu_ops->init(); if (ret) return ret; @@ -138,3 +147,13 @@ int pkvm_iommu_host_stage2_idmap(phys_addr_t start, ph= ys_addr_t end, =20 return pkvm_iommu_ops->host_stage2_idmap(start, end, pkvm_to_iommu_prot(p= rot)); } + +void *pkvm_iommu_donate_pages(u8 order) +{ + return hyp_alloc_pages(&iommu_pages_pool, order); +} + +void pkvm_iommu_reclaim_pages(void *ptr) +{ + hyp_put_page(&iommu_pages_pool, ptr); +} diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index c6698ecea4a2..39d59ff30f34 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -23,6 +23,8 @@ =20 unsigned long hyp_nr_cpus; =20 +unsigned int hyp_kvm_iommu_pages; + #define hyp_percpu_size ((unsigned long)__per_cpu_end - \ (unsigned long)__per_cpu_start) =20 @@ -34,6 +36,7 @@ static void *selftest_base; static void *ffa_proxy_pages; static struct kvm_pgtable_mm_ops pkvm_pgtable_mm_ops; static struct hyp_pool hpool; +static void *iommu_base; =20 static int divide_memory_pool(void *virt, unsigned long size) { @@ -71,6 +74,12 @@ static int divide_memory_pool(void *virt, unsigned long = size) if (!ffa_proxy_pages) return -ENOMEM; =20 + if (hyp_kvm_iommu_pages) { + iommu_base =3D hyp_early_alloc_contig(hyp_kvm_iommu_pages); + if (!iommu_base) + return -ENOMEM; + } + return 0; } =20 @@ -339,7 +348,7 @@ void __noreturn __pkvm_init_finalise(void) * that would be leaked if the hypervisor fails after as there is no * remove_iommu_driver() ops at the moment, that can be added later if ne= cessary. */ - ret =3D pkvm_iommu_init(); + ret =3D pkvm_iommu_init(iommu_base, hyp_kvm_iommu_pages); if (ret) goto out; =20 diff --git a/arch/arm64/kvm/iommu.c b/arch/arm64/kvm/iommu.c index 6492285012a4..c80663b506c0 100644 --- a/arch/arm64/kvm/iommu.c +++ b/arch/arm64/kvm/iommu.c @@ -7,10 +7,11 @@ #include =20 extern struct pkvm_iommu_ops *kvm_nvhe_sym(pkvm_iommu_ops); +extern unsigned int kvm_nvhe_sym(hyp_kvm_iommu_pages); =20 static DEFINE_MUTEX(pkvm_iommu_reg_lock); =20 -int __init pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops) +int pkvm_iommu_register_driver(struct pkvm_iommu_ops *hyp_ops, unsigned in= t nr_pages) { guard(mutex)(&pkvm_iommu_reg_lock); =20 @@ -20,6 +21,45 @@ int __init pkvm_iommu_register_driver(struct pkvm_iommu_= ops *hyp_ops) if (kvm_nvhe_sym(pkvm_iommu_ops)) return -EBUSY; =20 + /* See kvm_iommu_pages() */ + if (nr_pages > kvm_nvhe_sym(hyp_kvm_iommu_pages)) { + kvm_err("IOMMU pool needs 0x%x pages, check kvm-arm.iommu_pgt_mem", nr_p= ages); + return -ENOMEM; + } + kvm_nvhe_sym(pkvm_iommu_ops) =3D hyp_ops; return 0; } + +unsigned int pkvm_iommu_pages(void) +{ + /* + * This is used very early during setup_arch() before any initcalls + * or any drivers are registered. + * This value is set by a command line option. + * Later, when the driver is registered, it will pass the number + * pages needed for it's page tables, if it was more than what + * the system has already allocated, it will fail registration. + */ + return kvm_nvhe_sym(hyp_kvm_iommu_pages); +} + +/* Number of pages to reserve for iommu pool*/ +static int __init early_iommu_pgt_mem(char *arg) +{ + unsigned long long requested_size; + + if (!arg) + return -EINVAL; + + requested_size =3D memparse(arg, NULL); + + if (requested_size > UINT_MAX) { + kvm_err("kvm-arm.iommu_pgt_mem is too large\n"); + return -EINVAL; + } + + kvm_nvhe_sym(hyp_kvm_iommu_pages) =3D requested_size >> PAGE_SHIFT; + return 0; +} +early_param("kvm-arm.iommu_pgt_mem", early_iommu_pgt_mem); diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c index 053e4f733e4b..72ca1393c504 100644 --- a/arch/arm64/kvm/pkvm.c +++ b/arch/arm64/kvm/pkvm.c @@ -63,6 +63,7 @@ void __init kvm_hyp_reserve(void) hyp_mem_pages +=3D hyp_vmemmap_pages(STRUCT_HYP_PAGE_SIZE); hyp_mem_pages +=3D pkvm_selftest_pages(); hyp_mem_pages +=3D hyp_ffa_proxy_pages(); + hyp_mem_pages +=3D pkvm_iommu_pages(); =20 /* * Try to allocate a PMD-aligned region to reduce TLB pressure once --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7841546AEDB for ; Wed, 15 Jul 2026 11:59:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116769; cv=none; b=IXJCjjeVGGYps0BJf6TAilNfIImPCTwJ2Y03Z5kRs0YuKE0MufWn+y45fDfyHi7nFa8kkKWrcptzHO0TC6rQEnNcEicqI6jeZYSkkIX7WcyGdmxc66aYbCqO+EqUZhRvsPyWsRwof2mQ1iox+VNr/xPd8iraEEor5STgruu9Qo0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116769; c=relaxed/simple; bh=rlfx3qfyVRl3021fgkWqxMtUcvwBDWvKZgNJzUwlcHU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bF7+aesL44u0AkW3HmfpHdeAAkLixqas5TlmPfyH5KeKVCIWSbL/FbBzZYv62LnBXP8QE78JTiDX3JRFNJezvrP+5vNAY2e8RbjlTySdJEKufXkmleu+K9vOEoI1QlfpJBXmNI5g10TYtJAtSNS9Q+CBGag86PAWvAINcvePIrI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rNqZAQSJ; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rNqZAQSJ" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-493ce4b7777so39511205e9.3 for ; Wed, 15 Jul 2026 04:59:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116764; x=1784721564; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=nojqcuetD+/ikKOr3q9aYEmPsN9V6B5kzgOM/WeBKUw=; b=rNqZAQSJwyl+7bFtpwkNcAjdDAfmZNVxAi2o+xCd128Yx/+vEcx11QXEleaOAgJVrC rsf7Inv0Q9FWG/hew/328B82Y/aqi1atf9v8uJQqt6SKX1bK+lsW7M9dpGI8S/AAjj/F nz1hDB+4zB+HR3Bl8s4UHMwaEkATiWX1DXAQORv2CXVC4EDud8F2YQSJnNr84yY+HF0b vQEaIl7i23+V5WYNF+HtP4E/BN2TyMMZareFrDGYqtKsvtirpWNzycCa9kmhvIUW8Mlz PITiS+36/29gIllYwz+BqJV5PGxiSAl4oIdmB+oVb2SyJ9Pdoe1sRvI874QIywBiGimF wA/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116764; x=1784721564; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=nojqcuetD+/ikKOr3q9aYEmPsN9V6B5kzgOM/WeBKUw=; b=jHlMZHxCm8GPfYz7dd8cB0SR+2WWL2P6jl7PjtCr4/yeZhv5b9MduhC9T35mcN5aXn KEcJiTXkk64S7Ka7yr0YvaS6nofQ22Vc1Up8h1dH6ViSAoUgygVF55xx0VjnqrXJnqdW c3pRFaV3xw2X+8bvEXZqbgLHmGi0g6IQd+5E68Wathp0TVtlILzFjaXlW2B9YzyN49bi N5ZuyQ8D4fCug243RQLhIj/zg3oJYRCbQdGwwYf8kHf3cQjTdzo0imPVk1CS8bKSYn/b 0u6e64+f/J09VbWMuHU5KqGPZ/JtTnl42gX7VS0jSdFG9wCi5Xv3LEsistFgdsI1/3ct o/8g== X-Forwarded-Encrypted: i=1; AHgh+RqVdHRx/zBgwY3U86b77HYlGdBEOEe2YIOPBZ+G5tlXSnyl5nygFkVlvdy7sM/454qT48AcykDy8mRG3eQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yy8u78+bz01vE3lzfeQErJ64I6LryABrkxbpjjGBb6QZYbk+kq0 u7zuX3Ro6StCJ3ZMbLUa5MhZZ+1V9F80/J79zoD502EJOGUmCE1lMQrozENGJj9sy3azdMJt5Cu cUTAxrnW0H0shTw== X-Received: from wmbdx5.prod.google.com ([2002:a05:600c:63c5:b0:493:f781:b52f]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:5656:b0:493:c77c:108a with SMTP id 5b1f17b1804b1-4953c2897b9mr17716955e9.36.1784116763927; Wed, 15 Jul 2026 04:59:23 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:50 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-10-smostafa@google.com> Subject: [PATCH v7 09/24] KVM: arm64: iommu: Support DABT for IOMMU From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The pKVM SMMUv3 driver needs to trap and emulate access to the MMIO space of the SMMUv3 to provide emulation for the kernel driver. Add a handler for DABTs for IOMMU drivers to be able to do so. In case the host causes a data abort, check if it's part of IOMMU emulation first. Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/hyp/include/nvhe/iommu.h | 3 ++- arch/arm64/kvm/hyp/nvhe/iommu.c | 15 +++++++++++++++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 15 +++++++++++++++ 3 files changed, 32 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/iommu.h b/arch/arm64/kvm/hyp/i= nclude/nvhe/iommu.h index 028e89a3448d..8b7521135792 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/iommu.h +++ b/arch/arm64/kvm/hyp/include/nvhe/iommu.h @@ -8,6 +8,7 @@ struct pkvm_iommu_ops { int (*init)(void); int (*host_stage2_idmap)(phys_addr_t start, phys_addr_t end, int prot); + bool (*dabt_handler)(struct user_pt_regs *regs, u64 esr, u64 addr); }; =20 int pkvm_iommu_init(void *pool_base, unsigned int nr_pages); @@ -19,5 +20,5 @@ int pkvm_iommu_host_stage2_idmap(phys_addr_t start, phys_= addr_t end, void *pkvm_iommu_donate_pages(u8 order); /* Free pages from pkvm_iommu_donate_pages(). Must use same size */ void pkvm_iommu_reclaim_pages(void *ptr); - +bool pkvm_iommu_host_dabt_handler(struct user_pt_regs *regs, u64 esr, u64 = addr); #endif /* __ARM64_KVM_NVHE_IOMMU_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/iommu.c b/arch/arm64/kvm/hyp/nvhe/iomm= u.c index 8891cb7920c4..8ea9c8c23985 100644 --- a/arch/arm64/kvm/hyp/nvhe/iommu.c +++ b/arch/arm64/kvm/hyp/nvhe/iommu.c @@ -4,6 +4,10 @@ * * Copyright (C) 2022 Linaro Ltd. */ +#include + +#include + #include #include =20 @@ -157,3 +161,14 @@ void pkvm_iommu_reclaim_pages(void *ptr) { hyp_put_page(&iommu_pages_pool, ptr); } + +bool pkvm_iommu_host_dabt_handler(struct user_pt_regs *regs, u64 esr, u64 = addr) +{ + if (pkvm_iommu_ops && pkvm_iommu_ops->dabt_handler && + pkvm_iommu_ops->dabt_handler(regs, esr, addr)) { + /* DABT handled by the driver, skip to next instruction. */ + kvm_skip_host_instr(); + return true; + } + return false; +} diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index ce610274bda0..b798a36e0ddb 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -751,6 +751,12 @@ static void host_inject_mem_abort(struct kvm_cpu_conte= xt *host_ctxt) inject_host_exception(esr); } =20 +static bool is_mmio_dabt(u64 esr) +{ + return (ESR_ELx_EC(esr) =3D=3D ESR_ELx_EC_DABT_LOW) && + (esr & ESR_ELx_ISV); +} + void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; @@ -773,6 +779,15 @@ void handle_host_mem_abort(struct kvm_cpu_context *hos= t_ctxt) BUG_ON(!(fault.hpfar_el2 & HPFAR_EL2_NS)); addr =3D FIELD_GET(HPFAR_EL2_FIPA, fault.hpfar_el2) << 12; =20 + /* + * Emulate data aborts for IOMMU drivers, other access will be denied + * by host_stage2_adjust_range() + */ + if (is_mmio_dabt(esr) && !addr_is_memory(addr) && + pkvm_iommu_host_dabt_handler(&host_ctxt->regs, + esr, addr | FAR_TO_FIPA_OFFSET(fault.far_el2))) + return; + switch (host_stage2_idmap(addr)) { case -EPERM: host_inject_mem_abort(host_ctxt); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 296604657DA for ; Wed, 15 Jul 2026 11:59:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116772; cv=none; b=uDVpuz1k1F3oh4uUONAXbJ1EtBbLderCSu/YvdMlNAvIl2dBNOvF6KUpB3IL1DGaINeu3qgwfcaQ8RDl6kmd1IXXCHRuytjl+Gt4qfqMbq1AOB6kDnDVZ7rk7VrjTe7yemgUMVUMGG5Bas8MiK4x3ns806GD5SoV0zuS1ISFrmw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116772; c=relaxed/simple; bh=WC65vWXV3XqwhhIiAJrzrK4KiHn25jMkRXgG6O/emvA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VK7xh1enBL0awkn41EmB5Cj53GL95B5dYm9nnLMVwdASpILCBnziqDnBme60X4wctp+TESRuZdIL1UhXDiLCT2Zt3qi+v7ZLo5dIhAAdANIeua34gy2RUo0bdK+/Xx8H7juZi/YrI93BoVPlqpBrpSLo5YmjDXXK6Q1Yb72Um5c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iCaX+/Kl; arc=none smtp.client-ip=209.85.218.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iCaX+/Kl" Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-c1686e23b9cso12637166b.0 for ; Wed, 15 Jul 2026 04:59:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116765; x=1784721565; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=KjYDN+7EM7Yvzl2aRWCfwJrzMD6Nz8HQwAfKQduR34E=; b=iCaX+/KlHkY2cnVqf+ywa8VQcA09FQmL7YbzHJD/iEO+Eq06VltgcHvQpY2Ue9Yhw5 n9K4lxf1GPaD1TQEZdKdG0gTh7Kpsn6e/zLCPuiQyOYdEj0e9m1ERPMtjiDvvkq12UUq CcG66A+dsQh2I4rMTu2rPJ5rVNVBLsREhX2bnZsyNYnazJn6E7dn5dCale042StMJDn8 A+udKPssTr04CXf7Yz8lHvinJAMiZwEkwlPdfB8FpxXziNmeOTYkZ1X16jWZpCAEcEQ2 KtdDFjGWsO5+UMXM/WtX07SxnEd1tbemlITE9sPoMjtEibzmLz1YYp14Mm1FZ+gy2hvb ofOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116765; x=1784721565; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=KjYDN+7EM7Yvzl2aRWCfwJrzMD6Nz8HQwAfKQduR34E=; b=ge0KmSIG91CAYEyPnE45T25TTBey44AXEROPyuI3AVfPkdamwrCGW6/GTGy4FD+4x9 Ghbk8DmXVfSiJlSzT0UkbYhOSeJbcPhPedOzfWj4T1DcyVebPdDipN23q3kPldlO80+L S77+WO0jV1zNM9Pc8VGrF48ju/i00cg2lRd5nm0oqt5u89SIC5hypOo3q2ZPZyBo9yJ9 vWir0mQ9JesqYrN9d+djmitxpZKHMAdK8hJS9sW0kYkP9/Unz2D4Gu8kljUY7TDpCL6a jAl41Rao+Vyz/0Y7GoF8SCdaRqs0NLb6jXdzwQl6Jt4kQGX9Gh2MOvIZqxxvW4m2Hh3O 1X9Q== X-Forwarded-Encrypted: i=1; AHgh+RqSt+IAn2LS5t5cw7wqIC2IvoJs02ZCdL+OIIdxxy/JzWR5B/3x4NP7XEmF4KhBATMnyCTXwbVcuQoducY=@vger.kernel.org X-Gm-Message-State: AOJu0YwedhT54uQYXVEbuqqRint/i2oGQ6hlwlXpz6QseqdaxgzcM2Qt jVQTSFP64SMfCBeSOaSasOoD1tGYv50ciRDnCqyRXxLP5hF1LVwo6Iu38zWmShwXIsQVGpInMvi +/MBNevE4HDBWRQ== X-Received: from ejjr20.prod.google.com ([2002:a17:906:7054:b0:c12:9227:4a93]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:1804:b0:c12:34ed:da0b with SMTP id a640c23a62f3a-c1667b099abmr363811866b.55.1784116765282; Wed, 15 Jul 2026 04:59:25 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:51 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-11-smostafa@google.com> Subject: [PATCH v7 10/24] iommu/arm-smmu-v3-kvm: Add SMMUv3 driver From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Jean-Philippe Brucker , Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jean-Philippe Brucker Add the skeleton for an Arm SMMUv3 driver at EL2. The driver rely on an array of SMMUv3s on the system, where at init it will donate the array and the resources of the SMMUv3s so they can't be changed by the host after de-privilege. This array will be populated in the next patch. Signed-off-by: Jean-Philippe Brucker Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/hyp/nvhe/Makefile | 5 ++ drivers/iommu/arm/Kconfig | 9 ++ .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 87 +++++++++++++++++++ .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 27 ++++++ 4 files changed, 128 insertions(+) create mode 100644 drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c create mode 100644 drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Mak= efile index b3e7473f18d2..dcb75fb5b4f1 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -33,6 +33,11 @@ hyp-obj-$(CONFIG_LIST_HARDENED) +=3D list_debug.o hyp-obj-$(CONFIG_NVHE_EL2_TRACING) +=3D trace.o events.o hyp-obj-y +=3D $(lib-objs) =20 +HYP_SMMU_V3_DRV_PATH =3D ../../../../../drivers/iommu/arm/arm-smmu-v3 + +hyp-obj-$(CONFIG_ARM_SMMU_V3_PKVM) +=3D $(HYP_SMMU_V3_DRV_PATH)/pkvm/arm-s= mmu-v3.o \ + $(HYP_SMMU_V3_DRV_PATH)/arm-smmu-v3-common-lib.o + # Path to simple_ring_buffer.c CFLAGS_trace.nvhe.o +=3D -I$(srctree)/kernel/trace/ =20 diff --git a/drivers/iommu/arm/Kconfig b/drivers/iommu/arm/Kconfig index 5fac08b89dee..916f4723238d 100644 --- a/drivers/iommu/arm/Kconfig +++ b/drivers/iommu/arm/Kconfig @@ -141,3 +141,12 @@ config QCOM_IOMMU select ARM_DMA_USE_IOMMU help Support for IOMMU on certain Qualcomm SoCs. + +config ARM_SMMU_V3_PKVM + bool "ARM SMMUv3 support for protected Virtual Machines" + depends on KVM && ARM64 && ARM_SMMU_V3=3Dy + help + Enable a SMMUv3 driver in the KVM hypervisor, to protect VMs against + memory accesses from devices owned by the host. + + Say Y here if you intend to enable KVM in protected mode. diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c new file mode 100644 index 000000000000..fee5db6c9c20 --- /dev/null +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * pKVM hyp driver for the Arm SMMUv3 + * + * Copyright (C) 2022 Linaro Ltd. + */ +#include + +#include +#include + +#include "arm_smmu_v3.h" + +size_t __ro_after_init kvm_hyp_arm_smmu_v3_count; +struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; + +#define for_each_smmu(smmu) \ + for ((smmu) =3D kvm_hyp_arm_smmu_v3_smmus; \ + (smmu) !=3D &kvm_hyp_arm_smmu_v3_smmus[kvm_hyp_arm_smmu_v3_count]; \ + (smmu)++) + +/* Put the device in a state that can be probed by the host driver. */ +static void smmu_deinit_device(struct hyp_arm_smmu_v3_device *smmu) +{ + WARN_ON(__pkvm_hyp_donate_host_mmio(smmu->mmio_addr, smmu->mmio_size)); + smmu->base =3D NULL; +} + +static int smmu_init_device(struct hyp_arm_smmu_v3_device *smmu) +{ + unsigned long haddr; + int ret; + + if (!PAGE_ALIGNED(smmu->mmio_addr | smmu->mmio_size)) + return -EINVAL; + + ret =3D __pkvm_host_donate_hyp_mmio(smmu->mmio_addr, smmu->mmio_size, &ha= ddr); + if (ret) + return ret; + + smmu->base =3D (void __iomem *)haddr; + + return 0; +} + +/* Called while is the host is still trusted. */ +static int smmu_init(void) +{ + size_t smmu_arr_size =3D PAGE_ALIGN(sizeof(*kvm_hyp_arm_smmu_v3_smmus) * + kvm_hyp_arm_smmu_v3_count); + struct hyp_arm_smmu_v3_device *smmu; + u64 pfn, nr_pages; + int ret; + + kvm_hyp_arm_smmu_v3_smmus =3D kern_hyp_va(kvm_hyp_arm_smmu_v3_smmus); + pfn =3D hyp_virt_to_pfn(kvm_hyp_arm_smmu_v3_smmus); + nr_pages =3D smmu_arr_size >> PAGE_SHIFT; + + ret =3D __pkvm_host_donate_hyp(pfn, nr_pages); + if (ret) + return ret; + + for_each_smmu(smmu) { + ret =3D smmu_init_device(smmu); + if (ret) + goto out_reclaim_smmu; + } + + return 0; + +out_reclaim_smmu: + while (smmu !=3D kvm_hyp_arm_smmu_v3_smmus) + smmu_deinit_device(--smmu); + WARN_ON(__pkvm_hyp_donate_host(pfn, nr_pages)); + return ret; +} + +static int smmu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, int = prot) +{ + return 0; +} + +/* Shared with the kernel driver in EL1 */ +struct pkvm_iommu_ops smmu_ops =3D { + .init =3D smmu_init, + .host_stage2_idmap =3D smmu_host_stage2_idmap, +}; diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h new file mode 100644 index 000000000000..0d9e48b201f5 --- /dev/null +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_ARM_SMMU_V3_H +#define __KVM_ARM_SMMU_V3_H + +#include + +/* + * Parameters from the trusted host: + * @mmio_addr base address of the SMMU registers + * @mmio_size size of the registers resource + * + * Other members are filled and used at runtime by the SMMU driver. + * @base Virtual address of SMMU registers + */ +struct hyp_arm_smmu_v3_device { + phys_addr_t mmio_addr; + size_t mmio_size; + void __iomem *base; +}; + +extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); +#define kvm_hyp_arm_smmu_v3_count kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count) + +extern struct hyp_arm_smmu_v3_device *kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_smm= us); +#define kvm_hyp_arm_smmu_v3_smmus kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_smmus) + +#endif /* __KVM_ARM_SMMU_V3_H */ --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCB2847277C for ; Wed, 15 Jul 2026 11:59:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116773; cv=none; b=PPsI5kWADeECqPiCcVfZzySoW5wwef/WzBzvilX5BNzzokA0o78nPTaV8CeNKrINRySZevePivqWw7I2Qqy+q/DdytESxF7p3KtyjrlgXmBvrWx3BoyqaMDi52uvAG1Toc+yMCARfGn1AUw7acSBjt7hcZSzw3EoSBOwbeSD5LU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116773; c=relaxed/simple; bh=RDXLlQm+YXHmI97/58mng4V0anO13EP7q1SkxoaFHpc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NAp4huZf8o9GceGGHddZFM9edb234/xndPxT0c6cr6Yo0yrheT+2dY7MQr7/FsZlKO2F/tE/6q+ghMK0cSVJc61VkVK1YVzJ4spk5Yk73/76BMAIeiPNgECWQvCkM2UHS17NWiuzM4Vm2pz+BvHCg9woT/bPbeClUnflwCQ2mGg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wQBZPR19; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wQBZPR19" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-49244130073so57068345e9.1 for ; Wed, 15 Jul 2026 04:59:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116767; x=1784721567; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=fxl8rp8wqmkZLCdj3/pqUays/JOlw0FMpgaoC2sElPo=; b=wQBZPR19r74FKtfBUQZN7zQc6Z5Sl0Woscu6fFevR+/nbdjbn3/dWzfUmaOU42JF6H X4Jkc0X7PxyMNY296eBEpnssEgtrmsZ70psInF8l1Qh4qIOGKCnPre/pDFONeFsUf0fO t80ZROXykcWxRhwgmv0FUJIEPtfF0zNu1Nif22uK8aQJR3KksSnxA/q5p2/7iBkSchw9 XnaWrlGHrGXe08EUysd/MTPyopFq8IO/jCDFmgcwXr4z+f+y5ARZYSb2hkygSCZ1Q8Me JORJ4VjIhMF6qO/qnfJoTW1x9eWEx08LbNHwBvxWy93fk4Ia4k61M7axDQI1GRYo11uK bE2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116767; x=1784721567; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fxl8rp8wqmkZLCdj3/pqUays/JOlw0FMpgaoC2sElPo=; b=Oo6k8ujyPqpApJ9JgiXjJ/OzvrrV48pZfWXKVKdAT3nb5xg5HPr+OpdVV6T4thT5xO oNCs+p93RCd1J6xCGATrHun8JuKIJazIbuOUozOEIMPJvKPyayWBMzWUZJBLTOWGPKg+ AHCVJBC8xaP0tcLpuqpnX4/CigenwB+DG2/4TiwCaJLMa2wgZSmSLpz2xbZ0KfiuBfVA PhQyOc0h9E8hEygqKSHiaXI6tjcI0tSnWKYCDfO/Z3ONXxF+XdGNF1RdIvTkOG8E8P2o nNSvxC3ilY+vKgW/j5gJnIsdZXsczJWydxMNMB1IuGqg5+LNNfqb7vCaeqaiOSXnAyBT VSFg== X-Forwarded-Encrypted: i=1; AHgh+Rog0E4JoiDIgG2KNL/emwH00L39L0bNIDQ4kJMnLcylJqCC+AUBp6RTJA4WUp4u5CtuvhkaKtrpWWJ/QaY=@vger.kernel.org X-Gm-Message-State: AOJu0Yxum+wxLJ+8aQLWbuskB1Jpe9M+Qhp3v4YHLJwxBSrRnjsmtpLo WwQ0OymC213zds11bq/4WrMp6x1gljfaui2Ac05yptqydLGc+kTjjt7pE2CHPGx4/o+2diRcOWb UlvayYcq8HjzVRg== X-Received: from wmbdt5.prod.google.com ([2002:a05:600c:6305:b0:493:e190:29a1]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3486:b0:493:b7a6:3dac with SMTP id 5b1f17b1804b1-4953b156ee2mr37101555e9.33.1784116766396; Wed, 15 Jul 2026 04:59:26 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:52 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-12-smostafa@google.com> Subject: [PATCH v7 11/24] iommu/arm-smmu-v3-kvm: Add the kernel driver From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When KVM runs in protected mode, and CONFIG_ARM_SMMU_V3_PKVM is enabled, it will manage the SMMUv3 HW using trap and emulate and present emulated SMMUs to the host kernel. In that case, those SMMUs will be on the aux bus, so make it possible to the driver to probe those devices. Otherwise, everything else is the same as the KVM emulation complies with the architecture,so the driver doesn't need to be modified. Suggested-by: Jason Gunthorpe Signed-off-by: Mostafa Saleh --- drivers/iommu/arm/arm-smmu-v3/Makefile | 1 + .../iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c | 189 ++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 43 ++++ .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 2 + 4 files changed, 235 insertions(+) create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c diff --git a/drivers/iommu/arm/arm-smmu-v3/Makefile b/drivers/iommu/arm/arm= -smmu-v3/Makefile index c9ce392e6d31..c3fc5c4a4a1e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/Makefile +++ b/drivers/iommu/arm/arm-smmu-v3/Makefile @@ -4,5 +4,6 @@ arm_smmu_v3-y :=3D arm-smmu-v3.o arm-smmu-v3-common-lib.o arm_smmu_v3-$(CONFIG_ARM_SMMU_V3_IOMMUFD) +=3D arm-smmu-v3-iommufd.o arm_smmu_v3-$(CONFIG_ARM_SMMU_V3_SVA) +=3D arm-smmu-v3-sva.o arm_smmu_v3-$(CONFIG_TEGRA241_CMDQV) +=3D tegra241-cmdqv.o +arm_smmu_v3-$(CONFIG_ARM_SMMU_V3_PKVM) +=3D arm-smmu-v3-kvm.o =20 obj-$(CONFIG_ARM_SMMU_V3_KUNIT_TEST) +=3D arm-smmu-v3-test.o diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-kvm.c new file mode 100644 index 000000000000..68b78ed933d4 --- /dev/null +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c @@ -0,0 +1,189 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * pKVM host driver for the Arm SMMUv3 + * + * Copyright (C) 2022 Linaro Ltd. + */ +#include +#include + +#include +#include +#include +#include + +#include "arm-smmu-v3.h" +#include "pkvm/arm_smmu_v3.h" + +extern struct pkvm_iommu_ops kvm_nvhe_sym(smmu_ops); + +static size_t kvm_arm_smmu_count; +static struct hyp_arm_smmu_v3_device *kvm_arm_smmu_array; +static size_t kvm_arm_smmu_cur; + +static void kvm_arm_smmu_array_free(void) +{ + int order; + + order =3D get_order(kvm_arm_smmu_count * sizeof(*kvm_arm_smmu_array)); + free_pages((unsigned long)kvm_arm_smmu_array, order); +} + +static int kvm_arm_smmu_array_alloc(void) +{ + int smmu_order; + struct device_node *np; + + for_each_compatible_node(np, NULL, "arm,smmu-v3") + kvm_arm_smmu_count++; + + if (!kvm_arm_smmu_count) + return -ENODEV; + smmu_order =3D get_order(kvm_arm_smmu_count * sizeof(*kvm_arm_smmu_array)= ); + kvm_arm_smmu_array =3D (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, = smmu_order); + if (!kvm_arm_smmu_array) + return -ENOMEM; + return 0; +} + +static unsigned int smmu_hyp_pgt_pages(void) +{ + struct device_node *np =3D of_find_compatible_node(NULL, NULL, "arm,smmu-= v3"); + + /* + * SMMUv3 uses the same format as the CPU stage-2 and hence have the same= memory + * requirements, we add extra 500 pages for L2 STEs. + * Only one set of memory is allocated as the page table is shared betwee= n all + * the SMMUs. + */ + if (np) { + of_node_put(np); + return host_s2_pgtable_pages() + 500; + } + + return 0; +} + +static struct platform_driver smmuv3_nesting_driver; +static int smmuv3_nesting_probe(struct platform_device *pdev) +{ + struct hyp_arm_smmu_v3_device *smmu =3D &kvm_arm_smmu_array[kvm_arm_smmu_= cur]; + struct device *dev =3D &pdev->dev; + struct resource *res; + + /* Only device tree, ACPI not supported. */ + if (!dev->of_node) + return -EINVAL; + + if (kvm_arm_smmu_cur >=3D kvm_arm_smmu_count) + return -ENOSPC; + + res =3D platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) + return -ENODEV; + + if (of_property_read_bool(dev->of_node, "cavium,cn9900-broken-page1-regsp= ace")) + return -EINVAL; + + smmu->mmio_addr =3D res->start; + smmu->mmio_size =3D resource_size(res); + if (smmu->mmio_size < SZ_128K) { + dev_err(dev, "MMIO region too small(%pr)\n", res); + return -EINVAL; + } + + if (of_dma_is_coherent(dev->of_node)) + smmu->features |=3D ARM_SMMU_FEAT_COHERENCY; + + kvm_arm_smmu_cur++; + return 0; +} + +static int __init kvm_arm_smmu_v3_register(void) +{ + size_t nr_pages =3D smmu_hyp_pgt_pages(); + int ret; + + if (!is_protected_kvm_enabled() || !nr_pages) + return 0; + + ret =3D kvm_arm_smmu_array_alloc(); + if (ret) + goto out_err; + + ret =3D platform_driver_probe(&smmuv3_nesting_driver, smmuv3_nesting_prob= e); + if (ret) + goto out_free; + + ret =3D pkvm_iommu_register_driver(kern_hyp_va(lm_alias(&kvm_nvhe_sym(smm= u_ops))), + nr_pages); + if (ret) + goto out_unregister; + + /* + * These variables are stored in the nVHE image, and won't be accessible + * after KVM initialization. Ownership of kvm_arm_smmu_array will be + * transferred to the hypervisor as well. + */ + kvm_hyp_arm_smmu_v3_smmus =3D kvm_arm_smmu_array; + kvm_hyp_arm_smmu_v3_count =3D kvm_arm_smmu_cur; + return ret; + +out_unregister: + platform_driver_unregister(&smmuv3_nesting_driver); +out_free: + kvm_arm_smmu_array_free(); +out_err: + kvm_arm_smmu_count =3D 0; + kvm_arm_smmu_array =3D NULL; + return ret; +}; + +static int smmu_create_aux_device(struct device *dev, void *data) +{ + static int dev_id; + struct auxiliary_device *auxdev; + + auxdev =3D __devm_auxiliary_device_create(dev, "protected_kvm", + "smmu_v3_emu", NULL, dev_id++); + if (!auxdev) + return -ENODEV; + + auxdev->dev.parent =3D dev; + return 0; +} + +static int kvm_arm_smmu_v3_post_init(void) +{ + if (!kvm_arm_smmu_count) + return 0; + + /* + * If the hypervisor part of the driver fails, KVM will not initialise. + */ + if (!is_kvm_arm_initialised()) { + kvm_arm_smmu_array_free(); + platform_driver_unregister(&smmuv3_nesting_driver); + return 0; + } + + WARN_ON(driver_for_each_device(&smmuv3_nesting_driver.driver, NULL, + NULL, smmu_create_aux_device)); + + return 0; +} + +static const struct of_device_id smmuv3_nested_of_match[] =3D { + { .compatible =3D "arm,smmu-v3", }, + { }, +}; + +static struct platform_driver smmuv3_nesting_driver =3D { + .driver =3D { + .name =3D "smmuv3-nesting", + .of_match_table =3D smmuv3_nested_of_match, + .suppress_bind_attrs =3D true, + }, +}; +late_initcall(kvm_arm_smmu_v3_post_init); +subsys_initcall(kvm_arm_smmu_v3_register); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 674ff98706f6..49728d8076d9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -11,6 +11,7 @@ =20 #include #include +#include #include #include #include @@ -5311,6 +5312,48 @@ static struct platform_driver arm_smmu_driver =3D { module_driver(arm_smmu_driver, platform_driver_register, arm_smmu_driver_unregister); =20 +#ifdef CONFIG_ARM_SMMU_V3_PKVM +/* + * Now we have 2 devices, the aux device bound to this driver, and pdev + * which is the physical platform device bound to the KVM driver but not u= sed. + * However, this driver keeps using the platform device for 2 reasons: + * 1) Simplicity: Avoiding changing big parts of the code assuming + * the underlying device is a platform device. + * 2) Dealing with DMA-API, irqs(MSIs), RPM... requires the physical devic= e. + */ + +static int arm_smmu_device_probe_emu(struct auxiliary_device *auxdev, + const struct auxiliary_device_id *id) +{ + struct device *parent =3D auxdev->dev.parent; + + dev_info(&auxdev->dev, "Probing from %s\n", dev_name(parent)); + return arm_smmu_device_probe(to_platform_device(parent)); +} + +static void arm_smmu_device_remove_emu(struct auxiliary_device *auxdev) +{ + arm_smmu_device_remove(to_platform_device(auxdev->dev.parent)); +} + +const struct auxiliary_device_id arm_smmu_aux_table[] =3D { + { .name =3D "protected_kvm.smmu_v3_emu" }, + { }, +}; + +struct auxiliary_driver arm_smmu_driver_emu =3D { + .driver =3D { + .suppress_bind_attrs =3D true, + }, + .name =3D "arm-smmu-v3-emu", + .id_table =3D arm_smmu_aux_table, + .probe =3D arm_smmu_device_probe_emu, + .remove =3D arm_smmu_device_remove_emu, +}; + +module_auxiliary_driver(arm_smmu_driver_emu); +#endif + MODULE_DESCRIPTION("IOMMU API for ARM architected SMMUv3 implementations"); MODULE_AUTHOR("Will Deacon "); MODULE_ALIAS("platform:arm-smmu-v3"); diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h index 0d9e48b201f5..744ee2b7f0b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -8,6 +8,7 @@ * Parameters from the trusted host: * @mmio_addr base address of the SMMU registers * @mmio_size size of the registers resource + * @features Features of SMMUv3, subset of the main driver * * Other members are filled and used at runtime by the SMMU driver. * @base Virtual address of SMMU registers @@ -16,6 +17,7 @@ struct hyp_arm_smmu_v3_device { phys_addr_t mmio_addr; size_t mmio_size; void __iomem *base; + u32 features; }; =20 extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42C6844CADF for ; Wed, 15 Jul 2026 11:59:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116776; cv=none; b=thtRPlZ/sfvjesixPXd/mZB3cF5VFg336L1xWyUbe/ZV+VBtPFlhdKeuTASa2k+kZfk8uy7IkYdIim1CNKfYPV7DsGfcT1hfVI2HQzMmdQz8SjcblDcaRGFlhloRMg88/zc0zGMWeIHTdYYQdTkvMOrFETF4xC3KxeDfpoVWRXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116776; c=relaxed/simple; bh=avKKUR3ivX1o4eXWAvFx7D4B/bGHctrz504BKkAdasI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZI8EC6yvMz1g3mdCXghj57k9YCaGzeHGJBKxthfAdiNu3dlwptQdmo1AboaOhJzNtvzaXHea7Q7AcPal6Ae8dTYxvqPtz8mpGlmbe9ZgC9VIGNwgvi3xSZN6atnHMB7j35CdfSimO+YSkZin61C9tjN9q1Qtit4Np9bU2AMAit8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=n1m7OhFM; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n1m7OhFM" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-695f710d929so2516268a12.2 for ; Wed, 15 Jul 2026 04:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116768; x=1784721568; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=U+wIzMWf2cSB6EX/Jc8f2fGh3ASjjz6WosFQJCh8Jwc=; b=n1m7OhFMYzJ1n4M95KxVcch5aHdSsPCcpa2KTjmIqjfWdAAtrIYdBEX0rHzcq/Q0WL jGmKlW1NnteE5i1hjq7BfXbnwmN3CLK433OAkW/szhn6gDuZk44TyrO4uUf8XL/1TCEK K9rrCoSkA1liuHXqp6XJof0BA7oAalGXJhDHgUAYMHU0JHU+ontL0x2jlX1VXaOZJ9qp MEeRMSlNqFzfde3QAAPlV/CC7psJLiWj44PYMmW0AGW23C2M9Pf4UmXcqi63wArlxPPK iME2RXKi/kazwL0jmtDG75P3V8nQANMojS85TtwYa3ECNwBEIUdIDqGq7aP8WE7S0PVa OU3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116768; x=1784721568; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=U+wIzMWf2cSB6EX/Jc8f2fGh3ASjjz6WosFQJCh8Jwc=; b=kP9qP2UKzR6i7srv+BD7hAQdbwry+2ppN8NgFRJydxpcSItMRtxhqg69xIG9BiLDEE 1OAyTdYM2hZD99CL/oD52IIlLpbvev8MH9uxVKB01LS8YHWMQv3X5ZC0sL2c6nX9Hui/ S5Znyk/aI+q8GWHdDyDPiAFUWT6kw2Sztno3YslPBCFq2fs84DDtD3hK+o1NAN2xU3g/ TgMhtViQRNV+R4cnlIo/nkAPOcmCsxCY3B95qpvxcgcqmgJ+xSkYH1eKArpKCEVXAAqu G3qk/uklUkkH76H85GBh087MM1Qx+jeyhmVXJnHt+OjheuaLv9NNZ7dmO0uddyfVo0EQ 8jNA== X-Forwarded-Encrypted: i=1; AHgh+RrAo9QErYpXgfjnO5HlBUAD/6NuzTbDwj8l1tpIMUerLz72+Qx2COSNp15UKK8W4CJIuiRRlsvyYzUQkA8=@vger.kernel.org X-Gm-Message-State: AOJu0YzObo6wbUrWUkvKKoxEjJhYl35LHsFPl880AjMFXI5yCQgmyfev T+DJdZBnfC1Yr+/BC7jrboLaguygQvI3m7ULcYwpR/EzEZtwEZH1iu3xmW1vrdH8egkPfX1Sqtv rf9NFY++GSgIAGQ== X-Received: from edou7.prod.google.com ([2002:aa7:d0c7:0:b0:69c:458d:1598]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:1d4b:b0:69c:7754:f8b7 with SMTP id 4fb4d7f45d1cf-69c7754f940mr8902542a12.38.1784116767878; Wed, 15 Jul 2026 04:59:27 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:53 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-13-smostafa@google.com> Subject: [PATCH v7 12/24] iommu/arm-smmu-v3-kvm: Probe SMMU HW From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Probe SMMU features from the IDR register space, most of the logic is common with the kernel. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 54 +++++++++++++++++++ .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 6 +++ 2 files changed, 60 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index fee5db6c9c20..be5922d80184 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -10,6 +10,7 @@ #include =20 #include "arm_smmu_v3.h" +#include "../arm-smmu-v3.h" =20 size_t __ro_after_init kvm_hyp_arm_smmu_v3_count; struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; @@ -26,6 +27,53 @@ static void smmu_deinit_device(struct hyp_arm_smmu_v3_de= vice *smmu) smmu->base =3D NULL; } =20 +/* + * Mini-probe and validation for the hypervisor. + */ +static int smmu_probe(struct hyp_arm_smmu_v3_device *smmu) +{ + u32 reg; + + /* Similar to the kernel, rely on firmware override. */ + if (!(smmu->features & ARM_SMMU_FEAT_COHERENCY)) + return -EINVAL; + + /* IDR0 */ + reg =3D readl_relaxed(smmu->base + ARM_SMMU_IDR0); + + smmu->features |=3D smmu_idr0_features(reg); + if (!(smmu->features & (ARM_SMMU_FEAT_TT_LE | ARM_SMMU_FEAT_TT_BE))) + return -ENXIO; + + reg =3D readl_relaxed(smmu->base + ARM_SMMU_IDR1); + if (reg & (IDR1_TABLES_PRESET | IDR1_QUEUES_PRESET | IDR1_REL)) + return -EINVAL; + + smmu->sid_bits =3D FIELD_GET(IDR1_SIDSIZE, reg); + /* Follows the kernel logic */ + if (smmu->sid_bits <=3D STRTAB_SPLIT) + smmu->features &=3D ~ARM_SMMU_FEAT_2_LVL_STRTAB; + + reg =3D readl_relaxed(smmu->base + ARM_SMMU_IDR3); + smmu->features |=3D smmu_idr3_features(reg); + + reg =3D readl_relaxed(smmu->base + ARM_SMMU_IDR5); + smmu->pgsize_bitmap =3D smmu_idr5_to_pgsize(reg); + + smmu->oas =3D smmu_idr5_to_oas(reg); + if (smmu->oas =3D=3D 52) + smmu->pgsize_bitmap |=3D 1ULL << 42; + else if (!smmu->oas) + smmu->oas =3D 48; + + reg =3D readl_relaxed(smmu->base + ARM_SMMU_IIDR); + smmu->features =3D smmu_iidr_features(reg, smmu->features); + if (!(smmu->features & ARM_SMMU_FEAT_NESTING)) + return -ENXIO; + + return 0; +} + static int smmu_init_device(struct hyp_arm_smmu_v3_device *smmu) { unsigned long haddr; @@ -39,8 +87,14 @@ static int smmu_init_device(struct hyp_arm_smmu_v3_devic= e *smmu) return ret; =20 smmu->base =3D (void __iomem *)haddr; + ret =3D smmu_probe(smmu); + if (ret) + goto out_ret; =20 return 0; +out_ret: + smmu_deinit_device(smmu); + return ret; } =20 /* Called while is the host is still trusted. */ diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h index 744ee2b7f0b4..82b84673e85b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -12,12 +12,18 @@ * * Other members are filled and used at runtime by the SMMU driver. * @base Virtual address of SMMU registers + * @oas PA size + * @pgsize_bitmap Supported page sizes + * @sid_bits Max number of SID bits supported */ struct hyp_arm_smmu_v3_device { phys_addr_t mmio_addr; size_t mmio_size; void __iomem *base; u32 features; + unsigned long oas; + unsigned long pgsize_bitmap; + unsigned int sid_bits; }; =20 extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D928A4657F5 for ; Wed, 15 Jul 2026 11:59:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116776; cv=none; b=Aac3DflN9cNtD/jAOZ1cw/lpNn0qL6lkQc20k8+DZZeaH6qR7LHmw4eSngUW45wKPRi1oiWjY5BE0N857LUg+tJvYiqFjPXXuDdaXAjCALGX/mX2i/DPQaAaxo2sMJ3c55rFfsWAwp5Rzwa1N9B05ZPi95j1F6gCLDyJFygYljU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116776; c=relaxed/simple; bh=Vc51+tDXkTxNkQKt0ujm+ycuJbTsubABwtHPycKRIE0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=nW3A3LBOqA5rkerye0VzMr4fvMGuwgFHGHAcDkvsxSdSLkP0a+RujfAH05gG6Z/KmTlqTNJ4huUGGnyiAPIdJHAQhvQgu3s8DNU0y47HHCzLJ3TYos4K6xeJRB6xrMsTlRj88kQUcsh2jgVWLyOZ/wEUn1xuTyTpNF2IQnZATkQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IFRgPtkQ; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IFRgPtkQ" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493c526df6bso18092755e9.3 for ; Wed, 15 Jul 2026 04:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116769; x=1784721569; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=wRJ18s+161s2edooAsExJtsANoe4PXbtE7ux+6Z/9CQ=; b=IFRgPtkQgA9ui1gfUJgbMJr7ToK0+ml/a8cpuWcKDEnTZ632mrRlX6RYC+e8yST76G OWxSZ0PQnAiywih8YfGBLFID1XGOzSexlNgFMiDgh8UUDw6O1VYZ6kl5z1IGks/QcF2z d7Le+5vXi+jt1M/5KiKaV4VDkNpuzcjrDAptjOB/2RnHwwVbMnevEi0QPzv53Kvr20hz feslzIJI290B9W3bcssoheOuapM/CFuU4YbppwZXVJ/n0BIItNeZdjyEN1tcqSM4wBLE v4uX8pCtXqC0h96SA8LZvAu6Ajjm/EoX5UEOGx+eTNzrTPJlw/XzG0j76NBjuPvm1VJ3 Akrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116769; x=1784721569; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=wRJ18s+161s2edooAsExJtsANoe4PXbtE7ux+6Z/9CQ=; b=WBjlsflKOde8TTbGgX4KZMDAg2pLqn622jNkIoO2Z1qLF0zZkrS1mG1c8zv3SzuU/O JbnHDgRN1so9XB6Ql7GKhED/82KMaNqaRQB9qjIyuuTs9zUb4o4mblgP/VIwIJllqT7I R5aVm4beYyeXysppWRL3LN4Uk6cRDHk4sz85C+/gOPMSj87NHr/z0GrEVZf7AbLKzLzA eDixSVf3LZETPxqGX5fGi+chmkyencN1aTbqfl0PMTA1lwCHkBhAAp/rsvjwoiiNaWvq E6AzYQZd4i5kRQTGKcKqIrfGY8QJUomdELHVJdNFE5HuBG2DtE661FubYPLQ0srnEugi xtSA== X-Forwarded-Encrypted: i=1; AHgh+Rp0Yu12rmRyigdDXSCTRjLZiUV12l/4YM553OVimMD2H/iLn8jZwpLj6aZ5uo9u9I5vKALdN++k6+uxVD8=@vger.kernel.org X-Gm-Message-State: AOJu0Yyv+4s9nj1GdApfnuUAZOYmKLM4x/RfVmAiOav1YNn9d5AWHTyy V5Zkua0MfN+tiYy+nwYG3S3Ejb+xmNiMX0JLhMRFLFWfbSWXJ3ayOr1pxxzn5YTrkrjUobCIg1j Ne1gytn7jn0VYhw== X-Received: from wmaq7.prod.google.com ([2002:a05:600c:6c87:b0:493:b328:830b]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3e19:b0:493:f28e:462a with SMTP id 5b1f17b1804b1-493f87e6ba2mr184102835e9.12.1784116769008; Wed, 15 Jul 2026 04:59:29 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:54 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-14-smostafa@google.com> Subject: [PATCH v7 13/24] iommu/arm-smmu-v3-kvm: Add MMIO emulation From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add data abort handler for the SMMUs, at the moment most registers are just passthrough. In the next patches CMDQ/STE emulation will be added which inserts logic to some register access. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 159 ++++++++++++++++++ .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 14 ++ 2 files changed, 173 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index be5922d80184..af06c832fc6f 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -8,6 +8,7 @@ =20 #include #include +#include =20 #include "arm_smmu_v3.h" #include "../arm-smmu-v3.h" @@ -82,6 +83,8 @@ static int smmu_init_device(struct hyp_arm_smmu_v3_device= *smmu) if (!PAGE_ALIGNED(smmu->mmio_addr | smmu->mmio_size)) return -EINVAL; =20 + hyp_spin_lock_init(&smmu->lock); + hyp_spin_lock_init(&smmu->hw_lock); ret =3D __pkvm_host_donate_hyp_mmio(smmu->mmio_addr, smmu->mmio_size, &ha= ddr); if (ret) return ret; @@ -120,6 +123,8 @@ static int smmu_init(void) goto out_reclaim_smmu; } =20 + BUILD_BUG_ON(sizeof(hyp_spinlock_t) !=3D sizeof(u32)); + return 0; =20 out_reclaim_smmu: @@ -129,6 +134,159 @@ static int smmu_init(void) return ret; } =20 +static bool smmu_dabt_device(struct hyp_arm_smmu_v3_device *smmu, + struct user_pt_regs *regs, + u64 esr, u32 off) +{ + bool is_write =3D esr & ESR_ELx_WNR; + unsigned int len =3D BIT((esr & ESR_ELx_SAS) >> ESR_ELx_SAS_SHIFT); + int rd =3D (esr & ESR_ELx_SRT_MASK) >> ESR_ELx_SRT_SHIFT; + const u64 read_write =3D -1ULL; + const u64 no_access =3D 0; + u64 mask =3D no_access; + const u64 read_only =3D is_write ? no_access : read_write; + bool is_xzr =3D (rd =3D=3D 31); + u64 val =3D is_xzr ? 0 : regs->regs[rd]; + + switch (off) { + case ARM_SMMU_IDR0: + if (len !=3D sizeof(u32)) + break; + /* + * Clear stage-2 support, hide MSI to avoid write back to cmdq, + * and hide ATS as it is not emulated or handled in TLB invalidation. + */ + mask =3D read_only & ~(IDR0_S2P | IDR0_VMID16 | IDR0_MSI | IDR0_HYP | ID= R0_ATS); + break; + /* Passthrough the register access for bisectiblity, handled later */ + case ARM_SMMU_CMDQ_BASE: + case ARM_SMMU_CMDQ_PROD: + case ARM_SMMU_CMDQ_CONS: + case ARM_SMMU_STRTAB_BASE: + case ARM_SMMU_STRTAB_BASE_CFG: + case ARM_SMMU_GBPA: + mask =3D read_write; + break; + case ARM_SMMU_CR0: + if (len !=3D sizeof(u32)) + break; + mask =3D read_write; + break; + case ARM_SMMU_CR1: { + if (len !=3D sizeof(u32)) + break; + /* Don't mess with shareability/cacheability. */ + if (is_write && + (val !=3D (FIELD_PREP(CR1_TABLE_SH, ARM_SMMU_SH_ISH) | + FIELD_PREP(CR1_TABLE_OC, CR1_CACHE_WB) | + FIELD_PREP(CR1_TABLE_IC, CR1_CACHE_WB) | + FIELD_PREP(CR1_QUEUE_SH, ARM_SMMU_SH_ISH) | + FIELD_PREP(CR1_QUEUE_OC, CR1_CACHE_WB) | + FIELD_PREP(CR1_QUEUE_IC, CR1_CACHE_WB)))) + break; + + mask =3D read_write; + break; + } + + /* Allowed 32 bit registers. */ + case ARM_SMMU_EVTQ_IRQ_CFG1: + case ARM_SMMU_EVTQ_IRQ_CFG2: + case ARM_SMMU_GERROR_IRQ_CFG1: + case ARM_SMMU_GERROR_IRQ_CFG2: + case ARM_SMMU_PRIQ_IRQ_CFG1: + case ARM_SMMU_PRIQ_IRQ_CFG2: + /* These are RES0 as MSI support is hidden. */ + val =3D 0; + if (!is_write) + goto out_update_regs; + fallthrough; + case ARM_SMMU_EVTQ_PROD + SZ_64K: + case ARM_SMMU_EVTQ_CONS + SZ_64K: + case ARM_SMMU_PRIQ_PROD + SZ_64K: + case ARM_SMMU_PRIQ_CONS + SZ_64K: + case ARM_SMMU_GERRORN: + case ARM_SMMU_IRQ_CTRLACK: + case ARM_SMMU_IRQ_CTRL: + case ARM_SMMU_CR0ACK: + case ARM_SMMU_CR2: + if (len !=3D sizeof(u32)) + break; + mask =3D read_write; + break; + /* Allowed 64 bit registers. */ + case ARM_SMMU_EVTQ_IRQ_CFG0: + case ARM_SMMU_PRIQ_IRQ_CFG0: + case ARM_SMMU_GERROR_IRQ_CFG0: + /* These are RES0 as MSI support is hidden. */ + val =3D 0; + if (!is_write) + goto out_update_regs; + fallthrough; + case ARM_SMMU_EVTQ_BASE: + case ARM_SMMU_PRIQ_BASE: + if (len !=3D sizeof(u64)) + break; + mask =3D read_write; + break; + /* Allowed RO 32 bit registers. */ + case ARM_SMMU_IIDR: + case ARM_SMMU_IDR5: + case ARM_SMMU_IDR3: + case ARM_SMMU_IDR1: + case ARM_SMMU_GERROR: + if (len !=3D sizeof(u32)) + break; + mask =3D read_only; + }; + + if (WARN_ON(!mask)) + goto out_ret; + + hyp_spin_lock(&smmu->hw_lock); + if (is_write) { + if (len =3D=3D sizeof(u64)) + writeq_relaxed(val & mask, smmu->base + off); + else + writel_relaxed(val & mask, smmu->base + off); + + hyp_spin_unlock(&smmu->hw_lock); + return true; + } + + if (len =3D=3D sizeof(u64)) + val =3D readq_relaxed(smmu->base + off) & mask; + else + val =3D readl_relaxed(smmu->base + off) & mask; + hyp_spin_unlock(&smmu->hw_lock); +out_update_regs: + /* + * Device might be read senstive, so do it but ignore writing + * back for xzr. + */ + if (!is_xzr) + regs->regs[rd] =3D val; + +out_ret: + return true; +} + +static bool smmu_dabt_handler(struct user_pt_regs *regs, u64 esr, u64 addr) +{ + struct hyp_arm_smmu_v3_device *smmu; + bool ret; + + for_each_smmu(smmu) { + if (addr < smmu->mmio_addr || addr >=3D smmu->mmio_addr + smmu->mmio_siz= e) + continue; + hyp_spin_lock(&smmu->lock); + ret =3D smmu_dabt_device(smmu, regs, esr, addr - smmu->mmio_addr); + hyp_spin_unlock(&smmu->lock); + return ret; + } + return false; +} + static int smmu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, int = prot) { return 0; @@ -138,4 +296,5 @@ static int smmu_host_stage2_idmap(phys_addr_t start, ph= ys_addr_t end, int prot) struct pkvm_iommu_ops smmu_ops =3D { .init =3D smmu_init, .host_stage2_idmap =3D smmu_host_stage2_idmap, + .dabt_handler =3D smmu_dabt_handler, }; diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h index 82b84673e85b..2bda6e03c96c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -4,6 +4,10 @@ =20 #include =20 +#ifdef __KVM_NVHE_HYPERVISOR__ +#include +#endif + /* * Parameters from the trusted host: * @mmio_addr base address of the SMMU registers @@ -15,6 +19,9 @@ * @oas PA size * @pgsize_bitmap Supported page sizes * @sid_bits Max number of SID bits supported + * @lock Lock to protect SMMU emulation + * @hw_lock Lock to protect SMMU HW (as CMDQ) + Order smmu.lock =3D> host_mmu.lock =3D> smmu.hw_lock */ struct hyp_arm_smmu_v3_device { phys_addr_t mmio_addr; @@ -24,6 +31,13 @@ struct hyp_arm_smmu_v3_device { unsigned long oas; unsigned long pgsize_bitmap; unsigned int sid_bits; +#ifdef __KVM_NVHE_HYPERVISOR__ + hyp_spinlock_t lock; + hyp_spinlock_t hw_lock; +#else + u32 lock; + u32 hw_lock; +#endif }; =20 extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4BD4477999 for ; Wed, 15 Jul 2026 11:59:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116777; cv=none; b=JKbi5REgx7PdnZdw18AVJQSEtgSeeK4eH+nfY8+T+5qbCKNruhl8ux9dgus3RmqkJZUO6FrLat/oUB5t1kLfuvS5aushrGayekiKH1+FRN9e0QBb6GNElrkYXTUW+oVA58+HR9+QtZG/2enthAPwxZi8BL6aujmH+jvN2eokEwc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116777; c=relaxed/simple; bh=RC6AhUBEICYjjzzQL9okCYv+kzzw0t+Dy9J5qJicsbI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KxND4l3Uu70nDMQqd/3YpyUwNrif6wH8pAEtJWNzzrslrWnmfL4zvjCQChP+OPhpLMCQlvIhI4NlFpfJVgunSl2KsSmHPiDS3OvyoPW6KxnRGMdID/vMQcRSzj5ZA4z3R7/wiShAuq0djkhKCf9gdLlYLX9cT6Gnu4BJH4ped0E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IL0XXZoh; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IL0XXZoh" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-49402f0af24so25483425e9.0 for ; Wed, 15 Jul 2026 04:59:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116771; x=1784721571; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:from:to:cc :subject:date:message-id:reply-to:content-type; bh=pVGMaYEcniIvMPOgxM8+baJR4b4kWmoCjAgYLXKet98=; b=IL0XXZohxgfkOz2y0jIVbnrbpGFtqOYdbEkmj5YyzwU/nVaRUQ0KxFkBCN9L/7RUVc MKl1cKqJowxkZhQLRwWyYK7kdeUgtyfqjfkM+h5v2qgH6gUIt7CfNzbyQFzSLsaZ7YCw L0jAyCZcC0uJCYLGL/7Y+XNRLJgm14aC42UTTe8Czgn4lxElU+DWHWiwmCBOU52DNNV9 RunrBjHx8AoroWH1qNwwn+4dTpTxw2qM43TbHtCSkvNB2UQIICKR2vCjpbz+lMTT4PhB 4nXS/Zo+GTXAiI2xfuAn7SOvQzmxMFktpHwe02F6rc8KvM2+rYSzQE0T1JnKZfRR8iWa 79zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116771; x=1784721571; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=pVGMaYEcniIvMPOgxM8+baJR4b4kWmoCjAgYLXKet98=; b=jATyyyuCilAlzUomGHaTpO0BymgsWmJ6xgHiO5HizDn3BLh5FAURZAvo6/r0fQDlh8 ORa1XUeceENpWIvH4to4C9QOqzX4ZVOV2LPJb3/V/FgMeXP6kp8tWzVXZRSPYUXr7LYe /P0zKNR5OGP5zcyxEhkSuBuqopRGqSIDLCqiBfMasynWdqKtZ36941Gor9LTB5/axqr/ tpsI5CTlwoyABGxs/GZJ7xbgp96RdPLFN0QbNmSAeSfrqs7PmJ3jyi5yNqNvXWmgs4dG m2sDyamTJzbyCCOGTS96DDANfyvmrLySEJBOLkjoEp1h7dsXJKqFlSfxA9FEQit6bM3F 9lgw== X-Forwarded-Encrypted: i=1; AHgh+Rrso1Oz9UfThZHFTRb5cOIi70+Zu8zFJYFvMV7ga+hIKFaw0dW7G5VSAGFu7x3bKgQy//cbjNpAYeVUJ6Q=@vger.kernel.org X-Gm-Message-State: AOJu0YymeSNPA55VHs5g5aGhrfiSXy7pTGGTIHrU+bCO84roRn6GzWHG JiVIpkd9GGc6LKbIdglffVBDUGwY0YW9en2kXypsbH2XRPcjEHRasZfx8U3n0MVGSBzUr1QdyOg pR1aRu3ZwMx6Iug== X-Received: from wmnb8.prod.google.com ([2002:a05:600c:6c8:b0:493:d3c4:6a28]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e49:b0:490:e5c1:b8b9 with SMTP id 5b1f17b1804b1-4953905c584mr69823465e9.0.1784116771104; Wed, 15 Jul 2026 04:59:31 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:55 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-15-smostafa@google.com> Subject: [PATCH v7 14/24] iommu/arm-smmu-v3-kvm: Shadow the command queue From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" At boot allocate a command queue per SMMU which is used as a shadow by the hypervisor. The command queue size is 64K which is more than enough, as the hypervisor would consume all the entries per a command queue prod write, which means it can handle up to 4096 at a time. Then, the host command queue needs to be pinned in a shared state, so it can't be donated to VMs, and avoid tricking the hypervisor into accessing them. This is done each time the command queue is enabled, and undone each time the command queue is disabled. The hypervisor won=E2=80=99t access the host command queue when it is disab= led from the host. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c | 25 ++++ .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 123 +++++++++++++++++- .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 8 ++ 3 files changed, 155 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-kvm.c index 68b78ed933d4..81a4cd539415 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c @@ -15,6 +15,8 @@ #include "arm-smmu-v3.h" #include "pkvm/arm_smmu_v3.h" =20 +#define SMMU_KVM_CMDQ_ORDER 4 + extern struct pkvm_iommu_ops kvm_nvhe_sym(smmu_ops); =20 static size_t kvm_arm_smmu_count; @@ -24,6 +26,15 @@ static size_t kvm_arm_smmu_cur; static void kvm_arm_smmu_array_free(void) { int order; + int i; + + for (i =3D 0 ; i < kvm_arm_smmu_cur ; ++i) { + struct hyp_arm_smmu_v3_device *smmu =3D &kvm_arm_smmu_array[i]; + + if (smmu->cmdq.base_dma) + free_pages((unsigned long)phys_to_virt(smmu->cmdq.base_dma), + SMMU_KVM_CMDQ_ORDER); + } =20 order =3D get_order(kvm_arm_smmu_count * sizeof(*kvm_arm_smmu_array)); free_pages((unsigned long)kvm_arm_smmu_array, order); @@ -70,6 +81,7 @@ static int smmuv3_nesting_probe(struct platform_device *p= dev) struct hyp_arm_smmu_v3_device *smmu =3D &kvm_arm_smmu_array[kvm_arm_smmu_= cur]; struct device *dev =3D &pdev->dev; struct resource *res; + void *cmdq_base; =20 /* Only device tree, ACPI not supported. */ if (!dev->of_node) @@ -92,6 +104,19 @@ static int smmuv3_nesting_probe(struct platform_device = *pdev) return -EINVAL; } =20 + /* + * Allocate the shadow command queue, it doesn't have to be the same + * size as the host. + * Only populate base_dma and llq.max_n_shift, the hypervisor will init + * the rest. + */ + cmdq_base =3D (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, SMMU_KVM_= CMDQ_ORDER); + if (!cmdq_base) + return -ENOMEM; + + smmu->cmdq.base_dma =3D virt_to_phys(cmdq_base); + smmu->cmdq.llq.max_n_shift =3D SMMU_KVM_CMDQ_ORDER + PAGE_SHIFT - CMDQ_EN= T_SZ_SHIFT; + if (of_dma_is_coherent(dev->of_node)) smmu->features |=3D ARM_SMMU_FEAT_COHERENCY; =20 diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index af06c832fc6f..9f76f4e82341 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -11,7 +11,6 @@ #include =20 #include "arm_smmu_v3.h" -#include "../arm-smmu-v3.h" =20 size_t __ro_after_init kvm_hyp_arm_smmu_v3_count; struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; @@ -21,10 +20,68 @@ struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmu= s; (smmu) !=3D &kvm_hyp_arm_smmu_v3_smmus[kvm_hyp_arm_smmu_v3_count]; \ (smmu)++) =20 +#define cmdq_size(cmdq) ((1 << ((cmdq)->llq.max_n_shift)) * CMDQ_ENT_DWORD= S * 8) + +static bool is_cmdq_enabled(struct hyp_arm_smmu_v3_device *smmu) +{ + return FIELD_GET(CR0_CMDQEN, smmu->cr0); +} + +/* + * CMDQ, STE host copies are accessed by the hypervisor, we share them to + * - Prevent the host from passing protected VM memory. + * - Having them mapped in the hyp page table. + */ +static int smmu_share_pages(phys_addr_t addr, size_t size) +{ + size_t nr_pages =3D PAGE_ALIGN(size + (addr & ~PAGE_MASK)) >> PAGE_SHIFT; + phys_addr_t base =3D addr & PAGE_MASK; + int i, ret; + + for (i =3D 0 ; i < nr_pages ; ++i) { + if (__pkvm_host_share_hyp((base + i * PAGE_SIZE) >> PAGE_SHIFT)) { + while (i--) + __pkvm_host_unshare_hyp((base + i * PAGE_SIZE) >> PAGE_SHIFT); + return -EPERM; + } + } + + ret =3D hyp_pin_shared_mem(hyp_phys_to_virt(base), + hyp_phys_to_virt(base + nr_pages * PAGE_SIZE)); + if (ret) { + for (i =3D 0 ; i < nr_pages ; ++i) + __pkvm_host_unshare_hyp((base + i * PAGE_SIZE) >> PAGE_SHIFT); + } + + return ret; +} + +static int smmu_unshare_pages(phys_addr_t addr, size_t size) +{ + size_t nr_pages =3D PAGE_ALIGN(size + (addr & ~PAGE_MASK)) >> PAGE_SHIFT; + phys_addr_t base =3D addr & PAGE_MASK; + int i, ret; + + hyp_unpin_shared_mem(hyp_phys_to_virt(base), + hyp_phys_to_virt(base + nr_pages * PAGE_SIZE)); + + for (i =3D 0 ; i < nr_pages ; ++i) { + ret =3D __pkvm_host_unshare_hyp((base + i * PAGE_SIZE) >> PAGE_SHIFT); + if (ret) + return ret; + } + + return 0; +} + /* Put the device in a state that can be probed by the host driver. */ static void smmu_deinit_device(struct hyp_arm_smmu_v3_device *smmu) { WARN_ON(__pkvm_hyp_donate_host_mmio(smmu->mmio_addr, smmu->mmio_size)); + + if (smmu->cmdq.base) + WARN_ON(__pkvm_hyp_donate_host(smmu->cmdq.base_dma >> PAGE_SHIFT, + cmdq_size(&smmu->cmdq) >> PAGE_SHIFT)); smmu->base =3D NULL; } =20 @@ -75,6 +132,31 @@ static int smmu_probe(struct hyp_arm_smmu_v3_device *sm= mu) return 0; } =20 +/* + * The kernel part of the driver will allocate the shadow cmdq, + * and zero it. This function only donates it. + */ +static int smmu_init_cmdq(struct hyp_arm_smmu_v3_device *smmu) +{ + size_t cmdq_nr_pages =3D cmdq_size(&smmu->cmdq) >> PAGE_SHIFT; + int ret; + + ret =3D __pkvm_host_donate_hyp(smmu->cmdq.base_dma >> PAGE_SHIFT, cmdq_nr= _pages); + if (ret) + return ret; + + smmu->cmdq.base =3D hyp_phys_to_virt(smmu->cmdq.base_dma); + smmu->cmdq.prod_reg =3D smmu->base + ARM_SMMU_CMDQ_PROD; + smmu->cmdq.cons_reg =3D smmu->base + ARM_SMMU_CMDQ_CONS; + smmu->cmdq.q_base =3D smmu->cmdq.base_dma | + FIELD_PREP(Q_BASE_LOG2SIZE, smmu->cmdq.llq.max_n_shift); + smmu->cmdq.ent_dwords =3D CMDQ_ENT_DWORDS; + writel_relaxed(0, smmu->cmdq.prod_reg); + writel_relaxed(0, smmu->cmdq.cons_reg); + writeq_relaxed(smmu->cmdq.q_base, smmu->base + ARM_SMMU_CMDQ_BASE); + return 0; +} + static int smmu_init_device(struct hyp_arm_smmu_v3_device *smmu) { unsigned long haddr; @@ -94,7 +176,12 @@ static int smmu_init_device(struct hyp_arm_smmu_v3_devi= ce *smmu) if (ret) goto out_ret; =20 + ret =3D smmu_init_cmdq(smmu); + if (ret) + goto out_ret; + return 0; + out_ret: smmu_deinit_device(smmu); return ret; @@ -134,6 +221,23 @@ static int smmu_init(void) return ret; } =20 +static void smmu_emulate_cmdq_enable(struct hyp_arm_smmu_v3_device *smmu) +{ + u32 shift =3D smmu->cmdq_host.q_base & Q_BASE_LOG2SIZE; + + smmu->cmdq_host.llq.max_n_shift =3D min(shift, 19); + smmu->cmdq_host.base_dma =3D smmu->cmdq_host.q_base & Q_BASE_ADDR_MASK; + smmu->cmdq_host.base_dma &=3D ~(cmdq_size(&smmu->cmdq_host) - 1); + WARN_ON(smmu_share_pages(smmu->cmdq_host.base_dma, + cmdq_size(&smmu->cmdq_host))); +} + +static void smmu_emulate_cmdq_disable(struct hyp_arm_smmu_v3_device *smmu) +{ + WARN_ON(smmu_unshare_pages(smmu->cmdq_host.base_dma, + cmdq_size(&smmu->cmdq_host))); +} + static bool smmu_dabt_device(struct hyp_arm_smmu_v3_device *smmu, struct user_pt_regs *regs, u64 esr, u32 off) @@ -160,6 +264,14 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_de= vice *smmu, break; /* Passthrough the register access for bisectiblity, handled later */ case ARM_SMMU_CMDQ_BASE: + if (is_write) { + /* Not allowed by the architecture */ + if (is_cmdq_enabled(smmu)) + break; + smmu->cmdq_host.q_base =3D val; + } + mask =3D read_write; + break; case ARM_SMMU_CMDQ_PROD: case ARM_SMMU_CMDQ_CONS: case ARM_SMMU_STRTAB_BASE: @@ -170,6 +282,15 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_de= vice *smmu, case ARM_SMMU_CR0: if (len !=3D sizeof(u32)) break; + if (is_write) { + bool last_cmdq_en =3D is_cmdq_enabled(smmu); + + smmu->cr0 =3D val; + if (!last_cmdq_en && is_cmdq_enabled(smmu)) + smmu_emulate_cmdq_enable(smmu); + else if (last_cmdq_en && !is_cmdq_enabled(smmu)) + smmu_emulate_cmdq_disable(smmu); + } mask =3D read_write; break; case ARM_SMMU_CR1: { diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h index 2bda6e03c96c..74a7f62d93eb 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -8,6 +8,8 @@ #include #endif =20 +#include "../arm-smmu-v3.h" + /* * Parameters from the trusted host: * @mmio_addr base address of the SMMU registers @@ -22,6 +24,9 @@ * @lock Lock to protect SMMU emulation * @hw_lock Lock to protect SMMU HW (as CMDQ) Order smmu.lock =3D> host_mmu.lock =3D> smmu.hw_lock + * @cmdq CMDQ as observed by HW + * @cmdq_host Host view of the CMDQ, only q_base and llq used. + * @cr0 Last value of CR0 */ struct hyp_arm_smmu_v3_device { phys_addr_t mmio_addr; @@ -38,6 +43,9 @@ struct hyp_arm_smmu_v3_device { u32 lock; u32 hw_lock; #endif + struct arm_smmu_queue cmdq; + struct arm_smmu_queue cmdq_host; + u32 cr0; }; =20 extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 552214779A3 for ; Wed, 15 Jul 2026 11:59:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116778; cv=none; b=giUu9obdN8LH0Z/mg0RVmDJH9F8yHLZwaIkkMrK7PlxHOD6RCyzwTZ/Fa2iOKUil7Q8SM0RVGjlRmfFPQNkfcSiXsPWDN4ch2Srj7tN5ZAiog0DY6etD2rrFDvymFBxYQ376bFlU7P/9KzVH6Vq4+CTo0Oiki2LAmVhgoogfHMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116778; c=relaxed/simple; bh=TTW+FNhgm8rDj6yxtOrw2Lhsb+fDiJsDpk/4fctaI6g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uGb0xwhS88u9K08kw4EBHykE18fo8mu/LiSNA2z4wkCvW7WDxv6BKDBDMHIxWpNsdLc6G5zqF4odkUwSYGmYZdRVkyDrTXZ08PN/Oiz0yUAPENwdUKh29vXL7lGIyxLTik1BA5DFottbRunbdtfK8v/LQZxY8GvRmbqV03Rql3U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NKLQOHLB; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NKLQOHLB" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-492714d002cso14416665e9.0 for ; Wed, 15 Jul 2026 04:59:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116772; x=1784721572; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=75EB2f6MAXW5J4vafHy4b5awBXMBAYUr8yFrAesJgTk=; b=NKLQOHLBkYFYTV3HnrZwmq59/aIbZb8X6uSDK/l4l2GiOAM+dolPGg+rx5QGvfY56j jMQ55FraVjm5RrbHiWK4fyhVDJsHZ8wcsp54Rrcn/0NDHvlHmCJfurMu6joJNUTqTQ6c h47rdZHq0oeG6BSm0RPxyAkO+GdvdL94oKm1XOAhXM4YwWh71E60Ohzf9DLXlAoEsHNr ozOcNtUimY31fF5TMxXFTa05Y3E2GV8lyOhOEmK373vGZNKSiBlnPRgHlGjU6XgysVqr Y7YmtXm4Bsb5Q3xALaS41MISWYlLBQAJF3sJ20XYlSxWYdJts6ANo14I0llVWUS0rniS Zpfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116772; x=1784721572; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=75EB2f6MAXW5J4vafHy4b5awBXMBAYUr8yFrAesJgTk=; b=qTeS9VWI+1ZkXUglFZlXfBStZHorefAMu1Ro6K4mTTYVTI93WcJ/vdiuVO8FIUg/6L 1pYzL8CqZWI4KehxROUpMsy9ffjhPLXEpSYQU3IbCG7cerKickASjfoRMhJsJ+qk01CO TiO9QUEJoutqPnCHMDIhz9yBstb+9jxCfb/L6hSgrwC0l0LzbFQt4X0vi02enNb61ize VXknwS847bBeRCws+6f/VKUsjarwUUJeBDQ96v+By3hzxK8LFglZLE5R6Fb4jfMG9nW9 ZhMrao78av6y9iu/HZOtE4jjVKK6v3GUPMijKkmts5fuc27MsAxqJI9qlJ9sfgFa4k4+ 2CZw== X-Forwarded-Encrypted: i=1; AHgh+RpDkwHDcmcR+us1RBLEsmlfveW0yBlKM+c9ctxNYG4hDApezhulXLCMaF9ltFNU0k7yUS819NAbPSMZ9EE=@vger.kernel.org X-Gm-Message-State: AOJu0Yzjs6G6i/COYtUz6qKs1eDf6FpjuXmJlPsDhU8UeVmtI33fqwS1 igYF5HJRWciQ/jznpAzJ8RI2XJQxnQbG6QWqf3wJgrZ93UoCBpynS3fJ9DjNaU9wk3bVu7kwHrN m0wsvTNZ1pNU7eg== X-Received: from wmiv11.prod.google.com ([2002:a05:600c:e40b:b0:493:ab17:3e61]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600d:8444:10b0:493:bd2a:93bb with SMTP id 5b1f17b1804b1-493f87d7eefmr140336125e9.3.1784116772290; Wed, 15 Jul 2026 04:59:32 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:56 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-16-smostafa@google.com> Subject: [PATCH v7 15/24] iommu/arm-smmu-v3-kvm: Add CMDQ functions From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add functions to access the command queue, there are 2 main usage: - Hypervisor's own commands, as TLB invalidation, would use functions as smmu_send_cmd(), which creates and sends a command. - Add host commands to the shadow command queue, after being filtered, these will be added with smmu_add_cmd_raw. Signed-off-by: Mostafa Saleh --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 14 ++- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 109 ++++++++++++++++++ 2 files changed, 117 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 842d0c9b883c..cbc4589e89a8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -1226,19 +1226,21 @@ unsigned long smmu_iidr_features(u32 reg, unsigned = long features); unsigned long smmu_iidr_options(u32 reg, unsigned long options); =20 /* Queue functions shared between kernel and hyp. */ -static inline bool queue_has_space(struct arm_smmu_ll_queue *q, u32 n) +static inline u32 queue_space(struct arm_smmu_ll_queue *q) { - u32 space, prod, cons; + u32 prod, cons; =20 prod =3D Q_IDX(q, q->prod); cons =3D Q_IDX(q, q->cons); =20 if (Q_WRP(q, q->prod) =3D=3D Q_WRP(q, q->cons)) - space =3D (1 << q->max_n_shift) - (prod - cons); - else - space =3D cons - prod; + return (1 << q->max_n_shift) - (prod - cons); + return cons - prod; +} =20 - return space >=3D n; +static inline bool queue_has_space(struct arm_smmu_ll_queue *q, u32 n) +{ + return queue_space(q) >=3D n; } =20 static inline bool queue_full(struct arm_smmu_ll_queue *q) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index 9f76f4e82341..8e798fd8fdaa 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -6,6 +6,7 @@ */ #include =20 +#include #include #include #include @@ -22,6 +23,36 @@ struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; =20 #define cmdq_size(cmdq) ((1 << ((cmdq)->llq.max_n_shift)) * CMDQ_ENT_DWORD= S * 8) =20 +#define ARM_SMMU_EL2_POLL_TIMEOUT_US 1000 +/* + * Wait until @cond is true, can use WFE, if polling on an SMMU and + * event that supports it. + * Return 0 on success, or -ETIMEDOUT + */ +#define smmu_wait(__use_wfe, __cond) \ +({ \ + int __ret =3D 0; \ + u64 start =3D hyp_clock_ns(); \ + u64 timeout =3D ARM_SMMU_EL2_POLL_TIMEOUT_US * 1000; \ + \ + while (!(__cond)) { \ + if (__use_wfe) { \ + wfe(); \ + if (__cond) \ + break; \ + } else { \ + cpu_relax(); \ + } \ + if ((hyp_clock_ns() - start) >=3D timeout) { \ + if (__cond) \ + break; \ + __ret =3D -ETIMEDOUT; \ + break; \ + } \ + } \ + __ret; \ +}) + static bool is_cmdq_enabled(struct hyp_arm_smmu_v3_device *smmu) { return FIELD_GET(CR0_CMDQEN, smmu->cr0); @@ -74,6 +105,84 @@ static int smmu_unshare_pages(phys_addr_t addr, size_t = size) return 0; } =20 +__maybe_unused +static bool smmu_cmdq_has_space(struct arm_smmu_queue *cmdq, u32 n) +{ + struct arm_smmu_ll_queue *llq =3D &cmdq->llq; + + WRITE_ONCE(llq->cons, readl_relaxed(cmdq->cons_reg)); + return queue_has_space(llq, n); +} + +static bool smmu_cmdq_full(struct arm_smmu_queue *cmdq) +{ + struct arm_smmu_ll_queue *llq =3D &cmdq->llq; + + WRITE_ONCE(llq->cons, readl_relaxed(cmdq->cons_reg)); + return queue_full(llq); +} + +static bool smmu_cmdq_empty(struct arm_smmu_queue *cmdq) +{ + struct arm_smmu_ll_queue *llq =3D &cmdq->llq; + + WRITE_ONCE(llq->cons, readl_relaxed(cmdq->cons_reg)); + return queue_empty(llq); +} + +static void smmu_add_cmd_raw(struct hyp_arm_smmu_v3_device *smmu, + u64 *cmd) +{ + struct arm_smmu_queue *q =3D &smmu->cmdq; + struct arm_smmu_ll_queue *llq =3D &q->llq; + + queue_write(Q_ENT(q, llq->prod), cmd, CMDQ_ENT_DWORDS); + llq->prod =3D queue_inc_prod_n(llq, 1); +} + +static int smmu_add_cmd(struct hyp_arm_smmu_v3_device *smmu, + struct arm_smmu_cmd *cmd) +{ + int ret; + + hyp_assert_lock_held(&smmu->hw_lock); + ret =3D smmu_wait(false, !smmu_cmdq_full(&smmu->cmdq)); + if (ret) + return ret; + + smmu_add_cmd_raw(smmu, cmd->data); + writel(smmu->cmdq.llq.prod, smmu->cmdq.prod_reg); + return 0; +} + +static int smmu_sync_cmd(struct hyp_arm_smmu_v3_device *smmu) +{ + int ret; + struct arm_smmu_cmd cmd; + + hyp_assert_lock_held(&smmu->hw_lock); + cmd =3D arm_smmu_make_cmd_sync(CMDQ_SYNC_0_CS_SEV, 0); + ret =3D smmu_add_cmd(smmu, &cmd); + if (ret) + return ret; + + return smmu_wait(smmu->features & ARM_SMMU_FEAT_SEV, + smmu_cmdq_empty(&smmu->cmdq)); +} + +__maybe_unused +static int smmu_send_cmd(struct hyp_arm_smmu_v3_device *smmu, + struct arm_smmu_cmd *cmd) +{ + int ret =3D smmu_add_cmd(smmu, cmd); + + hyp_assert_lock_held(&smmu->hw_lock); + if (ret) + return ret; + + return smmu_sync_cmd(smmu); +} + /* Put the device in a state that can be probed by the host driver. */ static void smmu_deinit_device(struct hyp_arm_smmu_v3_device *smmu) { --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17D40477E24 for ; Wed, 15 Jul 2026 11:59:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116779; cv=none; b=p290Hym7e8JjJJgI0MrzL9erSrQE5nKWZdy/7rozdPvzSMeM4Tfxgfyg5Si2JAKtMZUkKUytliSe6kBTk8n8/zvBpVgNqTXucmTTzbsnjiqeAcIjCFwbIY5IwgEYN+4vIRFQ19VV9sN0lY1Or1OBSED2CV6vuIvW+1QJl3tDXb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116779; c=relaxed/simple; bh=YePeblpzE2wwy186w59Vlsd4xF4gXljPMBZRLISQyw0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NMoa+VOLN81MUTu8tTjP2P1vGvg8BjUxEVhJd70MJgE6YTB29sRe94lzMul7T1u+LuFHU6nbjjlel6l4xveFY8Q2QW+0U4E0rQHEQFeqqUY78vZc8Yjb/6ThdoqoCGlNTmHSih/IqDaK2rzu40veacilDkw17t4En9hZgzB5sE4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NrQ1gN1b; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NrQ1gN1b" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493bfc3b84aso26421975e9.0 for ; Wed, 15 Jul 2026 04:59:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116774; x=1784721574; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:from:to:cc :subject:date:message-id:reply-to:content-type; bh=XAFQ5BsXp17z8wDAJzX50F8XvcTlSxFTjmAveWqMmq0=; b=NrQ1gN1bBfD1jyD+UOukpDQkEdMEEHvsANFJZBnehl+xyIPNwa/PogO7Lgu+FCNlh2 F+6KTlbDNYeD5p3axhRws/+2ebtbLTrKGooE0nNni3Lc5PNVaisQVZyS0gcqGegMxpIG WlENr+ZtjSC6sph+C89ULRVbTENe9dQfuWhDF321AFqEgYWqDlOkTv0NCU7UL3HZGlP1 i6DpNVGreERxlIfzwhQiIx0hjwPSXp9x7Ea9Hol5n4bQ8pgMMSseFAJxCsSXC+TDRy7p aClchEgAfg++bXqdaWmXn7X4ZXDgm2ZEaAHmrSkmId003G9n5y8zrW9pXpvJV2TlFLYM RcMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116774; x=1784721574; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=XAFQ5BsXp17z8wDAJzX50F8XvcTlSxFTjmAveWqMmq0=; b=n3Rha2Vx9CaUeB2xF5t7d+QaV1AQ1qXSsvBEggOY/jJNDOiIe+7eU7h2Tac5zB3e9k /ZschWRP8lLPUrp0qxQ/apMhLDqe0PXQRVuEYx8sKPbEfdXU/zRUg9YNBdyQ5BJZ1BI6 MrqyuxQ38tDfj3rGvjmaW+6jREmPEjCLD+gZaKIxFv4i4RuvodWrP9ybEiANFYyMRgVF GVZyAAai8991zQfJPjxWyzJ5R0unbN85mrC/jEh6jnnhqtxaIWdg0nS6osdmuKspCItC i90hmHPIsUJJqzs2pyALq4yhFTrlXlpIZqPly2odXuBqls84YLiJHEr9j9P9XhvNY3nK ysTA== X-Forwarded-Encrypted: i=1; AHgh+Rp1c+2+XoU/luZR0OUUHa8k5i8YygSisKpEvbcl5y0lN+7Apk1QtfOHFHzT/+W24pD12hzFgV/Xr1Xjaqg=@vger.kernel.org X-Gm-Message-State: AOJu0YyUg8VZCbiq3MoT1vEOBLRqJ+LA2aEDrGzaQYdZgrCFsLQp3ct5 tSpdJMxdmtdimho88k54F8bj4BJ7mWpC25D+/sNUhrwcmfpp71KNFikl3aLxfC4ysZmxuRW2mgG 9inCqx+D8g+4vDg== X-Received: from wmga21.prod.google.com ([2002:a05:600c:2d55:b0:490:b58a:2c9b]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3486:b0:493:b7a6:3dac with SMTP id 5b1f17b1804b1-4953b156ee2mr37104465e9.33.1784116773557; Wed, 15 Jul 2026 04:59:33 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:57 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-17-smostafa@google.com> Subject: [PATCH v7 16/24] iommu/arm-smmu-v3-kvm: Emulate CMDQ for host From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don=E2=80=99t allow access to the command queue from the host: - ARM_SMMU_CMDQ_BASE: Only allowed to be written when CMDQ is disabled, we use it to keep track of the host command queue base. Reads return the saved value. - ARM_SMMU_CMDQ_PROD: Writes trigger command queue emulation which sanitise and filters the whole range. Reads returns the host copy. - ARM_SMMU_CMDQ_CONS: Writes move the sw copy of the cons, but the host can=E2=80=99t skip commands once submitted. Reads return the emulated val= ue and the error bits in the actual cons. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 133 +++++++++++++++++- 1 file changed, 129 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index 8e798fd8fdaa..f62c9e8f2c59 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -105,7 +105,6 @@ static int smmu_unshare_pages(phys_addr_t addr, size_t = size) return 0; } =20 -__maybe_unused static bool smmu_cmdq_has_space(struct arm_smmu_queue *cmdq, u32 n) { struct arm_smmu_ll_queue *llq =3D &cmdq->llq; @@ -330,6 +329,99 @@ static int smmu_init(void) return ret; } =20 +static bool smmu_filter_command(struct hyp_arm_smmu_v3_device *smmu, u64 *= command) +{ + u64 type =3D FIELD_GET(CMDQ_0_OP, command[0]); + + switch (type) { + case CMDQ_OP_CFGI_STE: + /* TBD: SHADOW_STE*/ + break; + case CMDQ_OP_CFGI_ALL: + { + /* + * Linux doesn't use range STE invalidation, and only use this + * for CFGI_ALL, which is done on reset and not on an new STE + * being used. + * Although, this is not architectural we rely on the current Linux + * implementation. + */ + if ((FIELD_GET(CMDQ_CFGI_1_RANGE, command[1]) !=3D 31)) + return true; + break; + } + case CMDQ_OP_TLBI_NH_ASID: + case CMDQ_OP_TLBI_NH_VA: + case 0x13: /* CMD_TLBI_NH_VAA: Not used by Linux */ + { + /* Only allow VMID =3D 0 */ + if (FIELD_GET(CMDQ_TLBI_0_VMID, command[0]) !=3D 0) + return true; + break; + } + case CMDQ_OP_PREFETCH_CFG: + case CMDQ_OP_CFGI_CD: + case CMDQ_OP_CFGI_CD_ALL: + case CMDQ_OP_TLBI_NH_ALL: + case CMDQ_OP_TLBI_NSNH_ALL: + break; + case CMDQ_OP_CMD_SYNC: + if (FIELD_GET(CMDQ_SYNC_0_CS, command[0]) =3D=3D CMDQ_SYNC_0_CS_IRQ) { + /* Allow it, but let the host timeout, as this should never happen. */ + command[0] &=3D ~CMDQ_SYNC_0_CS; + command[0] |=3D FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_SEV); + command[1] &=3D ~CMDQ_SYNC_1_MSIADDR_MASK; + } + break; + default: + /* Deny unknown commands */ + return true; + } + + return false; +} + +static int smmu_emulate_cmdq_insert(struct hyp_arm_smmu_v3_device *smmu) +{ + u64 *host_cmdq =3D hyp_phys_to_virt(smmu->cmdq_host.base_dma); + bool use_wfe =3D smmu->features & ARM_SMMU_FEAT_SEV, skip; + u64 cmd[CMDQ_ENT_DWORDS]; + int idx, ret; + u32 space; + + if (!is_cmdq_enabled(smmu)) + return 0; + + space =3D (1 << (smmu->cmdq_host.llq.max_n_shift)) - queue_space(&smmu->c= mdq_host.llq); + + /* Wait for the command queue to have some space. */ + ret =3D smmu_wait(use_wfe, smmu_cmdq_has_space(&smmu->cmdq, space)); + if (ret) + return ret; + hyp_spin_lock(&smmu->hw_lock); + while (space--) { + int i; + + idx =3D Q_IDX(&smmu->cmdq_host.llq, smmu->cmdq_host.llq.cons); + queue_inc_cons(&smmu->cmdq_host.llq); + + /* Copy the command to local buffer avoiding TOCTOU */ + for (i =3D 0 ; i < CMDQ_ENT_DWORDS ; ++i) + cmd[i] =3D le64_to_cpu(READ_ONCE(host_cmdq[idx * CMDQ_ENT_DWORDS + i])); + + skip =3D smmu_filter_command(smmu, cmd); + if (WARN_ON(skip)) + continue; + smmu_add_cmd_raw(smmu, cmd); + } + + writel(smmu->cmdq.llq.prod, smmu->cmdq.prod_reg); + + ret =3D smmu_wait(use_wfe, smmu_cmdq_empty(&smmu->cmdq)); + hyp_spin_unlock(&smmu->hw_lock); + return ret; +} + static void smmu_emulate_cmdq_enable(struct hyp_arm_smmu_v3_device *smmu) { u32 shift =3D smmu->cmdq_host.q_base & Q_BASE_LOG2SIZE; @@ -371,18 +463,51 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_d= evice *smmu, */ mask =3D read_only & ~(IDR0_S2P | IDR0_VMID16 | IDR0_MSI | IDR0_HYP | ID= R0_ATS); break; - /* Passthrough the register access for bisectiblity, handled later */ case ARM_SMMU_CMDQ_BASE: + /* + * Although allowed to use smaller size, we rely on the SMMUv3 driver + * using 64-bit store instruction for simplicity. + */ + if (len !=3D sizeof(u64)) + break; if (is_write) { /* Not allowed by the architecture */ if (is_cmdq_enabled(smmu)) break; smmu->cmdq_host.q_base =3D val; + goto out_ret; + } else { + val =3D smmu->cmdq_host.q_base; + goto out_update_regs; } - mask =3D read_write; - break; case ARM_SMMU_CMDQ_PROD: + if (len !=3D sizeof(u32)) + break; + if (is_write) { + smmu->cmdq_host.llq.prod =3D val; + WARN_ON(smmu_emulate_cmdq_insert(smmu)); + goto out_ret; + } else { + val =3D smmu->cmdq_host.llq.prod; + goto out_update_regs; + } case ARM_SMMU_CMDQ_CONS: + if (len !=3D sizeof(u32)) + break; + if (is_write) { + if (WARN_ON(is_cmdq_enabled(smmu))) + break; + + smmu->cmdq_host.llq.cons =3D val; + goto out_ret; + } else { + /* Propagate errors back to the host.*/ + u32 cons =3D readl_relaxed(smmu->base + ARM_SMMU_CMDQ_CONS); + + val =3D smmu->cmdq_host.llq.cons | (CMDQ_CONS_ERR & cons); + goto out_update_regs; + } + /* Passthrough the register access for bisectiblity, handled later */ case ARM_SMMU_STRTAB_BASE: case ARM_SMMU_STRTAB_BASE_CFG: case ARM_SMMU_GBPA: --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4974447884E for ; Wed, 15 Jul 2026 11:59:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116780; cv=none; b=ZoCN2HM3+XaUMp57uY1FR2Wkw1tu/p5Fm8l5uLnRf0kKhV3XUjB/x5HNpiWGNQLr0QwzqfAveVS5sIwwOPXkSMK8bMCLq4qevnM6BO7mpQh97XyWKRSLYEeeB/N0BNLe9xtTdTkTZFxnJVEq56QLjxd6VxpU2/N8sSbS0J3Jo3w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116780; c=relaxed/simple; bh=qSwbMCTvMcx1q1S8+AnJgpwJf3eG58WbAdUNotD1jCo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Y+5dZ5Cj98xY6/Lt2wotMNavbBiAhHZoecPKpHuaKYb7udEt/s+E1hkEYLXeskBxGsH9WhN9CuKcoDYyh+TfaZOOQ6qPNKlr4lRdA0EirFMKb9YUnh2U0ozuaSBtI0j2Zuo5rRW0IfwEY6Wt5FIiKZ4MkJ8dNMifzqsZzTANzts= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BBmDBhuA; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BBmDBhuA" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-6984787eeddso5335789a12.1 for ; Wed, 15 Jul 2026 04:59:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116775; x=1784721575; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=QhVA7+fH5iuDTjPK4Ro+fz5qNNNKxyVJWdTpr47ZOpA=; b=BBmDBhuAQLJ6D+2y5KNXgPNlhigDHoKLtPF3Kv4/maUErqY8VwL9JOzIanBWJpqeQF hPzoWROiDtbLq205JK22hAvhRTqJQ7cDPotQWqurYjCR+mkJdkdcixmmeJZboBILv8ZU chMe3FY73EDV02WMdVwQvesPIqFRBLCj9xi7T7oa1ZenC03wCRt0mpCeYkaw+uNOc2mT txb7J9ku6npL+7NRq5GdPfsRQIXuDBN2twlJFEBDEi+60lK7bMnMbT5gTU3hPIFKUtnm BeThsHyqTHQ5whj+NgZFzjzXIhmwE5R5gDn9mWXIO2i3QWrRwapmYvbQ2hKA35HSC10L peZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116775; x=1784721575; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=QhVA7+fH5iuDTjPK4Ro+fz5qNNNKxyVJWdTpr47ZOpA=; b=WAznXsfBc/nUAuFeFtschnBods+nesq/Uw/hVRDUlNCkCQrDSguzUZkz2eRT/jyPkx 5o6Zzd5qCxSwq9fRUpCAtFKag17azQlzexnBRV8XpL20jXbwafuO7RcMtpmJweGw9bRv bFu1lvns5Rl8fHLaU3Rx7/Lyl0ZAq/W7fgflEC8EnVd0kYa+PxcnH6fww3o2tJpO3Mr9 gWajxBOD08r4y+BzUmKz3YY4slhHiKYqqlnJ9+PbUBBXqGyuXORJg6x6baKdSQEGZ90q sB9zutRACgpM8/EHTySBGkYwtREqCRVvz4dgKwAyRwMmkWsEPLklRmeSdyCAJvOl1BeM Opng== X-Forwarded-Encrypted: i=1; AHgh+Rr5f2ViTwYNHJnAb2gDHyEsI1auH2uAgVAhbaGUSuozwEsGmxfy9ckw6JP18PDipTcILR6inpY+lTpaQFo=@vger.kernel.org X-Gm-Message-State: AOJu0YwMCjLxmeEne6rF9kO5A4MbxZ7aQSPITqC+SvwwQa4Gu3LV3HPQ cAlOkb0lhf2ggPVsb/dMHsLLrnMt703c0f6xwzmRWj/na1wzbpHUaLHprAnqOuPE53GCHtesW4C I10obIkAme3ufWQ== X-Received: from edr22.prod.google.com ([2002:a05:6402:44d6:b0:695:9e2c:a1ca]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:1ed6:b0:698:52b4:c2c9 with SMTP id 4fb4d7f45d1cf-69cd6f069dcmr3544598a12.31.1784116774920; Wed, 15 Jul 2026 04:59:34 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:58 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-18-smostafa@google.com> Subject: [PATCH v7 17/24] iommu/arm-smmu-v3-kvm: Shadow stream table From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allocate the shadow stream table per SMMU. We choose the size of that table to be 1MB which is the max size used by host in the case of 2 levels. All the host writes are still paththrough for bisectibility, that is changed next where CFGI commands will be trapped and used to update the shadow copy hypervisor that will be used by HW. Similar to the command queue, the host stream table is shared/unshared each time the SMMU is enabled/disabled. Handling of L2 tables is also done in the next patch when the shadowing is added. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c | 21 ++- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 122 ++++++++++++++++++ .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 10 ++ 3 files changed, 152 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-kvm.c index 81a4cd539415..2cafb03995ea 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c @@ -16,6 +16,13 @@ #include "pkvm/arm_smmu_v3.h" =20 #define SMMU_KVM_CMDQ_ORDER 4 +/* + * Use the max value of L1 the kernel uses, that also covers the worst case + * for linear tables as it is mandatory according to the spec to support 2 + * lvl tables if SIDSIZE >=3D 7 + */ +#define SMMU_KVM_STRTAB_ORDER (get_order(STRTAB_MAX_L1_ENTRIES * \ + sizeof(struct arm_smmu_strtab_l1))) =20 extern struct pkvm_iommu_ops kvm_nvhe_sym(smmu_ops); =20 @@ -34,6 +41,9 @@ static void kvm_arm_smmu_array_free(void) if (smmu->cmdq.base_dma) free_pages((unsigned long)phys_to_virt(smmu->cmdq.base_dma), SMMU_KVM_CMDQ_ORDER); + if (smmu->strtab_dma) + free_pages((unsigned long)phys_to_virt(smmu->strtab_dma), + SMMU_KVM_STRTAB_ORDER); } =20 order =3D get_order(kvm_arm_smmu_count * sizeof(*kvm_arm_smmu_array)); @@ -80,8 +90,8 @@ static int smmuv3_nesting_probe(struct platform_device *p= dev) { struct hyp_arm_smmu_v3_device *smmu =3D &kvm_arm_smmu_array[kvm_arm_smmu_= cur]; struct device *dev =3D &pdev->dev; + void *cmdq_base, *strtab; struct resource *res; - void *cmdq_base; =20 /* Only device tree, ACPI not supported. */ if (!dev->of_node) @@ -117,6 +127,15 @@ static int smmuv3_nesting_probe(struct platform_device= *pdev) smmu->cmdq.base_dma =3D virt_to_phys(cmdq_base); smmu->cmdq.llq.max_n_shift =3D SMMU_KVM_CMDQ_ORDER + PAGE_SHIFT - CMDQ_EN= T_SZ_SHIFT; =20 + strtab =3D (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, SMMU_KVM_STR= TAB_ORDER); + if (!strtab) { + free_pages((unsigned long)cmdq_base, SMMU_KVM_CMDQ_ORDER); + return -ENOMEM; + } + + smmu->strtab_dma =3D virt_to_phys(strtab); + smmu->strtab_size =3D PAGE_SIZE << SMMU_KVM_STRTAB_ORDER; + if (of_dma_is_coherent(dev->of_node)) smmu->features |=3D ARM_SMMU_FEAT_COHERENCY; =20 diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index f62c9e8f2c59..3b133f24b4ca 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -16,6 +16,14 @@ size_t __ro_after_init kvm_hyp_arm_smmu_v3_count; struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; =20 +/* strtab accessors */ +#define strtab_log2size(smmu) (FIELD_GET(STRTAB_BASE_CFG_LOG2SIZE, (smmu)-= >host_ste_cfg)) +#define strtab_size(smmu) ((1UL << strtab_log2size(smmu)) * STRTAB_STE_DWO= RDS * 8) +#define strtab_host_base(smmu) ((smmu)->host_ste_base & STRTAB_BASE_ADDR_M= ASK) +#define strtab_split(smmu) (FIELD_GET(STRTAB_BASE_CFG_SPLIT, (smmu)->host_= ste_cfg)) +#define strtab_l1_size(smmu) ((1UL << (strtab_log2size(smmu) - strtab_spli= t(smmu))) * \ + (sizeof(struct arm_smmu_strtab_l1))) + #define for_each_smmu(smmu) \ for ((smmu) =3D kvm_hyp_arm_smmu_v3_smmus; \ (smmu) !=3D &kvm_hyp_arm_smmu_v3_smmus[kvm_hyp_arm_smmu_v3_count]; \ @@ -58,6 +66,11 @@ static bool is_cmdq_enabled(struct hyp_arm_smmu_v3_devic= e *smmu) return FIELD_GET(CR0_CMDQEN, smmu->cr0); } =20 +static bool is_smmu_enabled(struct hyp_arm_smmu_v3_device *smmu) +{ + return FIELD_GET(CR0_SMMUEN, smmu->cr0); +} + /* * CMDQ, STE host copies are accessed by the hypervisor, we share them to * - Prevent the host from passing protected VM memory. @@ -190,6 +203,11 @@ static void smmu_deinit_device(struct hyp_arm_smmu_v3_= device *smmu) if (smmu->cmdq.base) WARN_ON(__pkvm_hyp_donate_host(smmu->cmdq.base_dma >> PAGE_SHIFT, cmdq_size(&smmu->cmdq) >> PAGE_SHIFT)); + + if (smmu->strtab_cfg.linear.table || + smmu->strtab_cfg.l2.l1tab) + WARN_ON(__pkvm_hyp_donate_host(hyp_phys_to_pfn(smmu->strtab_dma), + smmu->strtab_size >> PAGE_SHIFT)); smmu->base =3D NULL; } =20 @@ -265,6 +283,45 @@ static int smmu_init_cmdq(struct hyp_arm_smmu_v3_devic= e *smmu) return 0; } =20 +static int smmu_init_strtab(struct hyp_arm_smmu_v3_device *smmu) +{ + struct arm_smmu_strtab_cfg *cfg =3D &smmu->strtab_cfg; + int ret; + u32 reg; + + ret =3D __pkvm_host_donate_hyp(hyp_phys_to_pfn(smmu->strtab_dma), + smmu->strtab_size >> PAGE_SHIFT); + if (ret) + return ret; + + if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) { + unsigned int last_sid_idx =3D + arm_smmu_strtab_l1_idx((1ULL << smmu->sid_bits) - 1); + + cfg->l2.l1tab =3D hyp_phys_to_virt(smmu->strtab_dma); + cfg->l2.l1_dma =3D smmu->strtab_dma; + cfg->l2.num_l1_ents =3D min(last_sid_idx + 1, STRTAB_MAX_L1_ENTRIES); + + reg =3D FIELD_PREP(STRTAB_BASE_CFG_FMT, + STRTAB_BASE_CFG_FMT_2LVL) | + FIELD_PREP(STRTAB_BASE_CFG_LOG2SIZE, + ilog2(cfg->l2.num_l1_ents) + STRTAB_SPLIT) | + FIELD_PREP(STRTAB_BASE_CFG_SPLIT, STRTAB_SPLIT); + } else { + cfg->linear.table =3D hyp_phys_to_virt(smmu->strtab_dma); + cfg->linear.ste_dma =3D smmu->strtab_dma; + cfg->linear.num_ents =3D 1UL << smmu->sid_bits; + reg =3D FIELD_PREP(STRTAB_BASE_CFG_FMT, + STRTAB_BASE_CFG_FMT_LINEAR) | + FIELD_PREP(STRTAB_BASE_CFG_LOG2SIZE, smmu->sid_bits); + } + + writeq_relaxed((smmu->strtab_dma & STRTAB_BASE_ADDR_MASK) | STRTAB_BASE_R= A, + smmu->base + ARM_SMMU_STRTAB_BASE); + writel_relaxed(reg, smmu->base + ARM_SMMU_STRTAB_BASE_CFG); + return 0; +} + static int smmu_init_device(struct hyp_arm_smmu_v3_device *smmu) { unsigned long haddr; @@ -288,6 +345,10 @@ static int smmu_init_device(struct hyp_arm_smmu_v3_dev= ice *smmu) if (ret) goto out_ret; =20 + ret =3D smmu_init_strtab(smmu); + if (ret) + goto out_ret; + return 0; =20 out_ret: @@ -422,6 +483,46 @@ static int smmu_emulate_cmdq_insert(struct hyp_arm_smm= u_v3_device *smmu) return ret; } =20 +static int smmu_update_ste_shadow(struct hyp_arm_smmu_v3_device *smmu, boo= l enabled) +{ + size_t strtab_size; + u32 fmt =3D FIELD_GET(STRTAB_BASE_CFG_FMT, smmu->host_ste_cfg); + + /* Linux doesn't change the fmt nor size of the strtab in the run time. */ + if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) { + if ((fmt !=3D STRTAB_BASE_CFG_FMT_2LVL) || + (strtab_split(smmu) !=3D STRTAB_SPLIT) || + (strtab_log2size(smmu) > (ilog2(STRTAB_MAX_L1_ENTRIES) + STRTAB_SPL= IT)) || + (strtab_split(smmu) >=3D strtab_log2size(smmu))) + return -EINVAL; + strtab_size =3D strtab_l1_size(smmu); + } else { + if ((fmt !=3D STRTAB_BASE_CFG_FMT_LINEAR) || + (strtab_log2size(smmu) > smmu->sid_bits)) + return -EINVAL; + strtab_size =3D strtab_size(smmu); + } + + if (enabled) + return smmu_share_pages(strtab_host_base(smmu), strtab_size); + + return smmu_unshare_pages(strtab_host_base(smmu), strtab_size); +} + +static void smmu_emulate_enable(struct hyp_arm_smmu_v3_device *smmu) +{ + /* Enabling SMMU without CMDQ, means TLB invalidation won't work. */ + if (WARN_ON(!is_cmdq_enabled(smmu))) + return; + + WARN_ON(smmu_update_ste_shadow(smmu, true)); +} + +static void smmu_emulate_disable(struct hyp_arm_smmu_v3_device *smmu) +{ + WARN_ON(smmu_update_ste_shadow(smmu, false)); +} + static void smmu_emulate_cmdq_enable(struct hyp_arm_smmu_v3_device *smmu) { u32 shift =3D smmu->cmdq_host.q_base & Q_BASE_LOG2SIZE; @@ -509,7 +610,23 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_de= vice *smmu, } /* Passthrough the register access for bisectiblity, handled later */ case ARM_SMMU_STRTAB_BASE: + if (is_write) { + /* Must only be written when SMMU_CR0.SMMUEN =3D=3D 0.*/ + if (is_smmu_enabled(smmu)) + break; + smmu->host_ste_base =3D val; + } + mask =3D read_write; + break; case ARM_SMMU_STRTAB_BASE_CFG: + if (is_write) { + /* Must only be written when SMMU_CR0.SMMUEN =3D=3D 0.*/ + if (is_smmu_enabled(smmu)) + break; + smmu->host_ste_cfg =3D val; + } + mask =3D read_write; + break; case ARM_SMMU_GBPA: mask =3D read_write; break; @@ -518,12 +635,17 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_d= evice *smmu, break; if (is_write) { bool last_cmdq_en =3D is_cmdq_enabled(smmu); + bool last_smmu_en =3D is_smmu_enabled(smmu); =20 smmu->cr0 =3D val; if (!last_cmdq_en && is_cmdq_enabled(smmu)) smmu_emulate_cmdq_enable(smmu); else if (last_cmdq_en && !is_cmdq_enabled(smmu)) smmu_emulate_cmdq_disable(smmu); + if (!last_smmu_en && is_smmu_enabled(smmu)) + smmu_emulate_enable(smmu); + else if (last_smmu_en && !is_smmu_enabled(smmu)) + smmu_emulate_disable(smmu); } mask =3D read_write; break; diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h index 74a7f62d93eb..085aead009b6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -15,6 +15,8 @@ * @mmio_addr base address of the SMMU registers * @mmio_size size of the registers resource * @features Features of SMMUv3, subset of the main driver + * @strtab_dma Phys address of stream table + * @strtab_size Stream table size * * Other members are filled and used at runtime by the SMMU driver. * @base Virtual address of SMMU registers @@ -27,6 +29,9 @@ * @cmdq CMDQ as observed by HW * @cmdq_host Host view of the CMDQ, only q_base and llq used. * @cr0 Last value of CR0 + * @host_ste_cfg Host stream table config + * @host_ste_base Host stream table base + * @strtab_cfg Stream table as seen by HW */ struct hyp_arm_smmu_v3_device { phys_addr_t mmio_addr; @@ -46,6 +51,11 @@ struct hyp_arm_smmu_v3_device { struct arm_smmu_queue cmdq; struct arm_smmu_queue cmdq_host; u32 cr0; + dma_addr_t strtab_dma; + size_t strtab_size; + u64 host_ste_cfg; + u64 host_ste_base; + struct arm_smmu_strtab_cfg strtab_cfg; }; =20 extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25BBA478E51 for ; Wed, 15 Jul 2026 11:59:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116781; cv=none; b=Pi1vlfSRrzZQwypgxPij6kgDs8rLJxis/03jvw9uke75vqFCcRIHFr3nh5oaj+LkPWpA6y00jpEdYdeZulRIhJlEP/ksgPMBNNn+rUbuj8tRmKYvDqyn9LB4nCKV8boD/U7EOdVsI1OnQdRp/NtI3o0wpty/jmEvyQHM5aryP5A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116781; c=relaxed/simple; bh=7psO0ivKckhYkqo2jOqG+o3dj9A5xFsGWXBUAqXdSWM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bkUjyC5ZerXRvcBbGDUtBlR1Vg/ogLi1i2+8dey19OPwR9vrL/AUEHRFDvD9O6dn/i11pOsIn08tbHfcTvaXbB8gFVeXnOSLKKnPN+YviNpxs2623dbq33Pkyxkbz0b5NZYF+MNbc2AhNb8RGwOdli8SU7mzyVgeaJ9zl+ltr5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YnKtInSu; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YnKtInSu" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-493fa6e28a7so30673085e9.1 for ; Wed, 15 Jul 2026 04:59:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116776; x=1784721576; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=3arABq/LbCnYkg13T0a9RWMUKBWAd4ICzVlPWgUKHjA=; b=YnKtInSupK5NQfzn9uGRRj8RajWob3uaS80N5o9BSYHYASKmwkRsVzMkvBOHZY/a0x GXn8EnjtD8U52s6mi3WPin40XymQ4tLKuXhOHmDnNbxIrInJGEx9zxmobAGYiW1TsBn+ dsJpagVmsdq0ZV4WPGn/fawWEsrHmXn/uJ5ZSLZxJNCEuIgHwtwiUXSO/N1LyOFgpk4/ qN2n3ns5Ule3kajWUM1762Sna/kr1+bSYX7piRysRCKNvMuOl7EbXPvTyPNtGjt6399V cp2omlJV8qPlVRrdoW975F9NH8D+me3uFCjfpuuPujSUGs1XJbGD+/c88md1vPwMemFQ pKig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116776; x=1784721576; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=3arABq/LbCnYkg13T0a9RWMUKBWAd4ICzVlPWgUKHjA=; b=S8EKQ6G3D7Px0qjnXwljPvZHBOvtIm99Au4CGFHBzQXqD3F7LER5vJ+WZ822U5O3pN dAFRo191OzAWGDQuBdzvLzuFdJMIyDDIxNBSnC4f4Ss6CBJZwws9kgOFEvqsLRM9Hc4a R2QmL1uk+jlqN+HsAyl0ga1PId5HuOw+B6G0BEozEUV3cqj7ZWPI80yYM6tQNwNqITuu G4Q5COe4fBXDvHxDeWtrUIYDds0F8EwzMKK5Y/a4Qt+hMa3oPavwioY61rXlNt8VNGro xBdD+MvqCfUoydtdaUrh3XlpdAzX7QSVpqTI0FNZNYjc3fyGmQQLxGRDy/ZkmHQ+d2MR 1gDw== X-Forwarded-Encrypted: i=1; AHgh+Rqj6UTZXT6PjS5i3Fl1Z26hTxOOLDlu+O/GaT8sBZSgmsAMBMCP/bfCy0nyiijbtCGjzjWfW41J96nH4MQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzHpwNqlwRkX41cEh7/vOfQG81+Dcs0jMy0fMp4m+YGgsIv93z/ s1LIIUoF/6kslfDzSw+JacIPORuRsEtimvSPwib2uBaFxZoiAWFXFdZSkqBCAwtaiOqAZOG1mOP 0UTTi/vUftbsqRA== X-Received: from wmbdx5.prod.google.com ([2002:a05:600c:63c5:b0:493:f781:b52f]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600d:6446:20b0:493:e034:a3b5 with SMTP id 5b1f17b1804b1-4953c27ab18mr21837125e9.24.1784116776358; Wed, 15 Jul 2026 04:59:36 -0700 (PDT) Date: Wed, 15 Jul 2026 11:58:59 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-19-smostafa@google.com> Subject: [PATCH v7 18/24] iommu/arm-smmu-v3-kvm: Shadow STEs From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add STE emulation, when the host sends the CFGI_STE command. Copy the STE as is to the shadow owned by the hypervisor, in the next patch, stage-2 page table will be attached. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 127 +++++++++++++++++- 1 file changed, 121 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index 3b133f24b4ca..fba3b3e15780 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -23,6 +23,9 @@ struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; #define strtab_split(smmu) (FIELD_GET(STRTAB_BASE_CFG_SPLIT, (smmu)->host_= ste_cfg)) #define strtab_l1_size(smmu) ((1UL << (strtab_log2size(smmu) - strtab_spli= t(smmu))) * \ (sizeof(struct arm_smmu_strtab_l1))) +#define strtab_hyp_base(smmu) ((smmu)->features & ARM_SMMU_FEAT_2_LVL_STRT= AB ? \ + (u64 *)(smmu)->strtab_cfg.l2.l1tab :\ + (u64 *)(smmu)->strtab_cfg.linear.table) =20 #define for_each_smmu(smmu) \ for ((smmu) =3D kvm_hyp_arm_smmu_v3_smmus; \ @@ -283,6 +286,94 @@ static int smmu_init_cmdq(struct hyp_arm_smmu_v3_devic= e *smmu) return 0; } =20 +static int smmu_get_host_l2_ste(struct hyp_arm_smmu_v3_device *smmu, u32 s= id, + struct arm_smmu_ste *host_ste_out) +{ + u64 *host_ste_base =3D hyp_phys_to_virt(strtab_host_base(smmu)); + struct arm_smmu_strtab_l1 host_l1_desc; + struct arm_smmu_strtab_l2 *l2ptr; + phys_addr_t host_l2_tab; + int ret, i; + + host_l1_desc.l2ptr =3D READ_ONCE(host_ste_base[arm_smmu_strtab_l1_idx(sid= )]); + if (!(le64_to_cpu(host_l1_desc.l2ptr) & STRTAB_L1_DESC_SPAN)) + return -EINVAL; + + host_l2_tab =3D le64_to_cpu(host_l1_desc.l2ptr) & STRTAB_L1_DESC_L2PTR_MA= SK; + /* Share and pin the table before accessing it. */ + ret =3D smmu_share_pages(host_l2_tab, sizeof(struct arm_smmu_strtab_l2)); + if (ret) + return ret; + + l2ptr =3D hyp_phys_to_virt(host_l2_tab); + + for (i =3D 0 ; i < STRTAB_STE_DWORDS ; ++i) + host_ste_out->data[i] =3D + READ_ONCE(l2ptr->stes[arm_smmu_strtab_l2_idx(sid)].data[i]); + + WARN_ON(smmu_unshare_pages(host_l2_tab, sizeof(struct arm_smmu_strtab_l2)= )); + return 0; +} + +static int smmu_reshadow_ste(struct hyp_arm_smmu_v3_device *smmu, u32 sid,= bool leaf) +{ + struct arm_smmu_strtab_cfg *cfg =3D &smmu->strtab_cfg; + struct arm_smmu_ste *hyp_ste_ptr, *host_ste_ptr, host_ste_copy; + u64 *hyp_ste_base =3D strtab_hyp_base(smmu); + int ret, i; + + /* + * Linux only uses leaf =3D 1, when leaf is 0, we need to verify that this + * is a 2 level table and reshadow of l2. + * Also, we rely on Linux only issuing CFGI_STE to attach a device when + * the SMMU is enabled. + */ + if (!leaf || !is_smmu_enabled(smmu) || + (sid >=3D (1UL << strtab_log2size(smmu)))) + return -EINVAL; + + if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB)) { + struct arm_smmu_ste *hyp_table =3D (struct arm_smmu_ste *)hyp_ste_base; + u64 *host_ste_base =3D hyp_phys_to_virt(strtab_host_base(smmu)); + struct arm_smmu_ste *host_table =3D (struct arm_smmu_ste *)host_ste_base; + + if (sid >=3D cfg->linear.num_ents) + return -E2BIG; + + hyp_ste_ptr =3D &hyp_table[sid]; + host_ste_ptr =3D &host_table[sid]; + } else { + struct arm_smmu_strtab_l1 *l1tab =3D (struct arm_smmu_strtab_l1 *)hyp_st= e_base; + u32 l1_idx =3D arm_smmu_strtab_l1_idx(sid); + struct arm_smmu_strtab_l2 *l2ptr; + + if (l1_idx >=3D cfg->l2.num_l1_ents) + return -E2BIG; + + host_ste_ptr =3D &host_ste_copy; + ret =3D smmu_get_host_l2_ste(smmu, sid, host_ste_ptr); + if (ret) + return ret; + + if (!l1tab[l1_idx].l2ptr) { + struct arm_smmu_strtab_l2 *l2table; + + /* No hypervisor entry, first time the L2 is populated. */ + l2table =3D pkvm_iommu_donate_pages(get_order(sizeof(*l2table))); + if (!l2table) + return -ENOMEM; + arm_smmu_write_strtab_l1_desc(&l1tab[l1_idx], hyp_virt_to_phys(l2table)= ); + } + l2ptr =3D hyp_phys_to_virt(le64_to_cpu(l1tab[l1_idx].l2ptr) & + STRTAB_L1_DESC_L2PTR_MASK); + hyp_ste_ptr =3D &l2ptr->stes[arm_smmu_strtab_l2_idx(sid)]; + } + + for (i =3D 0 ; i < STRTAB_STE_DWORDS ; ++i) + WRITE_ONCE(hyp_ste_ptr->data[i], host_ste_ptr->data[i]); + return 0; +} + static int smmu_init_strtab(struct hyp_arm_smmu_v3_device *smmu) { struct arm_smmu_strtab_cfg *cfg =3D &smmu->strtab_cfg; @@ -396,8 +487,24 @@ static bool smmu_filter_command(struct hyp_arm_smmu_v3= _device *smmu, u64 *comman =20 switch (type) { case CMDQ_OP_CFGI_STE: - /* TBD: SHADOW_STE*/ + { + u32 leaf =3D FIELD_GET(CMDQ_CFGI_1_LEAF, command[1]); + u32 sid =3D FIELD_GET(CMDQ_CFGI_0_SID, command[0]); + bool ret; + + /* + * If STE update is required flush the CMDQ and drop the lock as that + * might require to update the host page table and aquire its lock. + */ + writel(smmu->cmdq.llq.prod, smmu->cmdq.prod_reg); + hyp_spin_unlock(&smmu->hw_lock); + ret =3D smmu_reshadow_ste(smmu, sid, leaf); + hyp_spin_lock(&smmu->hw_lock); + if (ret) + return true; + break; + } case CMDQ_OP_CFGI_ALL: { /* @@ -608,25 +715,33 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_d= evice *smmu, val =3D smmu->cmdq_host.llq.cons | (CMDQ_CONS_ERR & cons); goto out_update_regs; } - /* Passthrough the register access for bisectiblity, handled later */ case ARM_SMMU_STRTAB_BASE: + if (len !=3D sizeof(u64)) + break; if (is_write) { /* Must only be written when SMMU_CR0.SMMUEN =3D=3D 0.*/ if (is_smmu_enabled(smmu)) break; smmu->host_ste_base =3D val; + goto out_ret; + } else { + val =3D smmu->host_ste_base; + goto out_update_regs; } - mask =3D read_write; - break; case ARM_SMMU_STRTAB_BASE_CFG: + if (len !=3D sizeof(u32)) + break; if (is_write) { /* Must only be written when SMMU_CR0.SMMUEN =3D=3D 0.*/ if (is_smmu_enabled(smmu)) break; smmu->host_ste_cfg =3D val; + goto out_ret; + } else { + val =3D smmu->host_ste_cfg; + goto out_update_regs; } - mask =3D read_write; - break; + /* Passthrough the register access for bisectiblity, handled later */ case ARM_SMMU_GBPA: mask =3D read_write; break; --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8E574657C6 for ; Wed, 15 Jul 2026 11:59:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116783; cv=none; b=Thdo4MmK+nk32fR2iwJ2ea8gFVRhuCfA/zJpViPmi8sulzEMuLxlAkCZqVr0edP//I+s06D3vBKCAjZTt+1McIn70Hnzh5zifPoB5HgTn4MRxpDLa1JJDrrz+7kOOT69D/4IdhOyhis6HhscCE55PyVqvj/fvZAvSIbcOov870s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116783; c=relaxed/simple; bh=YQYjynOVyJgoO02um8UcNKToZLA1zwRwRv/O62BjZAU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BhWRxVlpg9gG5z/1jaEDeJFcIpatKccOFjNogr1n4G8hr5S06Vai4KSU9SnxTIlgy6NRQnoKXxkU8QywRN7hUKsnEQxvlmHBkr93OP6LtJHSyErlw4zDvl+wtzgkI9wkDmdVLe9DmvvbymumjoUUDiz1L9rj8vMIBxyRpnZthsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nDWsAqx5; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nDWsAqx5" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-47f4bff865cso844769f8f.0 for ; Wed, 15 Jul 2026 04:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116777; x=1784721577; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=cZv8wMHHgtVyVaoJdrUTfCe907vJcKIQX+U9Fwaj/aI=; b=nDWsAqx5vR2ysFfUq/SzOimozVdcjd/LKW+QA3/vyXPNWPH3Bxcb+l8zwVTA/Cm16W euUwh3O20cBwrZJiRM2FRfQAIAMlo8C572UHXa+sSCPZHew1Rm28iEpXXd3rFytz+IlA x+HfBdXDlP0CTZAwhAEqIA3x/lsBq+GF72J22oThYZKnQweVwueAUXr+CAtGtBo3Uc1K mtc2hNnsn/5uwBqfGwiyWJlF4D4KyUVMdBd+c2+VQB82FQgoHS786i6CzR4rxAZudMX2 D9EqouFpfOV1lhBlO0APX6ffn9NJUXXCoJFeV5QW1DpJjCgpwzzb2RKN7XzPVmA0iW8I ov3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116777; x=1784721577; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=cZv8wMHHgtVyVaoJdrUTfCe907vJcKIQX+U9Fwaj/aI=; b=oXrq+G75xtOojQoIweNMwaNDyzDQDcJi3e37bSQgL/YTEeI8yD8pUVPUZ10uM60x/d 1F26YBo6NKpdlaiNT2QHoJDRGYeDnDrMel3I/jKx5CKePr2WApmcQ+k9P5wUu0TuEeok aCzH7liuooGzod1lLfpsXZlFfhEiGx1LKvThKn+86KXV8gIXE2C6t/XA8HlIzVo8GLQN hecqNmkLJx8B2xnWAAVqWIFFqNL/DpWNIb654bi1yq0v8l4F1luwozL94pPUfzKubTfo 7/wJXjPNZH4wCh37JfUeea/RgZNxo6PXtoOU1nlQW5Q8hGKoODqOePmnIzFQM2lxV3YZ cHQA== X-Forwarded-Encrypted: i=1; AHgh+RoAJUaWeI4lP+V6uWKy9dCMALLIM8/Ri8bDbSMydIwMb0Dcb/4PiMlSN5HoxbkZGQrpC3AjfOPVsDDc7oI=@vger.kernel.org X-Gm-Message-State: AOJu0YyzI1yFHNmJjSiudKEWdjBp1L7PY5nvPawVkLto3G3x/r5EAWrK 9upnd3f8aYJJtZDFlsE0gR17DvsAoGvfHNbWXjUYahC+8rtqE8BNF/9KdD2kaT4eiyxWNBD11ih hulfXNY5UfQ29Ew== X-Received: from wmpj23.prod.google.com ([2002:a05:600c:4897:b0:493:f522:9fbe]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:283:b0:490:688b:f9f8 with SMTP id 5b1f17b1804b1-49518306db9mr62910255e9.27.1784116777309; Wed, 15 Jul 2026 04:59:37 -0700 (PDT) Date: Wed, 15 Jul 2026 11:59:00 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-20-smostafa@google.com> Subject: [PATCH v7 19/24] iommu/arm-smmu-v3-kvm: Share other queues From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Other queues as PRIQ and EVTQ doesn't need to be shadowed. However, we need to make sure they are in a state that disallow them to be donated to the hypervisor or guests. So, keep track of those and share them when they get enabled. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 78 +++++++++++++++++-- .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 8 ++ 2 files changed, 81 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index fba3b3e15780..11de73640a6f 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -74,6 +74,16 @@ static bool is_smmu_enabled(struct hyp_arm_smmu_v3_devic= e *smmu) return FIELD_GET(CR0_SMMUEN, smmu->cr0); } =20 +static bool is_evtq_enabled(struct hyp_arm_smmu_v3_device *smmu) +{ + return FIELD_GET(CR0_EVTQEN, smmu->cr0); +} + +static bool is_priq_enabled(struct hyp_arm_smmu_v3_device *smmu) +{ + return FIELD_GET(CR0_PRIQEN, smmu->cr0); +} + /* * CMDQ, STE host copies are accessed by the hypervisor, we share them to * - Prevent the host from passing protected VM memory. @@ -647,6 +657,18 @@ static void smmu_emulate_cmdq_disable(struct hyp_arm_s= mmu_v3_device *smmu) cmdq_size(&smmu->cmdq_host))); } =20 +static void smmu_emulate_queue(struct hyp_arm_smmu_v3_device *smmu, + unsigned long q_base, size_t ent_size_shift) +{ + /* Q_BASE_ADDR_MASK is not enough as the SMMU also ignores bits > OAS */ + phys_addr_t base =3D q_base & Q_BASE_ADDR_MASK & ((1ULL << smmu->oas) - 1= ); + size_t size =3D 1UL << (FIELD_GET(Q_BASE_LOG2SIZE, q_base) + ent_size_shi= ft); + + /* Queues are aligned to the size also. */ + base &=3D ~(size - 1); + WARN_ON(smmu_share_pages(base, size)); +} + static bool smmu_dabt_device(struct hyp_arm_smmu_v3_device *smmu, struct user_pt_regs *regs, u64 esr, u32 off) @@ -751,12 +773,34 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_d= evice *smmu, if (is_write) { bool last_cmdq_en =3D is_cmdq_enabled(smmu); bool last_smmu_en =3D is_smmu_enabled(smmu); + bool last_evtq_en =3D is_evtq_enabled(smmu); + bool last_priq_en =3D is_priq_enabled(smmu); =20 smmu->cr0 =3D val; if (!last_cmdq_en && is_cmdq_enabled(smmu)) smmu_emulate_cmdq_enable(smmu); else if (last_cmdq_en && !is_cmdq_enabled(smmu)) smmu_emulate_cmdq_disable(smmu); + + /* + * Share PRI and EVTQ to avoid the host using them to write to + * protected memory. However, do not unshare the queues at disable + * as that is more complicated, unsharing from here can lead to + * use-after-unshare issues, and requires ordering with cr0ack. + * The host can disable those queue during shutdown, but it nevers + * changes the base address (even with RPM), so leave the queue + * shared and assert that multiple host writes does not change it. + */ + if (!last_evtq_en && is_evtq_enabled(smmu) && !smmu->evtq_shared) { + smmu_emulate_queue(smmu, smmu->evtq_base, EVTQ_ENT_SZ_SHIFT); + smmu->evtq_shared =3D true; + } + + if (!last_priq_en && is_priq_enabled(smmu) && !smmu->priq_shared) { + smmu_emulate_queue(smmu, smmu->priq_base, PRIQ_ENT_SZ_SHIFT); + smmu->priq_shared =3D true; + } + if (!last_smmu_en && is_smmu_enabled(smmu)) smmu_emulate_enable(smmu); else if (last_smmu_en && !is_smmu_enabled(smmu)) @@ -780,6 +824,33 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_de= vice *smmu, mask =3D read_write; break; } + case ARM_SMMU_EVTQ_BASE: + if (len !=3D sizeof(u64)) + break; + + if (is_write) { + /* See ARM_SMMU_CR0 */ + if (is_evtq_enabled(smmu) || + (smmu->evtq_shared && (smmu->evtq_base !=3D val))) + break; + smmu->evtq_base =3D val; + } + mask =3D read_write; + break; + + case ARM_SMMU_PRIQ_BASE: + if (len !=3D sizeof(u64)) + break; + + if (is_write) { + /* See ARM_SMMU_CR0 */ + if (is_priq_enabled(smmu) || + (smmu->priq_shared && (smmu->priq_base !=3D val))) + break; + smmu->priq_base =3D val; + } + mask =3D read_write; + break; =20 /* Allowed 32 bit registers. */ case ARM_SMMU_EVTQ_IRQ_CFG1: @@ -810,15 +881,12 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_d= evice *smmu, case ARM_SMMU_EVTQ_IRQ_CFG0: case ARM_SMMU_PRIQ_IRQ_CFG0: case ARM_SMMU_GERROR_IRQ_CFG0: + if (len !=3D sizeof(u64)) + break; /* These are RES0 as MSI support is hidden. */ val =3D 0; if (!is_write) goto out_update_regs; - fallthrough; - case ARM_SMMU_EVTQ_BASE: - case ARM_SMMU_PRIQ_BASE: - if (len !=3D sizeof(u64)) - break; mask =3D read_write; break; /* Allowed RO 32 bit registers. */ diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h index 085aead009b6..d96801e433ef 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -32,6 +32,10 @@ * @host_ste_cfg Host stream table config * @host_ste_base Host stream table base * @strtab_cfg Stream table as seen by HW + * @evtq_base Host evtq base reg + * @priq_base Host priq base reg + * @evtq_shared Whether the EVTQ was setup + * @priq_shared Whether the PRIQ was setup */ struct hyp_arm_smmu_v3_device { phys_addr_t mmio_addr; @@ -56,6 +60,10 @@ struct hyp_arm_smmu_v3_device { u64 host_ste_cfg; u64 host_ste_base; struct arm_smmu_strtab_cfg strtab_cfg; + unsigned long evtq_base; + unsigned long priq_base; + bool evtq_shared; + bool priq_shared; }; =20 extern size_t kvm_nvhe_sym(kvm_hyp_arm_smmu_v3_count); --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18AB947CC69 for ; Wed, 15 Jul 2026 11:59:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116783; cv=none; b=gzUKTjvEMQ+rKrk4MRJPAOTiYQegiG1d5hVDALae46QXDDZeqQbfYCUwo3huxOPayePqimhJdFd36HCDSdRY8zHHRX809GCsHm/r6q4X5FfCC90qs8ukiS8s1LoGk8BO9R5xEfMwTlZH1gMojE6nFXzoIyKlHnk1MzdhaZkty1M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116783; c=relaxed/simple; bh=i5CStW7O9rQaAZkBUoHFyUYObK2PXVqrXOInPeWi3Ys=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lUInF+sbCFNbcMa/Re047j/H1fdLjVnYrTrdIJro2W6oCOGzcf8+Kd4JReqmP0Lw7B2Buk+Dxzq5Eolt8PGKvvzL7ro8v2K68cIaSMClnGVsFcvAIEWfnfwGWdKhYCcfHeuvFhWlWbThL7uLc59qy4umosF2i55blSsvPO/gjY0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=b9Cr2xAQ; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="b9Cr2xAQ" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-47345535410so5619376f8f.1 for ; Wed, 15 Jul 2026 04:59:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116779; x=1784721579; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:from:to:cc :subject:date:message-id:reply-to:content-type; bh=8eFC7jmqsBzC+eazaElzsGpHmLo+EQxKGjp4/qK1rg8=; b=b9Cr2xAQ+/4j0+gkeE5AsObeWsv+m+QhY/lLblr8db9Nf6i44AdvUMbdkhnIG2YsnS xuh1deD7zuWpT9DcWJTz1eLe0svNBl9BhkP5NY2jJn0BWsYntNpVY3WHlWAbHglhhqfA eFt0Jv93bQMIyOgt4M+Z8PNn/pkqpqQdZCGmpGdMljAz6VXDNGUCmufXeJzj2FsR8qhl vqxVcBJlUeXWKk8WYNivj0TAorrZ6JKTMYJ48Xg/q7tkkuWaCzek3qjPDjOOc5xb226k MaeM6VniJxFCVcYjT5o3Kv/GoPVOZ3AUyILGd+wSnQqU0i9OxcpXSshOs4ZlsVNqapK8 1ThQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116779; x=1784721579; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=8eFC7jmqsBzC+eazaElzsGpHmLo+EQxKGjp4/qK1rg8=; b=PmKgfxuNMaTOZA7gK2BI8vfI6AIDJVTYVq5g1jTbZHDWXs1rv5StjeULfp70FEfpTe aXTJvw/67iT1qlPEPnycREh+36zd5M6HDNWWiJ0zDFNHeKDgCzgAmDTNrH/KjOX9vhdh GqlOPePEvCGPT7omAr+8dMK3hSAxu5WL2YErfc8jNlDFSfymvjV2pv+FjVLKymFln5R4 dmWQcqv0z6JsI/+50hTtMS/touuT9ya8xJSh2SL+HXCVEB42/quVGIihfQl9whp+kKTk 7+0TFf1k5m7TuSBz1ZC5apJLaKaby9GFclf7sgwlv+Caz94/EWUz633EfSN0hWhH9T24 dMDA== X-Forwarded-Encrypted: i=1; AHgh+RpoIK1aMnblv3uPm9WK8Svri9SvSd4+ecSJ7jv+/OkO6MZiXwFuW4/4dtvVU+fn0ILXbGmTYb68aYRy5gc=@vger.kernel.org X-Gm-Message-State: AOJu0Yw9syZ7RLh/YY0t0Xunm/Is5YGGKR9qZH2R/yafwOeJG/YRBqfO pK3yGTunOtFLyN150xGDJzrS40mrwIdZSxL1sqo8WflXpcUU45clcEK3T5oY3BgWrx68cZun5H3 oR4SI1O06otgXkw== X-Received: from wrrg11.prod.google.com ([2002:adf:fc8b:0:b0:47f:4e4d:2531]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:310d:b0:474:83d4:9916 with SMTP id ffacd0b85a97d-47f4885698emr7936900f8f.17.1784116778676; Wed, 15 Jul 2026 04:59:38 -0700 (PDT) Date: Wed, 15 Jul 2026 11:59:01 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-21-smostafa@google.com> Subject: [PATCH v7 20/24] iommu/arm-smmu-v3-kvm: Emulate GBPA From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The last bit of emulation is GBPA. it must be always set to ABORT, as when the SMMU is disabled it=E2=80=99s not allowed for the host to bypass the SMMU. That's is done by setting the GBPA to ABORT at init time, and host writes are always ignored and host reads always return ABORT. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 32 +++++++++++++++++-- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index 11de73640a6f..45dbab1b18ad 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -139,6 +139,22 @@ static bool smmu_cmdq_has_space(struct arm_smmu_queue = *cmdq, u32 n) return queue_has_space(llq, n); } =20 +static int smmu_abort_gbpa(struct hyp_arm_smmu_v3_device *smmu) +{ + int ret; + u32 reg; + + ret =3D smmu_wait(false, + (readl_relaxed(smmu->base + ARM_SMMU_GBPA) & GBPA_UPDATE) =3D=3D 0); + if (ret) + return ret; + + reg =3D readl_relaxed(smmu->base + ARM_SMMU_GBPA); + writel_relaxed(GBPA_UPDATE | GBPA_ABORT | reg, smmu->base + ARM_SMMU_GBPA= ); + return smmu_wait(false, + (readl_relaxed(smmu->base + ARM_SMMU_GBPA) & GBPA_UPDATE) =3D=3D 0); +} + static bool smmu_cmdq_full(struct arm_smmu_queue *cmdq) { struct arm_smmu_ll_queue *llq =3D &cmdq->llq; @@ -450,6 +466,10 @@ static int smmu_init_device(struct hyp_arm_smmu_v3_dev= ice *smmu) if (ret) goto out_ret; =20 + ret =3D smmu_abort_gbpa(smmu); + if (ret) + goto out_ret; + return 0; =20 out_ret: @@ -763,10 +783,16 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_d= evice *smmu, val =3D smmu->host_ste_cfg; goto out_update_regs; } - /* Passthrough the register access for bisectiblity, handled later */ case ARM_SMMU_GBPA: - mask =3D read_write; - break; + if (len !=3D sizeof(u32)) + break; + + /* Ignore write, always read to abort. */ + if (!is_write) { + val =3D GBPA_ABORT; + goto out_update_regs; + } + goto out_ret; case ARM_SMMU_CR0: if (len !=3D sizeof(u32)) break; --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2863944D68A for ; Wed, 15 Jul 2026 11:59:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116784; cv=none; b=LUc/sh1hdrLRYMXZuBfOMi8753Wvr5Zkrv1dC1YxVL4md9Fd/niqOR5vwQGOhWJxhAlgLegHsiS9iw7AcQognm8auOAA2SO5+4Zmy1mHYSZeo5PVVDTQmm2EC0FlVle2Px2n5dS1Ff9DP88ZGig1ymrUltEjLnZWSG2mhESQTKs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116784; c=relaxed/simple; bh=yEEUCGjMm6Cq047Eod0q14m6OWNvFshd3GgoogvNQCM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LH9f+FufK7MPfqhycXw3mdMIVfO5v9an0wVYFDpbJahmoyir0mUydalwYGJYNDW9TWBysf7dlEWMAkDoGdzJSkEspmmbujDCFfMaB5FE+Kh02I0LY95ga7CoqjKK1cR2Bw8muiWAbk5tfUJRYX+p/n5yP3cBxcTU5Gjh5kOWpHU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UpVMl0b3; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UpVMl0b3" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-493ba771d1bso20794655e9.2 for ; Wed, 15 Jul 2026 04:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116780; x=1784721580; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=JkwZN65XPQxPyKvbylt/w2oKgmc1B/VRyR24GVjyEdM=; b=UpVMl0b33wT1N0HCvaY4VXs/WNNKITpF7HFGvEB7J+ejRkMSKbvn7FbdlVCQmxUBvq DUxvDfOQoK0TrPKCzaCJWhgtWU9yCm46H+hh0bBeMn20P9tgx6TM5BKJWOTRIoAyRDNt LoX9E/rWj2Ikm/bAjrR+xRm6qkzu3W9k2omO7gdH1SOZj2OaiZGzkIAj6RBwpcfURpIx VR3sp4AqzHfZ4vIJSkHuLAAA0t88AVa2VzHOFjCkwD9dlOyk+PT1GcgUZt0iGr9IW6Ra ebgcFMaPuJ+Eznj4LVBTL2Dvp8jnSJIHJuHVCuANop8OofRmdvNv7CxfbJl0tD2OzCmW O09w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116780; x=1784721580; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=JkwZN65XPQxPyKvbylt/w2oKgmc1B/VRyR24GVjyEdM=; b=Vc/tiNP8eI+1ydcm9I3P8f92ZbqAz1Zw3JZTaTn5dkAitVPaD93MsMcLQAFoFEC3co df4ew/ImaKco4mnbK6hDpXRXOQNdnDMuGhD+Zci1sELp+0TaEK+YtJLB6WmF04KBwpxH 2pBH6eDr96hFudLLIqKO56V2ULTAdeLQ5vvASTaJLBPy4oq7Exwh83v0VdbQksw7yDsj j+L7wLLpIjr/opOYiTherKpS+rSXZCBTFdDJ54oH9jZDf05Z/6XpPD+/MFXDQkwX3Dff taOmoQrLTj263KDGpaF3Rw/ljApcHSSY1ywsZ5JqOzPlluU7DF+6IoBsNr3LI7tVPYk1 AVhw== X-Forwarded-Encrypted: i=1; AHgh+RqPEzlBkGPsbi1w7XJ7+niywwy2r30FfYlZD656ktekYzh+TLETfqXwea1j+rOOZ+PWfq6zLcL83JRiRls=@vger.kernel.org X-Gm-Message-State: AOJu0YzX2uS4ZvdFF5AmnKI0CoLIb92/WQK6ocg617D0bVDFAgq7kPcM FZnxrcHPJjiXJE7ETnK9WPbr9irKw+psSBRidVtTVgzlrU7i3/XlWFrqNygLhMhHvsZ8brpvDze RmhnPPWr05opygw== X-Received: from wmop7.prod.google.com ([2002:a05:600c:4687:b0:493:f738:6344]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a111:b0:490:9782:3eb8 with SMTP id 5b1f17b1804b1-493f881d11amr144285835e9.25.1784116780019; Wed, 15 Jul 2026 04:59:40 -0700 (PDT) Date: Wed, 15 Jul 2026 11:59:02 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-22-smostafa@google.com> Subject: [PATCH v7 21/24] iommu/io-pgtable-arm: Support io-pgtable-arm in the hypervisor From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To be able to populate the shadow stage-2 in the hypervisor, compile io-pgtable-arm object for the hypervisor. That requires the hypervisor to support iommu-pages, re-defining some functions as address conversion and stubbing some others as DMA-API. Signed-off-by: Mostafa Saleh --- arch/arm64/kvm/hyp/nvhe/Makefile | 3 +- .../arm/arm-smmu-v3/pkvm/io-pgtable-arm-hyp.h | 68 +++++++++++++++++++ drivers/iommu/io-pgtable-arm.c | 39 ++++++++--- drivers/iommu/io-pgtable-arm.h | 17 +++++ 4 files changed, 117 insertions(+), 10 deletions(-) create mode 100644 drivers/iommu/arm/arm-smmu-v3/pkvm/io-pgtable-arm-hyp.h diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Mak= efile index dcb75fb5b4f1..35ac274ab2c0 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -36,7 +36,8 @@ hyp-obj-y +=3D $(lib-objs) HYP_SMMU_V3_DRV_PATH =3D ../../../../../drivers/iommu/arm/arm-smmu-v3 =20 hyp-obj-$(CONFIG_ARM_SMMU_V3_PKVM) +=3D $(HYP_SMMU_V3_DRV_PATH)/pkvm/arm-s= mmu-v3.o \ - $(HYP_SMMU_V3_DRV_PATH)/arm-smmu-v3-common-lib.o + $(HYP_SMMU_V3_DRV_PATH)/arm-smmu-v3-common-lib.o \ + $(HYP_SMMU_V3_DRV_PATH)/../../io-pgtable-arm.o =20 # Path to simple_ring_buffer.c CFLAGS_trace.nvhe.o +=3D -I$(srctree)/kernel/trace/ diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/io-pgtable-arm-hyp.h b/driv= ers/iommu/arm/arm-smmu-v3/pkvm/io-pgtable-arm-hyp.h new file mode 100644 index 000000000000..c9d20e781aed --- /dev/null +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/io-pgtable-arm-hyp.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef IO_PGTABLE_ARM_HYP_H_ +#define IO_PGTABLE_ARM_HYP_H_ + +#include +#include + +#undef WARN_ONCE +#define WARN_ONCE(condition, format...) WARN_ON(condition) + +static inline void *iommu_alloc_pages_node_sz(int nid, gfp_t gfp, size_t s= ize) +{ + return pkvm_iommu_donate_pages(get_order(size)); +} + +static inline void iommu_free_pages(void *virt) +{ + pkvm_iommu_reclaim_pages(virt); +} + +static inline void *arm_lpae_alloc_data(size_t size) +{ + return pkvm_iommu_donate_pages(get_order(size)); +} + +static inline void arm_lpae_free_data(void *p) +{ + pkvm_iommu_reclaim_pages(p); +} + +#undef phys_to_virt +#define phys_to_virt(x) hyp_phys_to_virt(x) + +#undef virt_to_phys +#define virt_to_phys(x) hyp_virt_to_phys(x) + +/* Stubs used by io-pgtable-arm but no relevant for hypervisor object. */ +#undef dma_map_single +#define dma_map_single(d, p, s, dir) (virt_to_phys(p)) + +#undef dma_unmap_single +#define dma_unmap_single(d, a, s, r) do { (void)(d); (void)(a); (void)(s);= (void)(r); } while (0) + +#define dma_mapping_error(...) (0) + +#undef dev_err +#define dev_err(dev, format, ...) + +static inline int iommu_pages_start_incoherent(void *virt, struct device *= dma_dev) +{ + return 0; +} + +static inline void iommu_pages_free_incoherent(void *virt, + struct device *dma_dev) +{ +} + +static inline void iommu_pages_flush_incoherent(struct device *dma_dev, + void *virt, size_t offset, + size_t len) +{ +} + +struct io_pgtable_ops *kvm_alloc_io_pgtable_ops(enum io_pgtable_fmt fmt, + struct io_pgtable_cfg *cfg, + void *cookie); +#endif /* IO_PGTABLE_ARM_HYP_H_ */ diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 476c0e25631a..cf71087e4d3a 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -20,7 +20,6 @@ #include =20 #include "io-pgtable-arm.h" -#include "iommu-pages.h" =20 #define ARM_LPAE_MAX_ADDR_BITS 52 #define ARM_LPAE_S2_MAX_CONCAT_PAGES 16 @@ -301,6 +300,7 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t = gfp, { struct device *dev =3D cfg->iommu_dev; void *pages; + int nid =3D dev ? dev_to_node(dev) : NUMA_NO_NODE; =20 /* * For very small starting-level translation tables the HW requires a @@ -311,7 +311,7 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t = gfp, if (cfg->alloc) return __arm_lpae_cfg_alloc(size, gfp, cfg, cookie); =20 - pages =3D iommu_alloc_pages_node_sz(dev_to_node(dev), gfp, size); + pages =3D iommu_alloc_pages_node_sz(nid, gfp, size); if (!pages) return NULL; =20 @@ -350,8 +350,7 @@ static void __arm_lpae_free_pages(void *pages, size_t s= ize, static void __arm_lpae_sync_pte(arm_lpae_iopte *ptep, int num_entries, struct io_pgtable_cfg *cfg) { - dma_sync_single_for_device(cfg->iommu_dev, __arm_lpae_dma_addr(ptep), - sizeof(*ptep) * num_entries, DMA_TO_DEVICE); + iommu_pages_flush_incoherent(cfg->iommu_dev, ptep, 0, sizeof(*ptep) * num= _entries); } =20 static void __arm_lpae_clear_pte(arm_lpae_iopte *ptep, struct io_pgtable_c= fg *cfg, int num_entries) @@ -650,7 +649,7 @@ static void arm_lpae_free_pgtable(struct io_pgtable *io= p) struct arm_lpae_io_pgtable *data =3D io_pgtable_to_data(iop); =20 __arm_lpae_free_pgtable(data, data->start_level, data->pgd); - kfree(data); + arm_lpae_free_data(data); } =20 static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data, @@ -964,7 +963,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg) if (cfg->oas > ARM_LPAE_MAX_ADDR_BITS) return NULL; =20 - data =3D kmalloc_obj(*data); + data =3D arm_lpae_alloc_data(sizeof(*data)); if (!data) return NULL; =20 @@ -1091,7 +1090,7 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *c= fg, void *cookie) return &data->iop; =20 out_free_data: - kfree(data); + arm_lpae_free_data(data); return NULL; } =20 @@ -1187,7 +1186,7 @@ arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *c= fg, void *cookie) return &data->iop; =20 out_free_data: - kfree(data); + arm_lpae_free_data(data); return NULL; } =20 @@ -1266,7 +1265,7 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cf= g, void *cookie) return &data->iop; =20 out_free_data: - kfree(data); + arm_lpae_free_data(data); return NULL; } =20 @@ -1299,3 +1298,25 @@ struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_= init_fns =3D { .alloc =3D arm_mali_lpae_alloc_pgtable, .free =3D arm_lpae_free_pgtable, }; + +#ifdef __KVM_NVHE_HYPERVISOR__ +struct io_pgtable_ops *kvm_alloc_io_pgtable_ops(enum io_pgtable_fmt fmt, + struct io_pgtable_cfg *cfg, + void *cookie) +{ + struct io_pgtable *iop; + + if (fmt !=3D ARM_64_LPAE_S2) + return NULL; + + iop =3D arm_64_lpae_alloc_pgtable_s2(cfg, cookie); + if (!iop) + return NULL; + + iop->fmt =3D fmt; + iop->cookie =3D cookie; + iop->cfg =3D *cfg; + + return &iop->ops; +} +#endif \ No newline at end of file diff --git a/drivers/iommu/io-pgtable-arm.h b/drivers/iommu/io-pgtable-arm.h index ba7cfdf7afa0..fd5c5ff2b550 100644 --- a/drivers/iommu/io-pgtable-arm.h +++ b/drivers/iommu/io-pgtable-arm.h @@ -27,4 +27,21 @@ #define ARM_LPAE_TCR_PS_48_BIT 0x5ULL #define ARM_LPAE_TCR_PS_52_BIT 0x6ULL =20 +#ifndef __KVM_NVHE_HYPERVISOR__ +#include "iommu-pages.h" + +static inline void *arm_lpae_alloc_data(size_t size) +{ + return kmalloc(size, GFP_KERNEL); +} + +static inline void arm_lpae_free_data(void *p) +{ + kfree(p); +} + +#else +#include "arm/arm-smmu-v3/pkvm/io-pgtable-arm-hyp.h" +#endif + #endif /* IO_PGTABLE_ARM_H_ */ --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2845447DFA7 for ; Wed, 15 Jul 2026 11:59:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116787; cv=none; b=MKXjAYfOxGYy12af4pDefkMgoccg/DtQ7EcZfnpd6XFfNGT3ZtrmVQSPJc7s3tUlB/zbAph7Wn5xR5fF/FeyzbI8IkFNQzAKzmLXaBf3YshyMcT7avdQJ7Vc+spaT8awU7nlGlp33ne961g6HbcDWaZ2Hwl3JzWekrb2/acQUSk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116787; c=relaxed/simple; bh=aU72uGmuLsWZIhqfP4j5OpYMdCvgPXygDx8qh7XcAuM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VB7KWHnoTGs+NK1MVGdeMXhz51DUbl2kZlfLAA8/EZBvUJR5DiBicaPj37CpvZJQuVzETQHST8jI+PMW9ov/8aOTi1CqxO5xnPtTWQlpCGqBoC+wdX4GUZCgq8NM0NLepgVFjXZ9tPWOWpHtr6/JNM0+ZLgf1ObK0LwG9hw8fd8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SmrRYTwd; arc=none smtp.client-ip=209.85.218.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SmrRYTwd" Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-c15c32294e1so606802866b.3 for ; Wed, 15 Jul 2026 04:59:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116782; x=1784721582; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ol6N7Vx1NTlVrCf/IVQ7ie/xZNl8BBwxYM7kichYdRY=; b=SmrRYTwdJaKgqPZCVx2OJCLfUMGvn4UvCZoA05mPSy2wQERx01nBj/cOrH989p0kEZ QFhK2r4X/UXVqWU8wGFqOR1NH+cMPlQ3Au5OczSLA4bU35XyBkci/nDO9jlTqHsH2pl3 shQu2CzsiH6S/gggAjJ/rwieDlXYLNUx9YI/JZP96f4jaHsL8YD/JsEkasDzhPN1MfGF fkEVpRTleMcp4K4huu+Wlx+qcgmAfnL73wfNaxbprzhSm/IoVNwXSRDvj44iMkVtzSC2 3Ulx8zTjzEcHJ780n03x7W4RZ9Azi9KK7yjSqaJyZy8f7La7bI+6kkOQxrFSre90Ch0x kWOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116782; x=1784721582; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ol6N7Vx1NTlVrCf/IVQ7ie/xZNl8BBwxYM7kichYdRY=; b=EC/VOAeJdDTDog+ssmdzvBdjSml9o/LeEhuQaMdZsm/CU9UnAZZHGYqcYwHIzlqEk6 OMeAvqcgKTCVCTmo8HNghKytmmkTgEFXcBRsAStmGPMpXclujTdaGXcjGi2XuVKwjzEi lUlFfF+DRb2eWuwT8oP0fA6vVT03sodrGmHDbEatfloRsaC8zI8Z6lnDsoAm1DcyxwEZ pwPLEzm81Ar9EhUvZLJSTpzQn0yUTQV7Q0SXDMFkdTHIPtJfChwrODtJiXRG6kFW65NV GYbH7EqJe6mbJ9zgqlRDq3uXELX8STjIrVX/DcP8Fd87t1GIVruxBaeno8WShmyI2G+6 X2kw== X-Forwarded-Encrypted: i=1; AHgh+RoC/mPZXvQME9sTd/gNRdIz7qroXOHJvePg4uf65Vvil2b6TFbrofQ6mEKhsHw+ElL/0fbx3J+azHdGyOc=@vger.kernel.org X-Gm-Message-State: AOJu0YwRcc6mT8hCu1lQtGZDaWQU/g1igfQqzJu6gVDpdDgrNNLZsLUT xlp6Wk4AES5uggOWQLsdIHF8eS8iJtD4CA02NTc8rx/nRWS02Z3pywh3FTCJqMJCU1nLVkO7jOX gOI8EcoDoRAxIFA== X-Received: from ejbwn11.prod.google.com ([2002:a17:907:68b:b0:c15:cc45:3883]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:72d1:b0:c15:df6d:4b45 with SMTP id a640c23a62f3a-c1661879330mr426876166b.36.1784116781057; Wed, 15 Jul 2026 04:59:41 -0700 (PDT) Date: Wed, 15 Jul 2026 11:59:03 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-23-smostafa@google.com> Subject: [PATCH v7 22/24] iommu/arm-smmu-v3-kvm: Shadow the CPU stage-2 page table From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Based on the callbacks from the hypervisor, update the SMMUv3 Identity mapped page table. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 249 +++++++++++++++++- .../iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h | 2 + 2 files changed, 249 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index 45dbab1b18ad..f30757dd9b11 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -13,6 +13,9 @@ =20 #include "arm_smmu_v3.h" =20 +#include +#include "../../../io-pgtable-arm.h" + size_t __ro_after_init kvm_hyp_arm_smmu_v3_count; struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; =20 @@ -64,6 +67,9 @@ struct hyp_arm_smmu_v3_device *kvm_hyp_arm_smmu_v3_smmus; __ret; \ }) =20 +/* Protected by host_mmu.lock from core code. */ +static struct io_pgtable *idmap_pgtable; + static bool is_cmdq_enabled(struct hyp_arm_smmu_v3_device *smmu) { return FIELD_GET(CR0_CMDQEN, smmu->cr0); @@ -211,7 +217,6 @@ static int smmu_sync_cmd(struct hyp_arm_smmu_v3_device = *smmu) smmu_cmdq_empty(&smmu->cmdq)); } =20 -__maybe_unused static int smmu_send_cmd(struct hyp_arm_smmu_v3_device *smmu, struct arm_smmu_cmd *cmd) { @@ -224,6 +229,69 @@ static int smmu_send_cmd(struct hyp_arm_smmu_v3_device= *smmu, return smmu_sync_cmd(smmu); } =20 +static void __smmu_add_cmd(void *__opaque, struct arm_smmu_cmdq_batch *unu= sed, + struct arm_smmu_cmd *cmd) +{ + struct hyp_arm_smmu_v3_device *smmu =3D (struct hyp_arm_smmu_v3_device *)= __opaque; + + WARN_ON(smmu_add_cmd(smmu, cmd)); +} + +static int smmu_tlb_inv_range_smmu(struct hyp_arm_smmu_v3_device *smmu, + struct arm_smmu_cmd *cmd, + unsigned long iova, size_t size, size_t granule, + bool leaf) +{ + arm_smmu_tlb_inv_build(cmd, iova, size, granule, + PAGE_SHIFT, smmu->features & ARM_SMMU_FEAT_RANGE_INV, + smmu, leaf, __smmu_add_cmd, NULL); + return smmu_sync_cmd(smmu); +} + +static void smmu_tlb_inv_range(unsigned long iova, size_t size, size_t gra= nule, + bool leaf) +{ + struct arm_smmu_cmd cmd_s1 =3D arm_smmu_make_cmd_op(CMDQ_OP_TLBI_NH_ALL); + struct hyp_arm_smmu_v3_device *smmu; + + for_each_smmu(smmu) { + struct arm_smmu_cmd cmd =3D arm_smmu_make_cmd_op(CMDQ_OP_TLBI_S2_IPA); + + hyp_spin_lock(&smmu->hw_lock); + /* + * Don't bother if SMMU is disabled, this would be useful for the case + * when RPM is supported to avoid touching the SMMU MMIO when disabled. + * The hypervisor also asserts CMDQEN is enabled before the SMMU is + * enabled. As otherwise the host can prevent the hypervisor from doing + * TLB invalidations. + * When the SMMU is re-enabled the hypervisor clean the TLBs. + */ + if (smmu->active) { + WARN_ON(smmu_tlb_inv_range_smmu(smmu, &cmd, iova, size, granule, leaf)); + WARN_ON(smmu_send_cmd(smmu, &cmd_s1)); + } + hyp_spin_unlock(&smmu->hw_lock); + } +} + +static void smmu_tlb_flush_walk(unsigned long iova, size_t size, + size_t granule, void *cookie) +{ + smmu_tlb_inv_range(iova, size, granule, false); +} + +static void smmu_tlb_add_page(struct iommu_iotlb_gather *gather, + unsigned long iova, size_t granule, + void *cookie) +{ + smmu_tlb_inv_range(iova, granule, granule, true); +} + +static const struct iommu_flush_ops smmu_tlb_ops =3D { + .tlb_flush_walk =3D smmu_tlb_flush_walk, + .tlb_add_page =3D smmu_tlb_add_page, +}; + /* Put the device in a state that can be probed by the host driver. */ static void smmu_deinit_device(struct hyp_arm_smmu_v3_device *smmu) { @@ -477,6 +545,38 @@ static int smmu_init_device(struct hyp_arm_smmu_v3_dev= ice *smmu) return ret; } =20 +static int smmu_init_pgt(void) +{ + /* Default values overridden based on SMMUs common features. */ + struct io_pgtable_cfg cfg =3D (struct io_pgtable_cfg) { + .tlb =3D &smmu_tlb_ops, + .pgsize_bitmap =3D -1, + .ias =3D 48, + .oas =3D 48, + .coherent_walk =3D true, + .quirks =3D IO_PGTABLE_QUIRK_NO_WARN, + }; + struct hyp_arm_smmu_v3_device *smmu; + struct io_pgtable_ops *ops; + + for_each_smmu(smmu) { + cfg.ias =3D min(cfg.ias, smmu->oas); + cfg.oas =3D min(cfg.oas, smmu->oas); + cfg.pgsize_bitmap &=3D smmu->pgsize_bitmap; + cfg.coherent_walk &=3D !!(smmu->features & ARM_SMMU_FEAT_COHERENCY); + } + + /* At least PAGE_SIZE must be supported by all SMMUs*/ + if ((cfg.pgsize_bitmap & PAGE_SIZE) =3D=3D 0) + return -EINVAL; + + ops =3D kvm_alloc_io_pgtable_ops(ARM_64_LPAE_S2, &cfg, NULL); + if (!ops) + return -ENOMEM; + idmap_pgtable =3D io_pgtable_ops_to_pgtable(ops); + return 0; +} + /* Called while is the host is still trusted. */ static int smmu_init(void) { @@ -502,7 +602,10 @@ static int smmu_init(void) =20 BUILD_BUG_ON(sizeof(hyp_spinlock_t) !=3D sizeof(u32)); =20 - return 0; + ret =3D smmu_init_pgt(); + if (ret) + goto out_reclaim_smmu; + return ret; =20 out_reclaim_smmu: while (smmu !=3D kvm_hyp_arm_smmu_v3_smmus) @@ -646,6 +749,34 @@ static int smmu_update_ste_shadow(struct hyp_arm_smmu_= v3_device *smmu, bool enab return smmu_unshare_pages(strtab_host_base(smmu), strtab_size); } =20 +static int smmu_flush_all_tlb(struct hyp_arm_smmu_v3_device *smmu) +{ + int ret; + u32 cr0; + struct arm_smmu_cmd cmd =3D arm_smmu_make_cmd_op(CMDQ_OP_TLBI_NSNH_ALL); + + hyp_spin_lock(&smmu->hw_lock); + /* + * This must be called when the SMMU is getting enabled. + * First enable the cmdq and then invalidate the TLB. + */ + cr0 =3D readl_relaxed(smmu->base + ARM_SMMU_CR0); + if (!(cr0 & CR0_CMDQEN)) { + cr0 |=3D CR0_CMDQEN; + writel_relaxed(cr0, smmu->base + ARM_SMMU_CR0); + ret =3D smmu_wait(false, + readl_relaxed(smmu->base + ARM_SMMU_CR0ACK) =3D=3D cr0); + if (ret) { + hyp_spin_unlock(&smmu->hw_lock); + return ret; + } + } + + ret =3D smmu_send_cmd(smmu, &cmd); + hyp_spin_unlock(&smmu->hw_lock); + return ret; +} + static void smmu_emulate_enable(struct hyp_arm_smmu_v3_device *smmu) { /* Enabling SMMU without CMDQ, means TLB invalidation won't work. */ @@ -653,6 +784,8 @@ static void smmu_emulate_enable(struct hyp_arm_smmu_v3_= device *smmu) return; =20 WARN_ON(smmu_update_ste_shadow(smmu, true)); + /* Clean the TLBs each time the SMMU is enabled. */ + WARN_ON(smmu_flush_all_tlb(smmu)); } =20 static void smmu_emulate_disable(struct hyp_arm_smmu_v3_device *smmu) @@ -673,6 +806,13 @@ static void smmu_emulate_cmdq_enable(struct hyp_arm_sm= mu_v3_device *smmu) =20 static void smmu_emulate_cmdq_disable(struct hyp_arm_smmu_v3_device *smmu) { + /* + * We can not enable the SMMU if the CMDQ is enabled and similarly + * we can not disable the CMDQ if the SMMU is enabled, as that can + * lead to stale TLBs. + */ + WARN_ON(is_smmu_enabled(smmu)); + WARN_ON(smmu_unshare_pages(smmu->cmdq_host.base_dma, cmdq_size(&smmu->cmdq_host))); } @@ -936,6 +1076,18 @@ static bool smmu_dabt_device(struct hyp_arm_smmu_v3_d= evice *smmu, else writel_relaxed(val & mask, smmu->base + off); =20 + /* + * Make sure writes to CR0 are immediately observed, that is important + * when synchronizing with TLB invalidation as reading CR0 is enough + * to deduce the SMMU state, and we have to enforce the ack with the + * hw_lock aquired. + */ + if (off =3D=3D ARM_SMMU_CR0) { + WARN_ON(smmu_wait(false, + readl_relaxed(smmu->base + ARM_SMMU_CR0ACK) =3D=3D (val & mask)= )); + smmu->active =3D !!(val & CR0_CMDQEN); + } + hyp_spin_unlock(&smmu->hw_lock); return true; } @@ -973,8 +1125,101 @@ static bool smmu_dabt_handler(struct user_pt_regs *r= egs, u64 esr, u64 addr) return false; } =20 +static size_t smmu_pgsize_idmap(size_t size, u64 paddr, size_t pgsize_bitm= ap) +{ + size_t pgsizes; + + /* Remove page sizes that are larger than the current size */ + pgsizes =3D pgsize_bitmap & GENMASK_ULL(__fls(size), 0); + + /* Remove page sizes that the address is not aligned to. */ + if (likely(paddr)) + pgsizes &=3D GENMASK_ULL(__ffs(paddr), 0); + + WARN_ON(!pgsizes); + + /* Return the largest page size that fits. */ + return BIT(__fls(pgsizes)); +} + static int smmu_host_stage2_idmap(phys_addr_t start, phys_addr_t end, int = prot) { + size_t pgsize =3D PAGE_SIZE, pgcount, size; + struct io_pgtable *pgtable =3D idmap_pgtable; + int ret =3D 0; + + end =3D min(end, BIT(pgtable->cfg.oas)); + if (start >=3D end) + return 0; + + size =3D end - start; + if (prot) { + size_t mapped; + + if (!(prot & IOMMU_MMIO)) + prot |=3D IOMMU_CACHE; + + while (size) { + mapped =3D 0; + /* + * We handle pages size for memory and MMIO differently: + * - memory: Map everything with PAGE_SIZE, that is guaranteed to + * find memory as we allocated enough pages to cover the entire + * memory, we do that as io-pgtable-arm doesn't support + * split_blk_unmap logic any more, so we can't break blocks once + * mapped to tables. + * - MMIO: Unlike memory, pKVM allocate 1G to for all MMIO, while + * the MMIO space can be large, as it is assumed to cover the + * whole IAS that is not memory, we have to use block mappings, + * that is fine for MMIO as it is never donated at the moment, + * so we never need to unmap MMIO at the run time triggereing + * split block logic. + */ + if (prot & IOMMU_MMIO) + pgsize =3D smmu_pgsize_idmap(size, start, pgtable->cfg.pgsize_bitmap); + + pgcount =3D size / pgsize; + ret =3D pgtable->ops.map_pages(&pgtable->ops, start, start, + pgsize, pgcount, prot, 0, &mapped); + size -=3D mapped; + start +=3D mapped; + + if (ret =3D=3D -EEXIST) { + /* + * It is possible to get EEXIST when a VM dies with pages + * in a shared state. + */ + ret =3D 0; + size -=3D pgsize; + start +=3D pgsize; + continue; + } + /* Map failures doesn't impact security, tolerate it. */ + if (!mapped || ret) + break; + } + } else { + struct iommu_iotlb_gather gather; + size_t unmapped; + + while (size) { + pgcount =3D size / pgsize; + iommu_iotlb_gather_init(&gather); + unmapped =3D pgtable->ops.unmap_pages(&pgtable->ops, start, + pgsize, pgcount, &gather); + size -=3D unmapped; + start +=3D unmapped; + if (!unmapped) + break; + } + } + + if (ret) + return ret; + + if (WARN_ON(size)) + return -EINVAL; + return 0; } =20 diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h index d96801e433ef..9599809b46fc 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm_smmu_v3.h @@ -29,6 +29,7 @@ * @cmdq CMDQ as observed by HW * @cmdq_host Host view of the CMDQ, only q_base and llq used. * @cr0 Last value of CR0 + * @active Is SMMU HW cmdq usable, protected by hw_lock * @host_ste_cfg Host stream table config * @host_ste_base Host stream table base * @strtab_cfg Stream table as seen by HW @@ -55,6 +56,7 @@ struct hyp_arm_smmu_v3_device { struct arm_smmu_queue cmdq; struct arm_smmu_queue cmdq_host; u32 cr0; + bool active; dma_addr_t strtab_dma; size_t strtab_size; u64 host_ste_cfg; --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2B7044C65F for ; Wed, 15 Jul 2026 11:59:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116787; cv=none; b=DujBdctM1acouxirySbZnpYB+0E/gxRHObJr/ePrMr4pqhcR2q0aDart/PoLSP0RDtSFoBnHybXDiFQxZFAI+C9A+qxd1Sh44J+/3F8FBFiMybu/KfRbQn+QpJP1cdw+iMLxOP6XGA9q/NsEGXnvMIgd9iRVfR50aVEjDY1FenY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116787; c=relaxed/simple; bh=fhQT+JAbH2Qy2Y5/pbDfbmISq0pN5McJ8TxGoZqW/QI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MzIsvcUNuAMlmWFoJPZMs96n9ySveEsWbcqZKCrxYpeVaYKCEDnxx8JRaveLFRIr0SHrJWvpZBZg8EH1bpMnfMO3CXlisr1O/sPEb18Mh1ulvAQ7mUhybG4XP08CPqZesyaFMPPf9aSU5AXs1ACFu9lipEP+VdKxJM6Q8aYqtYQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EU7gYHbK; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EU7gYHbK" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-69a1ab55774so6694755a12.1 for ; Wed, 15 Jul 2026 04:59:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116783; x=1784721583; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=UCxykfAjLEYMMvCVcdT1GfjF39XVffctpOWmsmN/ev4=; b=EU7gYHbK+eYUPYkmuIPZ2SHfhUn8f+IIod6vm4/bqGEEHfGFc+U2NoBClboNGNALYV FszdVi9JJlDB9isSNArpHYM/9KNIx38lgaV3KPbe0NeUDnZE1zuyZSo6rfgZf/KwS6fJ NUUhkUIJBTWLJHa5BsFZpf82sEBrQfc3Nb6S1XbgtoCdagjqf/WD3+LXN4mTilNg8d7Z 9TS5c3aaCWZuurrzZZJ4nlO9IpZAcMZMYM53iLu/daRAy4KNTEmVK92Y7kkxZsw4+e+2 m0TdFjpK1GnuJ1q7ia/aAv5D/xAc32g3sIgFOcmrxN+p1wAfL7nIE8X9/XagCA4ktP0A gPdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116783; x=1784721583; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=UCxykfAjLEYMMvCVcdT1GfjF39XVffctpOWmsmN/ev4=; b=EdgbTFKj+wnF597w+NoBnKh52chQn61uPfbQ5JHnMk8i4RkEZUEzzT/3i0DA2Lno9w yT6MLslTLBeP+xvBX5Oyt7RODmxEGm8DhyGdZgEos7M3/7IUm17l5WnazR+B9zfLFmln 1Buflk4wPk4zKSZT7TlMVlrCk+dpZkgKpPu08VGS4U6fGgwj4Xpgh4gcGVmf7AkEsa7N 8rnDxMYkYa4LKEIYcftySgOJjZkWR9bVQS5POBAJ/XtVC/aNSOk766ZX8y8hTo42+6vg an1/ENoHIE1d8raEZ6E011NqSCDT3IZ81xJKcGBdIWW2NgEjj4asO3Naov1pGES7Zk6k LSZA== X-Forwarded-Encrypted: i=1; AHgh+Rpwfy6jsni+/WhKDKiI8fFR3TNrS+ylSkV7qvpNpXnHp6FSJSNjNnzCC7mRB/UOO6/2XuW5+5rVQ8SFMw4=@vger.kernel.org X-Gm-Message-State: AOJu0YwuVDzUFNPFTmy9pM2stBqUbhD6meQOujSHpZPeReAos08WGDjp A/gzlUMffIf126tDazu2R9eCt+snQOOBrpqoM6hsD4pPgJEQBR+1CFwYq/tYvY1koefSiyQzOIT BwiBMnCyg63ybVQ== X-Received: from edvw11.prod.google.com ([2002:a05:6402:128b:b0:69c:79d3:df42]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:a292:20b0:69c:76c6:9bef with SMTP id 4fb4d7f45d1cf-69e197771femr873479a12.10.1784116782673; Wed, 15 Jul 2026 04:59:42 -0700 (PDT) Date: Wed, 15 Jul 2026 11:59:04 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-24-smostafa@google.com> Subject: [PATCH v7 23/24] iommu/arm-smmu-v3-kvm: Enable nesting From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now, as the hypervisor controls the command queue, stream table, and shadows the stage-2 page table. Enable stage-2 in case the host puts an STE in bypass or stage-1. Signed-off-by: Mostafa Saleh --- .../iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c | 109 ++++++++++++++++-- 1 file changed, 102 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c b/drivers/iom= mu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c index f30757dd9b11..4625240a5de2 100644 --- a/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/pkvm/arm-smmu-v3.c @@ -380,6 +380,59 @@ static int smmu_init_cmdq(struct hyp_arm_smmu_v3_devic= e *smmu) return 0; } =20 +static int smmu_attach_stage_2(struct arm_smmu_ste *ste) +{ + unsigned long vttbr; + unsigned long ts, sl, ic, oc, sh, tg, ps; + unsigned long cfg; + struct io_pgtable_cfg *pgt_cfg =3D &idmap_pgtable->cfg; + + cfg =3D FIELD_GET(STRTAB_STE_0_CFG, le64_to_cpu(ste->data[0])); + if (!FIELD_GET(STRTAB_STE_0_V, le64_to_cpu(ste->data[0])) || + (cfg =3D=3D STRTAB_STE_0_CFG_ABORT)) { + ste->data[2] =3D 0; + ste->data[3] =3D 0; + return 0; + } + /* S2 is not advertised, that should never be attempted. */ + if (cfg =3D=3D STRTAB_STE_0_CFG_NESTED) + return -EINVAL; + vttbr =3D pgt_cfg->arm_lpae_s2_cfg.vttbr; + ps =3D pgt_cfg->arm_lpae_s2_cfg.vtcr.ps; + tg =3D pgt_cfg->arm_lpae_s2_cfg.vtcr.tg; + sh =3D pgt_cfg->arm_lpae_s2_cfg.vtcr.sh; + oc =3D pgt_cfg->arm_lpae_s2_cfg.vtcr.orgn; + ic =3D pgt_cfg->arm_lpae_s2_cfg.vtcr.irgn; + sl =3D pgt_cfg->arm_lpae_s2_cfg.vtcr.sl; + ts =3D pgt_cfg->arm_lpae_s2_cfg.vtcr.tsz; + + ste->data[1] &=3D ~cpu_to_le64(STRTAB_STE_1_SHCFG); + ste->data[1] |=3D cpu_to_le64(FIELD_PREP(STRTAB_STE_1_SHCFG, STRTAB_STE_1= _SHCFG_INCOMING)); + + ste->data[1] &=3D ~cpu_to_le64(STRTAB_STE_1_EATS | STRTAB_STE_1_S2FWB); + + /* The host shouldn't write dwords 2 and 3, overwrite them. */ + ste->data[2] =3D cpu_to_le64(FIELD_PREP(STRTAB_STE_2_VTCR, + FIELD_PREP(STRTAB_STE_2_VTCR_S2PS, ps) | + FIELD_PREP(STRTAB_STE_2_VTCR_S2TG, tg) | + FIELD_PREP(STRTAB_STE_2_VTCR_S2SH0, sh) | + FIELD_PREP(STRTAB_STE_2_VTCR_S2OR0, oc) | + FIELD_PREP(STRTAB_STE_2_VTCR_S2IR0, ic) | + FIELD_PREP(STRTAB_STE_2_VTCR_S2SL0, sl) | + FIELD_PREP(STRTAB_STE_2_VTCR_S2T0SZ, ts)) | + FIELD_PREP(STRTAB_STE_2_S2VMID, 0) | + STRTAB_STE_2_S2AA64 | STRTAB_STE_2_S2R | + #ifdef __BIG_ENDIAN + STRTAB_STE_2_S2ENDI | +#endif + STRTAB_STE_2_S2PTW); + + ste->data[3] =3D cpu_to_le64(vttbr & STRTAB_STE_3_S2TTB_MASK); + /* Convert S1 =3D> nested and bypass =3D> S2 */ + ste->data[0] |=3D cpu_to_le64(FIELD_PREP(STRTAB_STE_0_CFG, cfg | BIT(1))); + return 0; +} + static int smmu_get_host_l2_ste(struct hyp_arm_smmu_v3_device *smmu, u32 s= id, struct arm_smmu_ste *host_ste_out) { @@ -412,8 +465,12 @@ static int smmu_get_host_l2_ste(struct hyp_arm_smmu_v3= _device *smmu, u32 sid, static int smmu_reshadow_ste(struct hyp_arm_smmu_v3_device *smmu, u32 sid,= bool leaf) { struct arm_smmu_strtab_cfg *cfg =3D &smmu->strtab_cfg; - struct arm_smmu_ste *hyp_ste_ptr, *host_ste_ptr, host_ste_copy; + struct arm_smmu_ste *hyp_ste_ptr; u64 *hyp_ste_base =3D strtab_hyp_base(smmu); + struct arm_smmu_ste target =3D {}; + struct arm_smmu_cmd cfgi_cmd =3D arm_smmu_make_cmd_cfgi_ste(sid, true); + bool cur_valid, target_valid; + u32 target_cfg; int ret, i; =20 /* @@ -435,7 +492,7 @@ static int smmu_reshadow_ste(struct hyp_arm_smmu_v3_dev= ice *smmu, u32 sid, bool return -E2BIG; =20 hyp_ste_ptr =3D &hyp_table[sid]; - host_ste_ptr =3D &host_table[sid]; + memcpy(target.data, host_table[sid].data, STRTAB_STE_DWORDS << 3); } else { struct arm_smmu_strtab_l1 *l1tab =3D (struct arm_smmu_strtab_l1 *)hyp_st= e_base; u32 l1_idx =3D arm_smmu_strtab_l1_idx(sid); @@ -444,8 +501,7 @@ static int smmu_reshadow_ste(struct hyp_arm_smmu_v3_dev= ice *smmu, u32 sid, bool if (l1_idx >=3D cfg->l2.num_l1_ents) return -E2BIG; =20 - host_ste_ptr =3D &host_ste_copy; - ret =3D smmu_get_host_l2_ste(smmu, sid, host_ste_ptr); + ret =3D smmu_get_host_l2_ste(smmu, sid, &target); if (ret) return ret; =20 @@ -463,9 +519,48 @@ static int smmu_reshadow_ste(struct hyp_arm_smmu_v3_de= vice *smmu, u32 sid, bool hyp_ste_ptr =3D &l2ptr->stes[arm_smmu_strtab_l2_idx(sid)]; } =20 - for (i =3D 0 ; i < STRTAB_STE_DWORDS ; ++i) - WRITE_ONCE(hyp_ste_ptr->data[i], host_ste_ptr->data[i]); - return 0; + /* + * Summary of each host emulated state vs real HW. + * | Host | HW | + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D + * | V=3D0 | V=3D0 | + * | Abort | Abort | + * | Bypass | S2 | + * | S1 | S1+S2 | + * + * For the host, any V=3D0 transition is not hitless, all other permutati= ons of + * (abort, bypass, S1) transitions are hitless. + * For the HW state, any V=3D0 transition is not hitless, as all the S2 c= onfig is + * always the same (ttbr, vtcr...), all other transitions should be hitle= ss too. + * However, the host is not trusted, which means that any V=3D0 <=3D> V= =3D1 transitions + * or any transition to an abort STE we need to enforce writing order of = the STE + * dword 0 and add CFGI. + * Otherwise, we write the STE in the opposite order to cover cases from = abort + * to S2 or nested. + */ + ret =3D smmu_attach_stage_2(&target); + if (ret) + return ret; + hyp_spin_lock(&smmu->hw_lock); + cur_valid =3D FIELD_GET(STRTAB_STE_0_V, le64_to_cpu(hyp_ste_ptr->data[0])= ); + target_cfg =3D FIELD_GET(STRTAB_STE_0_CFG, le64_to_cpu(target.data[0])); + target_valid =3D FIELD_GET(STRTAB_STE_0_V, le64_to_cpu(target.data[0])); + if ((cur_valid && !target_valid) || + (target_cfg =3D=3D STRTAB_STE_0_CFG_ABORT)) { + WRITE_ONCE(hyp_ste_ptr->data[0], target.data[0]); + WARN_ON(smmu_send_cmd(smmu, &cfgi_cmd)); + for (i =3D 1; i < STRTAB_STE_DWORDS; i++) + WRITE_ONCE(hyp_ste_ptr->data[i], target.data[i]); + } else { + for (i =3D 1; i < STRTAB_STE_DWORDS; i++) + WRITE_ONCE(hyp_ste_ptr->data[i], target.data[i]); + WARN_ON(smmu_send_cmd(smmu, &cfgi_cmd)); + WRITE_ONCE(hyp_ste_ptr->data[0], target.data[0]); + } + + ret =3D smmu_send_cmd(smmu, &cfgi_cmd); + hyp_spin_unlock(&smmu->hw_lock); + return ret; } =20 static int smmu_init_strtab(struct hyp_arm_smmu_v3_device *smmu) --=20 2.55.0.141.g00534a21ce-goog From nobody Sat Jul 25 17:33:44 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B200147ECFC for ; Wed, 15 Jul 2026 11:59:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116789; cv=none; b=ZclW5mRx86JSDfxnrjyJZXQd/hwCLDqC8stFfP7/HsbCRWOlaEXzneNKMI1QtASSeVVHoVVhafyvSwaQeHfAPhwX54AdQ+mlRItchVALNh8RNBjzE2tXq9RDDeUSM+XlMpX5nlluBfBEtaOdXYMMgZAaf2wEqAejV3TqkHCdZTY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784116789; c=relaxed/simple; bh=X92nv7EtYWCgdhcWR41Vd3TMlIXF4weejIJGzxYYk/k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=V7MXI6mPutMlPf8fhQoLDnoj/hHdFaedeOS8DYIR9zdsEHWo3YlCl5glZvSIQM/CAuB/Y3w0HO8w0dBSAxnRSzPoFjBcRFKtpdTaXX0M9aRPc/ELGlzO7IcmG3VUjDHuz4Sp91TybxJf05VpTbrBqv7+HzRXN0RFXgzHfQ2gC2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dXg3qgYL; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--smostafa.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dXg3qgYL" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-493dc8408fdso16997905e9.1 for ; Wed, 15 Jul 2026 04:59:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784116784; x=1784721584; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=5C0qOSsi5b2SM4RQ0BVtOxMAlSvEVPuXW/TZ/ZTmFdI=; b=dXg3qgYLe4p0ArjP5std1aY67E+77dt318MKmzVtJ9VX0os8e/mn8sqbKqplrzSlSo HBF39yPIPgtCP0PHJWzkYoyeXe1lcTkLJj1uG3Hk9qS4cBmytZyh0Dci26NB5QaWGbQ1 S2OVAw7fgSCzVHgqGBxwsZ+mr6OrLFwnoiqoNx44Cj5Go/wnEGG2OVPKbpTvS2pZW//7 MTPcXUQx+s+/lGAmGqzCHbC9an/HkTxMhkROvjDNyC2HrMVUH7vMz9SoJXDZx49i4i0S ZgYxMdCa8vrmteJVxCiQCKFb6Phdo0HaLA3UxTrNzH2EtLAjxyKtc/pCNT0QVusmFrra VpGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784116784; x=1784721584; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=5C0qOSsi5b2SM4RQ0BVtOxMAlSvEVPuXW/TZ/ZTmFdI=; b=c/2OdsRu2bowWf4jF0OIldMNcYDHI6+J+Cjjd6PZ/Rrrz1HJoz77NIZp/J1Nmn1J5t FA0fn30vmWsZZYPsqfFooByVPu+9wb3bHO7u10jPXsslfU/mItNqPkkUJlEd8w7FS/zO 8tWihjWNbS5ZxbIfvfJSQ5IZ1Nb2LQ6NGlZnOGdBKwA6kPS4vLja03VWyXqigTj8vgWH gLkLUg44dsPdVlfAyaSVaJ3YCMIRLoSvpH7aamnf9jIi+r1scrcvjhEy88NKreZ8l8Ns Aei51MukccB3moiJWZeDXkZ028sr6RxLTWtw2Y/bX0FwqbewdlNUVp0GXVQ8TIzwn3Dr 79lQ== X-Forwarded-Encrypted: i=1; AHgh+RpR+iGvuWhr3O60mpknIKVA8Pc6d6Hb5+iPDhCFCkSX0WMH7EezKrz+eYy7NHk1aFhJAtg1O7YW2kxxAw8=@vger.kernel.org X-Gm-Message-State: AOJu0YxJLdCXCyI4QYs0ysKF+6KFtfYUl057YxAOAEN8o2JDMY1R36CV RKXLS78D3glrjeruN5b2hmxW1iuQG1icd+0XmxvUMEsXqvdTci4qfl+wXvyWmSPZ9UTUnOBRXkp SVohJlHQBNsOwPA== X-Received: from wmiv11.prod.google.com ([2002:a05:600c:e40b:b0:493:ab17:3e61]) (user=smostafa job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:16ca:b0:493:c1bc:79bd with SMTP id 5b1f17b1804b1-493f88180f9mr114853395e9.20.1784116783668; Wed, 15 Jul 2026 04:59:43 -0700 (PDT) Date: Wed, 15 Jul 2026 11:59:05 +0000 In-Reply-To: <20260715115906.2664882-1-smostafa@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260715115906.2664882-1-smostafa@google.com> X-Mailer: git-send-email 2.55.0.141.g00534a21ce-goog Message-ID: <20260715115906.2664882-25-smostafa@google.com> Subject: [PATCH v7 24/24] KVM: arm64: Add documentation for pKVM DMA isolation From: Mostafa Saleh To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev Cc: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, joro@8bytes.org, jgg@ziepe.ca, mark.rutland@arm.com, qperret@google.com, tabba@google.com, vdonnefort@google.com, sebastianene@google.com, keirf@google.com, Mostafa Saleh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Populate the section for DMA isolation in pKVM with the newly added KVM IOMMU and pKVM SMMUv3 driver details. Signed-off-by: Mostafa Saleh --- Documentation/virt/kvm/arm/pkvm.rst | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/Documentation/virt/kvm/arm/pkvm.rst b/Documentation/virt/kvm/a= rm/pkvm.rst index 514992a79a83..ce67e2ae1f9a 100644 --- a/Documentation/virt/kvm/arm/pkvm.rst +++ b/Documentation/virt/kvm/arm/pkvm.rst @@ -77,7 +77,28 @@ Status: **Unimplemented.** DMA isolation using an IOMMU ---------------------------- =20 -Status: **Unimplemented.** +Status: Supported for devices behind SMMUv3 supporting dual stages +of translation. + +With ``CONFIG_ARM_SMMU_V3_PKVM``, the hypervisor will take over the SMMUs +on the system and provide an architectural emulation to the kernel SMMUv3 +driver. + +If some devices are not behind an IOMMU or behind another IOMMU architectu= re, +DMA isolation is not supported, as a driver must be provided for that. + +DMA isolation is enforced by dual stages of translation; similar to the CPU +where a driver can register their ops through ``kvm_iommu_register_driver`` +and implement ``host_stage2_idmap`` to shadow the CPU page table. + +The page tables for the stage-2 SMMUv3 will be allocated from a pool creat= ed +at boot with size configured from the command line "kvm-arm.iommu_pgt_mem" +The size must cover the worst case scenario for leaf mappings of system me= mory. + +This implementation trusts the system firmware not to allow the untrusted +host kernel to bypass the SMMUv3. +For example, by resetting the power. In that case, it is the firmware's +responsibility to save/restore the SMMUv3 state. =20 Proxying of Trustzone services ------------------------------ --=20 2.55.0.141.g00534a21ce-goog