From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 694DB435AB2 for ; Fri, 7 Aug 2026 16:43:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.69 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121025; cv=none; b=Idir2erubm3BDkE/xctI1lVgcv06GD0BX0oi/UIN0GLSHnWE55bERHfjnpC/y1c3JZmMondH+UPetr/2+TUjIFILK35HNkwzGuCkBdDYxR4Pd9r604/QLfy2bsX0S5xX9XNYkpgu1Npc1shD1+rh3UrDiZ+pYVunJ5e4oweV8Kg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121025; c=relaxed/simple; bh=WoOZLp8mrxOOu0Pc3Lyh8+preRrxVya03MwTvMjg4l4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PJSbsofO5/i0xJFX4XA/lQTzfooa27YEKsW54dh7hCK6dFKp+HkMldt2GNARugRaKWK9e4WdSvakviofTA79z7cXuLl4Lwl4UOjhYC+8gegQ9kZKRoslS/2h0K45R3zK1ozS/8hDEI/+CwgfIsx2RftuBE88iqA8638AypHq9pE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=G/nYVLyv; arc=none smtp.client-ip=209.85.128.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="G/nYVLyv" Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-495474a5fbcso31228405e9.1 for ; Fri, 07 Aug 2026 09:43:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121020; x=1786725820; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=8GYrEvMvff5zvEdtopJDUiIvGuJM4d42FcGqOwQV2a4=; b=G/nYVLyv25Yj9LC/zJr7RQ+34rkCDk/LK2GPXOSoWZt6fUPC1ggXSYDBNccWiOqiGM dB4clBwpcAWAFu1CwZizuRI6K7+4aQHC7E2zvc2ZNZEkGO0Xazo6L4CvcA7beDgbmkfb xfbvF4DiFMC2+Y9+bqIdvmoAy7uSvswkCcWBZJhZd/0Oh3lBapFfeOcBPHqKxzOzWRap 1Vli+Us36bJ1tGVA2+lN3TrXZBVfQpp2QKC1D0eHclyGCedsXUAGbHq6m9CK5ZCPhCjA UQA14ArEmH7+OVR+9u76SxoaGDci1+FZrUqlaxo2PbzBC3C4c6WBC82wkJkKqcXPBM0K EAEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121020; x=1786725820; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=8GYrEvMvff5zvEdtopJDUiIvGuJM4d42FcGqOwQV2a4=; b=YGrdVQjfD/9Nqxu3dKB5v5zi5dMmOAhqnR8+bFrmKNmuUtbBLOjIhPYE5Fl9J/BA2y f/RYxT7osEwA8+hnh4KrlYORt+obdyyuiD6fygKXbT3hTFW3jN3YSsndua66e0iIQcNX ghr2z9wRc3RmDdlEb3zv4yr/37eC96MyLuwfCZ0+ezfkS9zR3rDX4W0DGDEb0p1GGO5t Oc13O/eiFtcSrnW2AMxX/Eo+1AgS42mJTbsSChk+8DbEwSB+lX0kRwwqUKNM9JDUP2P4 EAp/R2p/hGzuLulNnf8VMjiI2LY0gE6/oiWdwkVNtvx8tX6Sn6iEtFum5x24h62c7GfR kT7A== X-Forwarded-Encrypted: i=1; AHgh+RqVtA6XuG+VMUfvjJPBdTE8fLIr0tFwk8vp7eN3wr/vTSMy1vBnlTS9TVw1QP7h2MKYaIQxk1UntSK6tVE=@vger.kernel.org X-Gm-Message-State: AOJu0Yw3CMMq5MxHR9yJHq5j9yzLPKyp40a1qPCUe9r1hCLgduhDvcAS 9bV3hMVbeddPg2VbGJ1zrHGOY3vrHppdmwTe+16rtYvhkssF2NWpWgHz48WBUrN6mjtTNRsqso/ VWpAyooQ0elq7DCO3+TIohAo0d4IS9g== X-Received: from wmbb14.prod.google.com ([2002:a05:600c:588e:b0:495:4a25:69f7]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1d23:b0:499:51cc:4e57 with SMTP id 5b1f17b1804b1-499617f7699mr13301535e9.0.1786121019899; Fri, 07 Aug 2026 09:43:39 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:11 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-3-sebastianene@google.com> Subject: [PATCH v2 01/13] KVM: arm64: Donate MMIO to the hypervisor From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mostafa Saleh Extend the pKVM API to allow the donation of MMIO from the host address space to the hypervisor linear map. Initialize the host s2 page table with an invalid leaf with the owner ID of the hypervisor to prevent the host from mapping the page on faults. Prevent kvm_pgtable_stage2_unmap() from removing owner ID from stage-2 PTEs, as this can be triggered from recycle logic under memory pressure. Signed-off-by: Mostafa Saleh Signed-off-by: Sebastian Ene --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 7 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 137 +++++++++++++++++- arch/arm64/kvm/hyp/pgtable.c | 11 +- 3 files changed, 148 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm= /hyp/include/nvhe/mem_protect.h index 29935c7da1de..6aa83b129e61 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -36,6 +36,13 @@ int __pkvm_guest_share_host(struct pkvm_hyp_vcpu *vcpu, = u64 gfn); int __pkvm_guest_unshare_host(struct pkvm_hyp_vcpu *vcpu, u64 gfn); int __pkvm_host_unshare_hyp(u64 pfn); int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages); +/* + * Donate MMIO range to the hypervisor, it will be mapped in the hyperviso= r's + * linea map and unmapped from the host stage-2. + */ +int __pkvm_host_donate_hyp_mmio(phys_addr_t addr, size_t size); +/* Remaps MMIO range in the host, typically used in error path. */ +int __pkvm_hyp_donate_host_mmio(phys_addr_t addr, size_t size); int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages); int __pkvm_host_share_ffa(u64 pfn, u64 nr_pages); int __pkvm_host_unshare_ffa(u64 pfn, u64 nr_pages); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 4e329e39a695..5cf7c4a0ed20 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -378,7 +378,11 @@ static int host_stage2_unmap_dev_all(void) u64 addr =3D 0; int i, ret; =20 - /* Unmap all non-memory regions to recycle the pages */ + /* + * Unmap all non-memory regions to recycle the pages. + * That relies on kvm_pgtable_stage2_unmap() not clearing + * counted PTEs which include hypervisor MMIO. + */ for (i =3D 0; i < hyp_memblock_nr; i++, addr =3D reg->base + reg->size) { reg =3D &hyp_memory[i]; ret =3D kvm_pgtable_stage2_unmap(pgt, addr, reg->base - addr); @@ -1119,6 +1123,137 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) return ret; } =20 +int __pkvm_host_donate_hyp_mmio(phys_addr_t addr, size_t size) +{ + kvm_pte_t pte; + u64 offset; + void *virt; + int ret; + + /* Only before de-privilege. */ + if (static_branch_unlikely(&kvm_protected_mode_initialized)) + return -EPERM; + + if (!PAGE_ALIGNED(addr | size) || + !pfn_range_is_valid(hyp_phys_to_pfn(addr), size >> PAGE_SHIFT)) + return -EINVAL; + + host_lock_component(); + hyp_lock_component(); + + for (offset =3D 0; offset < size; offset +=3D PAGE_SIZE) { + if (addr_is_memory(addr + offset)) { + ret =3D -EINVAL; + goto err_with_mapping; + } + + ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, addr + offset, &pte, NULL); + if (ret) + goto err_with_mapping; + + if (pte && !kvm_pte_valid(pte)) { + ret =3D -EPERM; + goto err_with_mapping; + } + + virt =3D __hyp_va(addr + offset); + ret =3D kvm_pgtable_get_leaf(&pkvm_pgtable, (u64)virt, &pte, NULL); + if (ret) + goto err_with_mapping; + if (pte) { + ret =3D -EBUSY; + goto err_with_mapping; + } + + ret =3D pkvm_create_mappings_locked(virt, virt + PAGE_SIZE, PAGE_HYP_DEV= ICE); + if (ret) + goto err_with_mapping; + } + + /* + * We set HYP as the owner of the MMIO pages in the host stage-2, for: + * - host aborts: host_stage2_adjust_range() would fail for invalid non z= ero PTEs. + * - recycle under memory pressure: host_stage2_unmap_dev_all() would call + * kvm_pgtable_stage2_unmap() which will not clear non zero invalid pte= s (counted). + * - other MMIO donation: Would fail as we check that the PTE is valid or= empty. + */ + ret =3D host_stage2_try(kvm_pgtable_stage2_annotate, &host_mmu.pgt, + addr, size, &host_s2_pool, + KVM_HOST_INVALID_PTE_TYPE_DONATION, + FIELD_PREP(KVM_HOST_DONATION_PTE_OWNER_MASK, PKVM_ID_HYP)); + if (ret) + goto err_with_mapping; +unlock: + hyp_unlock_component(); + host_unlock_component(); + return ret; +err_with_mapping: + if (!offset) + goto unlock; + + while (offset) { + offset -=3D PAGE_SIZE; + virt =3D __hyp_va(addr + offset); + WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, (u64)virt, PAGE_SIZE) !=3D = PAGE_SIZE); + } + goto unlock; +} + +int __pkvm_hyp_donate_host_mmio(phys_addr_t addr, size_t size) +{ + kvm_pte_t pte; + u64 offset; + int ret =3D 0; + void *virt; + + if (static_branch_unlikely(&kvm_protected_mode_initialized)) + return -EPERM; + + if (!PAGE_ALIGNED(addr | size) || + !pfn_range_is_valid(hyp_phys_to_pfn(addr), size >> PAGE_SHIFT)) + return -EINVAL; + + host_lock_component(); + hyp_lock_component(); + + for (offset =3D 0; offset < size; offset +=3D PAGE_SIZE) { + if (addr_is_memory(addr + offset)) { + ret =3D -EINVAL; + goto err_with_unmap; + } + ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, addr + offset, &pte, NULL); + if (ret) + goto err_with_unmap; + if (!pte || kvm_pte_valid(pte)) { + ret =3D -EINVAL; + goto err_with_unmap; + } + if (FIELD_GET(KVM_HOST_DONATION_PTE_OWNER_MASK, pte) !=3D PKVM_ID_HYP) { + ret =3D -EPERM; + goto err_with_unmap; + } + + virt =3D __hyp_va(addr + offset); + if (kvm_pgtable_hyp_unmap(&pkvm_pgtable, (u64)virt, PAGE_SIZE) !=3D PAGE= _SIZE) + goto err_with_unmap; + } + WARN_ON(host_stage2_idmap_locked(addr, size, PKVM_HOST_MMIO_PROT)); +unlock: + hyp_unlock_component(); + host_unlock_component(); + return ret; +err_with_unmap: + if (!offset) + goto unlock; + + while (offset) { + offset -=3D PAGE_SIZE; + virt =3D __hyp_va(addr + offset); + WARN_ON(pkvm_create_mappings_locked(virt, virt + PAGE_SIZE, PAGE_HYP_DEV= ICE)); + } + goto unlock; +} + int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) { u64 phys =3D hyp_pfn_to_phys(pfn); diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b74dd5ce1efd..7638213bd893 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1161,13 +1161,12 @@ static int stage2_unmap_walker(const struct kvm_pgt= able_visit_ctx *ctx, kvm_pte_t *childp =3D NULL; bool need_flush =3D false; =20 - if (!kvm_pte_valid(ctx->old)) { - if (stage2_pte_is_counted(ctx->old)) { - kvm_clear_pte(ctx->ptep); - mm_ops->put_page(ctx->ptep); - } + /* + * That also ignores stage2_pte_is_counted() instead of clearing + * the PTE as the MMIO can be owned by the hypervisor. + */ + if (!kvm_pte_valid(ctx->old)) return 0; - } =20 if (kvm_pte_table(ctx->old, ctx->level)) { childp =3D kvm_pte_follow(ctx->old, mm_ops); --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC48C3BCD38 for ; Fri, 7 Aug 2026 16:43:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.69 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121025; cv=none; b=VfBPjV5sWJI0pfyQ5GxI/prEoe1Aan6/ONVk/rHP2jFmZEMHcJZzJyln1sZMSvvfFYVkVPQ/fS70lsF8Nw7GTJ2RlDgniC5L4lxlpgpu4dMB1fGZZlz7NXCXZMIiHV6OvtEU9i1FnHebwbcHAIWv7QRZh2fI3+P7/vxrg6trTR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121025; c=relaxed/simple; bh=fW4c7LYnBzLD1FiWd4akwgLLfkkoP/cLS3UvI0SBd6c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OyUn9HEWsylXOxuXZx28dCxrJD8d00hRBY48YlmZ11p1jN9qlqnrHnVlON6UwUMYwS1LTxPnc3MF1Wd2bxYoHtRK+qscjL2LERQswV1u9QZcQMkQCg+GyYfltrLb2HpY5eL+v1WzpAGdC2TU3Xva26OwZcBCpC/sudNC3zRHpGQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jokGr9Up; arc=none smtp.client-ip=209.85.128.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jokGr9Up" Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-49953abe51fso16094785e9.1 for ; Fri, 07 Aug 2026 09:43:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121021; x=1786725821; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=9LMvLSIyRJoS17MqrfOagB9BtwOQBhza88CokriU1a0=; b=jokGr9Up7dToVH5qX52cFxXfkk2YxCZS0/9PN+mNep7Z8Y2WVoul2FTwTGY7NHGzOW 1SseYfc1haoRlrIkVX4Cwp38+nXHz1QtAApt0Y7WQrQSENoJpuSAA7mzkfwzcY6MAsG7 pQq2jiC74PPdHGpLKQPwjiCACU+PB7ckw4cM5eTqGQnRzlsLfwEPm2k3ZlvIe0PzmWeq t7e96QCJi5sXV84pkO96ZBqifvk2MLVcfMl5K0ShYZjvwsOGuhwbrA91P+YKx65zgfvd 8ubGFEnDBkj0ucmQZp4EV4pqny2+VHFvvIE61s9hWgaiEdu1a+zdO8p47v5MczZlLx0n xT5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121021; x=1786725821; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=9LMvLSIyRJoS17MqrfOagB9BtwOQBhza88CokriU1a0=; b=j2Y85CbEkeL5r2fXAYgbRBFIaEEitsJbgdT6qQnK4Y2dN1p7c2QNnRZIULMLt7PFO9 1Iyzs+fRv742fvYV+A4S8Zv/xZ3dyREHYvbzeeysJudHmXfk/n4zz7cau94BBAkGld8b K2IwcKv1WXeJkQdRveNvfYJFLhmxWEDyz/nKRV5HRxxQR2hA7akW+E1SY9Cpoi+mKSIR MTTFnKp1Jf7mfW9XRy0/GbMQ52KvldRvD7aBQI8H+HVAIVdhwlUuqFnJzwFY+Pv/l67i qoyUbBLnaEyKo/B/7obGgGyAxkFUiFtAN08Ix5yrlKIWdg9HvYn+W3zplbh955jvt8Yl RIsA== X-Forwarded-Encrypted: i=1; AHgh+Rpc0js1BEuWIR6OegL7MdCt+cnCN/rctNhtH8LZPl1NiYYeBqgRyBmHBtjtrD04WjqR1CO6D1d8wmeN9Gg=@vger.kernel.org X-Gm-Message-State: AOJu0Yx40C4X8Km/XJ0ymyoedljXEAu77LTUTUpYTZYeSx8eZxoRaq8o OeTMYY7Hawii7XddzTMfM698KkBL3t8wYbsHW6Tj8VKI8+kf5qGGtar9wRu1LIVAR5GR1zpnUZx 3BNGBOcBk2WF6sb0K11IvrqVvwY+trg== X-Received: from wrrq16.prod.google.com ([2002:adf:f950:0:b0:47f:8ed4:b07f]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e12:b0:499:61bb:6c78 with SMTP id 5b1f17b1804b1-49961bb71abmr15660795e9.14.1786121021370; Fri, 07 Aug 2026 09:43:41 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:12 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-4-sebastianene@google.com> Subject: [PATCH v2 02/13] KVM: arm64: Track host-unmapped MMIO regions in a static array From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce a registry to track protected MMIO regions that are unmapped from the host stage-2 page tables. These regions are stored in a fixed-size array and their ownership is donated to the hypervisor during initialization to ensure host-exclusion and persistent tracking. Signed-off-by: Sebastian Ene --- arch/arm64/include/asm/kvm_pkvm.h | 11 +++++++++++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 3 +++ arch/arm64/kvm/hyp/nvhe/setup.c | 24 ++++++++++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index 74fedd9c5ff0..ab26bec079d6 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -17,6 +17,17 @@ =20 #define HYP_MEMBLOCK_REGIONS 128 =20 +/* The maximum number of hypervisor protected regions from the host */ +#define PKVM_PROTECTED_REGS_NUM 8 + +struct pkvm_protected_reg { + u64 pfn; + u64 nr_pages; +}; + +extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[]; +extern unsigned int kvm_nvhe_sym(num_protected_reg); + int pkvm_init_host_vm(struct kvm *kvm, unsigned long type); int pkvm_create_hyp_vm(struct kvm *kvm); bool pkvm_hyp_vm_is_created(struct kvm *kvm); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 5cf7c4a0ed20..500c18c2fd48 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -28,6 +28,9 @@ struct host_mmu host_mmu; =20 static struct hyp_pool host_s2_pool; =20 +unsigned int num_protected_reg; +struct pkvm_protected_reg pkvm_protected_regs[PKVM_PROTECTED_REGS_NUM]; + static DEFINE_PER_CPU(struct pkvm_hyp_vm *, __current_vm); #define current_vm (*this_cpu_ptr(&__current_vm)) =20 diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index 75b00c323310..64c0290da888 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -284,6 +284,26 @@ static int fix_hyp_pgtable_refcnt(void) &walker); } =20 +static int donate_protected_mmio_regions(void) +{ + int ret; + int i; + + for (i =3D 0; i < num_protected_reg; i++) { + ret =3D __pkvm_host_donate_hyp_mmio(hyp_pfn_to_phys(pkvm_protected_regs[= i].pfn), + pkvm_protected_regs[i].nr_pages << PAGE_SHIFT); + if (ret) + goto err_setup; + } + + return 0; +err_setup: + while (--i >=3D 0) + __pkvm_hyp_donate_host_mmio(hyp_pfn_to_phys(pkvm_protected_regs[i].pfn), + pkvm_protected_regs[i].nr_pages << PAGE_SHIFT); + return ret; +} + void __noreturn __pkvm_init_finalise(void) { struct kvm_cpu_context *host_ctxt =3D host_data_ptr(host_ctxt); @@ -324,6 +344,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; =20 + ret =3D donate_protected_mmio_regions(); + if (ret) + goto out; + ret =3D hyp_ffa_init(ffa_proxy_pages); if (ret) goto out; --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89A264746B7 for ; Fri, 7 Aug 2026 16:43:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.71 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121026; cv=none; b=BAXnQCS0bq7wGt9j+Xs2k4sLW5G7MSRDRLNt3WbHbz5f7FMs8yyB1Xeaaak5yYAVMMrE6ALDCbOtZsY23u8VOfIABwGlRk0m8P92C6dWej5nBYcrEiufbP0S5QJvwcAVx5XyWVEpvjZbrSDw+Ob/HohmOJw0uOlWj2WW59IxYKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121026; c=relaxed/simple; bh=/+oBPRPor12nAw5j501NP9Ed7vddxDnJfcm2D2u6JbQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GpfxZpavWSr7nrcHdxpOfSS2naGlpdSOIjUtXiyO7FSU5xpebWUXv+QFH7pJnHxz+HVQbo/0pR3yZlL3HyDkGJOyrQkP1/eJIfxweNcwO1Vf0xSGWn8pv82+osYlDaX66tZl18ZyL4g1M528kMFBzA1VoGNcyod7/pkNY5xckxs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wZpKwYCN; arc=none smtp.client-ip=209.85.128.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wZpKwYCN" Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4955fd77c18so20719205e9.2 for ; Fri, 07 Aug 2026 09:43:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121023; x=1786725823; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:from:to:cc :subject:date:message-id:reply-to:content-type; bh=DRnRzIglH4bre6fymvnhTQIWpGXgBbzyHJfeoNchwKg=; b=wZpKwYCNSPjspAKsdx43bgj3qQsKB4uHevb6bGYRv7P6S8lzb1l7rhVSwzpTngVIb3 RtvMqoWvZKi4CzJGYANZzmyvTUiX6yY68/e1475xl8olcnGqOxu2g8+CWW9vo6/pfcd0 sJUt6ut97pdULwJ5GIR89poDIJbdcdRIfeWv4bkkv/1Z+/dq2xRUmrazP2fhp1arcjQH ro/URIxyJTl6D0XC8nFlbMLD2i2zOtyu0aCx9z0MkMNqwarYFwmb0aNlanqy/w1wpnG4 JJzZJt5tYTSr/QmDuP8m7i+Gu+oOwszZqE10y2fLjT3xf5V9CTBqPgGxbwnKXDwCGPcw 3BBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121023; x=1786725823; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=DRnRzIglH4bre6fymvnhTQIWpGXgBbzyHJfeoNchwKg=; b=dbat1CtaAT5rzEWmlaKxBeAoc4K8EOSFfd0rJPA0E673e0bznHMNLJla4uuQbNcdrt loU6Wlhw7kxoHG/PlIeZq6HaVGDi8KhlLLMurj9wC7eWxSybdteP5cXR0PldMVvayPYZ X7nLaCLvNlpafD4gyrxIMyUlwG9FX7sH2qXzvzenCeCtZIqa8+PwRJJyiwIn2MdAYSEJ ZT5ORfPbbq5x1baJkolySsmm8nYANn/u1n8L8kVAnsIvz9X6KEc4qb1hARGLrK3UoFDm 7PRiovxnCtd4t92sKMKxq84sIRAIJJ6a+N8IokgRO4k90E71rAbbAYxAswe2u6NGUFq3 Vy2A== X-Forwarded-Encrypted: i=1; AHgh+RoQo0UrT+zpzSk08vcjIKQF3F/DLm4L2tXOTVG82ioYvgzyTEbNUBeB6yIH4x2AZuqlI+GRW2lqc5DC9Xo=@vger.kernel.org X-Gm-Message-State: AOJu0YwX74BZPSD7GQNlPyjsREJY/EQn4NnrQn2zXrNlA/H5b7iLi2Mr MrD0+c+1F/0LntfXnhE7KoHAj1LgSOQxLT5BgryhF2dz1z+n9/mquQUIje30hPUY4Qu5O9zPaoI R3OQRicrsj/uZnrocb1SPXrcVOX/cog== X-Received: from wmbhu19.prod.google.com ([2002:a05:600c:a293:b0:493:f83f:e304]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c297:b0:494:596e:e8c4 with SMTP id 5b1f17b1804b1-4994e7d3395mr238197545e9.17.1786121022494; Fri, 07 Aug 2026 09:43:42 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:13 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-5-sebastianene@google.com> Subject: [PATCH v2 03/13] KVM: arm64: Support host MMIO trap handlers for unmapped devices From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hook a handler to the host mem abort so that the hypervisor can intercept host accesses to unmapped memory regions. When a Stage-2 fault occurs on a registered device region, the hypervisor will look if there is any registered function that can handle the access. On the back of this, mediate host accesses to devices and emulate them in pKVM. Signed-off-by: Sebastian Ene Signed-off-by: Bart=C5=82omiej Grzesik --- arch/arm64/include/asm/kvm_arm.h | 2 ++ arch/arm64/include/asm/kvm_pkvm.h | 4 +++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 49 +++++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/setup.c | 3 ++ 4 files changed, 58 insertions(+) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_= arm.h index 3f9233b5a130..6360c90f9855 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -304,6 +304,8 @@ =20 /* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */ #define HPFAR_MASK (~UL(0xf)) +#define FAR_MASK GENMASK_ULL(11, 0) + /* * We have * PAR [PA_Shift - 1 : 12] =3D PA [PA_Shift - 1 : 12] diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index ab26bec079d6..0a471564be00 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -20,9 +20,13 @@ /* The maximum number of hypervisor protected regions from the host */ #define PKVM_PROTECTED_REGS_NUM 8 =20 +struct pkvm_protected_reg; +typedef void(pkvm_emulate_handler)(struct pkvm_protected_reg *region, u64 = offset, + bool write, u64 *reg, u8 reg_size); struct pkvm_protected_reg { u64 pfn; u64 nr_pages; + pkvm_emulate_handler *cb; }; =20 extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[]; diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 500c18c2fd48..7e978e0c44b9 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -14,6 +14,7 @@ #include =20 #include +#include =20 #include #include @@ -752,6 +753,50 @@ static void host_inject_mem_abort(struct kvm_cpu_conte= xt *host_ctxt) inject_host_exception(esr); } =20 +static bool handle_host_mmio_trap(struct kvm_cpu_context *host_ctxt, u64 e= sr, u64 addr) +{ + u64 offset, reg_value =3D 0, start, end; + u8 reg_size, reg_index; + bool write; + int i; + + for (i =3D 0; i < num_protected_reg; i++) { + if (!pkvm_protected_regs[i].pfn || !pkvm_protected_regs[i].nr_pages || + !pkvm_protected_regs[i].cb) + continue; + + start =3D PFN_PHYS(pkvm_protected_regs[i].pfn); + end =3D start + PFN_PHYS(pkvm_protected_regs[i].nr_pages); + reg_size =3D BIT((esr & ESR_ELx_SAS) >> ESR_ELx_SAS_SHIFT); + + if (start > addr || addr + reg_size > end) + continue; + + reg_index =3D (esr & ESR_ELx_SRT_MASK) >> ESR_ELx_SRT_SHIFT; + write =3D (esr & ESR_ELx_WNR) =3D=3D ESR_ELx_WNR; + offset =3D addr - start; + + if (write && reg_index !=3D 31) + reg_value =3D host_ctxt->regs.regs[reg_index]; + + pkvm_protected_regs[i].cb(&pkvm_protected_regs[i], offset, write, + ®_value, reg_size); + + if (!write && reg_index !=3D 31) + host_ctxt->regs.regs[reg_index] =3D reg_value; + + kvm_skip_host_instr(); + return true; + } + + return false; +} + +static bool is_dabt(u64 esr) +{ + return (ESR_ELx_EC(esr) =3D=3D ESR_ELx_EC_DABT_LOW) && (esr & ESR_ELx_ISV= ); +} + void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; @@ -774,6 +819,10 @@ void handle_host_mem_abort(struct kvm_cpu_context *hos= t_ctxt) BUG_ON(!(fault.hpfar_el2 & HPFAR_EL2_NS)); addr =3D FIELD_GET(HPFAR_EL2_FIPA, fault.hpfar_el2) << 12; =20 + if (is_dabt(esr) && !addr_is_memory(addr) && + handle_host_mmio_trap(host_ctxt, esr, addr | (fault.far_el2 & FAR_MAS= K))) + return; + switch (host_stage2_idmap(addr)) { case -EPERM: host_inject_mem_abort(host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index 64c0290da888..4395595b7f7e 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -294,6 +294,9 @@ static int donate_protected_mmio_regions(void) pkvm_protected_regs[i].nr_pages << PAGE_SHIFT); if (ret) goto err_setup; + + if (pkvm_protected_regs[i].cb) + pkvm_protected_regs[i].cb =3D kern_hyp_va(pkvm_protected_regs[i].cb); } =20 return 0; --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B69304756B3 for ; Fri, 7 Aug 2026 16:43:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.69 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121028; cv=none; b=Pj2n7z9L2BYNpyqYk9AnKaz5m/u9qGF1kC2la/YJjRUA8PUlgZn8Cx3gL/W3SfPP1FpZPdLD+fxag4We5ZCG/muL9NcXCmJtALdIgPVTlZHnrkl2oigf+o/bc8y6D68rcmPVMHXtNKsWbwO6V4/dJV4ILU0uaBoHdNYdqeJU90E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121028; c=relaxed/simple; bh=XILxJKy12TWaQHFCLBTQsYLQR5jF/oISDfaILeGY1YQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=k4GQaww5MQ728t3qTcPjhTsc6H2NUCWNsiwtM3i4chYe+6Vx0eO/0jFDEa1jnm6bePDsLJZRXf6mDTCy0WhPQBsQ58U49uGI7WYdORtMecPMzrDxGVW9HFioiWxsg80kOISgt4pry+L3jgELrSGgTfL9s4ww+5fgjIldUIv/L5o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iZc/RzGw; arc=none smtp.client-ip=209.85.218.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iZc/RzGw" Received: by mail-ej1-f69.google.com with SMTP id a640c23a62f3a-c15c32294e1so317916366b.3 for ; Fri, 07 Aug 2026 09:43:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121024; x=1786725824; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ubeHaXbMbCOzTTinYD2VelpFhUrhVALfoKhm+KzrKis=; b=iZc/RzGwO6KwpzgnoUFYEVtj2wmkrKaJ16nJ55h3ikVnfq24GWCJkyaIi6m/tX+plg wA4a9ghAfDv0iw1SVj+GOEcvRkJPOrN1Hv5WrmONZTOzbPCwoypBLLqXRK4+sDpYpG7o FpCw7hVomgX9pC3W9cj/4t2TCxaSLeFsq8BnhDKI7Cdi1VjZA9fFy/3UsGH6zaHywcdG jXI76wO6WBCmFMSrjyu9ABDtLEpS3RQsQ3XpTbckiKeTl9J8n6tbIbRre1QgFub7u8Ih /NqLRMb+bm63dJzeGcTCPcCjmcHvnbHDuPpkm6xUXgQEmPb2JKvpyVemj5Wsr1XYAwjS LbXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121024; x=1786725824; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ubeHaXbMbCOzTTinYD2VelpFhUrhVALfoKhm+KzrKis=; b=iMvrfRvX/CDpjR9TatadrJkZqeY2N5yfaO4BBnzTRh5HUYcHejmNdNTiM1S/8E1Jyc lRmYqro/zCeFqnAjwPvYp2/8uaAi0/xkUDBCg4+Ni7g1M8DX23ezho30kxxTuhB731bd gSYxKSnG3w5D7HLkxGCVR+fxnmyQov7VOQr24pg6nVyOOvnFa23sbF2cOxQhWS7upn3F quwy8TBLHI/zdVsFZWHfzn3I2E9ZjqtH5jpayWSeFQ4X2Epx24MN1BcT5Z90VrJ4Dlxj PNncSZe1+lsQz7W7HVU/JnefW4S/Hfpp0a1R/i7ENAJIUJesybG5g7Jyc552JOd6XxO3 5khg== X-Forwarded-Encrypted: i=1; AHgh+RppXoEc7dMusb/4kmoL0b+qDB3pEVOF+O5oW3gsLZ/N364e+XRHbXcHbqOR61tS96vxAkKprZXrQ8qC7Eo=@vger.kernel.org X-Gm-Message-State: AOJu0YxihNj2hGOioXqIKC7+hO8uKb9dUypmUMKLau1j/73m60PIZo2k EOtsD+ZfthZAW9Ecrsr57p/1IK0DSCKvkZRXCX/PoQbJZ2DvNV09Q69+spvipATch9TgHdhTwPC o+hFlmdRU9Gbpd1YXNnX079UVw1pfqg== X-Received: from ejfv7.prod.google.com ([2002:a17:906:3bc7:b0:c12:8ea1:b062]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:1b24:b0:c12:8b1c:454f with SMTP id a640c23a62f3a-c2039c0264bmr1298937866b.2.1786121023739; Fri, 07 Aug 2026 09:43:43 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:14 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-6-sebastianene@google.com> Subject: [PATCH v2 04/13] KVM: Parse the device tree and register the ITS region with pKVM From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Identify the ITS base address from the device tree and store it in the pkvm_protected_regs array so that it will be unmapped from the host address space. Register a callback to forward all the MMIO requests to the device to prevent breaking ITS functionality in this patch. The patch by itself shouldn't break any existing functionality even though all the accesses from the gic-ITS driver are now mediated inside pKVM. Signed-off-by: Sebastian Ene --- arch/arm64/include/asm/kvm_pkvm.h | 2 ++ arch/arm64/kvm/hyp/nvhe/Makefile | 3 +- arch/arm64/kvm/hyp/nvhe/its_emulate.c | 37 +++++++++++++++++++ arch/arm64/kvm/pkvm.c | 52 +++++++++++++++++++++++++++ 4 files changed, 93 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/its_emulate.c diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index 0a471564be00..370225f0e72c 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -31,6 +31,8 @@ struct pkvm_protected_reg { =20 extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[]; extern unsigned int kvm_nvhe_sym(num_protected_reg); +extern void kvm_nvhe_sym(its_emulate_forward_req)(struct pkvm_protected_re= g *region, u64 offset, + bool write, u64 *reg, u8 reg_size); =20 int pkvm_init_host_vm(struct kvm *kvm, unsigned long type); int pkvm_create_hyp_vm(struct kvm *kvm); diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Mak= efile index f57450ebcb49..70fbca325852 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -24,7 +24,8 @@ CFLAGS_switch.nvhe.o +=3D -Wno-override-init =20 hyp-obj-y :=3D timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o= host.o \ hyp-main.o hyp-smp.o psci-relay.o early_alloc.o page_alloc.o \ - cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o + cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o \ + its_emulate.o hyp-obj-y +=3D ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../en= try.o \ ../hyp-entry.o ../exception.o ../pgtable.o ../vgic-v5-sr.o hyp-obj-y +=3D ../../../kernel/smccc-call.o diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c new file mode 100644 index 000000000000..63a42f520ed2 --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include +#include + +void its_emulate_forward_req(struct pkvm_protected_reg *region, u64 offset= , bool write, u64 *reg, + u8 reg_size) +{ + void __iomem *addr =3D __hyp_va(PFN_PHYS(region->pfn) + offset); + + switch (reg_size) { + case 1: + if (!write) + *reg =3D readb_relaxed(addr); + else + writeb_relaxed(*reg, addr); + break; + case 2: + if (!write) + *reg =3D readw_relaxed(addr); + else + writew_relaxed(*reg, addr); + break; + case 4: + if (!write) + *reg =3D readl_relaxed(addr); + else + writel_relaxed(*reg, addr); + break; + case 8: + if (!write) + *reg =3D readq_relaxed(addr); + else + writeq_relaxed(*reg, addr); + break; + } +} diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c index 428723b1b0f5..4bfffbedac4c 100644 --- a/arch/arm64/kvm/pkvm.c +++ b/arch/arm64/kvm/pkvm.c @@ -9,8 +9,13 @@ #include #include #include +#include #include #include +#include +#include +#include +#include =20 #include =20 @@ -39,6 +44,47 @@ static int __init register_memblock_regions(void) return 0; } =20 +static int __init register_its_emulated_region(void) +{ + struct device_node *np; + struct resource res; + int i =3D 0; + int ret; + + for_each_compatible_node(np, NULL, "arm,gic-v3-its") { + ret =3D of_address_to_resource(np, 0, &res); + if (ret) + goto out_fail; + + if (i >=3D PKVM_PROTECTED_REGS_NUM) { + kvm_err("Out of protected region slots\n"); + ret =3D -ENOSPC; + goto out_fail; + } + + /* + * Note: don't unmap the entire animal from the host because devices need + * to be able to access GITS_TRANSLATER to raise MSIs. If the + * page where GITS_TRANSLATER is given to HYP, devices won't be + * able to map it in their IOMMU when the IOMMU is managed by + * pKVM. + */ + kvm_nvhe_sym(pkvm_protected_regs)[i].pfn =3D PHYS_PFN(res.start); + kvm_nvhe_sym(pkvm_protected_regs)[i].cb =3D + lm_alias(&kvm_nvhe_sym(its_emulate_forward_req)); + kvm_nvhe_sym(pkvm_protected_regs)[i].nr_pages =3D + PFN_DOWN(min_t(u64, resource_size(&res), PAGE_ALIGN_DOWN(GITS_TRANSLATE= R))); + + i++; + } + + kvm_nvhe_sym(num_protected_reg) =3D i; + return 0; +out_fail: + of_node_put(np); + return ret; +} + void __init kvm_hyp_reserve(void) { u64 hyp_mem_pages =3D 0; @@ -57,6 +103,12 @@ void __init kvm_hyp_reserve(void) return; } =20 + ret =3D register_its_emulated_region(); + if (ret) { + kvm_err("Failed to register ITS region %d\n", ret); + return; + } + hyp_mem_pages +=3D hyp_s1_pgtable_pages(); hyp_mem_pages +=3D host_s2_pgtable_pages(); hyp_mem_pages +=3D hyp_vm_table_pages(); --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05A164766B4 for ; Fri, 7 Aug 2026 16:43:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121029; cv=none; b=RjCxGUpaMbFowyTweHF7Al6RSOBDdev1QYIGCLQeA1/dZxq+0aV17nSDfgfGSyGcknZWb2XLjwZvKlapAbFI8gTC6RIjN3vdsiRMPvV0uRy00/DGg3iy8Grz6v6DczCzDjknwjcEI6bHbxJ/C9haZEk1NB/uWAlxo8Lfbb+dH08= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121029; c=relaxed/simple; bh=yGBjsaJ9hgV8fJYzvxdhdFeS0ETWRMPWiiqS4nncA1U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pYCYEUlaMgzd8j1P5PLkMRXmJmsagOMFQ1xTtCY9dJZIB7B2T1BkqJ7fZo6l5MBKO89cLGxSwYTvUc0UMpUAYtZgQpKVcbj3zwCHnV1hCtcHmrc0SDZYn8QMn2957Czv4re7N5p6hknNy3C9AQy4vdeKRG6EqONeCUJm9JeBsZ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dTRFTpt9; arc=none smtp.client-ip=209.85.128.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dTRFTpt9" Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-4955fd77c18so20719345e9.2 for ; Fri, 07 Aug 2026 09:43:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121025; x=1786725825; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:from:to:cc :subject:date:message-id:reply-to:content-type; bh=iZntm8VORvDjKObm+3e5kg7isnusBSNzR7FkdFCW7YM=; b=dTRFTpt92FVFrvuYD1dMUxzVJLnD6Q1lSP9YwuP4vXS2zeR3RkEJmBSC+t2psUaw/i LNCF6iFgpm8r4zp5eYRfwySe6FDzwt4+plGWoXj/5yjOkhH7g26rMLpdGRyv/kTvg72F Gp3TA0QQkUinW5RaihceA7shvtz9ZOoO8+JhnS7vwV1mCPbKYc0S8wI4dWCQnCPM1myE WH7PytH1EqTYGad61hAVnIAF27QT1sxL216Lxg/HZXJjVDiS3Vl/I0IwYXC5H7a/OnD6 /+NRkfnJBkpK9l7AsHiAsunn7klmJFX5H5QN4eTT3u0hPUXxaTKeLZah5EcEO6Ga33fw s5Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121025; x=1786725825; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=iZntm8VORvDjKObm+3e5kg7isnusBSNzR7FkdFCW7YM=; b=PhIucbWKL20xWJizc0ebdAk0siSkOlicuYHY4uP52JWsAmuKz6ogFdhTVAwrRESRL0 K1Za7q6kouOEmmd+yyDO73t9q9lzmpB7SmXOUDnzKX/xQWrkcbOzVtZs5Nykty3Qa3Xt ojTapRv6/Xu61lvqls58MjDKdwGPNqhI/4DqTyGE8i8senp8RwraoFf1LNonoduMWDdb K0W9kzZqLtflPQuIVCYv6snyOoTq/DGdo+/KM+uLtnp4eMQFeromBUQUW0w1O86HoEhE Ks/bEGfICOWVGVeBn9/kPBNJs/0Fzg6+p9cXKc0vmKP/ndQQYKBcqNWjn+AoYDd0aePR 0JDw== X-Forwarded-Encrypted: i=1; AHgh+Rq7Iw2u7XUB7SuJavNtdGy+LIj2N3Jp2jBNOeSIN2lb1skyXHQoTsT7Yj7HeN1jB8JhjgHVL47MMsNBbVw=@vger.kernel.org X-Gm-Message-State: AOJu0YxQjBAtBUMWvUy3ZkzS//41adkvWswGPv20Lp/tdbQJqvwTjX0D hr3TOq76foQSdZnC24bkCVuTYZvTO7MP9fxK5ySifliByL9st8g50r9lrGTjkVbn2xLDPkLLZPd ueYc21cCiNwcyTl0J/Uni6u9Xw/WcvA== X-Received: from wmsk23-n2.prod.google.com ([2002:a05:600d:8497:20b0:493:c773:ff79]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4ed3:b0:493:e983:806e with SMTP id 5b1f17b1804b1-4994e72f795mr337802465e9.3.1786121025032; Fri, 07 Aug 2026 09:43:45 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:15 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-7-sebastianene@google.com> Subject: [PATCH v2 05/13] irqchip/gic-v3-its: Add support for the ITS emulation setup From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce two new helper functions to allow locking the ITS and setting up a copy of the host ITS state that will be given to the pKVM emulation. The caller of these functions is responsible to implement a callback which will be used to setup the emulation layer. The calling flow is expected to do the following: pkvm_its_emulate_setup(its_phys, host) // allocate memory for the priv state of the ITS emulation // call the its emulation setup(its_phys, host, priv_state); pkvm_drop_host_privileges() its_emulate_acquire_locks(&flags); on_each_cpu(_kvm_host_prot_finalize, &ret, 1); its_emulate_release_locks(ret, &flags, pkvm_its_emulate_setup); Augment the its_baser structure with a new fiels that will hold a pointer to the base table copy. The gic ITS driver will use the pointer to the base table copy when emulation is enabled, as this allows us to hide away the original first level of an indirect table to prevent the following: // assumming an indirect Device Table layout 1. malicious host patches an entry in the 1st level table with an address that it wants to write to. 2. malicious host issues MAPD to install a DTE in the table pointed by the address from (1). As the driver only manipulates a copy of the table, the emulation is responsible for looking at the updates from the copy table, sanitizing them and updating the original table before talking to the hardware. In a simillar fashion, when emulation is in place we no longer let the gic ITS driver use the original command queue but we present the driver a copy of it and we hide away the original command queue from the driver as this will be used entirely by the emulation layer. Co-authored-by: Bart=C5=82omiej Grzesik Signed-off-by: Sebastian Ene --- drivers/irqchip/irq-gic-v3-its.c | 157 +++++++++++++++++++++++++++-- include/linux/irqchip/arm-gic-v3.h | 39 +++++++ 2 files changed, 185 insertions(+), 11 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index 6f5811aae59c..e74ae9220af5 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -78,17 +78,6 @@ struct its_collection { u16 col_id; }; =20 -/* - * The ITS_BASER structure - contains memory information, cached - * value of BASER register configuration and ITS page size. - */ -struct its_baser { - void *base; - u64 val; - u32 order; - u32 psz; -}; - struct its_device; =20 /* @@ -5226,6 +5215,152 @@ static int __init its_compute_its_list_map(struct i= ts_node *its) return its_number; } =20 +static void its_free_snapshot(struct its_host_state *snapshot) +{ + int i; + + if (snapshot->cmd_host_copy) + its_free_pages(snapshot->cmd_host_copy, get_order(ITS_CMD_QUEUE_SZ)); + + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { + if (!snapshot->tables[i].base_snapshot) + continue; + + its_free_pages(snapshot->tables[i].base_snapshot, snapshot->tables[i].or= der); + } + + its_free_pages(snapshot, 0); +} + +static struct its_host_state *its_snapshot_host_state(struct its_node *its) +{ + void *page; + struct its_host_state *snapshot; + int i; + + page =3D its_alloc_pages_node(its->numa_node, GFP_ATOMIC | __GFP_ZERO, 0); + if (!page) + return NULL; + + snapshot =3D (void *)page_address(page); + page =3D its_alloc_pages_node(its->numa_node, GFP_ATOMIC | __GFP_ZERO, + get_order(ITS_CMD_QUEUE_SZ)); + if (!page) + goto err_alloc; + + snapshot->cmd_host_copy =3D page_address(page); + snapshot->cmdq_len =3D ITS_CMD_QUEUE_SZ; + snapshot->cmd_original =3D its->cmd_base; + snapshot->cmd_write =3D its->cmd_write; + + memcpy(snapshot->tables, its->tables, sizeof(struct its_baser) * GITS_BAS= ER_NR_REGS); + + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { + if (!(snapshot->tables[i].val & GITS_BASER_VALID)) + continue; + + if (!(snapshot->tables[i].val & GITS_BASER_INDIRECT)) + continue; + + page =3D its_alloc_pages_node(its->numa_node, + GFP_ATOMIC | __GFP_ZERO, + snapshot->tables[i].order); + if (!page) + goto err_alloc; + + snapshot->tables[i].base_snapshot =3D page_address(page); + + memcpy(snapshot->tables[i].base_snapshot, snapshot->tables[i].base, + PAGE_ORDER_TO_SIZE(snapshot->tables[i].order)); + } + + return snapshot; + +err_alloc: + its_free_snapshot(snapshot); + return NULL; +} + +static int its_emulate_switch_queues_locked(struct its_node *its, its_emul= ate_setup cb) +{ + struct its_host_state *host_snaphsot, host; + int i, ret; + u64 baser_phys; + + host_snaphsot =3D its_snapshot_host_state(its); + if (!host_snaphsot) + return -ENOMEM; + + /* + * The snapshot of the ITS state will be given to the emulation, make a c= opy of it + * so that we don't go in weeds. + */ + memcpy(&host, host_snaphsot, sizeof(host)); + + ret =3D cb(its->phys_base, host_snaphsot); + if (ret) { + its_free_snapshot(host_snaphsot); + return ret; + } + + /* Switch the driver command queue to use the host copy and update the wr= ite index */ + its->cmd_write =3D (its->cmd_write - its->cmd_base) + + (struct its_cmd_block *)host.cmd_host_copy; + its->cmd_base =3D host.cmd_host_copy; + + /* + * Replace the first level of the indirect tables with the snapshot table= as the + * emulation layer will make it innaccessible to the host. + */ + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { + if (!(host.tables[i].val & GITS_BASER_INDIRECT)) + continue; + + baser_phys =3D virt_to_phys(host.tables[i].base_snapshot); + if (IS_ENABLED(CONFIG_ARM64_64K_PAGES) && (baser_phys >> 48)) + baser_phys =3D GITS_BASER_PHYS_52_to_48(baser_phys); + + its->tables[i].val &=3D ~GENMASK(47, 12); + its->tables[i].val |=3D baser_phys; + its->tables[i].base =3D host.tables[i].base_snapshot; + } + + return 0; +} + +void its_emulate_acquire_locks(unsigned long *flags) +{ + struct its_node *its; + + if (WARN_ON(!flags)) + return; + + raw_spin_lock_irqsave(&its_lock, *flags); + + list_for_each_entry(its, &its_nodes, entry) + raw_spin_lock(&its->lock); +} + +int its_emulate_release_locks(int ret_pkvm_finalize, unsigned long *flags,= its_emulate_setup cb) +{ + struct its_node *its; + int ret =3D 0; + + if (WARN_ON(!flags || !cb)) + ret =3D -EINVAL; + + list_for_each_entry(its, &its_nodes, entry) { + if (!ret_pkvm_finalize && !ret) + ret =3D its_emulate_switch_queues_locked(its, cb); + + raw_spin_unlock(&its->lock); + } + + raw_spin_unlock_irqrestore(&its_lock, *flags); + + return ret; +} + static int __init its_probe_one(struct its_node *its) { u64 baser, tmp; diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm= -gic-v3.h index ea5fd2374ebe..b75f82cef4bf 100644 --- a/include/linux/irqchip/arm-gic-v3.h +++ b/include/linux/irqchip/arm-gic-v3.h @@ -657,6 +657,45 @@ static inline bool gic_enable_sre(void) return !!(val & ICC_SRE_EL1_SRE); } =20 +/* + * The ITS_BASER structure - contains memory information, cached + * value of BASER register configuration and ITS page size. + */ +struct its_baser { + void *base; + + /* + * The table used when emulation is in place and indirect layout is + * configured. + */ + void *base_snapshot; + u64 val; + u32 order; + u32 psz; +}; + +struct its_host_state { + struct its_baser tables[GITS_BASER_NR_REGS]; + + /* The command queue used after the emulation is in place */ + void *cmd_host_copy; + + /* The command queue configured by the ITS driver at boot */ + void *cmd_original; + void *cmd_write; + size_t cmdq_len; +}; + +/* + * Callback used to initialize the emulation. It is expected to allocate m= emory for the private + * state of the emulation and receive as arguments copy of the host ITS dr= iver state along + * with the address of the ITS. + */ +typedef int (*its_emulate_setup)(phys_addr_t its_phys_base, struct its_hos= t_state *host); + +void its_emulate_acquire_locks(unsigned long *flags); +int its_emulate_release_locks(int ret_pkvm_finalize, unsigned long *flags,= its_emulate_setup cb); + #endif =20 #endif --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D9D347884E for ; Fri, 7 Aug 2026 16:43:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.71 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121031; cv=none; b=pibgVpnRI//E3HKxZd8xH3PBE1Nrm2xpZNAZTmvTJidCB5pg9elkv3ByAWihJmp0sxyYUZsHXQ5YH4UunJKD5fdtjOOL4W77U71KPd/FkF9cZ4PxTMYZgZUAUdZpnXdj8sxXhdszngfIu7MrNtsWWcrUtnEl5aVTADUMgy2Czcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121031; c=relaxed/simple; bh=FZjb20JofCESqDsnXMWtIjjk7phaXd3uuOYdnUvuix4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=J5wo7Ew36QsN3NbBvSkw9uzEV0WMHaUibXqL3COKf8urbVEK4YJPXC2kC3K7CMDDdVWbLhyAXD2TDY3S0Mt3oU1Dy2uKsyMKGDRaeA5cO7F1CsGu8gD/wvXTWaplfoGEFg/EteV9OU0KJOHRGYcRHOC03BCbr78sfzykzrCRh0s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uvzUClsB; arc=none smtp.client-ip=209.85.128.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uvzUClsB" Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-495569acf8dso22029385e9.1 for ; Fri, 07 Aug 2026 09:43:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121027; x=1786725827; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:from:to:cc :subject:date:message-id:reply-to:content-type; bh=ZQn1FQ7zxmEvhT55im1AaHwQn9RplPZn28ifPmSEdaY=; b=uvzUClsBDJ4j0+5HfVCGtoxXuRFvROLoTHyrUkEpksDbl26R6o6o7bjRuL6Le078i4 l2k87XJNGj/XiZ9aR8AOp7KKdjufRPYyFbh5WPOHUOA7M//4jPJv+vAMRyi0VOPEgNFv WsrVWYhyqm3aBu8SWy5VM6GrHNP+wD4MvWGOoHEvawRR3CApxY5IFRblUiiMA0AfUu22 7czvGrqCjS7Rf1fr4olZ7znR+LlUPHTLw00ET7yvziSn1rtKuERetJX1d05hj/g/Djmo iZDePWsApyAW2QJIyGWhfSwFgDi5Cs5bqqzV7LxZ/B8qFuYQN1MusWQrBrDDzPNPY59B oMHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121027; x=1786725827; h=content-transfer-encoding:content-type:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ZQn1FQ7zxmEvhT55im1AaHwQn9RplPZn28ifPmSEdaY=; b=Z9Ox+DiPG1BeT7vdBpVTxzWWUB++3GXb5M+B8E7OWMILOWGRdLaIKzfHV1SvTDKHP9 NG+/723tbjHvig4l3WZgiiI6UFrx47YrC5FX5KoH5YmR4MwzIw4itbJxvy3x0M0Se3D8 NaBvqVpFmlECMio4SRPUD1xZKhVMbVcIYFLTdnHb+ZG8nw0W8Lc2nHOKJV0Kac1XYK3q m8ME69Ei8EZzLfS5okSDhw5RYtZfBAu8FZI601mAUMcReCUQK50W9ipUK/C1Lcyw9QT+ Dj6v5JPNUQdjOtxW3WqqHi5CuvWk5TRJ0H49g1RZaFDKkw4RnDpy7PX69ibXEoRZk/bZ qp/w== X-Forwarded-Encrypted: i=1; AHgh+RrG9Cot8MG9aWKtJnxgn6usndzkNtD5Xve7tIBv8iXPoe6bHzq3PII5SH7JffsYMdBV9FZY+wHnC5dhyNc=@vger.kernel.org X-Gm-Message-State: AOJu0YxJ7t8EP5ySGy0mKohA8WlyMib2l/tzt5UQIwGwo6gwGSxmhpb8 VUWwtGH1FzEd3go2H9ix3X/Lp9AtwkphTE1Hn8802oL6ZQF/BsgmYjZ0ZmZgT+i66KZgYAKeZtL kR1GdQNSIhGOXdzrwFX2qv+bP4PChrA== X-Received: from wmol19.prod.google.com ([2002:a05:600c:47d3:b0:493:bdba:620b]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8b51:b0:495:6b55:f938 with SMTP id 5b1f17b1804b1-4994e7ba99emr358910455e9.10.1786121026918; Fri, 07 Aug 2026 09:43:46 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:16 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-8-sebastianene@google.com> Subject: [PATCH v2 06/13] KVM: arm64: Shadow the ITS command queue and setup emulation From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose two functions that will be used to setup the entry point into the pKVM ITS emulation. One will be called from an hvc to setup the ITS structures and the other one will be called from a data abort to handle the emulation. The later one will be stored in a pkvm_protected_reg as part of the register_its_emulated_region once the command emulation is in place. Donate two memory regions as part of the emulation setup phase. One holds GIC ITS driver state information and the other is used to store private state information for the emulation and it is zeroed out. Shadow the command queue by sharing a copy of it from the GIC ITS driver and donate the original queue to the hypervisor. The host will use a copy, while the emulation will use the original queue programmed in hardware. This makes sure that the original queue is not accessible to the host. When the GIC ITS driver writes a command, the emulation will trap the access to the CWRITER register and it will validate the command before copying it to the original queue. Re-use some of the definitions for command format and move them from the GIC ITS driver to the public header. Co-authored-by: Bart=C5=82omiej Grzesik Signed-off-by: Sebastian Ene --- arch/arm64/include/asm/kvm_pkvm.h | 1 + arch/arm64/kvm/hyp/include/nvhe/its_emulate.h | 14 + arch/arm64/kvm/hyp/nvhe/its_emulate.c | 285 ++++++++++++++++++ drivers/irqchip/irq-gic-v3-its.c | 12 - include/linux/irqchip/arm-gic-v3.h | 12 + 5 files changed, 312 insertions(+), 12 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/nvhe/its_emulate.h diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index 370225f0e72c..78597210a53c 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -27,6 +27,7 @@ struct pkvm_protected_reg { u64 pfn; u64 nr_pages; pkvm_emulate_handler *cb; + void *priv; }; =20 extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[]; diff --git a/arch/arm64/kvm/hyp/include/nvhe/its_emulate.h b/arch/arm64/kvm= /hyp/include/nvhe/its_emulate.h new file mode 100644 index 000000000000..29429feb30a9 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/nvhe/its_emulate.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __NVHE_ITS_EMULATE_H +#define __NVHE_ITS_EMULATE_H + +#include + +struct its_host_state; + +int pkvm_its_emulate_setup(phys_addr_t dev_addr, struct its_host_state *ho= st_state, void *priv, + size_t priv_num_pages); +void pkvm_its_emulate_handler(struct pkvm_protected_reg *region, u64 offse= t, bool write, u64 *reg, + u8 reg_size); +#endif /* __NVHE_ITS_EMULATE_H */ diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index 63a42f520ed2..e943ab972aa5 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -2,6 +2,9 @@ =20 #include #include +#include + +#include =20 void its_emulate_forward_req(struct pkvm_protected_reg *region, u64 offset= , bool write, u64 *reg, u8 reg_size) @@ -35,3 +38,285 @@ void its_emulate_forward_req(struct pkvm_protected_reg = *region, u64 offset, bool break; } } + +struct its_handler { + u64 offset; + u8 access_size; + void (*write)(struct pkvm_protected_reg *region, u64 offset, u64 value); + void (*read)(struct pkvm_protected_reg *region, u64 offset, u64 *read); +}; + +#define ITS_HANDLER(off, sz, write_cb, read_cb) \ +{ \ + .offset =3D (off), \ + .access_size =3D (sz), \ + .write =3D (write_cb), \ + .read =3D (read_cb), \ +} + +struct its_priv_state { + /* The location of the ITS in the hypervisor VA */ + void __iomem *base; + + /* ITS command queue use by the hardware */ + void *cmd_original; + void *cmd_host_copy; + u64 cmd_offset; + bool needs_flush; + hyp_spinlock_t its_lock; + + struct its_host_state *host_state; +}; + +#define GITS_CWRITER_RETRY BIT_ULL(0) +#define GITS_CWRITER_OFFSET GENMASK_ULL(19, 5) + +#define GITS_CREADR_STALLED BIT_ULL(0) +#define GITS_CREADR_OFFSET GENMASK_ULL(19, 5) + +static int submit_single_cmd(struct its_priv_state *its, bool retry) +{ + size_t cmdq_sz =3D its->host_state->cmdq_len; + u64 timeout =3D 1000; + u64 offset, cwriter, creadr; + + offset =3D (its->cmd_offset + sizeof(struct its_cmd_block)) % cmdq_sz; + + cwriter =3D offset & GITS_CWRITER_OFFSET; + cwriter |=3D FIELD_PREP(GITS_CWRITER_RETRY, retry); + writeq_relaxed(cwriter, its->base + GITS_CWRITER); + + while (its->cmd_offset !=3D offset) { + creadr =3D readq_relaxed(its->base + GITS_CREADR); + + /* Command failed. */ + if (FIELD_GET(GITS_CREADR_STALLED, creadr)) + return -EIO; + + its->cmd_offset =3D creadr & GITS_CREADR_OFFSET; + if (its->cmd_offset =3D=3D offset) + return 0; + + /* + * We can't spin here forever and we can't roll back + * the cmd queue pointer. Let's revert the cmd effects in the + * emulation layer and then go back to the driver to let it + * decide what to do next. + */ + if (!timeout--) + return -EBUSY; + } + + return 0; +} + +static int process_cmd(struct its_priv_state *its, struct its_cmd_block *c= md, + bool rollback) +{ + /* Passthrough everything for now */ + return 0; +} + +static void cwriter_write(struct pkvm_protected_reg *region, u64 offset, u= 64 value) +{ + struct its_priv_state *its =3D region->priv; + struct its_cmd_block cmd, raw; + u64 new_offset; + bool retry; + int i; + + new_offset =3D value & GITS_CWRITER_OFFSET; + if (new_offset >=3D its->host_state->cmdq_len) + return; + + retry =3D FIELD_GET(GITS_CWRITER_RETRY, value); + while (its->cmd_offset !=3D new_offset) { + memcpy(&raw, its->cmd_host_copy + its->cmd_offset, sizeof(raw)); + + for (i =3D 0; i < ARRAY_SIZE(cmd.raw_cmd); i++) + cmd.raw_cmd[i] =3D le64_to_cpu(raw.raw_cmd_le[i]); + + if (process_cmd(its, &cmd, /* rollback */ false)) + return; + + memcpy(its->cmd_original + its->cmd_offset, &raw, sizeof(struct its_cmd_= block)); + + if (its->needs_flush) + gic_flush_dcache_to_poc(its->cmd_original + its->cmd_offset, sizeof(cmd= )); + else + dsb(ishst); + + if (submit_single_cmd(its, retry)) { + WARN_ON(process_cmd(its, &cmd, /* rollback */ true)); + return; + } + } +} + +static void cwriter_read(struct pkvm_protected_reg *region, u64 offset, u6= 4 *read) +{ + struct its_priv_state *its =3D region->priv; + *read =3D readq_relaxed(its->base + GITS_CWRITER); +} + +static struct its_handler its_handlers[] =3D { + ITS_HANDLER(GITS_CWRITER, sizeof(u64), cwriter_write, cwriter_read), + {}, +}; + +void pkvm_its_emulate_handler(struct pkvm_protected_reg *region, u64 offse= t, bool write, u64 *reg, + u8 reg_size) +{ + struct its_priv_state *priv =3D region->priv; + struct its_handler *reg_handler; + + if (!priv || !IS_ALIGNED(offset, reg_size)) + return; + + for (reg_handler =3D its_handlers; reg_handler->access_size; reg_handler+= +) { + if (reg_handler->offset > offset || + reg_handler->offset + reg_handler->access_size <=3D offset) + continue; + + if (reg_handler->access_size < reg_size) + return; + + if (write && reg_handler->write) { + hyp_spin_lock(&priv->its_lock); + reg_handler->write(region, offset, *reg); + hyp_spin_unlock(&priv->its_lock); + return; + } + + if (!write && reg_handler->read) { + hyp_spin_lock(&priv->its_lock); + reg_handler->read(region, offset, reg); + hyp_spin_unlock(&priv->its_lock); + return; + } + + return; + } + + its_emulate_forward_req(region, offset, write, reg, reg_size); +} + +static int pkvm_setup_its_shadow_cmdq(struct its_host_state *host_state) +{ + u64 start_pfn, num_pages, i; + int ret; + + start_pfn =3D hyp_virt_to_pfn(host_state->cmd_host_copy); + num_pages =3D host_state->cmdq_len >> PAGE_SHIFT; + + for (i =3D 0; i < num_pages; i++) { + ret =3D __pkvm_host_share_hyp(start_pfn + i); + if (ret) + goto unshare_cmd_host; + } + + ret =3D hyp_pin_shared_mem(host_state->cmd_host_copy, + host_state->cmd_host_copy + host_state->cmdq_len); + if (ret) + goto unshare_cmd_host; + + ret =3D __pkvm_host_donate_hyp(hyp_virt_to_pfn(host_state->cmd_original),= num_pages); + if (ret) { + hyp_unpin_shared_mem(host_state->cmd_host_copy, + host_state->cmd_host_copy + host_state->cmdq_len); + goto unshare_cmd_host; + } + + return ret; +unshare_cmd_host: + if (i =3D=3D 0) + return ret; + + for (i =3D i - 1; i >=3D 0; i--) + __pkvm_host_unshare_hyp(start_pfn + i); + return ret; +} + +static struct pkvm_protected_reg *get_region(phys_addr_t dev_addr) +{ + int i; + + for (i =3D 0; i < num_protected_reg; i++) { + if (PFN_PHYS(pkvm_protected_regs[i].pfn) =3D=3D dev_addr) + return &pkvm_protected_regs[i]; + } + + return NULL; +} + +DEFINE_HYP_SPINLOCK(its_setup_lock); + +int pkvm_its_emulate_setup(phys_addr_t dev_addr, struct its_host_state *ho= st_state, void *priv, + size_t priv_num_pages) +{ + struct pkvm_protected_reg *its_reg; + struct its_priv_state *priv_state; + int ret; + + if (!PAGE_ALIGNED(host_state) || !PAGE_ALIGNED(priv) || !priv_num_pages) + return -EINVAL; + + host_state =3D kern_hyp_va(host_state); + priv =3D kern_hyp_va(priv); + + hyp_spin_lock(&its_setup_lock); + its_reg =3D get_region(dev_addr); + if (!its_reg) { + ret =3D -ENODEV; + goto err_unlock; + } + + if (its_reg->priv) { + ret =3D -EOPNOTSUPP; + goto err_unlock; + } + + ret =3D __pkvm_host_donate_hyp(hyp_virt_to_pfn(priv), priv_num_pages); + if (ret) + goto err_unlock; + + priv_state =3D priv; + memset(priv_state, 0, priv_num_pages << PAGE_SHIFT); + + ret =3D __pkvm_host_donate_hyp(hyp_virt_to_pfn(host_state), 1); + if (ret) + goto err_with_priv; + + host_state->cmd_original =3D kern_hyp_va(host_state->cmd_original); + host_state->cmd_host_copy =3D kern_hyp_va(host_state->cmd_host_copy); + + ret =3D pkvm_setup_its_shadow_cmdq(host_state); + if (ret) + goto err_with_host_state; + + hyp_spin_lock_init(&priv_state->its_lock); + + priv_state->host_state =3D host_state; + priv_state->base =3D (void __iomem *)__hyp_va(dev_addr); + priv_state->cmd_original =3D host_state->cmd_original; + priv_state->cmd_host_copy =3D host_state->cmd_host_copy; + + priv_state->cmd_offset =3D readq_relaxed(priv_state->base + GITS_CREADR) & + GITS_CREADR_OFFSET; + priv_state->needs_flush =3D + (readq_relaxed(priv_state->base + GITS_CBASER) & GITS_CBASER_SHAREABILIT= Y_MASK) !=3D + GITS_CBASER_InnerShareable; + + its_reg->priv =3D priv_state; + + hyp_spin_unlock(&its_setup_lock); + + return 0; +err_with_host_state: + WARN_ON(__pkvm_hyp_donate_host(hyp_virt_to_pfn(host_state), 1)); +err_with_priv: + WARN_ON(__pkvm_hyp_donate_host(hyp_virt_to_pfn(priv_state), 1)); +err_unlock: + hyp_spin_unlock(&its_setup_lock); + return ret; +} diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index e74ae9220af5..4736e49e3f2d 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -121,8 +121,6 @@ static DEFINE_PER_CPU(struct its_node *, local_4_1_its); #define is_v4_1(its) (!!((its)->typer & GITS_TYPER_VMAPP)) #define device_ids(its) (FIELD_GET(GITS_TYPER_DEVBITS, (its)->typer) + 1) =20 -#define ITS_ITT_ALIGN SZ_256 - /* The maximum number of VPEID bits supported by VLPI commands */ #define ITS_MAX_VPEID_BITS \ ({ \ @@ -515,16 +513,6 @@ struct its_cmd_desc { }; }; =20 -/* - * The ITS command block, which is what the ITS actually parses. - */ -struct its_cmd_block { - union { - u64 raw_cmd[4]; - __le64 raw_cmd_le[4]; - }; -}; - #define ITS_CMD_QUEUE_SZ SZ_64K #define ITS_CMD_QUEUE_NR_ENTRIES (ITS_CMD_QUEUE_SZ / sizeof(struct its_cmd= _block)) =20 diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm= -gic-v3.h index b75f82cef4bf..7f72632115b8 100644 --- a/include/linux/irqchip/arm-gic-v3.h +++ b/include/linux/irqchip/arm-gic-v3.h @@ -524,6 +524,8 @@ #define GITS_CMD_VSGI GITS_CMD_GICv4(3) #define GITS_CMD_INVDB GITS_CMD_GICv4(0xe) =20 +#define ITS_ITT_ALIGN SZ_256 + /* * ITS error numbers */ @@ -686,6 +688,16 @@ struct its_host_state { size_t cmdq_len; }; =20 +/* + * The ITS command block, which is what the ITS actually parses. + */ +struct its_cmd_block { + union { + u64 raw_cmd[4]; + __le64 raw_cmd_le[4]; + }; +}; + /* * Callback used to initialize the emulation. It is expected to allocate m= emory for the private * state of the emulation and receive as arguments copy of the host ITS dr= iver state along --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A00C347A87B for ; Fri, 7 Aug 2026 16:43:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.69 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121032; cv=none; b=NmWI6ksk2BgyMOAjcmlUbj53pu997xbKLC/rq/QGQRTC7nW8lFlKyOeIoh6PpvSut+JzQYvcIH4EK/00fsiGnZqzOno3Mgh9AxaveauWUMkHf3cMZuaElJIGPRagm+mHSRKDTCnKehO7PbgaBPG2mOQKvNXzjiLARQ7mWbfGAXI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121032; c=relaxed/simple; bh=B+kAmOvqka3yjYQ9VnbQJbNnM0ujY92oykRZepshn3M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ILVsEq4OHvcHB+Em+U2SjnWQ/msIIL6XabIVnPI3I4tAmFdkygS9QJEpcJE4kmFd2cPOwmTMrKr4LsZ2CftntGVJ8RxxgRfaE9LLIpOCm3pi+NAOmtwolAzHh5WJ9QOWjQyUjA+/xag1yR33av47M4w+DESGwfHBJ6d7JSznGcU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=tx5guLQq; arc=none smtp.client-ip=209.85.218.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tx5guLQq" Received: by mail-ej1-f69.google.com with SMTP id a640c23a62f3a-c15deb3377eso222661166b.3 for ; Fri, 07 Aug 2026 09:43:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121029; x=1786725829; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=Aee98ZEVCnU/nGC9n5IlQRH4Fbf+7bsHwlgyxI01O/I=; b=tx5guLQqnI8ZtcOhtWMn0/iWOuaGYmt3jvitnfAoGsX1KOGn9d24GPh2Ri3bJkJz7T jKoPOBx0Nr9chRgZLSpkBECGeMpKBWeWX6BDQMnUsqccVYP4EncfD9zSVg12ByIORSyF iIIhDvxEH/3Kqvj1jnKt2RZl07dvQCoBTuDGtcM5nW6lg9BHQmbNVNBAhVW/ZiNFoZFJ DiI/+MLolMRc89Jjv5PVkCTJFKsb2FB/dS+iMvTyPQcN+f+x3X0Ko55fdhlHR3oxAihu ekhPBmcHkbE4EqRhQruwI6aFTRKH/cVplml20PfwdMgoojS8wqejZm9P2mAslnMPd1Gk 7uHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121029; x=1786725829; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Aee98ZEVCnU/nGC9n5IlQRH4Fbf+7bsHwlgyxI01O/I=; b=E6waaa/iBd61lJXlRDYnmfQnjgxXVSQzN+pSRdhMYIxolJK0sZ+6zwbxd7AUYTVbLU JUwdrRzFSaPgmTNKoTFek+FMz/yti1lsmReJQtVWQC/E8li6f5pOz+hR4riomhde2QN1 5fETP4RfiYEPWqo+8ypXT0VQfGWnCl8nAo3OeeOAJ2xoL97CuA5ZBDVW8zXxNEracCYk 8qMjmwJBXcyvSU0IsMVn8a5ph23iVNKtzp+1GPHRl5v7NG39I6pryBsSyAn7icio9EXz me7VEizEiu777nz0R5aBzelDs0Gjhpkyoa3KtSTbB0Zgiq4Qe+2p11OpRzj6pAzU1lR8 qENw== X-Forwarded-Encrypted: i=1; AHgh+RrBK9NqPH19214qF08uf+XToGFcWHX56ngZpPcdCbEl0O5Pn3CbiZLzf7bR+csQYfjuHu0X/8ITO/DYKbo=@vger.kernel.org X-Gm-Message-State: AOJu0YxraHkypuPzpno1tuB9ZrhpPQp0+GwJrOFoHMgnkEsiq6uBDwq5 S3Pr0iHxmJXqa7rMxPKvJHKypdIFigwv30KbaHOmXy3EWLbhYf9a2M3eC/n0/2djSbxupOh/Qzj kW42qunqNt2TqXx6zv+VwVfM2f1iiwQ== X-Received: from ejcuc8.prod.google.com ([2002:a17:907:c888:b0:c16:7c0a:26a]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:198c:b0:c20:2165:f530 with SMTP id a640c23a62f3a-c2039d52e72mr1393700766b.28.1786121028433; Fri, 07 Aug 2026 09:43:48 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:17 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-9-sebastianene@google.com> Subject: [PATCH v2 07/13] KVM: arm64: Restrict host access to the private ITS tables From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make the last level of the tables(DeviceTable, Collection and vPE) inaccessible to the host by donating them to the hypervisor. This prevents a compromised host from patching an entry with an address that it wants to write to and then using an ITS command to write over the memory content from that address. When tables are configured with indirect layout, shadow the first layer by copying it to a separate table, update the gic ITS host driver to use the copy instead of the original table and share the copy between the host and the hypervisor. Make the original layer innaccessible to the host by donating the table memory from the host to the hypervisor. This ensures that the pKVM ITS emulation mediates the configuration written by the driver in the first layer of the table and sanitizes the entries before writing to the original table programmed in hardware. The update phase of the original table from the copy will be done when commands are sent to the ITS. Signed-off-by: Sebastian Ene --- arch/arm64/kvm/hyp/nvhe/its_emulate.c | 161 ++++++++++++++++++++++++++ 1 file changed, 161 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index e943ab972aa5..1ce2f9d8fcf9 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -237,6 +237,20 @@ static int pkvm_setup_its_shadow_cmdq(struct its_host_= state *host_state) return ret; } =20 +static void pkvm_teardown_its_shadow_cmdq(struct its_host_state *host_stat= e) +{ + u64 i, start_pfn, num_pages =3D host_state->cmdq_len >> PAGE_SHIFT; + + start_pfn =3D hyp_virt_to_pfn(host_state->cmd_host_copy); + hyp_unpin_shared_mem(host_state->cmd_host_copy, + host_state->cmd_host_copy + host_state->cmdq_len); + + for (i =3D 0; i < num_pages; i++) + WARN_ON(__pkvm_host_unshare_hyp(start_pfn + i)); + + WARN_ON(__pkvm_hyp_donate_host(hyp_virt_to_pfn(host_state->cmd_original),= num_pages)); +} + static struct pkvm_protected_reg *get_region(phys_addr_t dev_addr) { int i; @@ -249,6 +263,147 @@ static struct pkvm_protected_reg *get_region(phys_add= r_t dev_addr) return NULL; } =20 +static void pkvm_unshare_shadow_table(void *shadow, u64 nr_pages) +{ + u64 i, start_pfn =3D hyp_virt_to_pfn(shadow); + + hyp_unpin_shared_mem(shadow, shadow + (nr_pages << PAGE_SHIFT)); + + for (i =3D 0; i < nr_pages; i++) + WARN_ON(__pkvm_host_unshare_hyp(start_pfn + i)); +} + +static int pkvm_host_unmap_last_level(void *shadow, size_t num_pages, u32 = psz) +{ + phys_addr_t table_addr; + u64 *table =3D shadow; + int i, end; + int ret; + + end =3D (num_pages << PAGE_SHIFT) / sizeof(*table); + for (i =3D 0; i < end; i++) { + if (!(table[i] & GITS_BASER_VALID)) + continue; + + table_addr =3D table[i] & PHYS_MASK; + ret =3D __pkvm_host_donate_hyp(hyp_phys_to_pfn(table_addr), psz >> PAGE_= SHIFT); + if (ret) + goto err_donate; + } + + return 0; +err_donate: + for (i =3D i - 1; i >=3D 0; i--) { + if (!(table[i] & GITS_BASER_VALID)) + continue; + + table_addr =3D table[i] & PHYS_MASK; + __pkvm_hyp_donate_host(hyp_phys_to_pfn(table_addr), psz >> PAGE_SHIFT); + } + return ret; +} + +static int pkvm_share_shadow_table(void *shadow, u64 nr_pages) +{ + u64 i, ret, start_pfn =3D hyp_virt_to_pfn(shadow); + + for (i =3D 0; i < nr_pages; i++) { + ret =3D __pkvm_host_share_hyp(start_pfn + i); + if (ret) + goto unshare; + } + + ret =3D hyp_pin_shared_mem(shadow, shadow + (nr_pages << PAGE_SHIFT)); + if (ret) + goto unshare; + + return ret; +unshare: + while (i--) + __pkvm_host_unshare_hyp(start_pfn + i); + return ret; +} + +static void pkvm_host_map_last_level(void *shadow, size_t num_pages, u32 p= sz) +{ + u64 *table =3D shadow; + int i, end =3D (num_pages << PAGE_SHIFT) / sizeof(*table); + phys_addr_t table_addr; + + for (i =3D 0; i < end; i++) { + if (!(table[i] & GITS_BASER_VALID)) + continue; + + table_addr =3D table[i] & PHYS_MASK; + WARN_ON(__pkvm_hyp_donate_host(hyp_phys_to_pfn(table_addr), psz >> PAGE_= SHIFT)); + } +} + +static int pkvm_setup_its_shadow_baser(struct its_host_state *host_state) +{ + u64 baser_val, num_pages; + void *original_table, *snapshot_table; + int ret; + int i; + + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { + baser_val =3D host_state->tables[i].val; + if (!(baser_val & GITS_BASER_VALID)) + continue; + + original_table =3D kern_hyp_va(host_state->tables[i].base); + num_pages =3D (1 << host_state->tables[i].order); + + ret =3D __pkvm_host_donate_hyp(hyp_virt_to_pfn(original_table), num_page= s); + if (ret) + goto err_donate; + + if (baser_val & GITS_BASER_INDIRECT) { + if (!host_state->tables[i].base_snapshot) { + ret =3D -EINVAL; + goto err_with_donation; + } + + snapshot_table =3D kern_hyp_va(host_state->tables[i].base_snapshot); + ret =3D pkvm_share_shadow_table(snapshot_table, num_pages); + if (ret) + goto err_with_donation; + + ret =3D pkvm_host_unmap_last_level(original_table, num_pages, + host_state->tables[i].psz); + if (ret) + goto err_with_share; + } + } + + return 0; +err_with_share: + pkvm_unshare_shadow_table(snapshot_table, num_pages); +err_with_donation: + __pkvm_hyp_donate_host(hyp_virt_to_pfn(original_table), num_pages); +err_donate: + for (i =3D i - 1; i >=3D 0; i--) { + baser_val =3D host_state->tables[i].val; + if (!(baser_val & GITS_BASER_VALID)) + continue; + + original_table =3D kern_hyp_va(host_state->tables[i].base); + num_pages =3D (1 << host_state->tables[i].order); + + if (baser_val & GITS_BASER_INDIRECT) { + snapshot_table =3D kern_hyp_va(host_state->tables[i].base_snapshot); + pkvm_unshare_shadow_table(snapshot_table, num_pages); + + pkvm_host_map_last_level(original_table, num_pages, + host_state->tables[i].psz); + } + + WARN_ON(__pkvm_hyp_donate_host(hyp_virt_to_pfn(original_table), num_page= s)); + } + + return ret; +} + DEFINE_HYP_SPINLOCK(its_setup_lock); =20 int pkvm_its_emulate_setup(phys_addr_t dev_addr, struct its_host_state *ho= st_state, void *priv, @@ -294,6 +449,10 @@ int pkvm_its_emulate_setup(phys_addr_t dev_addr, struc= t its_host_state *host_sta if (ret) goto err_with_host_state; =20 + ret =3D pkvm_setup_its_shadow_baser(host_state); + if (ret) + goto err_with_shadow_cmdq; + hyp_spin_lock_init(&priv_state->its_lock); =20 priv_state->host_state =3D host_state; @@ -312,6 +471,8 @@ int pkvm_its_emulate_setup(phys_addr_t dev_addr, struct= its_host_state *host_sta hyp_spin_unlock(&its_setup_lock); =20 return 0; +err_with_shadow_cmdq: + pkvm_teardown_its_shadow_cmdq(host_state); err_with_host_state: WARN_ON(__pkvm_hyp_donate_host(hyp_virt_to_pfn(host_state), 1)); err_with_priv: --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05A1F47ACC7 for ; Fri, 7 Aug 2026 16:43:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.71 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121034; cv=none; b=sSRBGd/aDs3c/id/OxhbPOntCnh24fdMoB00TqkHg/gGFqPSJc0/qZ6hTAM/wj+BtAszR8mKWG2WGsGD+/9M/qghdaFF6166N9vwRP2ORY+ddVY3PQZ38pWFXlE1x0AKA5TGXeD/KzvYzTMrlGxgsmJ41P+RQMjMd3aXOpyLtvc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121034; c=relaxed/simple; bh=UTsxHC2ek2DT6ev0qCKTp9h2imFI+9o8uIR5wbfwqR4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dtCxOyYNBDOHKQuaGG6IR5SfjbRgX12pvl9VFiChjZHBr6xxI+kghs3hqI/pW37slfhCwutihgr+ANAsBAkC5G6sElaZYJYJdIAMeccqN0w59hZDFA8kcvHXi5s7jpovkCi1jrnxXCx5FegxbKfKBPm9Ojsi67aDal1gUUlrrDI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GLXaEQ7h; arc=none smtp.client-ip=209.85.221.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GLXaEQ7h" Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-47f7039fa59so2811294f8f.1 for ; Fri, 07 Aug 2026 09:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121030; x=1786725830; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=DGF2g3bOrwS0bBAJ3Ttr3oRulWgqFoQx+8Dbjow6I58=; b=GLXaEQ7hKkljQM7o9qmnfMEMiNe8/rkSq5yrzvKgGfiYhPvv2uWKh1DwO2ZMyNyGQu cgsIDYy29nIr0zKmUGEuuZ04GsG3rxR1jKuyaLUxf+3UktX09qziQmQUUALnH4MmdOUW IVImu5SbNE92OkrIb6oyBvHU1wKIRJ45HkcE7u1Oon/1N4IE72O8l2PtjsiO77lk1iiK NA1oRHdEP9Df9SMWgGEuZLFH3u9LBtgTvMq4G0Cm14iJewKT7vIBbn1LosILty6DNzeD /5PvMu44AnwrEQ0z0jAXxXLaS5s8jL/RiEW0s1vIjAUb/WSV4S2b172q3HROr7AbD0Xo tp4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121030; x=1786725830; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=DGF2g3bOrwS0bBAJ3Ttr3oRulWgqFoQx+8Dbjow6I58=; b=AfUCcBMlFhwqSH8it0gStwc1j2zw2LRzKkWMaePMVJX+HsuRNOlWCo+pl6kHF3CHhh YAFORXactExgwCf3E+Be5iYyPuEAkuyU4HzQJ+6ucLBzUmDqoUFPvB2elNN7PQmWqBAM FB2sBqP9ix9Lj8jyArvtDiy37Sm1orRjwiiiLhUu1njfSLXU9u+/JTav6ZKsd89e4QHY 1IHDZZhhBNN+uIYWs7QeoFakeKyzsmUqEvPThwKOdDP9s1/Tx/Y0DsLGh8cyMdrmPs6q XhlWLzvUr7QIiGVp/EE3eiR0cCysFptOAGIxbnCWhulmokjikO/B4KmwwWG6o9ftneoY aK8Q== X-Forwarded-Encrypted: i=1; AHgh+RqrEM/6Y66pOjSWQDa7hpRJJ+xnYBPIg4oerJA+c1j7FMAyPyEQqpKEOjH2np7wJ1x4tM2+z7Y3t66vOIs=@vger.kernel.org X-Gm-Message-State: AOJu0YwhaouBFBTydslMhsTuTWbF79JKYCbGGHu03bpjNl9lCv7U1LMR B+Cj8dZfUZkWShFvUZAB1FCsri2DIPLWwJsO5BkIEIz6HQdFY2QIdCwuKJdgYszQlg6jui0QAdL ZZMC3FPh7XBXZOkwxjmJ9baWxxKNRuQ== X-Received: from wrsn11.prod.google.com ([2002:a5d:484b:0:b0:46f:b032:2830]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:2501:b0:47f:6bdf:b13a with SMTP id ffacd0b85a97d-47fec523c73mr37443105f8f.18.1786121029869; Fri, 07 Aug 2026 09:43:49 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:18 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-10-sebastianene@google.com> Subject: [PATCH v2 08/13] KVM: arm64: Trap & emulate the ITS MAPD command From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Parse the MAPD command and extract the ITT address to sanitize it. When the command has the valid bit set, share and pin the memory that holds the ITT table with the hypervisor to prevent it from being given to someone else (eg. a VM). Use the pinning as a mechanism to get a grip to the page and to prevent other users of the pKVM API from sharing or donating the page for something else. This is to prevent a a situation where a page is given to someone else and then a MAPTI command is used to create an ITE entry in that page. Implement shadow table updates for the first level of the indirect tables when a MAPD command is issued. Compare the host view of the table for the entry identified by the deviceId with the original table at the same index and check if the valid bit is changed. If it didn't change, don't update the original table. If it changed, verify if the new entry has the valid bit set and donate the level2 table from the host to the hypervisor (with the address of the table used from the new entry). If the new entry has the valid bit cleared, donate the level2 table from the hypervisor to the host with the address of the table extracted from the original table managed by the hypervisor. Signed-off-by: Sebastian Ene --- arch/arm64/kvm/hyp/nvhe/its_emulate.c | 240 +++++++++++++++++++++++++- 1 file changed, 238 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index 1ce2f9d8fcf9..071a08d3602d 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -54,6 +54,11 @@ struct its_handler { .read =3D (read_cb), \ } =20 +struct dte_entry { + u32 device_id; + u64 itt_pfn; +}; + struct its_priv_state { /* The location of the ITS in the hypervisor VA */ void __iomem *base; @@ -66,6 +71,9 @@ struct its_priv_state { hyp_spinlock_t its_lock; =20 struct its_host_state *host_state; + u16 empty_entry; + u16 num_tracked_entries; + struct dte_entry tracked_entries[]; }; =20 #define GITS_CWRITER_RETRY BIT_ULL(0) @@ -110,11 +118,236 @@ static int submit_single_cmd(struct its_priv_state *= its, bool retry) return 0; } =20 +static int get_num_itt_pages(struct its_priv_state *its, u8 num_bits) +{ + u64 gits_typer, nr_ites; + size_t sz; + + gits_typer =3D readq_relaxed(its->base + GITS_TYPER); + if (num_bits > FIELD_GET(GITS_TYPER_IDBITS, gits_typer)) + return -EINVAL; + + nr_ites =3D BIT_ULL(num_bits + 1); + sz =3D nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, gits_typer) + 1); + sz =3D max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1; + + return PAGE_ALIGN(sz) >> PAGE_SHIFT; +} + +static struct its_baser *get_table_from_snapshot(struct its_host_state *ho= st, u64 baser_type) +{ + int i; + + for (i =3D 0; i < GITS_BASER_NR_REGS; i++) { + if (GITS_BASER_TYPE(host->tables[i].val) =3D=3D baser_type) + return &host->tables[i]; + } + + return NULL; +} + +static int check_table_update(struct its_priv_state *its, u32 device_id, u= 64 type, bool rollback) +{ + struct its_baser *table =3D get_table_from_snapshot(its->host_state, type= ); + size_t lvl2_entry_sz, lvl1_table_sz, num_lvl2_entries, num_lvl1_entries; + u64 *snapshot_table, *original_table; + u64 prev_entry, new_entry; + u32 new_entry_index; + int ret; + + if (!table) + return -EINVAL; + + /* We only do shadow udates for the first level of indirect tables */ + if (!(table->val & GITS_BASER_INDIRECT)) + return 0; + + lvl2_entry_sz =3D GITS_BASER_ENTRY_SIZE(table->val); + num_lvl2_entries =3D table->psz / lvl2_entry_sz; + + lvl1_table_sz =3D (1 << table->order) << PAGE_SHIFT; + num_lvl1_entries =3D lvl1_table_sz / sizeof(u64); + + new_entry_index =3D device_id / num_lvl2_entries; + if (new_entry_index >=3D num_lvl1_entries) + return -ENOSPC; + + snapshot_table =3D kern_hyp_va(table->base_snapshot); + original_table =3D kern_hyp_va(table->base); + + /* + * Look at the host table copy and if the entry hasn't changed the valid + * bit compared to the original table used by the hardwre, don't update a= nything. + */ + new_entry =3D snapshot_table[new_entry_index]; + prev_entry =3D original_table[new_entry_index]; + if (!((new_entry ^ prev_entry) & GITS_BASER_VALID)) + return 0; + + /* + * The host can play nasty tricks with read-modify-write after a + * rollback is triggered but we still hold on to the original tables + * which are hyp managed and we don't give back any other page to the + * host. + */ + if (rollback) + new_entry =3D new_entry ^ GITS_BASER_VALID; + + if (new_entry & GITS_BASER_VALID) + ret =3D __pkvm_host_donate_hyp(hyp_phys_to_pfn(new_entry & PHYS_MASK), + table->psz >> PAGE_SHIFT); + else + ret =3D __pkvm_hyp_donate_host(hyp_phys_to_pfn(prev_entry & PHYS_MASK), + table->psz >> PAGE_SHIFT); + if (ret) + return ret; + + original_table[new_entry_index] =3D new_entry; + return 0; +} + +static int track_pfn_add(struct its_priv_state *its, u32 device_id, u64 pf= n) +{ + void *virt =3D hyp_phys_to_virt(hyp_pfn_to_phys(pfn)); + struct dte_entry *entries =3D &its->tracked_entries[0]; + bool pfn_shared =3D false; + int ret; + int i; + + for (i =3D 0; i < its->num_tracked_entries; i++) { + if (entries[i].itt_pfn =3D=3D pfn) { + if (entries[i].device_id !=3D device_id) { + pfn_shared =3D true; + break; + } else { + return hyp_pin_shared_mem(virt, virt + PAGE_SIZE); + } + } + } + + if (its->empty_entry >=3D its->num_tracked_entries) + return -ENOSPC; + + if (!pfn_shared) { + ret =3D __pkvm_host_share_hyp(pfn); + if (ret) + return ret; + } + + ret =3D hyp_pin_shared_mem(virt, virt + PAGE_SIZE); + if (ret) { + __pkvm_host_unshare_hyp(pfn); + return ret; + } + + entries[its->empty_entry].itt_pfn =3D pfn; + entries[its->empty_entry].device_id =3D device_id; + + for (i =3D 0; i < its->num_tracked_entries; i++) { + if (!entries[i].itt_pfn && !entries[i].device_id) + break; + } + its->empty_entry =3D i; + return 0; +} + +static int track_pfn_remove(struct its_priv_state *its, u32 device_id, u64= pfn) +{ + void *virt =3D hyp_phys_to_virt(hyp_pfn_to_phys(pfn)); + struct dte_entry *entries =3D &its->tracked_entries[0]; + int ret; + int i; + + for (i =3D 0; i < its->num_tracked_entries; i++) { + if (entries[i].itt_pfn !=3D pfn || entries[i].device_id !=3D device_id) + continue; + + /* To decrement the refcount, first try to unshare it */ + ret =3D __pkvm_host_unshare_hyp(pfn); + if (ret =3D=3D -EBUSY) { + hyp_unpin_shared_mem(virt, virt + PAGE_SIZE); + ret =3D __pkvm_host_unshare_hyp(pfn); + if (ret =3D=3D -EBUSY) + return 0; + + WARN_ON(ret); + } + + memset(&entries[i], 0, sizeof(struct dte_entry)); + its->empty_entry =3D i; + return 0; + } + + return -EINVAL; +} + +static int track_pfn(struct its_priv_state *its, u32 device_id, u64 pfn, i= nt num_pages, + bool remove) +{ + int ret; + int i; + + for (i =3D 0; i < num_pages; i++) { + if (remove) + ret =3D track_pfn_remove(its, device_id, pfn + i); + else + ret =3D track_pfn_add(its, device_id, pfn + i); + + if (ret) + goto err_track_pfn; + } + + return 0; +err_track_pfn: + for (i =3D i - 1; i >=3D 0; i--) { + if (remove) + WARN_ON(track_pfn_add(its, device_id, pfn + i)); + else + WARN_ON(track_pfn_remove(its, device_id, pfn + i)); + } + return ret; +} + +static int process_its_mapd(struct its_priv_state *its, struct its_cmd_blo= ck *cmd, bool rollback) +{ + phys_addr_t itt_addr =3D cmd->raw_cmd[2] & GENMASK(51, 8); + bool remove =3D !(cmd->raw_cmd[2] & BIT(63)); + u8 size =3D cmd->raw_cmd[1] & GENMASK(4, 0); + u32 device_id =3D cmd->raw_cmd[0] >> 32; + int num_pages, ret; + u64 itt_pfn; + + if (rollback) + remove =3D !remove; + + itt_pfn =3D hyp_phys_to_pfn(itt_addr); + num_pages =3D get_num_itt_pages(its, size); + if (num_pages < 0) + return num_pages; + + ret =3D check_table_update(its, device_id, GITS_BASER_TYPE_DEVICE, rollba= ck); + if (ret) + return ret; + + return track_pfn(its, device_id, itt_pfn, num_pages, remove); +} + static int process_cmd(struct its_priv_state *its, struct its_cmd_block *c= md, bool rollback) { - /* Passthrough everything for now */ - return 0; + u8 req_type =3D cmd->raw_cmd[0] & GENMASK_ULL(7, 0); + int ret =3D 0; + + switch (req_type) { + case GITS_CMD_MAPD: + ret =3D process_its_mapd(its, cmd, rollback); + break; + default: + /* Passthrough everything for now */ + break; + } + + return ret; } =20 static void cwriter_write(struct pkvm_protected_reg *region, u64 offset, u= 64 value) @@ -459,6 +692,9 @@ int pkvm_its_emulate_setup(phys_addr_t dev_addr, struct= its_host_state *host_sta priv_state->base =3D (void __iomem *)__hyp_va(dev_addr); priv_state->cmd_original =3D host_state->cmd_original; priv_state->cmd_host_copy =3D host_state->cmd_host_copy; + priv_state->empty_entry =3D 0; + priv_state->num_tracked_entries =3D ((priv_num_pages << PAGE_SHIFT) - + offsetof(struct its_priv_state, tracked_entries)) / sizeof(struct dte_en= try); =20 priv_state->cmd_offset =3D readq_relaxed(priv_state->base + GITS_CREADR) & GITS_CREADR_OFFSET; --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 37D4B47CA8C for ; Fri, 7 Aug 2026 16:43:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121037; cv=none; b=sP3xhssyZCLOmOqaXIuu/mGWhsoliXc6uOVAgQotZCjNqDaLPHj+pNjIwhmCZAGGOH4EMkxHD4WqM96/udk1dStaGcN0mJkfdHTQtaP6QWZg/2Xq8hXSecm2+WijJi1V8NGlPryWqpeegPtEoTO4xvIDhqxsQ8SjY2jBrUM64TU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121037; c=relaxed/simple; bh=5ifxU6j26G5OaYQO/lDPi7ymHnNqbvnUe/Av1COaOCk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TQRE80gIq+h5sptzMtCIkPoVtU1uxnhNnUB4gGqMciKf1EbgGF8VQkO9DxD8YIU7HQh3NVUz8qU1iNyFJQhR7CF0Is638a39Vaa1cExlkqnQLYN0MwrEYqkLMg5pxZX0ak+9MtsuB/cb4cR1yVOK25XdCpdz5Bp36nI2vJcPrJ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MGEsMwq8; arc=none smtp.client-ip=209.85.208.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MGEsMwq8" Received: by mail-ed1-f72.google.com with SMTP id 4fb4d7f45d1cf-6a17c83732fso3471230a12.2 for ; Fri, 07 Aug 2026 09:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121032; x=1786725832; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=9zUqneyGoRi5zO+Pb+RiK7Bj3mze6rvm5S1Nt9aVy8g=; b=MGEsMwq8PT4oBhAUKlqpKnQXmZ8g5PFLyIDP5VE2WWLVVdzxpa5zx8N/oWU0xnSXNy +SEVnOIi1LvXZl+jwekGU1oOKnhooqyanlZw+YurYQf5lDVR6CtCnsiSOFLJrlWVOtOm tP0nwYmuVHQJn2jXafCrku9ntWySEP4G2u0p4oFaUnGI5LslQOvg7BR3dxXEMv4sT0OW 645lq62ilU8Jb8z4fy4rOBYojljaswtwayDlzuSjwXJ0egrLstB3c9evf64C1Tjl2bdg nJYtgbOSjkoMzmmBSs514h5pg3hGnhEVophf3rD++2ZoJ0K7MnCvjcu52EL+VEG7Y9SG mI6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121032; x=1786725832; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=9zUqneyGoRi5zO+Pb+RiK7Bj3mze6rvm5S1Nt9aVy8g=; b=Fr6Nq4w/WhFoUKplTlPm6DP2ty42FbMU+u2t4rP/8WatzKMBr2ZXn7FzIJjCW6AqLi cta7oQh7xBwuxGCCgw81j7bcv2vW3291IEbLXgfpUt8uSn7Dt02H9DAaWpzScNQL8Yeu QKXeanhbmk1VXFGaiqHF1X7sKwRGghgM+eWMEwiXuv3MAajqMNGF92CXUMa68/i/pZ8I 19tT4w8odHb3UWwgDARZ3coF6Xo16bSwzjGEzcWcRPvSbL+nKGgepTA1uKzBZocZooSV IAndJ0FqaQbS5wQe+uzOlzkfvZ6Dpu1hLy/yojRzKSTyDyM+nhMMmPckNMZJPIg+vn9j T2ew== X-Forwarded-Encrypted: i=1; AHgh+RqvyEDB6KCZDP6L0EQV+8ePbTJ962VOzXPxqXbZQCscvAupM6L+xRjcR5j5T3yLPTDUAhBArdabNBlSHHw=@vger.kernel.org X-Gm-Message-State: AOJu0Yz7LcBKZqdoo+rHbkeq6CYGbPx1BstnUSo2GzZXmOHQM88tp5wp 0KgK3YFZLCDcexRViqoPBm97M7rmSkVSDi7+00vZ2hvT1xaHD8/NfIQWU+CJtsa0mbFIG6R818w ma9u16qgReWuX0xEE0EG3lnXJ+O0Kng== X-Received: from edbfj6.prod.google.com ([2002:a05:6402:2b86:b0:697:8356:d9d6]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:2394:b0:6a1:7718:ad44 with SMTP id 4fb4d7f45d1cf-6a17718af13mr8635625a12.21.1786121031567; Fri, 07 Aug 2026 09:43:51 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:19 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-11-sebastianene@google.com> Subject: [PATCH v2 09/13] KVM: arm64: Trap & emulate the ITS MAPC command From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Parse the MAPC command and verify if we need to do any updates to the shadow collection table. Signed-off-by: Sebastian Ene --- arch/arm64/kvm/hyp/nvhe/its_emulate.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index 071a08d3602d..5629e2a070df 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -332,6 +332,13 @@ static int process_its_mapd(struct its_priv_state *its= , struct its_cmd_block *cm return track_pfn(its, device_id, itt_pfn, num_pages, remove); } =20 +static int process_its_mapc(struct its_priv_state *its, struct its_cmd_blo= ck *cmd, bool rollback) +{ + u32 icid =3D cmd->raw_cmd[2] & GENMASK(15, 0); + + return check_table_update(its, icid, GITS_BASER_TYPE_COLLECTION, rollback= ); +} + static int process_cmd(struct its_priv_state *its, struct its_cmd_block *c= md, bool rollback) { @@ -342,6 +349,10 @@ static int process_cmd(struct its_priv_state *its, str= uct its_cmd_block *cmd, case GITS_CMD_MAPD: ret =3D process_its_mapd(its, cmd, rollback); break; + + case GITS_CMD_MAPC: + ret =3D process_its_mapc(its, cmd, rollback); + break; default: /* Passthrough everything for now */ break; --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 424614766B4 for ; Fri, 7 Aug 2026 16:43:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.69 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121038; cv=none; b=Kgb0uKQ1WkjpaUeMZxb1lNoQX4yw0qjEjUpX5jmr6bGzqMCLlUYmk8GCv5CGjnLecpJcVhN+ENERxzh5hF84qQb42leC74JN5C5xw9eX4ljugCft9SVKKo96n1tpXiwVNq3QTC5CCd81ixuQAu7+irpAsZpTDtRJ4vlivxFnaEA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121038; c=relaxed/simple; bh=STYl0ghz5J/YkTbmptnKIYNWYygu2v7YgYHI7YW7274=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=phbE8vz7V1GRPSDPwh0NsGvW3bQPSnyjGrRK+5Dvgp/ivqVjvm8gjhX73lmHvB5B+YEK8uPaMif6GTIWqmdSBQsQGz3koX3HuHVgTVjSO10up36TG66qLjQBH/nTdSrdzFffwybw1T6xnOl/81dXniALRraoVw/g7A9B1mdwMfk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=S52o6N52; arc=none smtp.client-ip=209.85.128.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="S52o6N52" Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-492488f8583so30045755e9.2 for ; Fri, 07 Aug 2026 09:43:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121033; x=1786725833; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=pEop5HcfowkDeNq3FT2Q1c6RiGCFxpFvlXH/k0dA9E8=; b=S52o6N52ppIKKKvICls0j6QK1D2ZsqGggY/11PTHAn43pCXm5ZVzlSFvFWr+0E5CZm l+Ir/VL91EiSz9IQ8BO7n4i4/HK4p1tI5U+161o/0WzBYFAfL/T1PmZh4eg3WWLGFR1w uCdCOPzLB14LC+wJapzAYRSW3O5+OkWfraHl6vG82enA7xvZ7cYMVfFg0AQApdy4bq9L yQIGe8ksg/lyfOc1D1yCSWvJv1OirI+Pr06d3mDFpNikai+t6x/647171HAC0gx2rBnh 5TxYxnxh9JBXVHY2o7Hmz7eMfJKj2K1usa7JcIkp8GjlgbowPaIV04zAWFxnNxyF+MAR vUMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121033; x=1786725833; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=pEop5HcfowkDeNq3FT2Q1c6RiGCFxpFvlXH/k0dA9E8=; b=XNh7YAJdiVERnQqlahoqlI/kSi5YMLD4qeh6iTZAxj+H7zRcl4qXxAqrngbSMurr+L QrShP+C7h8QbVUpXZ4WnPMPCc9IPQsyUufgo2eVFivOfpo6tJ80hVdHGu04+Xc1P03CF yEL/tixkQO27SJYN91HVNwEK+TEZq80N39dlNqyTeGSqj2EzxRvIqm5N9zjEsuXIcMi6 XUAmXkVVqxTqVUD+ev3Yw8byymTkMHdQ6P2yw6/s2Poc54xtZRwXMAgeHgH2ED+SoYUT EBGs02WMcXRbMn0wcwr31ix8U1XnzpNWjV/wGXxcRx3cma+B9iOq2B0uQiiQpElycvYB mQ5A== X-Forwarded-Encrypted: i=1; AHgh+Rr7UHzMIj/wXhtFXuWbGhYv+mXq4ymq1Pie1VD10Fby4h6kctP9XSNaF3XDob6J3wa3G8ZIgHQ7FPQgcOc=@vger.kernel.org X-Gm-Message-State: AOJu0YwQoESGIbvNv2tRxottGW4casEw08ZJqWsfLI/bzs8Y47e4/5Yb Gp7qIGerzo3UaJG2IyVQzPoPJQ3mR7TOn7NqFgRhQJkxN6tw8u5KoUxTNlYN02BxYlTeR+60Y9J EXISPMBq5BCp0k7+jRey2HXyyvDCciQ== X-Received: from wmbhi12.prod.google.com ([2002:a05:600c:534c:b0:498:124b:1aa0]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:2e54:b0:495:4e1d:82df with SMTP id 5b1f17b1804b1-4994e7c1445mr233498895e9.10.1786121033123; Fri, 07 Aug 2026 09:43:53 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:20 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-12-sebastianene@google.com> Subject: [PATCH v2 10/13] KVM: arm64: Restrict host updates to GITS_CTLR From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Prevent unpredictable hardware behavior when the host tries to enable the ITS while it is not in quiescent state. Signed-off-by: Sebastian Ene --- arch/arm64/kvm/hyp/nvhe/its_emulate.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index 5629e2a070df..b9b71aa18d48 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -403,8 +403,35 @@ static void cwriter_read(struct pkvm_protected_reg *re= gion, u64 offset, u64 *rea *read =3D readq_relaxed(its->base + GITS_CWRITER); } =20 +static void ctlr_read(struct pkvm_protected_reg *region, u64 offset, u64 *= read) +{ + struct its_priv_state *its =3D region->priv; + *read =3D readl_relaxed(its->base + GITS_CTLR); +} + +static void ctlr_write(struct pkvm_protected_reg *region, u64 offset, u64 = value) +{ + struct its_priv_state *its =3D region->priv; + bool is_quiescent, is_enabled; + u32 ctlr; + + ctlr =3D readl_relaxed(its->base + GITS_CTLR); + is_quiescent =3D !!(ctlr & GITS_CTLR_QUIESCENT); + is_enabled =3D !!(ctlr & GITS_CTLR_ENABLE); + + /* + * If it's disabled and not in quiescent state and it tries to enable + * it, bail out. + */ + if (!is_enabled && (value & GITS_CTLR_ENABLE) && !is_quiescent) + return; + + writel_relaxed(value, its->base + GITS_CTLR); +} + static struct its_handler its_handlers[] =3D { ITS_HANDLER(GITS_CWRITER, sizeof(u64), cwriter_write, cwriter_read), + ITS_HANDLER(GITS_CTLR, sizeof(u32), ctlr_write, ctlr_read), {}, }; =20 --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE87F4756C9 for ; Fri, 7 Aug 2026 16:43:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.198 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121042; cv=none; b=auUhoL75nMuabC2/UMcZKFFCDyVKWzAjyPpSIY6qIePbIfrPy68CfPJwyTMIYNkG2548vTr5N0Vq7JYfz0ALtjQJUBRK57gape0YQbiTMZan/S2jRpMLT3+f1VogtDaDr4pDBITYn+Z6oLG6Skjr06j5xr7umju/Bj8TDxZAA2M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121042; c=relaxed/simple; bh=WJpZQ0SdOrH7RGvEvyDpMPhGUMqMI5kRtv8KfcRLpcY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dut9CIWdBeTNt2Qj4lqOefxE2Zk2TKG/Kg1rrPvRYOHqllPSRel3zurUM0SVeJM750LbfOm2yHNlfpwH0wmRn0jFR3ihhl1fT++ueYD7V7L/he85LOwKmsXYZvT4STwG0ePxGU0LIyUHtCIKLM6RL9oz0ReRpRjb45aXnj9gr1A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=d/1jx+7S; arc=none smtp.client-ip=209.85.208.198 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="d/1jx+7S" Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-39f72d86889so10742631fa.1 for ; Fri, 07 Aug 2026 09:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121035; x=1786725835; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=7TXLJIzu0sdUTbcYifBsweWW0qiwt6RC0AmUjCHWXcg=; b=d/1jx+7SP4m9918Xi55eshxB9isUi6qQkpw+sooQYQxyYh8jSL0cHMRokk/DFTK0gT vSsWJwqj+NEdXp4eEdGJybfL1qixVTz/lXp6JOKTPPnLg1GUUyA6ZwFU2AG/v53uM8oD W3PtbolOdGVKBGdQPotfgd6aON2/XEXdyQ+ojqJOOZy7ug4b8FAF5P8UNUdHXDQvj2ZT P+ywH6wVrEX3/QJDmJX24AhYAro1Og+1ekkZunghnFqzNqYAbkpBQ5mSYHyaMoz/jNw9 1xvp3df3JyEmLgXk9ei2OOMRu6ie3nwrQBkUQ2lacNoHJ0bYM56E7reYHhEhebITNc5t 6uvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121035; x=1786725835; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7TXLJIzu0sdUTbcYifBsweWW0qiwt6RC0AmUjCHWXcg=; b=aNL/BLtSgAd3MVltEWx4oJ3qzW8gQyniwzJ0HdwOHMNOEoGA38eUbd9R7+Z35ok4PA iaLdou/CF5ZIkftXhhxJctIPY6prLgIKNks1lW4fBbmE4V7t15ajrqFnGHkYTZveSk+N kJkbjRgt1DAKtOmMwkfisTiO7aHYJCf3ujvewJY+4qB3nq48iphn+UIICfSlS8gT+hqr rangYhIRvglJAJXLPcwq12xms/k5Mh5eJUi3BIXpsTua+/lnlH22ffo8Q2wPTb6Vl/iP XWH0EieV16vAHFsIBpYzhUydqHfecE2i4sNOz2QrvvQHIiVG+pFhoJPMTy5VlsWSERHd V2Qg== X-Forwarded-Encrypted: i=1; AHgh+RpiKCVasE1UQ4e9nRIDNsgFeWUxTvq5dLjelI5srGK5+GGQF6emhqhR53J1QECqkIDkFttprLTXGqTg7t0=@vger.kernel.org X-Gm-Message-State: AOJu0YxMY+zNcdptAAHyZSdlKhi9QxQpreIIaKoFZPwZ85R4ecTwnRRN zAKpK9/2VFETI2hLiJdH+ZMNeQ+RKzVb82wKXCd0bikEvU8u7mGheNe08ngMKmfxD5zBbi5FCOa gknooCemOQtzyb9e5nkRTmEM+cYyBBg== X-Received: from ljik18.prod.google.com ([2002:a2e:8892:0:b0:397:f415:3317]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a2e:bc0d:0:b0:39b:f64:1af3 with SMTP id 38308e7fff4ca-39fe56728b1mr2004231fa.8.1786121034668; Fri, 07 Aug 2026 09:43:54 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:21 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-13-sebastianene@google.com> Subject: [PATCH v2 11/13] KVM: arm64: Prevent the host from specifying a different command queue From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't let a malicious host re-program the command queue register with a different address and size to bypass the commands sanitization. Prevent unpredictable hardware behavior and restrict updates to the GITS_CBASER while the ITS is enabled or not in a quiescent state. Signed-off-by: Sebastian Ene --- arch/arm64/kvm/hyp/nvhe/its_emulate.c | 32 +++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index b9b71aa18d48..97cfa31d90d1 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -429,9 +429,41 @@ static void ctlr_write(struct pkvm_protected_reg *regi= on, u64 offset, u64 value) writel_relaxed(value, its->base + GITS_CTLR); } =20 +static void cbaser_write(struct pkvm_protected_reg *region, u64 offset, u6= 4 value) +{ + struct its_priv_state *its =3D region->priv; + int num_pages; + u64 ctlr; + + ctlr =3D readl_relaxed(its->base + GITS_CTLR); + if ((ctlr & GITS_CTLR_ENABLE) || !(ctlr & GITS_CTLR_QUIESCENT)) + return; + + num_pages =3D its->host_state->cmdq_len / SZ_4K; + + /* Don't let the host program a different command queue */ + value &=3D ~(GENMASK(7, 0) | GENMASK_ULL(51, 12)); + value |=3D (num_pages - 1) & GENMASK(7, 0); + value |=3D __hyp_pa(its->cmd_original) & GENMASK_ULL(51, 12); + its->needs_flush =3D (value & GITS_CBASER_SHAREABILITY_MASK) !=3D GITS_CB= ASER_InnerShareable; + + writeq_relaxed(value, its->base + GITS_CBASER); + + /* Restart the CMDQ to read from 0 */ + its->cmd_offset =3D 0; + writeq_relaxed(0, its->base + GITS_CWRITER); +} + +static void cbaser_read(struct pkvm_protected_reg *region, u64 offset, u64= *read) +{ + struct its_priv_state *its =3D region->priv; + *read =3D readq_relaxed(its->base + GITS_CBASER); +} + static struct its_handler its_handlers[] =3D { ITS_HANDLER(GITS_CWRITER, sizeof(u64), cwriter_write, cwriter_read), ITS_HANDLER(GITS_CTLR, sizeof(u32), ctlr_write, ctlr_read), + ITS_HANDLER(GITS_CBASER, sizeof(u64), cbaser_write, cbaser_read), {}, }; =20 --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E7EC4766A1 for ; Fri, 7 Aug 2026 16:43:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121045; cv=none; b=JvVQJe+5dWYQ3Cv/mOz99j9QHUHav4wZBR8KXboaVkMauhl6hwnahXOuaUJzSTETfFTEje/DTfdFytTui89n4/MpQaMzAoN10TmweQ5jsP+T5Espy+8SWBzqVGn8ZZJyf1hUqoXXcPAFwmf6RAgKLWQJzwGWC+Bs//vHlzL/f98= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121045; c=relaxed/simple; bh=evapBqF9fdH+H6PlBk1r5uqALms6dqHIsGbi9+k9YLk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JRq3gEdq8UEfCwboAzTrmPaGlTJNpGk5V30PLP+N3R51k6g+06oVDfGloaGRlUwSd8dvbdU/JgtP/DnGCcPYmZZXGYM5n41NiFtIvD2HG1pnTd6bq2DqRPuG8bHvXDBYwbSzn5Sy1PDsOQDeUzobbu2jEUx5JcMqkX4FAc45GKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pn214yGD; arc=none smtp.client-ip=209.85.218.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pn214yGD" Received: by mail-ej1-f72.google.com with SMTP id a640c23a62f3a-c1c232fa136so271842166b.0 for ; Fri, 07 Aug 2026 09:43:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121036; x=1786725836; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=kTVefapbptpCkdGjL+1kDLjlNC/ag2PzDpk3DPL7XNw=; b=pn214yGDFp4jn4WNxJ5LNAsL3508yQdHmJetiUmnqFaHqkoyQt+KggVjvRTUlTdU/V up2L96dNuvp7OH0H2jOZD+kOHY4Su0Crd+CRMhgfdU13zGZEhgwaOFSwRkCGdIbwmWjy Hm6KByn1V2TLpMMHRHoPpDftferCiCNXIMl6FhD/6HcCpLqX4pLq6CLmehA9btJeTlfi 6M5+W3PvmcQ0SEb8NBL+biu6T5ljdXLdv/5woC6nJOSLeN9Lo7oCcXmsQhcRx60MeZD7 ld6EcrN6KU27yn9IFxvqlziwvoow8PE4fEdO9nMTtiG3BEpchXAkP56nY+ogbp87/tZG NtpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121036; x=1786725836; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=kTVefapbptpCkdGjL+1kDLjlNC/ag2PzDpk3DPL7XNw=; b=nWajvnbearUyxmQDO2BgxcHtdlrF/3XB5MC0j2M4jGMwlcuryypIVhsUd3zP5/HzNz SkqmAcqqMfj+N+lp6+ZiO3yZl+e2tzletEeJR2A0BGvTjgvEqSsO/kb8bPPYmbuT7BLQ KoL6R0esEGffkQ5dPtO9zCLCeaufaP6z0F/QauYLX2iU4P58mY0nWpssbQaOSAhZqGxI UpOo5LeYGP97tGc//VZCUm1M02pcbWcJ6gIfqM2IuAAjlAA7VfauqgsLStO3hpmFVMiM Cnf++KiLfph+48BIvdE1B4U/jBI060OIo4SrkZv9m8bDK+kx3A/LJo96wPSEEWDJFyPc 4LkA== X-Forwarded-Encrypted: i=1; AHgh+RoXA1xnnGgUPulZlGZlJfSZDpmAWcDlZnyeRlylKW3VHZLTK77ny+WIRrAlI96tEo9IQHC3ja6Wm/1FHFQ=@vger.kernel.org X-Gm-Message-State: AOJu0YyxFVkYuDzHuY+9M3Tt6T4K5pE/3m/25RoNv9NuDLacIk7/JTRQ h/YUmSVt15CRUEKzvFFmMUDq5EaCN3g3LyGPXKA3BQTmFGGRHiU40z94t5vKunEprJQbXOI+xJ4 S9pJP7w6uSzhSw9bhBtIUDSHg5GNsww== X-Received: from ejoz15.prod.google.com ([2002:a17:906:668f:b0:c12:533f:6b88]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a17:906:2ad4:b0:c20:61e0:e3ab with SMTP id a640c23a62f3a-c2061e10ffbmr524125066b.3.1786121036039; Fri, 07 Aug 2026 09:43:56 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:22 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-14-sebastianene@google.com> Subject: [PATCH v2 12/13] KVM: arm64: Prevent the host from programming new GITS_BASER tables From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't allow the host to change the layout of the tables or to modify the address programmed in the GITS_BASER registers to point to new tables and bypass the sanitization. Prevent the host from updating the ITS tables while the ITS is enabled and the tables are set to prevent undefined behavior. Signed-off-by: Sebastian Ene --- arch/arm64/kvm/hyp/nvhe/its_emulate.c | 44 +++++++++++++++++++++++++-- 1 file changed, 41 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index 97cfa31d90d1..82dc60dcde68 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -42,18 +42,23 @@ void its_emulate_forward_req(struct pkvm_protected_reg = *region, u64 offset, bool struct its_handler { u64 offset; u8 access_size; + u8 num_registers; void (*write)(struct pkvm_protected_reg *region, u64 offset, u64 value); void (*read)(struct pkvm_protected_reg *region, u64 offset, u64 *read); }; =20 -#define ITS_HANDLER(off, sz, write_cb, read_cb) \ +#define ITS_HANDLER_REG_PAIR(off, sz, registers, write_cb, read_cb) \ { \ .offset =3D (off), \ .access_size =3D (sz), \ + .num_registers =3D (registers), \ .write =3D (write_cb), \ .read =3D (read_cb), \ } =20 +#define ITS_HANDLER(off, sz, write_cb, read_cb) \ + ITS_HANDLER_REG_PAIR(off, sz, 1, write_cb, read_cb) + struct dte_entry { u32 device_id; u64 itt_pfn; @@ -460,10 +465,42 @@ static void cbaser_read(struct pkvm_protected_reg *re= gion, u64 offset, u64 *read *read =3D readq_relaxed(its->base + GITS_CBASER); } =20 +static void baser_write(struct pkvm_protected_reg *region, u64 offset, u64= value) +{ + struct its_priv_state *its =3D region->priv; + u32 ctlr =3D readl_relaxed(its->base + GITS_CTLR); + int baser_idx; + u64 baser; + + if ((ctlr & GITS_CTLR_ENABLE) || !(ctlr & GITS_CTLR_QUIESCENT)) + return; + + baser_idx =3D (offset - GITS_BASER) >> 3; + baser =3D its->host_state->tables[baser_idx].val; + + /* Prevent if it tries to change from direct layout to indirect layout */ + if ((value & GITS_BASER_INDIRECT) !=3D (baser & GITS_BASER_INDIRECT)) + return; + + /* Don't allow the host to point to new tables or new attributes */ + value &=3D ~(GENMASK_ULL(47, 12) | GENMASK_ULL(9, 0)); + value |=3D (baser & GENMASK_ULL(47, 12)) | (baser & GENMASK_ULL(9, 0)); + + writeq_relaxed(value, its->base + offset); +} + +static void baser_read(struct pkvm_protected_reg *region, u64 offset, u64 = *read) +{ + struct its_priv_state *its =3D region->priv; + *read =3D readq_relaxed(its->base + offset); +} + static struct its_handler its_handlers[] =3D { ITS_HANDLER(GITS_CWRITER, sizeof(u64), cwriter_write, cwriter_read), ITS_HANDLER(GITS_CTLR, sizeof(u32), ctlr_write, ctlr_read), ITS_HANDLER(GITS_CBASER, sizeof(u64), cbaser_write, cbaser_read), + + ITS_HANDLER_REG_PAIR(GITS_BASER, sizeof(u64), 8, baser_write, baser_read), {}, }; =20 @@ -472,13 +509,14 @@ void pkvm_its_emulate_handler(struct pkvm_protected_r= eg *region, u64 offset, boo { struct its_priv_state *priv =3D region->priv; struct its_handler *reg_handler; + u64 end; =20 if (!priv || !IS_ALIGNED(offset, reg_size)) return; =20 for (reg_handler =3D its_handlers; reg_handler->access_size; reg_handler+= +) { - if (reg_handler->offset > offset || - reg_handler->offset + reg_handler->access_size <=3D offset) + end =3D reg_handler->offset + reg_handler->access_size * reg_handler->nu= m_registers; + if (reg_handler->offset > offset || end <=3D offset) continue; =20 if (reg_handler->access_size < reg_size) --=20 2.55.0.654.g21b8a5bc05-goog From nobody Wed Sep 30 13:54:50 2026 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF7D547D445 for ; Fri, 7 Aug 2026 16:43:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.72 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121045; cv=none; b=l76YA3ZksKUE8FbY//0JC28YPMCQ+AHYUEupu1eErAPTvDeXeJ/bXzuoCTzazvS9yQD8xlC9/DAzc5QV+AyoYOGxuJQV6XhgxM0hnNr4INVgwvUJ7MuMT9jX97dFpI3oBZZIeglePpWMqKj6v7EDGjXDN+4OKDjEvQ3CpE8CV7Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786121045; c=relaxed/simple; bh=dG75R9t60p8la9dDfP9piGZZE5hhgOort9SadpwYw1Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=O721vY5XXM1H95T3W2KQxYUKiyndBC794x8XeInDFNEFNcvt0gffK6l+B4b6B/s7VgeNuYR/+xXc6VxNwg03BNFyWtYuq79E3D6SLzsd2brXnXFLE7ByGiBVqQgVVs9GXF5rtGPtUFOgMxTBP4HSiIyf6GvCPgn+QAjT6WI8BRA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LorU/mcL; arc=none smtp.client-ip=209.85.128.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LorU/mcL" Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-496b6f97676so35192665e9.2 for ; Fri, 07 Aug 2026 09:43:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786121037; x=1786725837; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=qlULfceAkSyqJh1I9C7h7VPnh2+5JOHQToDbI7j6Lxc=; b=LorU/mcLh62ffHQzsIQSuL0YWwPSpcE7XhJZ4mTvni0Y/kexjIfdcV7adGnayQ2jot YR+iIiOI0B77PeDlNFjy5ojOytG3gd+79MxM8VnJuHhPtSWJaRYwaKCHcczZhFk1rfC9 pZwx/pASOUVexC6sb9wz7hwWn2Uj3p2p6pSQeN1v6IklDaYkBZiVMDvHb36Uh/hInr2L wKZcMq5CF97YtjKG3HrAdecV0ExUU1o/VXcb7R2gu09BP2dKaexYMfWm3iKC+CK7XXdO UAEr8i3T17axp1zCYAaNMzo8nZmpi0MoaI7Msf+yT5G2ej4PD7fTygeV9IRC0znZQ/CX WmGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786121037; x=1786725837; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=qlULfceAkSyqJh1I9C7h7VPnh2+5JOHQToDbI7j6Lxc=; b=fu9D16KDXK5SBymt8DIZoT1MumVOEckzAPXDKfv37rMCv9siu2otlSiRyb3vJvYSUC +ccHXgYo+CQLaIjntWb3AnlFMa/D7Tyrlf5aF2gUhUne1E75DUTVwbRv7ExOvi15k7/9 p8HUbfQnY465AUwf1oyg3zZkL9OVrc4/GModcYXoD1TFi6T9gUk2YvNOBcnDrfA68IHf 9AvQtv7AxYBOPmJRn+0Cimn5fl9h3oDt4GMsdrBjmFGjHIcmMawiF7/e31vcEIAcigGu PkXJj6qsulFDWe94jJYFpGz0rJLO62LYIlZtxZRdLMBLSpmJRKPoONS/Kol7pwdbg0Qe gEfg== X-Forwarded-Encrypted: i=1; AHgh+RoyjZjggldI2Ai5adGgvSjCjxbCjG2B9ortaZxZqjm9sedZuq1TmeTbtFy4dMG/K6JOReCBVC3RCuZzFhQ=@vger.kernel.org X-Gm-Message-State: AOJu0YynNTvD6ePP3+m64HWiUEq13hHe+2mdRCKPbIBurgZ4j3uGMNjY n7bYllHz5wSQged4rpGR8vNys0F9ugqrJEAYQiX+GtS1VB8n1k3esPge/Y0kepsiFnjnqjv+mA6 W0gRqcvASD44zsiAElX79w+jzGD0DcQ== X-Received: from wmbjx23.prod.google.com ([2002:a05:600c:5797:b0:495:58ee:fab7]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b01:b0:495:641a:bd3f with SMTP id 5b1f17b1804b1-4996199a076mr13517025e9.13.1786121037301; Fri, 07 Aug 2026 09:43:57 -0700 (PDT) Date: Fri, 7 Aug 2026 16:43:23 +0000 In-Reply-To: <20260807164322.2970811-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260807164322.2970811-2-sebastianene@google.com> X-Mailer: git-send-email 2.55.0.654.g21b8a5bc05-goog Message-ID: <20260807164322.2970811-15-sebastianene@google.com> Subject: [PATCH v2 13/13] KVM: arm64: Implement HVC interface for ITS emulation setup From: Sebastian Ene To: catalin.marinas@arm.com, fuad.tabba@linux.dev, joey.gouly@arm.com, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, rananta@google.com, Sascha.Bischoff@arm.com, suzuki.poulose@arm.com, will@kernel.org Cc: kvmarm@lists.linux.dev, android-kvm@google.com, bgrzesik@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, nathan@kernel.org, perlarsen@google.com, sebastianene@google.com, seiden@linux.ibm.com, smostafa@google.com, tglx@kernel.org, vdonnefort@google.com, vladimir.murzin@arm.com, yuzenghui@huawei.com, zenghui.yu@linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce a new HVC to allow the host to trigger the ITS emulation setup. Use the introduced API in the GIC ITS driver to call the driver to lock the ITS before pKVM finalize and to prepare for emulation setup. On the return path from the pKVM finalize, call into the driver to release the ITS locks which performs a switch in the driver to use a different command queue and a different set of level-1 indirect tables. Allocate memory that will be used by the emulation to track the internal state and send the snapshot state from the driver. Replace the initial "trap-and-forward" MMIO handler with a full-featured emulation handler. Signed-off-by: Sebastian Ene --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/include/asm/kvm_pkvm.h | 4 ++-- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 16 ++++++++++++++++ arch/arm64/kvm/hyp/nvhe/its_emulate.c | 4 ++-- arch/arm64/kvm/pkvm.c | 26 ++++++++++++++++++++++++-- 5 files changed, 45 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_= asm.h index 043495f7fc78..fcb2871b8a86 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -114,6 +114,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_load, __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_put, __KVM_HOST_SMCCC_FUNC___pkvm_tlb_flush_vmid, + __KVM_HOST_SMCCC_FUNC___pkvm_its_emulate_setup, =20 MARKER(__KVM_HOST_SMCCC_FUNC_MAX) }; diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index 78597210a53c..cc89e2bde468 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -32,8 +32,8 @@ struct pkvm_protected_reg { =20 extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[]; extern unsigned int kvm_nvhe_sym(num_protected_reg); -extern void kvm_nvhe_sym(its_emulate_forward_req)(struct pkvm_protected_re= g *region, u64 offset, - bool write, u64 *reg, u8 reg_size); +extern void kvm_nvhe_sym(pkvm_its_emulate_handler)(struct pkvm_protected_r= eg *region, u64 offset, + bool write, u64 *reg, u8 reg_size); =20 int pkvm_init_host_vm(struct kvm *kvm, unsigned long type); int pkvm_create_hyp_vm(struct kvm *kvm); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index d3df96ed8ba4..ad57b2076eee 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -16,6 +16,7 @@ #include =20 #include +#include #include #include #include @@ -705,6 +706,20 @@ static void handle___vgic_v5_restore_vmcr_apr(struct k= vm_cpu_context *host_ctxt) __vgic_v5_restore_vmcr_apr(kern_hyp_va(cpu_if)); } =20 +static void handle___pkvm_its_emulate_setup(struct kvm_cpu_context *host_c= txt) +{ + DECLARE_REG(phys_addr_t, dev_addr, host_ctxt, 1); + DECLARE_REG(struct its_host_state *, host_state, host_ctxt, 2); + DECLARE_REG(void *, priv_state, host_ctxt, 3); + DECLARE_REG(size_t, priv_state_num_pages, host_ctxt, 4); + + if (!is_protected_kvm_enabled()) + return; + + cpu_reg(host_ctxt, 1) =3D pkvm_its_emulate_setup(dev_addr, host_state, pr= iv_state, + priv_state_num_pages); +} + typedef void (*hcall_t)(struct kvm_cpu_context *); =20 #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] =3D (hcall_t)handle_##x @@ -762,6 +777,7 @@ static const hcall_t host_hcall[] =3D { HANDLE_FUNC(__pkvm_vcpu_load), HANDLE_FUNC(__pkvm_vcpu_put), HANDLE_FUNC(__pkvm_tlb_flush_vmid), + HANDLE_FUNC(__pkvm_its_emulate_setup), }; =20 static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index 82dc60dcde68..8c8acaee4d2b 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -6,8 +6,8 @@ =20 #include =20 -void its_emulate_forward_req(struct pkvm_protected_reg *region, u64 offset= , bool write, u64 *reg, - u8 reg_size) +static void its_emulate_forward_req(struct pkvm_protected_reg *region, u64= offset, bool write, + u64 *reg, u8 reg_size) { void __iomem *addr =3D __hyp_va(PFN_PHYS(region->pfn) + offset); =20 diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c index 4bfffbedac4c..a9ceb9ffe6a4 100644 --- a/arch/arm64/kvm/pkvm.c +++ b/arch/arm64/kvm/pkvm.c @@ -71,7 +71,7 @@ static int __init register_its_emulated_region(void) */ kvm_nvhe_sym(pkvm_protected_regs)[i].pfn =3D PHYS_PFN(res.start); kvm_nvhe_sym(pkvm_protected_regs)[i].cb =3D - lm_alias(&kvm_nvhe_sym(its_emulate_forward_req)); + lm_alias(&kvm_nvhe_sym(pkvm_its_emulate_handler)); kvm_nvhe_sym(pkvm_protected_regs)[i].nr_pages =3D PFN_DOWN(min_t(u64, resource_size(&res), PAGE_ALIGN_DOWN(GITS_TRANSLATE= R))); =20 @@ -312,8 +312,28 @@ static void __init _kvm_host_prot_finalize(void *arg) WRITE_ONCE(*err, -EINVAL); } =20 +#define ITS_PAGES (2UL) + +static int pkvm_init_its_emulation(phys_addr_t dev_addr, struct its_host_s= tate *host) +{ + size_t priv_state_sz =3D ITS_PAGES << PAGE_SHIFT; + void *priv_state; + int ret; + + priv_state =3D alloc_pages_exact(priv_state_sz, GFP_ATOMIC); + if (!priv_state) + return -ENOMEM; + + ret =3D kvm_call_hyp_nvhe(__pkvm_its_emulate_setup, dev_addr, host, priv_= state, ITS_PAGES); + if (ret) + free_pages_exact(priv_state, priv_state_sz); + + return ret; +} + static int __init pkvm_drop_host_privileges(void) { + unsigned long its_flags; int ret =3D 0; =20 /* @@ -321,8 +341,10 @@ static int __init pkvm_drop_host_privileges(void) * once the host stage 2 is installed. */ static_branch_enable(&kvm_protected_mode_initialized); + + its_emulate_acquire_locks(&its_flags); on_each_cpu(_kvm_host_prot_finalize, &ret, 1); - return ret; + return its_emulate_release_locks(ret, &its_flags, pkvm_init_its_emulation= ); } =20 static int __init finalize_pkvm(void) --=20 2.55.0.654.g21b8a5bc05-goog