From nobody Thu Apr 9 07:15:58 2026 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA3E73AD512 for ; Tue, 10 Mar 2026 12:49:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773146992; cv=none; b=uxLohph9wIyuec62Y1OxoWbc8ex8F5yZ2Q8SesBXlNEPS9bi3amlNBM2m671c4QNGDTo/Wy5mv+ony3WbPrcP+puUlpYqeRspQflluSQhUIR/JVGSfdTTnF8+tCdgA6qVZ32pWLVMF9m6fufNa4UIDy8fXeTDLYceOzuAY8xZRU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773146992; c=relaxed/simple; bh=BGZWw0pNpFUVe6fJJOqZpw/0eKKedefH/jfskrr/pLw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dS1K+eDkJiJOrw1ltoHkvXIkCZaO6kIg7+v8IOQ9gWSBowdKI33RJJ20atCcpP9MA4LJhilcxBosnn+aN52NuivCe3vtIPlckDeeI3yIY3AMRfWMtbxxn2pfan0+/Da1QR1RFzp2jr9MggC8S8226UmXkxDAYlcKdguIR1txiEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Tfpj+M8P; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Tfpj+M8P" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-b96ec4966cbso225337866b.0 for ; Tue, 10 Mar 2026 05:49:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1773146989; x=1773751789; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AuLKSluxKmhn480CzigIfSkaIIQucOGgq+FPielHBMw=; b=Tfpj+M8P6SwBW8oR4zT+DrEdpFk8zY9SLqmthesSqZ9TTDrQbWMZhQqRpfUXXVWF/L xgP2ZVmqeoTSzqgTm8AiwF5nkAsqIPWauq1YV80EZK/xqkKPKUKVkC3apLz4i7ybXQaK o2xJADvdpu7IE4+OkGglh9R8e73nXWgHs37Jt+4jAIsHPheSAIsEfqu/ZSecj7eFUIJJ FVHzqlO+Ds+VElTfTEEZjbJjXbXLmfxBZw06wXOKGi2lSa4Q4G3Vf90fKU1yzBsK6NEJ BPNHWyEA0wp5OxQ5zezBBHzhYeHcqRHD/W/fv+2aXj4MiNVirtX/Rzc0QrLvGwOLqLPI 5XUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1773146989; x=1773751789; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AuLKSluxKmhn480CzigIfSkaIIQucOGgq+FPielHBMw=; b=Vg/XKSde0+QLhP85qYZ9U/xsl/BnY9MHffp7cG/jS0tmcCEsriqXOho9wv3NvXBWpN kV85A1YKN1gzOV3PUgwguobGeM6iO36AEb6aX9o48CW1ANEQar//k8edHCBHjh4YJRqQ 8uZdOO3ImToBqEXpogFFOms+Oh1hMu8KgvP0fAq3O9MYqgxXcR+glmfsXfGlZKyLVW9C +qpv5NUir5BayHd54xk0Qo184ZXDaBNpCeh73QCYmWZqxkb4IRXS9/oXihJCCt/ceHQP 0SSy65Xe0tXhsBHGC0dvMUv2CE2pMGpwTkwnxzJqbeCGRk/6pv1WpFMI8siLxK1bvE9x dzCg== X-Forwarded-Encrypted: i=1; AJvYcCV9r6z31UCDDvGVApkVvbQuiNt8he1hM2P0IBUyRl25BxEiLL8EieuZsQTRd7OFm5H1IApGSxkRaXObMjc=@vger.kernel.org X-Gm-Message-State: AOJu0YzpMX80ExIHgOUeBjpBcbPxyRUNoxVMoRKxQcvP1H7bHWgPcZcG 3YCwSdd0eDQJDaTnGq8Bq9WBlOhJsEcVHbO8KNuYWZDRJZxHQgPkZmaFKFkrmDEyXzjid+fnEYh v5MUMJD4W63lBQLIZ3zdNNSY9kWZF4g== X-Received: from edfa4.prod.google.com ([2002:a50:9e84:0:b0:662:810d:badd]) (user=sebastianene job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:7290:b0:b8e:9a0:1bfb with SMTP id a640c23a62f3a-b942d58ea78mr829388866b.0.1773146989033; Tue, 10 Mar 2026 05:49:49 -0700 (PDT) Date: Tue, 10 Mar 2026 12:49:25 +0000 In-Reply-To: <20260310124933.830025-1-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260310124933.830025-1-sebastianene@google.com> X-Mailer: git-send-email 2.53.0.473.g4a7958ca14-goog Message-ID: <20260310124933.830025-7-sebastianene@google.com> Subject: [PATCH 06/14] KVM: arm64: Add infrastructure for ITS emulation setup From: Sebastian Ene To: alexandru.elisei@arm.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, android-kvm@google.com Cc: catalin.marinas@arm.com, dbrazdil@google.com, joey.gouly@arm.com, kees@kernel.org, mark.rutland@arm.com, maz@kernel.org, oupton@kernel.org, perlarsen@google.com, qperret@google.com, rananta@google.com, sebastianene@google.com, smostafa@google.com, suzuki.poulose@arm.com, tabba@google.com, tglx@kernel.org, vdonnefort@google.com, bgrzesik@google.com, will@kernel.org, yuzenghui@huawei.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Share the host command queue with the hypervisor. Donate the original command queue memory to the hypervisor to ensure host exclusion and trap accesses on GITS_CWRITE register. On a CWRITER write, the hypervisor copies commands from the host's queue to the protected queue before updating the hardware register. This ensures the hypervisor mediates all commands sent to the physical ITS. Signed-off-by: Sebastian Ene --- arch/arm64/include/asm/kvm_pkvm.h | 1 + arch/arm64/kvm/hyp/include/nvhe/its_emulate.h | 17 ++ arch/arm64/kvm/hyp/nvhe/its_emulate.c | 203 ++++++++++++++++++ 3 files changed, 221 insertions(+) create mode 100644 arch/arm64/kvm/hyp/include/nvhe/its_emulate.h diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index ef00c1bf7d00..dc5ef2f9ac49 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -28,6 +28,7 @@ struct pkvm_protected_reg { u64 start_pfn; size_t num_pages; pkvm_emulate_handler *cb; + void *priv; }; =20 extern struct pkvm_protected_reg kvm_nvhe_sym(pkvm_protected_regs)[]; diff --git a/arch/arm64/kvm/hyp/include/nvhe/its_emulate.h b/arch/arm64/kvm= /hyp/include/nvhe/its_emulate.h new file mode 100644 index 000000000000..6be24c723658 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/nvhe/its_emulate.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __NVHE_ITS_EMULATE_H +#define __NVHE_ITS_EMULATE_H + + +#include + + +struct its_shadow_tables; + +int pkvm_init_gic_its_emulation(phys_addr_t dev_addr, void *priv_state, + struct its_shadow_tables *shadow); + +void pkvm_handle_gic_emulation(struct pkvm_protected_reg *region, u64 offs= et, bool write, + u64 *reg, u8 reg_size); +#endif /* __NVHE_ITS_EMULATE_H */ diff --git a/arch/arm64/kvm/hyp/nvhe/its_emulate.c b/arch/arm64/kvm/hyp/nvh= e/its_emulate.c index 0eecbb011898..4a3ccc90a1a9 100644 --- a/arch/arm64/kvm/hyp/nvhe/its_emulate.c +++ b/arch/arm64/kvm/hyp/nvhe/its_emulate.c @@ -1,8 +1,75 @@ // SPDX-License-Identifier: GPL-2.0-only =20 #include +#include +#include #include =20 +struct its_priv_state { + void *base; + void *cmd_hyp_base; + void *cmd_host_base; + void *cmd_host_cwriter; + struct its_shadow_tables *shadow; + hyp_spinlock_t its_lock; +}; + +struct its_handler { + u64 offset; + u8 access_size; + void (*write)(struct its_priv_state *its, u64 offset, u64 value); + void (*read)(struct its_priv_state *its, u64 offset, u64 *read); +}; + +DEFINE_HYP_SPINLOCK(its_setup_lock); + +static void cwriter_write(struct its_priv_state *its, u64 offset, u64 valu= e) +{ + u64 cwriter_offset =3D value & GENMASK(19, 5); + int cmd_len, cmd_offset; + size_t cmdq_sz =3D its->shadow->cmdq_len; + + if (cwriter_offset > cmdq_sz) + return; + + cmd_offset =3D its->cmd_host_cwriter - its->cmd_host_base; + cmd_len =3D cwriter_offset - cmd_offset; + if (cmd_len < 0) + cmd_len =3D cmdq_sz - cmd_offset; + + if (cmd_offset + cmd_len > cmdq_sz) + return; + + memcpy(its->cmd_hyp_base + cmd_offset, its->cmd_host_cwriter, cmd_len); + + its->cmd_host_cwriter =3D its->cmd_host_base + + (cmd_offset + cmd_len) % cmdq_sz; + if (its->cmd_host_cwriter =3D=3D its->cmd_host_base) { + memcpy(its->cmd_hyp_base, its->cmd_host_base, cwriter_offset); + + its->cmd_host_cwriter =3D its->cmd_host_base + cwriter_offset; + } + + writeq_relaxed(value, its->base + GITS_CWRITER); +} + +static void cwriter_read(struct its_priv_state *its, u64 offset, u64 *read) +{ + *read =3D readq_relaxed(its->base + GITS_CWRITER); +} + +#define ITS_HANDLER(off, sz, write_cb, read_cb) \ +{ \ + .offset =3D (off), \ + .access_size =3D (sz), \ + .write =3D (write_cb), \ + .read =3D (read_cb), \ +} + +static struct its_handler its_handlers[] =3D { + ITS_HANDLER(GITS_CWRITER, sizeof(u64), cwriter_write, cwriter_read), + {}, +}; =20 void pkvm_handle_forward_req(struct pkvm_protected_reg *region, u64 offset= , bool write, u64 *reg, u8 reg_size) @@ -21,3 +88,139 @@ void pkvm_handle_forward_req(struct pkvm_protected_reg = *region, u64 offset, bool writeq_relaxed(*reg, addr); } } + +void pkvm_handle_gic_emulation(struct pkvm_protected_reg *region, u64 offs= et, bool write, + u64 *reg, u8 reg_size) +{ + struct its_priv_state *its_priv =3D region->priv; + void __iomem *addr; + struct its_handler *reg_handler; + + if (!its_priv) + return; + + addr =3D its_priv->base + offset; + for (reg_handler =3D its_handlers; reg_handler->access_size; reg_handler+= +) { + if (reg_handler->offset > offset || + reg_handler->offset + reg_handler->access_size <=3D offset) + continue; + + if (reg_handler->access_size & (reg_size - 1)) + continue; + + if (write && reg_handler->write) { + hyp_spin_lock(&its_priv->its_lock); + reg_handler->write(its_priv, offset, *reg); + hyp_spin_unlock(&its_priv->its_lock); + return; + } + + if (!write && reg_handler->read) { + hyp_spin_lock(&its_priv->its_lock); + reg_handler->read(its_priv, offset, reg); + hyp_spin_unlock(&its_priv->its_lock); + return; + } + + return; + } + + pkvm_handle_forward_req(region, offset, write, reg, reg_size); +} + +static struct pkvm_protected_reg *get_region(phys_addr_t dev_addr) +{ + int i; + u64 dev_pfn =3D dev_addr >> PAGE_SHIFT; + + for (i =3D 0; i < PKVM_PROTECTED_REGS_NUM; i++) { + if (pkvm_protected_regs[i].start_pfn =3D=3D dev_pfn) + return &pkvm_protected_regs[i]; + } + + return NULL; +} + +static int pkvm_setup_its_shadow_cmdq(struct its_shadow_tables *shadow) +{ + int ret, i, num_pages; + u64 shadow_start_pfn, original_start_pfn; + void *cmd_shadow_va =3D kern_hyp_va(shadow->cmd_shadow); + + shadow_start_pfn =3D hyp_virt_to_pfn(cmd_shadow_va); + original_start_pfn =3D hyp_virt_to_pfn(kern_hyp_va(shadow->cmd_original)); + num_pages =3D shadow->cmdq_len >> PAGE_SHIFT; + + for (i =3D 0; i < num_pages; i++) { + ret =3D __pkvm_host_share_hyp(shadow_start_pfn + i); + if (ret) + goto unshare_shadow; + } + + ret =3D hyp_pin_shared_mem(cmd_shadow_va, cmd_shadow_va + shadow->cmdq_le= n); + if (ret) + goto unshare_shadow; + + ret =3D __pkvm_host_donate_hyp(original_start_pfn, num_pages); + if (ret) + goto unpin_shadow; + + return ret; + +unpin_shadow: + hyp_unpin_shared_mem(cmd_shadow_va, cmd_shadow_va + shadow->cmdq_len); + +unshare_shadow: + for (i =3D i - 1; i >=3D 0; i--) + __pkvm_host_unshare_hyp(shadow_start_pfn + i); + + return ret; +} + +int pkvm_init_gic_its_emulation(phys_addr_t dev_addr, void *host_priv_stat= e, + struct its_shadow_tables *host_shadow) +{ + int ret; + struct its_priv_state *priv_state =3D kern_hyp_va(host_priv_state); + struct its_shadow_tables *shadow =3D kern_hyp_va(host_shadow); + struct pkvm_protected_reg *its_reg; + + hyp_spin_lock(&its_setup_lock); + its_reg =3D get_region(dev_addr); + if (!its_reg) + return -ENODEV; + + if (its_reg->priv) + return -EOPNOTSUPP; + + ret =3D __pkvm_host_donate_hyp(hyp_virt_to_pfn(priv_state), 1); + if (ret) + return ret; + + ret =3D __pkvm_host_donate_hyp(hyp_virt_to_pfn(shadow), 1); + if (ret) + goto err_with_state; + + ret =3D pkvm_setup_its_shadow_cmdq(shadow); + if (ret) + goto err_with_shadow; + + its_reg->priv =3D priv_state; + + hyp_spin_lock_init(&priv_state->its_lock); + priv_state->shadow =3D shadow; + priv_state->base =3D __hyp_va(dev_addr); + + priv_state->cmd_hyp_base =3D kern_hyp_va(shadow->cmd_original); + priv_state->cmd_host_base =3D kern_hyp_va(shadow->cmd_shadow); + priv_state->cmd_host_cwriter =3D priv_state->cmd_host_base; + + hyp_spin_unlock(&its_setup_lock); + + return 0; +err_with_shadow: + __pkvm_hyp_donate_host(hyp_virt_to_pfn(shadow), 1); +err_with_state: + __pkvm_hyp_donate_host(hyp_virt_to_pfn(priv_state), 1); + return ret; +} --=20 2.53.0.473.g4a7958ca14-goog