From nobody Tue Feb 10 22:00:05 2026 Received: from mail-ot1-f73.google.com (mail-ot1-f73.google.com [209.85.210.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3BF331D372 for ; Mon, 9 Feb 2026 22:40:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770676858; cv=none; b=TnQ+Xrkgqk34ujWZMYD7yZFy3gf6vAqyU08NxQXoENMcYMzis5VGyeB2lHN8O+KIm1fHxSPKBksNTjV9t5wKU0nMlMNdHuIGdncpy3z5N0GBpYWn+AgV6jgnPK+pmh2Ql2IAbK+woRSEB7y1YZlTdtLsqUTANdkt0At2jF5t3X8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770676858; c=relaxed/simple; bh=VlntSjT33zFT4cwsmNUlK5lpAzJYXAv17KA7vHnYPW8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tmJ41FjFG2tW86QLoG24vXOMRuFStcClkJBtjIQOC/yPfmoSPBxjoLnCzEOnJqlyf16YndX6Q0w9moOMZQdIZ0iDmZLvqMk3VXslOeUf46gCNiw+nUvbZGsicu//Kc1qeVGX48MITIgLyY6iFbpd176nOxCEhTt/eYxqKrr0ASw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--coltonlewis.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Lxmy9cH5; arc=none smtp.client-ip=209.85.210.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--coltonlewis.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Lxmy9cH5" Received: by mail-ot1-f73.google.com with SMTP id 46e09a7af769-7d498212845so623225a34.3 for ; Mon, 09 Feb 2026 14:40:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1770676854; x=1771281654; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1q4WN7z/SL7quCNEFM+ItoBlF4yxyRczz8LessGfPao=; b=Lxmy9cH5s438x/aSg/jAP0vd9MbnfNzRWx9myX25dDHvy6auC0aXnRWWtXozqqWev2 8V9BedJCp/q/02BROvUxW48VBaJHjVorxbPGvwC19PnZk1r+TTR535Zl887f23ik24Ug y1zV8asGsRqGlNyrk5X0o4pZQGRF4pSTg4gGCYvZcthBfJeR7DogXt0wVul60kEY3bcs DlwQbCkavaNLUHukzq3GCc9BBSjrlUquHTPs5cFIBoSVNJglnDaFXP9Qlr9TBOmsaqqG YK+sZAHgPk1UDlUXHh21J/qIgHgmcZICfO7NHo+fE74Fststvo0UXlZgB8SimHVEFm4b fblQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770676854; x=1771281654; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1q4WN7z/SL7quCNEFM+ItoBlF4yxyRczz8LessGfPao=; b=CnBpd8kXLMFJqndoIfdg8PpUmVYqXuez0LfGsggPtgIFjhLwzRP4kO7vvYP+XRGRtQ eU9wbszEMq3ESwNQbO/Ho3hGNmv9yNAc6eY6v/qJOBJ469vs+SJ4rbyzy63J2J8OX873 bPfdvRGWp5pm2mEg4G/8Mxqwefdyyp4jrsxQGd22/peiETR6ufvg/HRvFxGs6wdOeqg7 Zc+tWVtVGLplBM4Oc9rrQyk37l+ZK3iFg7ysexHijisVxl1US0NXTtselwkLrk9XIfya Y+0Ew8N2ywJJf9WAxcp2NomHxxV9J9fqopSzCAgkYGBzSuC+4TW+WqvDChiv/vXKRHbL Jf7w== X-Forwarded-Encrypted: i=1; AJvYcCVli38kJt4VC1yFtTzYy2D1ida3kbJPyrEyG/syUdVF241ZJMnLM7S46QSvAt0O8ljRLmZXL161nP4wD/M=@vger.kernel.org X-Gm-Message-State: AOJu0YyJX009jN9manz6uUSli9sqp8CLrrv0buk8A7jZT/aC7CcYEViQ 7yiBUfQB4Q/F9oUNDpntbO3QjnBS6PyMLPX9uFF9FktaTLr20yq07KR+32otTaDD8Xk48zJp+bn U67BmUzSrDXj6B+lVvpbtt7A6yw== X-Received: from iong8.prod.google.com ([2002:a5d:8c88:0:b0:957:4b76:541e]) (user=coltonlewis job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6820:6aca:b0:663:9b9:9297 with SMTP id 006d021491bc7-66d0c6685ccmr6026920eaf.64.1770676853645; Mon, 09 Feb 2026 14:40:53 -0800 (PST) Date: Mon, 9 Feb 2026 22:14:04 +0000 In-Reply-To: <20260209221414.2169465-1-coltonlewis@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260209221414.2169465-1-coltonlewis@google.com> X-Mailer: git-send-email 2.53.0.rc2.204.g2597b5adb4-goog Message-ID: <20260209221414.2169465-10-coltonlewis@google.com> Subject: [PATCH v6 09/19] KVM: arm64: Write fast path PMU register handlers From: Colton Lewis To: kvm@vger.kernel.org Cc: Alexandru Elisei , Paolo Bonzini , Jonathan Corbet , Russell King , Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Mingwei Zhang , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Mark Rutland , Shuah Khan , Ganapatrao Kulkarni , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-perf-users@vger.kernel.org, linux-kselftest@vger.kernel.org, Colton Lewis Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We may want a partitioned PMU but not have FEAT_FGT to untrap the specific registers that would normally be untrapped. Add a handler for those registers in the fast path so we can still get a performance boost from partitioning. The idea is to handle traps for all the PMU registers quickly by writing directly to the hardware when possible instead of hooking into the emulated vPMU as the standard handlers in sys_regs.c do. For registers that can't be written to hardware because they require special handling (PMEVTYPER and PMOVS), write to the virtual register. A later patch will ensure these are handled correctly at vcpu_load time. Signed-off-by: Colton Lewis --- arch/arm64/kvm/hyp/vhe/switch.c | 238 ++++++++++++++++++++++++++++++++ 1 file changed, 238 insertions(+) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switc= h.c index 9db3f11a4754d..154da70146d98 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -28,6 +28,8 @@ #include #include =20 +#include <../../sys_regs.h> + /* VHE specific context */ DEFINE_PER_CPU(struct kvm_host_data, kvm_host_data); DEFINE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); @@ -482,6 +484,239 @@ static bool kvm_hyp_handle_zcr_el2(struct kvm_vcpu *v= cpu, u64 *exit_code) return false; } =20 +/** + * kvm_hyp_handle_pmu_regs() - Fast handler for PMU registers + * @vcpu: Pointer to vcpu struct + * + * This handler immediately writes through certain PMU registers when + * we have a partitioned PMU (that is, MDCR_EL2.HPMN is set to reserve + * a range of counters for the guest) but the machine does not have + * FEAT_FGT to selectively untrap the registers we want. + * + * Return: True if the exception was successfully handled, false otherwise + */ +static bool kvm_hyp_handle_pmu_regs(struct kvm_vcpu *vcpu) +{ + struct sys_reg_params p; + u64 pmuser; + u64 pmselr; + u64 esr; + u64 val; + u64 mask; + u32 sysreg; + u8 nr_cnt; + u8 rt; + u8 idx; + bool ret; + + if (!kvm_vcpu_pmu_is_partitioned(vcpu)) + return false; + + pmuser =3D kvm_vcpu_read_pmuserenr(vcpu); + + if (!(pmuser & ARMV8_PMU_USERENR_EN)) + return false; + + esr =3D kvm_vcpu_get_esr(vcpu); + p =3D esr_sys64_to_params(esr); + sysreg =3D esr_sys64_to_sysreg(esr); + rt =3D kvm_vcpu_sys_get_rt(vcpu); + val =3D vcpu_get_reg(vcpu, rt); + nr_cnt =3D vcpu->kvm->arch.nr_pmu_counters; + + switch (sysreg) { + case SYS_PMCR_EL0: + mask =3D ARMV8_PMU_PMCR_MASK; + + if (p.is_write) { + write_sysreg(val & mask, pmcr_el0); + } else { + mask |=3D ARMV8_PMU_PMCR_N; + val =3D u64_replace_bits( + read_sysreg(pmcr_el0), + nr_cnt, + ARMV8_PMU_PMCR_N); + vcpu_set_reg(vcpu, rt, val & mask); + } + + ret =3D true; + break; + case SYS_PMUSERENR_EL0: + mask =3D ARMV8_PMU_USERENR_MASK; + + if (p.is_write) { + write_sysreg(val & mask, pmuserenr_el0); + } else { + val =3D read_sysreg(pmuserenr_el0); + vcpu_set_reg(vcpu, rt, val & mask); + } + + ret =3D true; + break; + case SYS_PMSELR_EL0: + mask =3D PMSELR_EL0_SEL_MASK; + val &=3D mask; + + if (p.is_write) { + write_sysreg(val & mask, pmselr_el0); + } else { + val =3D read_sysreg(pmselr_el0); + vcpu_set_reg(vcpu, rt, val & mask); + } + ret =3D true; + break; + case SYS_PMINTENCLR_EL1: + mask =3D kvm_pmu_accessible_counter_mask(vcpu); + + if (p.is_write) { + write_sysreg(val & mask, pmintenclr_el1); + } else { + val =3D read_sysreg(pmintenclr_el1); + vcpu_set_reg(vcpu, rt, val & mask); + } + ret =3D true; + + break; + case SYS_PMINTENSET_EL1: + mask =3D kvm_pmu_accessible_counter_mask(vcpu); + + if (p.is_write) { + write_sysreg(val & mask, pmintenset_el1); + } else { + val =3D read_sysreg(pmintenset_el1); + vcpu_set_reg(vcpu, rt, val & mask); + } + + ret =3D true; + break; + case SYS_PMCNTENCLR_EL0: + mask =3D kvm_pmu_accessible_counter_mask(vcpu); + + if (p.is_write) { + write_sysreg(val & mask, pmcntenclr_el0); + } else { + val =3D read_sysreg(pmcntenclr_el0); + vcpu_set_reg(vcpu, rt, val & mask); + } + + ret =3D true; + break; + case SYS_PMCNTENSET_EL0: + mask =3D kvm_pmu_accessible_counter_mask(vcpu); + + if (p.is_write) { + write_sysreg(val & mask, pmcntenset_el0); + } else { + val =3D read_sysreg(pmcntenset_el0); + vcpu_set_reg(vcpu, rt, val & mask); + } + + ret =3D true; + break; + case SYS_PMOVSCLR_EL0: + mask =3D kvm_pmu_accessible_counter_mask(vcpu); + + if (p.is_write) { + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, &=3D, ~(val & mask)); + } else { + val =3D __vcpu_sys_reg(vcpu, PMOVSSET_EL0); + vcpu_set_reg(vcpu, rt, val & mask); + } + + ret =3D true; + break; + case SYS_PMOVSSET_EL0: + mask =3D kvm_pmu_accessible_counter_mask(vcpu); + + if (p.is_write) { + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=3D, val & mask); + } else { + val =3D __vcpu_sys_reg(vcpu, PMOVSSET_EL0); + vcpu_set_reg(vcpu, rt, val & mask); + } + + ret =3D true; + break; + case SYS_PMCCNTR_EL0: + case SYS_PMXEVCNTR_EL0: + case SYS_PMEVCNTRn_EL0(0) ... SYS_PMEVCNTRn_EL0(30): + if (sysreg =3D=3D SYS_PMCCNTR_EL0) + idx =3D ARMV8_PMU_CYCLE_IDX; + else if (sysreg =3D=3D SYS_PMXEVCNTR_EL0) + idx =3D FIELD_GET(PMSELR_EL0_SEL, kvm_vcpu_read_pmselr(vcpu)); + else + idx =3D ((p.CRm & 3) << 3) | (p.Op2 & 7); + + if (idx =3D=3D ARMV8_PMU_CYCLE_IDX && + !(pmuser & ARMV8_PMU_USERENR_CR)) { + ret =3D false; + break; + } else if (!(pmuser & ARMV8_PMU_USERENR_ER)) { + ret =3D false; + break; + } + + if (idx >=3D nr_cnt && idx < ARMV8_PMU_CYCLE_IDX) { + ret =3D false; + break; + } + + pmselr =3D read_sysreg(pmselr_el0); + write_sysreg(idx, pmselr_el0); + + if (p.is_write) { + write_sysreg(val, pmxevcntr_el0); + } else { + val =3D read_sysreg(pmxevcntr_el0); + vcpu_set_reg(vcpu, rt, val); + } + + write_sysreg(pmselr, pmselr_el0); + ret =3D true; + break; + case SYS_PMCCFILTR_EL0: + case SYS_PMXEVTYPER_EL0: + case SYS_PMEVTYPERn_EL0(0) ... SYS_PMEVTYPERn_EL0(30): + if (sysreg =3D=3D SYS_PMCCFILTR_EL0) + idx =3D ARMV8_PMU_CYCLE_IDX; + else if (sysreg =3D=3D SYS_PMXEVTYPER_EL0) + idx =3D FIELD_GET(PMSELR_EL0_SEL, kvm_vcpu_read_pmselr(vcpu)); + else + idx =3D ((p.CRm & 3) << 3) | (p.Op2 & 7); + + if (idx =3D=3D ARMV8_PMU_CYCLE_IDX && + !(pmuser & ARMV8_PMU_USERENR_CR)) { + ret =3D false; + break; + } else if (!(pmuser & ARMV8_PMU_USERENR_ER)) { + ret =3D false; + break; + } + + if (idx >=3D nr_cnt && idx < ARMV8_PMU_CYCLE_IDX) { + ret =3D false; + break; + } + + if (p.is_write) { + __vcpu_assign_sys_reg(vcpu, PMEVTYPER0_EL0 + idx, val); + } else { + val =3D __vcpu_sys_reg(vcpu, PMEVTYPER0_EL0 + idx); + vcpu_set_reg(vcpu, rt, val); + } + + ret =3D true; + break; + default: + ret =3D false; + } + + if (ret) + __kvm_skip_instr(vcpu); + + return ret; +} + static bool kvm_hyp_handle_sysreg_vhe(struct kvm_vcpu *vcpu, u64 *exit_cod= e) { if (kvm_hyp_handle_tlbi_el2(vcpu, exit_code)) @@ -496,6 +731,9 @@ static bool kvm_hyp_handle_sysreg_vhe(struct kvm_vcpu *= vcpu, u64 *exit_code) if (kvm_hyp_handle_zcr_el2(vcpu, exit_code)) return true; =20 + if (kvm_hyp_handle_pmu_regs(vcpu)) + return true; + return kvm_hyp_handle_sysreg(vcpu, exit_code); } =20 --=20 2.53.0.rc2.204.g2597b5adb4-goog