From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 313B12BE038 for ; Wed, 6 Aug 2025 19:57:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510254; cv=none; b=V3re7nvh8dhgcBnzk2vl8D6HGLuMIEz2G2W910fKWUrQUE1CseiRkOiGgkqXrwkNkfL+yAgHy5OQ/LTRuj6gUyWkOFogZDqRIoinD7AC7Y+wrYzcSC9/yyiuzvLe6GcaWUhPMdEjUgjvRlfOExevSpD8zrT5oj+SE9NrZhgYb6M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510254; c=relaxed/simple; bh=N7QiJzpxt/oF3iR/QDRN+b3QtpGh0UIAL2mIbSM/BGQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XDU+X0tsIbxvtKVoI34m3L1tZGiicS8UIyvUwUGe3vsaP8PRHB499jeiHtTm1GCfWw55jNlzXqdr51uNvchxj/eCzIZD85J7wrFRdyY8KiSJo74VAfp2e9zIyqEXdfGSBUtZd2v85+EFKG7d/B+bQgepiBoDH5PPP/gw04LmDAE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0I2BzlDA; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0I2BzlDA" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2400499ab2fso1558725ad.0 for ; Wed, 06 Aug 2025 12:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510250; x=1755115050; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=mLCoApegT2C8Zn1SJf1wYwN1uRlNuEQSrOHklByprgU=; b=0I2BzlDAehUhPPwsBSye1tc0ocBcSeIxtswzpz0wkBDozmBy6sluPTWdo7tEON1oiS 4nMfuPr6EkKELT53PvvgvNvUcNgX4liHt4KVP9M3JScFLIOklFAqAyh1wddMV9Zmvl+6 oaHYlK+vwDGQXwRa7d2DR+3wAPeoszBDc/mRRecZOpB6tqhpZTP0Fr9cGnAhftPUc5eF +cHOzk9Y74RkhYm5+CEMkDkimSF8+OyrXPQ7tgRqDanctChFw55Va8wLLtCWQjlEmXbA Ie+BySPTtx3oSM7HMgV3Pj03B1y8VbQk1mvOMnavvi4K1zbOl4aTL3stYPw8nfQQvLso A92Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510250; x=1755115050; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mLCoApegT2C8Zn1SJf1wYwN1uRlNuEQSrOHklByprgU=; b=dXR7eyD/4arMuoFZz+e+2B6vD8SHIlMePM1UCB+c9ATKMztESBpSonp5Iv6fjbv8fu SkZBqpAPDmTAb1aHHy3SH5pO3ypy6iQufcSI+rsAKx+Dno8w5/QQviAbr6v5xcaRyF4q QhKJHaqYNyI1Qdzn9wZ2N+dlCIHx0eubh0IaRKU/R3RLw8YA6dJisl4CiDy7huykwyD/ 475A8/UVIC1OWPMzS+B+YcsAPjTBZ/iS31jEKbs1QH78s+LaQ6MzXKbjU5vCjfugenAP My9/tGlmQl7rO/NoHdqt9n95qf20t8OW7HJ0h9D3hde0WRYXfHiWUjRnBJXQbdSOonBw MAgw== X-Forwarded-Encrypted: i=1; AJvYcCWiFJGp8Kwu/fc68ehC66ckWbCBIh3AnhSiNBbG3CXtMXPLleit/9jLNuivY7JMo5FJgmkjg10j+mJC0XA=@vger.kernel.org X-Gm-Message-State: AOJu0YzDbRjG98hTM6k6jv+VRg5RWjPsZ/48VCN1X27iXo0yHNgELtZ6 L0ZadjRpYQlB0mbbAJXBhNNvn2U4GTH7moS5fL4D5Ct6wJjcUEql6gPSmCBeFcjQn6UwaEf8g4K cxNDKaQ== X-Google-Smtp-Source: AGHT+IGGAEizgonLm0FsoKt80WWuFMmxvZ9tdXyWc4+Z75ESLdyWsCnJtZKQl7SnHv99v5pbq5U5y7gC/zQ= X-Received: from plq1.prod.google.com ([2002:a17:903:2f81:b0:23f:ecf6:4649]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:144b:b0:240:2970:65e7 with SMTP id d9443c01a7336-2429eebe90emr51929475ad.25.1754510250416; Wed, 06 Aug 2025 12:57:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:23 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-2-seanjc@google.com> Subject: [PATCH v5 01/44] perf: Skip pmu_ctx based on event_type From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang To optimize the cgroup context switch, the perf_event_pmu_context iteration skips the PMUs without cgroup events. A bool cgroup was introduced to indicate the case. It can work, but this way is hard to extend for other cases, e.g. skipping non-mediated PMUs. It doesn't make sense to keep adding bool variables. Pass the event_type instead of the specific bool variable. Check both the event_type and related pmu_ctx variables to decide whether skipping a PMU. Event flags, e.g., EVENT_CGROUP, should be cleard in the ctx->is_active. Add EVENT_FLAGS to indicate such event flags. No functional change. Signed-off-by: Kan Liang Tested-by: Yongwei Ma Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- kernel/events/core.c | 74 ++++++++++++++++++++++++-------------------- 1 file changed, 40 insertions(+), 34 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 22fdf0c187cd..d4528554528d 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -164,7 +164,7 @@ enum event_type_t { /* see ctx_resched() for details */ EVENT_CPU =3D 0x10, EVENT_CGROUP =3D 0x20, - + EVENT_FLAGS =3D EVENT_CGROUP, /* compound helpers */ EVENT_ALL =3D EVENT_FLEXIBLE | EVENT_PINNED, EVENT_TIME_FROZEN =3D EVENT_TIME | EVENT_FROZEN, @@ -778,27 +778,37 @@ do { \ ___p; \ }) =20 -#define for_each_epc(_epc, _ctx, _pmu, _cgroup) \ +static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx, + enum event_type_t event_type) +{ + if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups) + return true; + return false; +} + +#define for_each_epc(_epc, _ctx, _pmu, _event_type) \ list_for_each_entry(_epc, &((_ctx)->pmu_ctx_list), pmu_ctx_entry) \ - if (_cgroup && !_epc->nr_cgroups) \ + if (perf_skip_pmu_ctx(_epc, _event_type)) \ continue; \ else if (_pmu && _epc->pmu !=3D _pmu) \ continue; \ else =20 -static void perf_ctx_disable(struct perf_event_context *ctx, bool cgroup) +static void perf_ctx_disable(struct perf_event_context *ctx, + enum event_type_t event_type) { struct perf_event_pmu_context *pmu_ctx; =20 - for_each_epc(pmu_ctx, ctx, NULL, cgroup) + for_each_epc(pmu_ctx, ctx, NULL, event_type) perf_pmu_disable(pmu_ctx->pmu); } =20 -static void perf_ctx_enable(struct perf_event_context *ctx, bool cgroup) +static void perf_ctx_enable(struct perf_event_context *ctx, + enum event_type_t event_type) { struct perf_event_pmu_context *pmu_ctx; =20 - for_each_epc(pmu_ctx, ctx, NULL, cgroup) + for_each_epc(pmu_ctx, ctx, NULL, event_type) perf_pmu_enable(pmu_ctx->pmu); } =20 @@ -963,8 +973,7 @@ static void perf_cgroup_switch(struct task_struct *task) return; =20 WARN_ON_ONCE(cpuctx->ctx.nr_cgroups =3D=3D 0); - - perf_ctx_disable(&cpuctx->ctx, true); + perf_ctx_disable(&cpuctx->ctx, EVENT_CGROUP); =20 ctx_sched_out(&cpuctx->ctx, NULL, EVENT_ALL|EVENT_CGROUP); /* @@ -980,7 +989,7 @@ static void perf_cgroup_switch(struct task_struct *task) */ ctx_sched_in(&cpuctx->ctx, NULL, EVENT_ALL|EVENT_CGROUP); =20 - perf_ctx_enable(&cpuctx->ctx, true); + perf_ctx_enable(&cpuctx->ctx, EVENT_CGROUP); } =20 static int perf_cgroup_ensure_storage(struct perf_event *event, @@ -2898,11 +2907,11 @@ static void ctx_resched(struct perf_cpu_context *cp= uctx, =20 event_type &=3D EVENT_ALL; =20 - for_each_epc(epc, &cpuctx->ctx, pmu, false) + for_each_epc(epc, &cpuctx->ctx, pmu, 0) perf_pmu_disable(epc->pmu); =20 if (task_ctx) { - for_each_epc(epc, task_ctx, pmu, false) + for_each_epc(epc, task_ctx, pmu, 0) perf_pmu_disable(epc->pmu); =20 task_ctx_sched_out(task_ctx, pmu, event_type); @@ -2922,11 +2931,11 @@ static void ctx_resched(struct perf_cpu_context *cp= uctx, =20 perf_event_sched_in(cpuctx, task_ctx, pmu); =20 - for_each_epc(epc, &cpuctx->ctx, pmu, false) + for_each_epc(epc, &cpuctx->ctx, pmu, 0) perf_pmu_enable(epc->pmu); =20 if (task_ctx) { - for_each_epc(epc, task_ctx, pmu, false) + for_each_epc(epc, task_ctx, pmu, 0) perf_pmu_enable(epc->pmu); } } @@ -3475,11 +3484,10 @@ static void ctx_sched_out(struct perf_event_context *ctx, struct pmu *pmu, enum event_= type_t event_type) { struct perf_cpu_context *cpuctx =3D this_cpu_ptr(&perf_cpu_context); + enum event_type_t active_type =3D event_type & ~EVENT_FLAGS; struct perf_event_pmu_context *pmu_ctx; int is_active =3D ctx->is_active; - bool cgroup =3D event_type & EVENT_CGROUP; =20 - event_type &=3D ~EVENT_CGROUP; =20 lockdep_assert_held(&ctx->lock); =20 @@ -3510,7 +3518,7 @@ ctx_sched_out(struct perf_event_context *ctx, struct = pmu *pmu, enum event_type_t * see __load_acquire() in perf_event_time_now() */ barrier(); - ctx->is_active &=3D ~event_type; + ctx->is_active &=3D ~active_type; =20 if (!(ctx->is_active & EVENT_ALL)) { /* @@ -3531,7 +3539,7 @@ ctx_sched_out(struct perf_event_context *ctx, struct = pmu *pmu, enum event_type_t =20 is_active ^=3D ctx->is_active; /* changed bits */ =20 - for_each_epc(pmu_ctx, ctx, pmu, cgroup) + for_each_epc(pmu_ctx, ctx, pmu, event_type) __pmu_ctx_sched_out(pmu_ctx, is_active); } =20 @@ -3687,7 +3695,7 @@ perf_event_context_sched_out(struct task_struct *task= , struct task_struct *next) raw_spin_lock_nested(&next_ctx->lock, SINGLE_DEPTH_NESTING); if (context_equiv(ctx, next_ctx)) { =20 - perf_ctx_disable(ctx, false); + perf_ctx_disable(ctx, 0); =20 /* PMIs are disabled; ctx->nr_no_switch_fast is stable. */ if (local_read(&ctx->nr_no_switch_fast) || @@ -3711,7 +3719,7 @@ perf_event_context_sched_out(struct task_struct *task= , struct task_struct *next) =20 perf_ctx_sched_task_cb(ctx, task, false); =20 - perf_ctx_enable(ctx, false); + perf_ctx_enable(ctx, 0); =20 /* * RCU_INIT_POINTER here is safe because we've not @@ -3735,13 +3743,13 @@ perf_event_context_sched_out(struct task_struct *ta= sk, struct task_struct *next) =20 if (do_switch) { raw_spin_lock(&ctx->lock); - perf_ctx_disable(ctx, false); + perf_ctx_disable(ctx, 0); =20 inside_switch: perf_ctx_sched_task_cb(ctx, task, false); task_ctx_sched_out(ctx, NULL, EVENT_ALL); =20 - perf_ctx_enable(ctx, false); + perf_ctx_enable(ctx, 0); raw_spin_unlock(&ctx->lock); } } @@ -4050,11 +4058,9 @@ static void ctx_sched_in(struct perf_event_context *ctx, struct pmu *pmu, enum event_t= ype_t event_type) { struct perf_cpu_context *cpuctx =3D this_cpu_ptr(&perf_cpu_context); + enum event_type_t active_type =3D event_type & ~EVENT_FLAGS; struct perf_event_pmu_context *pmu_ctx; int is_active =3D ctx->is_active; - bool cgroup =3D event_type & EVENT_CGROUP; - - event_type &=3D ~EVENT_CGROUP; =20 lockdep_assert_held(&ctx->lock); =20 @@ -4072,7 +4078,7 @@ ctx_sched_in(struct perf_event_context *ctx, struct p= mu *pmu, enum event_type_t barrier(); } =20 - ctx->is_active |=3D (event_type | EVENT_TIME); + ctx->is_active |=3D active_type | EVENT_TIME; if (ctx->task) { if (!(is_active & EVENT_ALL)) cpuctx->task_ctx =3D ctx; @@ -4087,13 +4093,13 @@ ctx_sched_in(struct perf_event_context *ctx, struct= pmu *pmu, enum event_type_t * in order to give them the best chance of going on. */ if (is_active & EVENT_PINNED) { - for_each_epc(pmu_ctx, ctx, pmu, cgroup) + for_each_epc(pmu_ctx, ctx, pmu, event_type) __pmu_ctx_sched_in(pmu_ctx, EVENT_PINNED); } =20 /* Then walk through the lower prio flexible groups */ if (is_active & EVENT_FLEXIBLE) { - for_each_epc(pmu_ctx, ctx, pmu, cgroup) + for_each_epc(pmu_ctx, ctx, pmu, event_type) __pmu_ctx_sched_in(pmu_ctx, EVENT_FLEXIBLE); } } @@ -4110,11 +4116,11 @@ static void perf_event_context_sched_in(struct task= _struct *task) =20 if (cpuctx->task_ctx =3D=3D ctx) { perf_ctx_lock(cpuctx, ctx); - perf_ctx_disable(ctx, false); + perf_ctx_disable(ctx, 0); =20 perf_ctx_sched_task_cb(ctx, task, true); =20 - perf_ctx_enable(ctx, false); + perf_ctx_enable(ctx, 0); perf_ctx_unlock(cpuctx, ctx); goto rcu_unlock; } @@ -4127,7 +4133,7 @@ static void perf_event_context_sched_in(struct task_s= truct *task) if (!ctx->nr_events) goto unlock; =20 - perf_ctx_disable(ctx, false); + perf_ctx_disable(ctx, 0); /* * We want to keep the following priority order: * cpu pinned (that don't need to move), task pinned, @@ -4137,7 +4143,7 @@ static void perf_event_context_sched_in(struct task_s= truct *task) * events, no need to flip the cpuctx's events around. */ if (!RB_EMPTY_ROOT(&ctx->pinned_groups.tree)) { - perf_ctx_disable(&cpuctx->ctx, false); + perf_ctx_disable(&cpuctx->ctx, 0); ctx_sched_out(&cpuctx->ctx, NULL, EVENT_FLEXIBLE); } =20 @@ -4146,9 +4152,9 @@ static void perf_event_context_sched_in(struct task_s= truct *task) perf_ctx_sched_task_cb(cpuctx->task_ctx, task, true); =20 if (!RB_EMPTY_ROOT(&ctx->pinned_groups.tree)) - perf_ctx_enable(&cpuctx->ctx, false); + perf_ctx_enable(&cpuctx->ctx, 0); =20 - perf_ctx_enable(ctx, false); + perf_ctx_enable(ctx, 0); =20 unlock: perf_ctx_unlock(cpuctx, ctx); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 713A22BE656 for ; Wed, 6 Aug 2025 19:57:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510258; cv=none; b=KF4a44mch1F/OEAMAaCbL6r6HTEBrEj4UOPJKztjWXi7tuwAeDTv0CBrC+XQY8dYpHskRffUZ4Wffy5ZMDDf+CJ3dNomdFTbvyEXq4wkPR8kgYawx22GI8Gf9Bvjs8kgG2RI7DbbdfRU2QaSFZnjrkqeGGa0I9KTFu/RHQITlOs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510258; c=relaxed/simple; bh=jUBl1OA0dbcCe9Exxx/O066eDYx/qkeyxg/XjKFA/3U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bm6cyZwp747He5OUuqAhX9Fy3hw6U9u2HU3lw9MaLNJk02AqmM/8rA9xjTQHlpDjU5SyhJLtLP3QcbiuWwxGPQsL4i3u8EAd3UMnvkkQNfR5/JtQRKW1VRQODKTQGx3JLgX/gtGBqxBEyHQ/+SqxJqpTaCfMQL8zw9Xa4Ug1Pp0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=krqFVzHx; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="krqFVzHx" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-76bef13c254so290741b3a.0 for ; Wed, 06 Aug 2025 12:57:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510252; x=1755115052; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3IAofGMjgJOHx6SdMcFjG4jFgSxJevKv5cJ/ErNkHsk=; b=krqFVzHx48sFUhkTmv5ZdxVvJJbo9yybHXbJml4FKAfQUxzOY4jH1x4IujuldA6FMr C0+zX0xKrvTclXv7jhBwPGfFMBwKz14JTJaEdBZ+70bCTbFbTQ5Xcu0limwYhQffnRT1 FCaCbpNmWYehwuLFuoJuSmpfOfKEHjXyqoWRlcCcYUhWpxWox9SHW9jhq5ivmdwOnlOi M8VHe2C3cxVOVnnB16o4ieO9NP8RP8SSHnDvNPO8i3ftSat6W9nnq7VOVBrTtE1bSHEA GW1sz/GYRCc8mL0hoHmzkx1cVT3yt0t/1gWrYDYckq2x7Gw1de223JzLbu7xVcmqkXsS Ng6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510252; x=1755115052; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3IAofGMjgJOHx6SdMcFjG4jFgSxJevKv5cJ/ErNkHsk=; b=fFB+F1ILeyP5dPRiLEcXZGcxNeHSHXCinYWby+YxP+QXmYKoVBhjItyOsoBgiLLua0 ICn6sSi2lRbxf77ETo6JsZNfITR9vhzskOiOpzIVs6bU1w8sq1+U3tEZwEN9jJ1M1JKN a4iFGBLwTJEdOKD2hjeWLPbuW48qNcKDRCfogSQJveK9cGRkM+iPOAugI25EivYpVDBA 60Ti0yyfRwJMkBuHkzsyoNisdktQWjHoETstiU9L5xfJpQ1kT+BD985DBLKC/2B51J7M pVoyjLaFISpR99iy71R2yohVoDqWmoo0k9eLVJCO7/pbZ7NgoxnGtrFSgFtOgqvfhmfO V79Q== X-Forwarded-Encrypted: i=1; AJvYcCWtU8oRxWrHqso5Ek62L8HEEzF8YUXVGYP994IThMdPrzOZu2OO61Ik86rngZlwh79G0vBMcPQn2lrJHug=@vger.kernel.org X-Gm-Message-State: AOJu0YwsHr3hBfYmi9YedIKaf1lW+8VDQyik0lO2K7To1J6S9X69/piX qAjYc5OMs5bXsoyWLMsiZ/HN2ycTJCvcwQwiFUbuUz21jDERzXXu6tXmvI4IsHU1YxnaeArx2Sf xgPVeRw== X-Google-Smtp-Source: AGHT+IGg8m1AYQ1T/bkneerdDiI2IWHPXHDhUcioTumR7faJkpYOSLNXQVSuuMxt1I5dkPrSi8WAHfNQ8i8= X-Received: from pgc3.prod.google.com ([2002:a05:6a02:2f83:b0:b42:2da2:8848]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:a103:b0:23f:f712:4103 with SMTP id adf61e73a8af0-24041364f38mr1167741637.18.1754510252085; Wed, 06 Aug 2025 12:57:32 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:24 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-3-seanjc@google.com> Subject: [PATCH v5 02/44] perf: Add generic exclude_guest support From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang Only KVM knows the exact time when a guest is entering/exiting. Expose two interfaces to KVM to switch the ownership of the PMU resources. All the pinned events must be scheduled in first. Extend the perf_event_sched_in() helper to support extra flag, e.g., EVENT_GUEST. Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- kernel/events/core.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index d4528554528d..3a98e11d8efc 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -2866,14 +2866,15 @@ static void task_ctx_sched_out(struct perf_event_co= ntext *ctx, =20 static void perf_event_sched_in(struct perf_cpu_context *cpuctx, struct perf_event_context *ctx, - struct pmu *pmu) + struct pmu *pmu, + enum event_type_t event_type) { - ctx_sched_in(&cpuctx->ctx, pmu, EVENT_PINNED); + ctx_sched_in(&cpuctx->ctx, pmu, EVENT_PINNED | event_type); if (ctx) - ctx_sched_in(ctx, pmu, EVENT_PINNED); - ctx_sched_in(&cpuctx->ctx, pmu, EVENT_FLEXIBLE); + ctx_sched_in(ctx, pmu, EVENT_PINNED | event_type); + ctx_sched_in(&cpuctx->ctx, pmu, EVENT_FLEXIBLE | event_type); if (ctx) - ctx_sched_in(ctx, pmu, EVENT_FLEXIBLE); + ctx_sched_in(ctx, pmu, EVENT_FLEXIBLE | event_type); } =20 /* @@ -2929,7 +2930,7 @@ static void ctx_resched(struct perf_cpu_context *cpuc= tx, else if (event_type & EVENT_PINNED) ctx_sched_out(&cpuctx->ctx, pmu, EVENT_FLEXIBLE); =20 - perf_event_sched_in(cpuctx, task_ctx, pmu); + perf_event_sched_in(cpuctx, task_ctx, pmu, 0); =20 for_each_epc(epc, &cpuctx->ctx, pmu, 0) perf_pmu_enable(epc->pmu); @@ -4147,7 +4148,7 @@ static void perf_event_context_sched_in(struct task_s= truct *task) ctx_sched_out(&cpuctx->ctx, NULL, EVENT_FLEXIBLE); } =20 - perf_event_sched_in(cpuctx, ctx, NULL); + perf_event_sched_in(cpuctx, ctx, NULL, 0); =20 perf_ctx_sched_task_cb(cpuctx->task_ctx, task, true); =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BABD82BE7B7 for ; Wed, 6 Aug 2025 19:57:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510257; cv=none; b=YpwAsY60BxtvZIR1F5Gd7jYZaKfuvZsh+fJxrWMD9oXrNapYoSAEr1ZwtBeWdBxcdfOFT6ZioYnaKjvFbi+a3U/O4E8R4y1xV+4hCtCdSYi+CUNsqyxz6fT8tfHKnmcOXN3hPRyx449MhRmQCRzXi0bRRWkWKnMrVoLImsLOWu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510257; c=relaxed/simple; bh=tS0WmC5r+kmTtxkhhYKlXDU+egOc68kS1ctzXr8LOA8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=nFu5u8AAIzEg0tYQnaKQ1fDzS6bRImGsNldt+zIdFDI2S/PBE0FgdIY27LZGCcRsjc8ZiA4WpYo1FzDZ7sbcM1YcisCa+LDmv1i+x394aZVNPcz4dREuUsfQX4tI7lDwKFcunazZVLgzXaoYkB3YcnjNubF5JfxKNF9GHOYTpRk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CcqyyqZT; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CcqyyqZT" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b2c36951518so245340a12.2 for ; Wed, 06 Aug 2025 12:57:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510254; x=1755115054; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=tP9GWyHDKyhyNham1AJeXmzoVBEhNFAEzN5Uklpk9I0=; b=CcqyyqZTQ9+tu9RFgIsmhK8g25sm1T5DiiFL+2GVwNSk1syEvZSf6LdcDIqXgOvI41 gQNYjs1UElWpwcKrpQnRV0alfOEx8/MHN4ilTPUYO+bCnxFXUy6TORaL3IytBor011yJ SFPcNAoXHsE51QXzOkrfgJhinKbMsfCLFNJBEzup9ttMWtu2rVjcdDXUHw9IZfNOHCrn EEwrVOklD2vysXLvc6jZM2RUEd71y6yZsUEfBkk+xXpnhQm+++9FWk00CnLwhxk/qLLL pkWA/vG3zqUcofDwL1dD4nWTtxnZUMm2Eg8hJIFeXfLpURozsDlQj4h5tbpD6XmN+IhV YpAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510254; x=1755115054; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tP9GWyHDKyhyNham1AJeXmzoVBEhNFAEzN5Uklpk9I0=; b=YG0vXLF2s/4Jdhe1iiY9gwCeAQ2+xI2TQWurh2WuoetWeXHRolLmWT7t5dvi9ggH/H jzwf+ViuZ62i93aA+Z4j07PxmAgS26sdJmnKrd/i9Dvocha25e3m9A0V777dfrYMxkpS vXvn0CfcLqKdpiCyjcfJ8XSzNMcNYrxGR7/gk51yITDlV1dSQrQQX5CgiwOVHv80CHNf p4/6sg8tPWntaAWj3S5eqwXOPNOsjqWjp/HylV8LiYyT//eSoAkWOliI+p2zosSmbh2v Q1ZGPAmjbaxU+1BD1iSKrg6po6bqM+PkW/YzqjwyppR/pEXYkfzthc1Wx/u3A8mGQpZW oqFw== X-Forwarded-Encrypted: i=1; AJvYcCWlsh/L3hNTs6J6znY9qbv6p9CBBBpVfV49IijiTP0Uye8FJYgHuLRqSD1ynk6W5MDwhkLlgUmPQAK3RIQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yxh4W73rBYnmS0xbBXSUYtcN4KXK30iaXgayFIT5EFIGyBl4Jpj EKFJoF2+c4J9niGJxdTCaCMXoTePofYgfPVWPTcfel9AjN42JsuvpOwcB5f6BBdkifl/vlMCLpK iXdsZjQ== X-Google-Smtp-Source: AGHT+IE0CEB3xtAs9hI54+H7B1IyMBLcepknW0FCY+2mxv/eXxKsKgLhnBmg0Z6EMAhWWlTkD97yQMZQQo8= X-Received: from pgg21.prod.google.com ([2002:a05:6a02:4d95:b0:b42:3ec5:d919]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:5493:b0:220:aea6:2154 with SMTP id adf61e73a8af0-2403125bcacmr7040722637.17.1754510253905; Wed, 06 Aug 2025 12:57:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:25 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-4-seanjc@google.com> Subject: [PATCH v5 03/44] perf: Move security_perf_event_free() call to __free_event() From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the freeing of any security state associated with a perf event from _free_event() to __free_event(), i.e. invoke security_perf_event_free() in the error paths for perf_event_alloc(). This will allow adding potential error paths in perf_event_alloc() that can occur after allocating security state. Note, kfree() and thus security_perf_event_free() is a nop if event->security is NULL, i.e. calling security_perf_event_free() even if security_perf_event_alloc() fails or is never reached is functionality ok. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- kernel/events/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 3a98e11d8efc..1753a97638a3 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -5596,6 +5596,8 @@ static void __free_event(struct perf_event *event) { struct pmu *pmu =3D event->pmu; =20 + security_perf_event_free(event); + if (event->attach_state & PERF_ATTACH_CALLCHAIN) put_callchain_buffers(); =20 @@ -5659,8 +5661,6 @@ static void _free_event(struct perf_event *event) =20 unaccount_event(event); =20 - security_perf_event_free(event); - if (event->rb) { /* * Can happen when we close an event with re-directed output. --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5D0220458A for ; Wed, 6 Aug 2025 19:57:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510261; cv=none; b=DMG8MtyI5jiXjUTldHrEg6nl9PPWpLkLmQaEcBH8leCfRXqDjNS+ZwYUumO6+2M3BWv8z35wSQ4/a+WF+nl2EA0PclH2NtuZ0P+bIla9HtBVZr+gJnnH0xakm4E+NEvAR3ESiVV1/WfrRHBTbaA/IiXNERNKZDx4gBmO8dn6bn0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510261; c=relaxed/simple; bh=UrgGnFtSniCNk74J85bXS6weeSIBAARTVjsD4XCec00=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hDKUgWCKWJPAhCEcYTYU1zB920eT6zoiOFx+WE1kyZBpIMki7UsHXnlCZELZ3MgnHVsdX7bKVwEQmxETPhQTPEk0CuYz/PzBGltrv0Y31UsGI7lRtKUQ3baIEVlBjMpzwLn1d9SCQPeiVOhOXSFOGAU5Mf0ayziEoYu0FHW5pGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2ZMhPiND; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2ZMhPiND" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31332dc2b59so326467a91.0 for ; Wed, 06 Aug 2025 12:57:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510255; x=1755115055; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=rYhmmFtojtIGRiNFVnTHHWtxjI2pG0Wx0DryIldmwBo=; b=2ZMhPiNDAErmnwI2zROGgL8Evr67rVZeucWYoBU1LCBpoSLaePFK67B1/KTPIt6A0q L0W474PI55LKgD+ls3PQ7OY59u8gHQ5LKgD4ZVVwMgPth6ke1rd0m4rKhBCma0piVHu8 6+KyQIacbW20biYZMFHnrBzppzKJvVVFNY4hVLxCl4O7JGXAkVzdGTJsfgOaE4hUeYfO 8Sp/veWbirp7nabFpeK8ovKmyqyRu4gXJ5CdjDW6UNmamzQ6Joqutt93IqG/gYwaT9R/ kL2myaYT2pWAHEwpVtPHzYqF3BjkRpL3AyuXfC4nBmGgZZRUDosfj2AC8B2fCaOsoJFR VMzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510255; x=1755115055; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=rYhmmFtojtIGRiNFVnTHHWtxjI2pG0Wx0DryIldmwBo=; b=QgTDjlqTvQVf6I8R/jWuB2i0WB9SK9VCddJ8mJuLpK2+fooAZdfhEEhL5KyWZ/2QZC a/94GVidxBad80kXcKoU9FfeUXR1fXQUkQ2bDMTT4OTFfXNBEfe8NXXVsyw/5LjiRy4Z M6F1xWPlSjWHZ+tCM1FMFgwMySAbLBrnTMA3tGG99YtvcnKLyqi9bbl1Ck2dqLuD2KOG GzR+UAJ7x0VmTYROs/5gM8/IG6XfPSHypbSbuEv4RVNf0ESewTh2ek4rGkwixn0mZ99+ B7fqE4m7+a0mQ5cpX9kgOrZrD+K3tDiE3N/MJKa8k7rWkY2ru0sCbzEgGH00Paqt/FIH SP5Q== X-Forwarded-Encrypted: i=1; AJvYcCVJN9Tp+rVLc4fnkrGP9BloDmOK6AiFCfk4dA0+8T8mHcbb7oy2JN5ZIR2rDeyN5Soh53Oa0i9Uk7zKzcg=@vger.kernel.org X-Gm-Message-State: AOJu0Yx+fRv0u/yNxEEscnkh7Mk0DDqofJx6la+m8y10kT8kdpaLzk9V akVObXbkONr3IOWS2s1+V1fZZXNY6IIrLD1b8eEKn6zEtopzenATqi7cqSx9HlIr7LuaRsvezKx Ugf6w5w== X-Google-Smtp-Source: AGHT+IE1JXMzx+3+mEhB/aRAtKWF6tsabKOHfV9vXr8D6RhEuIeRQytpFLmD0EN42AoRsIYuN8o0xUeAxdE= X-Received: from pjbso3.prod.google.com ([2002:a17:90b:1f83:b0:311:1a09:11ff]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4ac1:b0:31e:f193:1822 with SMTP id 98e67ed59e1d1-32166ccac82mr5176638a91.28.1754510255466; Wed, 06 Aug 2025 12:57:35 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:26 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-5-seanjc@google.com> Subject: [PATCH v5 04/44] perf: Add APIs to create/release mediated guest vPMUs From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang Currently, exposing PMU capabilities to a KVM guest is done by emulating guest PMCs via host perf events, i.e. by having KVM be "just" another user of perf. As a result, the guest and host are effectively competing for resources, and emulating guest accesses to vPMU resources requires expensive actions (expensive relative to the native instruction). The overhead and resource competition results in degraded guest performance and ultimately very poor vPMU accuracy. To address the issues with the perf-emulated vPMU, introduce a "mediated vPMU", where the data plane (PMCs and enable/disable knobs) is exposed directly to the guest, but the control plane (event selectors and access to fixed counters) is managed by KVM (via MSR interceptions). To allow host perf usage of the PMU to (partially) co-exist with KVM/guest usage of the PMU, KVM and perf will coordinate to a world switch between host perf context and guest vPMU context near VM-Enter/VM-Exit. Add two exported APIs, perf_{create,release}_mediated_pmu(), to allow KVM to create and release a mediated PMU instance (per VM). Because host perf context will be deactivated while the guest is running, mediated PMU usage will be mutually exclusive with perf analysis of the guest, i.e. perf events that do NOT exclude the guest will not behave as expected. To avoid silent failure of !exclude_guest perf events, disallow creating a mediated PMU if there are active !exclude_guest events, and on the perf side, disallowing creating new !exclude_guest perf events while there is at least one active mediated PMU. Exempt PMU resources that do not support mediated PMU usage, i.e. that are outside the scope/view of KVM's vPMU and will not be swapped out while the guest is running. Guard mediated PMU with a new kconfig to help readers identify code paths that are unique to mediated PMU support, and to allow for adding arch- specific hooks without stubs. KVM x86 is expected to be the only KVM architecture to support a mediated PMU in the near future (e.g. arm64 is trending toward a partitioned PMU implementation), and KVM x86 will select PERF_GUEST_MEDIATED_PMU unconditionally, i.e. won't need stubs. Immediately select PERF_GUEST_MEDIATED_PMU when KVM x86 is enabled so that all paths are compile tested. Full KVM support is on its way... Suggested-by: Sean Christopherson Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang [sean: add kconfig and WARNing, rewrite changelog, swizzle patch ordering] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/Kconfig | 1 + include/linux/perf_event.h | 6 +++ init/Kconfig | 4 ++ kernel/events/core.c | 82 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 93 insertions(+) diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 2c86673155c9..ee67357b5e36 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -37,6 +37,7 @@ config KVM_X86 select SCHED_INFO select PERF_EVENTS select GUEST_PERF_EVENTS + select PERF_GUEST_MEDIATED_PMU select HAVE_KVM_MSI select HAVE_KVM_CPU_RELAX_INTERCEPT select HAVE_KVM_NO_POLL diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index ec9d96025683..63097beb5f02 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -305,6 +305,7 @@ struct perf_event_pmu_context; #define PERF_PMU_CAP_EXTENDED_HW_TYPE 0x0100 #define PERF_PMU_CAP_AUX_PAUSE 0x0200 #define PERF_PMU_CAP_AUX_PREFER_LARGE 0x0400 +#define PERF_PMU_CAP_MEDIATED_VPMU 0x0800 =20 /** * pmu::scope @@ -1914,6 +1915,11 @@ extern int perf_event_account_interrupt(struct perf_= event *event); extern int perf_event_period(struct perf_event *event, u64 value); extern u64 perf_event_pause(struct perf_event *event, bool reset); =20 +#ifdef CONFIG_PERF_GUEST_MEDIATED_PMU +int perf_create_mediated_pmu(void); +void perf_release_mediated_pmu(void); +#endif + #else /* !CONFIG_PERF_EVENTS: */ =20 static inline void * diff --git a/init/Kconfig b/init/Kconfig index 666783eb50ab..1e3c90c3f24f 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1955,6 +1955,10 @@ config GUEST_PERF_EVENTS bool depends on HAVE_PERF_EVENTS =20 +config PERF_GUEST_MEDIATED_PMU + bool + depends on GUEST_PERF_EVENTS + config PERF_USE_VMALLOC bool help diff --git a/kernel/events/core.c b/kernel/events/core.c index 1753a97638a3..bf0347231bd9 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -5651,6 +5651,8 @@ static void __free_event(struct perf_event *event) call_rcu(&event->rcu_head, free_event_rcu); } =20 +static void mediated_pmu_unaccount_event(struct perf_event *event); + DEFINE_FREE(__free_event, struct perf_event *, if (_T) __free_event(_T)) =20 /* vs perf_event_alloc() success */ @@ -5660,6 +5662,7 @@ static void _free_event(struct perf_event *event) irq_work_sync(&event->pending_disable_irq); =20 unaccount_event(event); + mediated_pmu_unaccount_event(event); =20 if (event->rb) { /* @@ -6182,6 +6185,81 @@ u64 perf_event_pause(struct perf_event *event, bool = reset) } EXPORT_SYMBOL_GPL(perf_event_pause); =20 +#ifdef CONFIG_PERF_GUEST_MEDIATED_PMU +static atomic_t nr_include_guest_events __read_mostly; + +static atomic_t nr_mediated_pmu_vms __read_mostly; +static DEFINE_MUTEX(perf_mediated_pmu_mutex); + +/* !exclude_guest event of PMU with PERF_PMU_CAP_MEDIATED_VPMU */ +static inline bool is_include_guest_event(struct perf_event *event) +{ + if ((event->pmu->capabilities & PERF_PMU_CAP_MEDIATED_VPMU) && + !event->attr.exclude_guest) + return true; + + return false; +} + +static int mediated_pmu_account_event(struct perf_event *event) +{ + if (!is_include_guest_event(event)) + return 0; + + guard(mutex)(&perf_mediated_pmu_mutex); + + if (atomic_read(&nr_mediated_pmu_vms)) + return -EOPNOTSUPP; + + atomic_inc(&nr_include_guest_events); + return 0; +} + +static void mediated_pmu_unaccount_event(struct perf_event *event) +{ + if (!is_include_guest_event(event)) + return; + + atomic_dec(&nr_include_guest_events); +} + +/* + * Currently invoked at VM creation to + * - Check whether there are existing !exclude_guest events of PMU with + * PERF_PMU_CAP_MEDIATED_VPMU + * - Set nr_mediated_pmu_vms to prevent !exclude_guest event creation on + * PMUs with PERF_PMU_CAP_MEDIATED_VPMU + * + * No impact for the PMU without PERF_PMU_CAP_MEDIATED_VPMU. The perf + * still owns all the PMU resources. + */ +int perf_create_mediated_pmu(void) +{ + guard(mutex)(&perf_mediated_pmu_mutex); + if (atomic_inc_not_zero(&nr_mediated_pmu_vms)) + return 0; + + if (atomic_read(&nr_include_guest_events)) + return -EBUSY; + + atomic_inc(&nr_mediated_pmu_vms); + return 0; +} +EXPORT_SYMBOL_GPL(perf_create_mediated_pmu); + +void perf_release_mediated_pmu(void) +{ + if (WARN_ON_ONCE(!atomic_read(&nr_mediated_pmu_vms))) + return; + + atomic_dec(&nr_mediated_pmu_vms); +} +EXPORT_SYMBOL_GPL(perf_release_mediated_pmu); +#else +static int mediated_pmu_account_event(struct perf_event *event) { return 0= ; } +static void mediated_pmu_unaccount_event(struct perf_event *event) {} +#endif + /* * Holding the top-level event's child_mutex means that any * descendant process that has inherited this event will block @@ -13024,6 +13102,10 @@ perf_event_alloc(struct perf_event_attr *attr, int= cpu, if (err) return ERR_PTR(err); =20 + err =3D mediated_pmu_account_event(event); + if (err) + return ERR_PTR(err); + /* symmetric to unaccount_event() in _free_event() */ account_event(event); =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8206B2BEFFE for ; Wed, 6 Aug 2025 19:57:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510264; cv=none; b=FXS/KeeQOgChqHr4KcHYx+hYYKiZwNs03zi/4/FN50muaKKYX8NkBDVWqgzJjeK3dlDsz9pb7I1WPS/qOfjanZK99kooccBnqcwX4JU+lhxyCW/5k0g/M0RlKXa18GTVSu+n/g+qQMF9j6rWm98Z+H11gis5h7HYIRx3/to66cc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510264; c=relaxed/simple; bh=YYCe+4eLi35qElSGVX2EX5EoBQNPxGJxcMCI2Jk+YMU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hfi0KkoKmyJj5HOZ+zw7ylGyuwBHygBYJ0+zXHlD9dvXkgqsVLYwXsRjE/Gl3FPP6qJ4kfxjr+oQZsBm6sxzSZbuCNlmpCXbKlVgyqN0VADx4gHl80xCtqngFgROh9O/ESKsJbREM34lHmVtdIq1qwsftn8z4Mb7SgcIlZXwRPE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nccQ4gdO; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nccQ4gdO" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b423aba05feso137872a12.3 for ; Wed, 06 Aug 2025 12:57:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510258; x=1755115058; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RSqs7LjEbfI/o+XH+tbZdSeN9oFJhrETU4UrZ2e2+po=; b=nccQ4gdOx7uh0LSzRCCErY+WkYMUzWQPyrPzXgmeXUaMjpScuM3iuk/GWTr0JX77nQ 5eQOmW1WR/uWMWWxAq61fCwW5nb7+XcHa7PVTM7kTNF+w7OC75pNZLjGcfzp9MwHlvdd w61Hkzll6izQ1sfUNgiBjPAKnhSryGNtu0bYAmKuRjtH42myVvlhuOHqp6bxhv1MzExD hOlK92XLSXM6J1o9LYiNlvPY1wIbiMtQ4TusJ0a/803hrk9Rty/PzmEDcQtGE8G+RvUJ E++/mMN+0MHeEPSUSpjJOvZD2gBDZzKVJelAX+6ZGkKN0ccZhqLa5LMZW+3b9IQn8Sj9 w/QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510258; x=1755115058; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RSqs7LjEbfI/o+XH+tbZdSeN9oFJhrETU4UrZ2e2+po=; b=Q4NA+wUz5RSIAUxMZfxp2bOx0OB6bikQrGY+l4+ab368HzPMa3DlcYDb22CQ7FMmdp ANFKW5gm3EIRgJBzd/TsO1fLL3gTGJvpAXruz8cJNDLm5HTvffebCrwcUuBffvyyEGE4 jX5K/78OAaDe25p8bdPeuzghxEvEeRt6y+7IdlfrVGurkZVvFpyE5ym6OpP7DZsR6n9N F/POzDcfONPocHfVKDteSvkuuCoFK6G1thke2Xb9NKhUVOEO3IXtbZGUEICrojQiLzv2 EA6dQmaV/7N0dOfTpwGoZ8Y8NzDAKNO1kyRNOa/ZqeQX4R+cyPHyANRQIJW8zr3sKE+q V2lQ== X-Forwarded-Encrypted: i=1; AJvYcCVt+sWkcIds7zdi2cr7kyd2TgN8CpH6UgcJvuE4x7N4qfyAFhohD5Uuj4FXDKzWdW8iA/i6kvDdyvkoPy4=@vger.kernel.org X-Gm-Message-State: AOJu0Yx8MBwVCMLjvnpdU4KpMVKQ5A+p+PAj2FkoKHxSQjmokonILKTj Yf2gQNYuWjzTNt1J2OLe3O/hYBpkzv2bk0zGTLT+sb/p3oq/CmXbo9p+yPF9Q3DvXypZdrkLCMf QD16ZEQ== X-Google-Smtp-Source: AGHT+IHRaSY+k4TAZbl8ZMQSCfgVDMYJiwgIqFCqRbteQRg9trtxOd4uQTojLJyQdeJPXNfHx0P6bSsh+7Q= X-Received: from pjqq16.prod.google.com ([2002:a17:90b:5850:b0:31c:2fe4:33bd]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1c10:b0:311:ef19:824d with SMTP id 98e67ed59e1d1-32166df97d9mr4595576a91.2.1754510257253; Wed, 06 Aug 2025 12:57:37 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:27 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-6-seanjc@google.com> Subject: [PATCH v5 05/44] perf: Clean up perf ctx time From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang The current perf tracks two timestamps for the normal ctx and cgroup. The same type of variables and similar codes are used to track the timestamps. In the following patch, the third timestamp to track the guest time will be introduced. To avoid the code duplication, add a new struct perf_time_ctx and factor out a generic function update_perf_time_ctx(). No functional change. Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- include/linux/perf_event.h | 13 +++---- kernel/events/core.c | 70 +++++++++++++++++--------------------- 2 files changed, 39 insertions(+), 44 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 63097beb5f02..186a026fb7fb 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -999,6 +999,11 @@ struct perf_event_groups { u64 index; }; =20 +struct perf_time_ctx { + u64 time; + u64 stamp; + u64 offset; +}; =20 /** * struct perf_event_context - event context structure @@ -1037,9 +1042,7 @@ struct perf_event_context { /* * Context clock, runs when context enabled. */ - u64 time; - u64 timestamp; - u64 timeoffset; + struct perf_time_ctx time; =20 /* * These fields let us detect when two contexts have both @@ -1172,9 +1175,7 @@ struct bpf_perf_event_data_kern { * This is a per-cpu dynamically allocated data structure. */ struct perf_cgroup_info { - u64 time; - u64 timestamp; - u64 timeoffset; + struct perf_time_ctx time; int active; }; =20 diff --git a/kernel/events/core.c b/kernel/events/core.c index bf0347231bd9..2cf988bdabf0 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -815,6 +815,24 @@ static void perf_ctx_enable(struct perf_event_context = *ctx, static void ctx_sched_out(struct perf_event_context *ctx, struct pmu *pmu,= enum event_type_t event_type); static void ctx_sched_in(struct perf_event_context *ctx, struct pmu *pmu, = enum event_type_t event_type); =20 +static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 no= w, bool adv) +{ + if (adv) + time->time +=3D now - time->stamp; + time->stamp =3D now; + + /* + * The above: time' =3D time + (now - timestamp), can be re-arranged + * into: time` =3D now + (time - timestamp), which gives a single value + * offset to compute future time without locks on. + * + * See perf_event_time_now(), which can be used from NMI context where + * it's (obviously) not possible to acquire ctx->lock in order to read + * both the above values in a consistent manner. + */ + WRITE_ONCE(time->offset, time->time - time->stamp); +} + #ifdef CONFIG_CGROUP_PERF =20 static inline bool @@ -856,7 +874,7 @@ static inline u64 perf_cgroup_event_time(struct perf_ev= ent *event) struct perf_cgroup_info *t; =20 t =3D per_cpu_ptr(event->cgrp->info, event->cpu); - return t->time; + return t->time.time; } =20 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64= now) @@ -865,22 +883,11 @@ static inline u64 perf_cgroup_event_time_now(struct p= erf_event *event, u64 now) =20 t =3D per_cpu_ptr(event->cgrp->info, event->cpu); if (!__load_acquire(&t->active)) - return t->time; - now +=3D READ_ONCE(t->timeoffset); + return t->time.time; + now +=3D READ_ONCE(t->time.offset); return now; } =20 -static inline void __update_cgrp_time(struct perf_cgroup_info *info, u64 n= ow, bool adv) -{ - if (adv) - info->time +=3D now - info->timestamp; - info->timestamp =3D now; - /* - * see update_context_time() - */ - WRITE_ONCE(info->timeoffset, info->time - info->timestamp); -} - static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *c= puctx, bool final) { struct perf_cgroup *cgrp =3D cpuctx->cgrp; @@ -894,7 +901,7 @@ static inline void update_cgrp_time_from_cpuctx(struct = perf_cpu_context *cpuctx, cgrp =3D container_of(css, struct perf_cgroup, css); info =3D this_cpu_ptr(cgrp->info); =20 - __update_cgrp_time(info, now, true); + update_perf_time_ctx(&info->time, now, true); if (final) __store_release(&info->active, 0); } @@ -917,7 +924,7 @@ static inline void update_cgrp_time_from_event(struct p= erf_event *event) * Do not update time when cgroup is not active */ if (info->active) - __update_cgrp_time(info, perf_clock(), true); + update_perf_time_ctx(&info->time, perf_clock(), true); } =20 static inline void @@ -941,7 +948,7 @@ perf_cgroup_set_timestamp(struct perf_cpu_context *cpuc= tx) for (css =3D &cgrp->css; css; css =3D css->parent) { cgrp =3D container_of(css, struct perf_cgroup, css); info =3D this_cpu_ptr(cgrp->info); - __update_cgrp_time(info, ctx->timestamp, false); + update_perf_time_ctx(&info->time, ctx->time.stamp, false); __store_release(&info->active, 1); } } @@ -1562,20 +1569,7 @@ static void __update_context_time(struct perf_event_= context *ctx, bool adv) =20 lockdep_assert_held(&ctx->lock); =20 - if (adv) - ctx->time +=3D now - ctx->timestamp; - ctx->timestamp =3D now; - - /* - * The above: time' =3D time + (now - timestamp), can be re-arranged - * into: time` =3D now + (time - timestamp), which gives a single value - * offset to compute future time without locks on. - * - * See perf_event_time_now(), which can be used from NMI context where - * it's (obviously) not possible to acquire ctx->lock in order to read - * both the above values in a consistent manner. - */ - WRITE_ONCE(ctx->timeoffset, ctx->time - ctx->timestamp); + update_perf_time_ctx(&ctx->time, now, adv); } =20 static void update_context_time(struct perf_event_context *ctx) @@ -1593,7 +1587,7 @@ static u64 perf_event_time(struct perf_event *event) if (is_cgroup_event(event)) return perf_cgroup_event_time(event); =20 - return ctx->time; + return ctx->time.time; } =20 static u64 perf_event_time_now(struct perf_event *event, u64 now) @@ -1607,9 +1601,9 @@ static u64 perf_event_time_now(struct perf_event *eve= nt, u64 now) return perf_cgroup_event_time_now(event, now); =20 if (!(__load_acquire(&ctx->is_active) & EVENT_TIME)) - return ctx->time; + return ctx->time.time; =20 - now +=3D READ_ONCE(ctx->timeoffset); + now +=3D READ_ONCE(ctx->time.offset); return now; } =20 @@ -11991,7 +11985,7 @@ static void task_clock_event_update(struct perf_eve= nt *event, u64 now) =20 static void task_clock_event_start(struct perf_event *event, int flags) { - local64_set(&event->hw.prev_count, event->ctx->time); + local64_set(&event->hw.prev_count, event->ctx->time.time); perf_swevent_start_hrtimer(event); } =20 @@ -11999,7 +11993,7 @@ static void task_clock_event_stop(struct perf_event= *event, int flags) { perf_swevent_cancel_hrtimer(event); if (flags & PERF_EF_UPDATE) - task_clock_event_update(event, event->ctx->time); + task_clock_event_update(event, event->ctx->time.time); } =20 static int task_clock_event_add(struct perf_event *event, int flags) @@ -12019,8 +12013,8 @@ static void task_clock_event_del(struct perf_event = *event, int flags) static void task_clock_event_read(struct perf_event *event) { u64 now =3D perf_clock(); - u64 delta =3D now - event->ctx->timestamp; - u64 time =3D event->ctx->time + delta; + u64 delta =3D now - event->ctx->time.stamp; + u64 time =3D event->ctx->time.time + delta; =20 task_clock_event_update(event, time); } --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FC9E2BF3CA for ; Wed, 6 Aug 2025 19:57:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510265; cv=none; b=V9loKhMM27Xa7c9oxnO6gDJcIUBEggYdolWLh39RO3wXOx8uxG2SlhZY+8+f1L+77nFJBjSt7N0RHkYNy9WmIPt6FFfvc6qWzEHyB8k1+LuRQO4HTMQJ+AVXRU25wEOGQULvvF+sGt8rV+XTuS0GXZH4gqbUVoEsnNNi2vJa/aY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510265; c=relaxed/simple; bh=qXNnxwAFJGaumhjqTu3EpzzvDhsbJFUwsj0x4yAhN4E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Avj1vnROqpdi++JLG1GNexmeEdu3BMNuFl6gRl6m3VPrMz2MefgUDn20qphHrBtjwsG4WIllqpTvdsWX0SIpsG6QwCTR+m0CO0gFdTskTnZFPKZnG6J0A85ZR/EDBe0KMCJs7m5gLfdPwJo9vbXPSWT/rftE4mz/n7mZFp3elgk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TPqZJ7Ie; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TPqZJ7Ie" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31f65d519d3so468655a91.2 for ; Wed, 06 Aug 2025 12:57:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510260; x=1755115060; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=u7P8yyV+9ZMI+BI4Iw55kZCbOndqPSIcTJXU+Tv7Czo=; b=TPqZJ7IeYpGSjV/Lz336x9Y7dRKRKQZyJCJjNUR9HkqxQRR/Sa4SBvc/WH0ZThEdNk KAoB5RqRlrsPh5T3paBJdvRjBp4xJnt/RKvO/tHlMJ0f0pEbH4XKBDtOJSkJ/jSquQoi 6IqZxO5Mc/GuyHDHIDFgzuixUlfeCO4vQ0UaP6dXPhkzKgx4xJUxP5tDvJ46J1CibkOs uW6lLX7gXMbV/sBeYzc/cXYmcq+emp/H9Mrxpcu1GViFrVSGDkOD8Lm4/a1k8QYN5/u8 6ZHjkEYDUVk4HMQ1gHQNdZJMOypmlasF7Q580ABOfh+FU8Y8ITBgvTD4U2uPuYRWr9NY 9tnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510260; x=1755115060; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=u7P8yyV+9ZMI+BI4Iw55kZCbOndqPSIcTJXU+Tv7Czo=; b=fyVrRpE2esm62OAkO2zlHJUlByfv5dOVHVsuFlttqqGVIrd8pFjEhm2gGN76t6bEcG 7uxS6aZhHkCpAhEH1BbkEnuyKMXo5lD8Rg0vFSdjYW/waqSr4npS9oZFj+e5PKuJdYwV y3cIr8Jqdc2gwBxqJEV0kbM2mcP+NK3OWfzuepX0AoQnuvGEPncjl6YhUOJkcGTtk9U0 YwX0DdpNX7WbZjG0g1+9eF2YSA5f3+DOAk7Fm9JvtYqzEzZZSChQBvJgA2sF/XraR5iM B4AruuYzmHhHZ1j7tG51ENp90bSn48xkk3nArs48u8ckcKMRwy1cOI9lCXmIKSFqnvv+ pavg== X-Forwarded-Encrypted: i=1; AJvYcCVaeGLh0NzMCWs4efeYqVJ9LZIXwbeSGyliZBNV6ofCqF4TaZSZM2W3TRrIBsfTNlwLnJD7x6pIjIKm73w=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4DkCTqwEOHM5Es8ial4N6iFKl+FInbymrn03Ci+osms1NBOk4 rnd3KmWIA3wCtyNz3SvUN3ao6pZsdqwA6/cWtLBD17VysfGgcnfdfEi4LiTtx3BhgWnWU+gGPQs P0nuRqA== X-Google-Smtp-Source: AGHT+IG1ZiDUo/Q9d3HEUVIEffrei/1g0+Hi4k9oLFsDUgDDE2+7uLClBEV/7iTHQOJOUOiDxrBymUDpBDM= X-Received: from pjob20.prod.google.com ([2002:a17:90a:8c94:b0:31f:a0:fad4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1d46:b0:31a:9004:899d with SMTP id 98e67ed59e1d1-32166ca4402mr6079864a91.18.1754510259792; Wed, 06 Aug 2025 12:57:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:28 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-7-seanjc@google.com> Subject: [PATCH v5 06/44] perf: Add a EVENT_GUEST flag From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem with the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to an exclude_guest event when a guest is running (Intel way), or set the corresponding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running. However, either way doesn't work with the introduced mediated vPMU. A guest owns all the PMU counters when it's running. The host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwritten. Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest. It's possible that an exclude_guest event is created when a guest is running. The new event should not be scheduled in as well. The ctx time is shared among different PMUs. The time cannot be stopped when a guest is running. It is required to calculate the time for events from other PMUs, e.g., uncore events. Add timeguest to track the guest run time. For an exclude_guest event, the elapsed time equals the ctx time - guest time. Cgroup has dedicated times. Use the same method to deduct the guest time from the cgroup time as well. Co-developed-by: Peter Zijlstra (Intel) Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang [sean: massage comments] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- include/linux/perf_event.h | 6 + kernel/events/core.c | 232 ++++++++++++++++++++++++++++--------- 2 files changed, 186 insertions(+), 52 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 186a026fb7fb..0958b6d0a61c 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1044,6 +1044,11 @@ struct perf_event_context { */ struct perf_time_ctx time; =20 + /* + * Context clock, runs when in the guest mode. + */ + struct perf_time_ctx timeguest; + /* * These fields let us detect when two contexts have both * been cloned (inherited) from a common ancestor. @@ -1176,6 +1181,7 @@ struct bpf_perf_event_data_kern { */ struct perf_cgroup_info { struct perf_time_ctx time; + struct perf_time_ctx timeguest; int active; }; =20 diff --git a/kernel/events/core.c b/kernel/events/core.c index 2cf988bdabf0..6875b56ddd6b 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -164,7 +164,19 @@ enum event_type_t { /* see ctx_resched() for details */ EVENT_CPU =3D 0x10, EVENT_CGROUP =3D 0x20, - EVENT_FLAGS =3D EVENT_CGROUP, + + /* + * EVENT_GUEST is set when scheduling in/out events between the host + * and a guest with a mediated vPMU. Among other things, EVENT_GUEST + * is used: + * + * - In for_each_epc() to skip PMUs that don't support events in a + * MEDIATED_VPMU guest, i.e. don't need to be context switched. + * - To indicate the start/end point of the events in a guest. Guest + * running time is deducted for host-only (exclude_guest) events. + */ + EVENT_GUEST =3D 0x40, + EVENT_FLAGS =3D EVENT_CGROUP | EVENT_GUEST, /* compound helpers */ EVENT_ALL =3D EVENT_FLEXIBLE | EVENT_PINNED, EVENT_TIME_FROZEN =3D EVENT_TIME | EVENT_FROZEN, @@ -457,6 +469,11 @@ static cpumask_var_t perf_online_pkg_mask; static cpumask_var_t perf_online_sys_mask; static struct kmem_cache *perf_event_cache; =20 +static __always_inline bool is_guest_mediated_pmu_loaded(void) +{ + return false; +} + /* * perf event paranoia level: * -1 - not paranoid at all @@ -783,6 +800,9 @@ static bool perf_skip_pmu_ctx(struct perf_event_pmu_con= text *pmu_ctx, { if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups) return true; + if ((event_type & EVENT_GUEST) && + !(pmu_ctx->pmu->capabilities & PERF_PMU_CAP_MEDIATED_VPMU)) + return true; return false; } =20 @@ -833,6 +853,39 @@ static inline void update_perf_time_ctx(struct perf_ti= me_ctx *time, u64 now, boo WRITE_ONCE(time->offset, time->time - time->stamp); } =20 +static_assert(offsetof(struct perf_event_context, timeguest) - + offsetof(struct perf_event_context, time) =3D=3D + sizeof(struct perf_time_ctx)); + +#define T_TOTAL 0 +#define T_GUEST 1 + +static inline u64 __perf_event_time_ctx(struct perf_event *event, + struct perf_time_ctx *times) +{ + u64 time =3D times[T_TOTAL].time; + + if (event->attr.exclude_guest) + time -=3D times[T_GUEST].time; + + return time; +} + +static inline u64 __perf_event_time_ctx_now(struct perf_event *event, + struct perf_time_ctx *times, + u64 now) +{ + if (is_guest_mediated_pmu_loaded() && event->attr.exclude_guest) { + /* + * (now + times[total].offset) - (now + times[guest].offset) :=3D + * times[total].offset - times[guest].offset + */ + return READ_ONCE(times[T_TOTAL].offset) - READ_ONCE(times[T_GUEST].offse= t); + } + + return now + READ_ONCE(times[T_TOTAL].offset); +} + #ifdef CONFIG_CGROUP_PERF =20 static inline bool @@ -869,12 +922,16 @@ static inline int is_cgroup_event(struct perf_event *= event) return event->cgrp !=3D NULL; } =20 +static_assert(offsetof(struct perf_cgroup_info, timeguest) - + offsetof(struct perf_cgroup_info, time) =3D=3D + sizeof(struct perf_time_ctx)); + static inline u64 perf_cgroup_event_time(struct perf_event *event) { struct perf_cgroup_info *t; =20 t =3D per_cpu_ptr(event->cgrp->info, event->cpu); - return t->time.time; + return __perf_event_time_ctx(event, &t->time); } =20 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64= now) @@ -883,9 +940,21 @@ static inline u64 perf_cgroup_event_time_now(struct pe= rf_event *event, u64 now) =20 t =3D per_cpu_ptr(event->cgrp->info, event->cpu); if (!__load_acquire(&t->active)) - return t->time.time; - now +=3D READ_ONCE(t->time.offset); - return now; + return __perf_event_time_ctx(event, &t->time); + + return __perf_event_time_ctx_now(event, &t->time, now); +} + +static inline void __update_cgrp_guest_time(struct perf_cgroup_info *info,= u64 now, bool adv) +{ + update_perf_time_ctx(&info->timeguest, now, adv); +} + +static inline void update_cgrp_time(struct perf_cgroup_info *info, u64 now) +{ + update_perf_time_ctx(&info->time, now, true); + if (is_guest_mediated_pmu_loaded()) + __update_cgrp_guest_time(info, now, true); } =20 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *c= puctx, bool final) @@ -901,7 +970,7 @@ static inline void update_cgrp_time_from_cpuctx(struct = perf_cpu_context *cpuctx, cgrp =3D container_of(css, struct perf_cgroup, css); info =3D this_cpu_ptr(cgrp->info); =20 - update_perf_time_ctx(&info->time, now, true); + update_cgrp_time(info, now); if (final) __store_release(&info->active, 0); } @@ -924,11 +993,11 @@ static inline void update_cgrp_time_from_event(struct= perf_event *event) * Do not update time when cgroup is not active */ if (info->active) - update_perf_time_ctx(&info->time, perf_clock(), true); + update_cgrp_time(info, perf_clock()); } =20 static inline void -perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx) +perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest) { struct perf_event_context *ctx =3D &cpuctx->ctx; struct perf_cgroup *cgrp =3D cpuctx->cgrp; @@ -948,8 +1017,12 @@ perf_cgroup_set_timestamp(struct perf_cpu_context *cp= uctx) for (css =3D &cgrp->css; css; css =3D css->parent) { cgrp =3D container_of(css, struct perf_cgroup, css); info =3D this_cpu_ptr(cgrp->info); - update_perf_time_ctx(&info->time, ctx->time.stamp, false); - __store_release(&info->active, 1); + if (guest) { + __update_cgrp_guest_time(info, ctx->time.stamp, false); + } else { + update_perf_time_ctx(&info->time, ctx->time.stamp, false); + __store_release(&info->active, 1); + } } } =20 @@ -1153,7 +1226,7 @@ static inline int perf_cgroup_connect(pid_t pid, stru= ct perf_event *event, } =20 static inline void -perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx) +perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest) { } =20 @@ -1565,16 +1638,24 @@ static void perf_unpin_context(struct perf_event_co= ntext *ctx) */ static void __update_context_time(struct perf_event_context *ctx, bool adv) { - u64 now =3D perf_clock(); + lockdep_assert_held(&ctx->lock); + + update_perf_time_ctx(&ctx->time, perf_clock(), adv); +} =20 +static void __update_context_guest_time(struct perf_event_context *ctx, bo= ol adv) +{ lockdep_assert_held(&ctx->lock); =20 - update_perf_time_ctx(&ctx->time, now, adv); + /* must be called after __update_context_time(); */ + update_perf_time_ctx(&ctx->timeguest, ctx->time.stamp, adv); } =20 static void update_context_time(struct perf_event_context *ctx) { __update_context_time(ctx, true); + if (is_guest_mediated_pmu_loaded()) + __update_context_guest_time(ctx, true); } =20 static u64 perf_event_time(struct perf_event *event) @@ -1587,7 +1668,7 @@ static u64 perf_event_time(struct perf_event *event) if (is_cgroup_event(event)) return perf_cgroup_event_time(event); =20 - return ctx->time.time; + return __perf_event_time_ctx(event, &ctx->time); } =20 static u64 perf_event_time_now(struct perf_event *event, u64 now) @@ -1601,10 +1682,9 @@ static u64 perf_event_time_now(struct perf_event *ev= ent, u64 now) return perf_cgroup_event_time_now(event, now); =20 if (!(__load_acquire(&ctx->is_active) & EVENT_TIME)) - return ctx->time.time; + return __perf_event_time_ctx(event, &ctx->time); =20 - now +=3D READ_ONCE(ctx->time.offset); - return now; + return __perf_event_time_ctx_now(event, &ctx->time, now); } =20 static enum event_type_t get_event_type(struct perf_event *event) @@ -2427,20 +2507,23 @@ group_sched_out(struct perf_event *group_event, str= uct perf_event_context *ctx) } =20 static inline void -__ctx_time_update(struct perf_cpu_context *cpuctx, struct perf_event_conte= xt *ctx, bool final) +__ctx_time_update(struct perf_cpu_context *cpuctx, struct perf_event_conte= xt *ctx, + bool final, enum event_type_t event_type) { if (ctx->is_active & EVENT_TIME) { if (ctx->is_active & EVENT_FROZEN) return; + update_context_time(ctx); - update_cgrp_time_from_cpuctx(cpuctx, final); + /* vPMU should not stop time */ + update_cgrp_time_from_cpuctx(cpuctx, !(event_type & EVENT_GUEST) && fina= l); } } =20 static inline void ctx_time_update(struct perf_cpu_context *cpuctx, struct perf_event_context= *ctx) { - __ctx_time_update(cpuctx, ctx, false); + __ctx_time_update(cpuctx, ctx, false, 0); } =20 /* @@ -3506,7 +3589,7 @@ ctx_sched_out(struct perf_event_context *ctx, struct = pmu *pmu, enum event_type_t * * would only update time for the pinned events. */ - __ctx_time_update(cpuctx, ctx, ctx =3D=3D &cpuctx->ctx); + __ctx_time_update(cpuctx, ctx, ctx =3D=3D &cpuctx->ctx, event_type); =20 /* * CPU-release for the below ->is_active store, @@ -3532,7 +3615,18 @@ ctx_sched_out(struct perf_event_context *ctx, struct= pmu *pmu, enum event_type_t cpuctx->task_ctx =3D NULL; } =20 - is_active ^=3D ctx->is_active; /* changed bits */ + if (event_type & EVENT_GUEST) { + /* + * Schedule out all exclude_guest events of PMU + * with PERF_PMU_CAP_MEDIATED_VPMU. + */ + is_active =3D EVENT_ALL; + __update_context_guest_time(ctx, false); + perf_cgroup_set_timestamp(cpuctx, true); + barrier(); + } else { + is_active ^=3D ctx->is_active; /* changed bits */ + } =20 for_each_epc(pmu_ctx, ctx, pmu, event_type) __pmu_ctx_sched_out(pmu_ctx, is_active); @@ -3991,10 +4085,15 @@ static inline void group_update_userpage(struct per= f_event *group_event) event_update_userpage(event); } =20 +struct merge_sched_data { + int can_add_hw; + enum event_type_t event_type; +}; + static int merge_sched_in(struct perf_event *event, void *data) { struct perf_event_context *ctx =3D event->ctx; - int *can_add_hw =3D data; + struct merge_sched_data *msd =3D data; =20 if (event->state <=3D PERF_EVENT_STATE_OFF) return 0; @@ -4002,13 +4101,22 @@ static int merge_sched_in(struct perf_event *event,= void *data) if (!event_filter_match(event)) return 0; =20 - if (group_can_go_on(event, *can_add_hw)) { + /* + * Don't schedule in any host events from PMU with + * PERF_PMU_CAP_MEDIATED_VPMU, while a guest is running. + */ + if (is_guest_mediated_pmu_loaded() && + event->pmu_ctx->pmu->capabilities & PERF_PMU_CAP_MEDIATED_VPMU && + !(msd->event_type & EVENT_GUEST)) + return 0; + + if (group_can_go_on(event, msd->can_add_hw)) { if (!group_sched_in(event, ctx)) list_add_tail(&event->active_list, get_event_list(event)); } =20 if (event->state =3D=3D PERF_EVENT_STATE_INACTIVE) { - *can_add_hw =3D 0; + msd->can_add_hw =3D 0; if (event->attr.pinned) { perf_cgroup_event_disable(event, ctx); perf_event_set_state(event, PERF_EVENT_STATE_ERROR); @@ -4031,11 +4139,15 @@ static int merge_sched_in(struct perf_event *event,= void *data) =20 static void pmu_groups_sched_in(struct perf_event_context *ctx, struct perf_event_groups *groups, - struct pmu *pmu) + struct pmu *pmu, + enum event_type_t event_type) { - int can_add_hw =3D 1; + struct merge_sched_data msd =3D { + .can_add_hw =3D 1, + .event_type =3D event_type, + }; visit_groups_merge(ctx, groups, smp_processor_id(), pmu, - merge_sched_in, &can_add_hw); + merge_sched_in, &msd); } =20 static void __pmu_ctx_sched_in(struct perf_event_pmu_context *pmu_ctx, @@ -4044,9 +4156,9 @@ static void __pmu_ctx_sched_in(struct perf_event_pmu_= context *pmu_ctx, struct perf_event_context *ctx =3D pmu_ctx->ctx; =20 if (event_type & EVENT_PINNED) - pmu_groups_sched_in(ctx, &ctx->pinned_groups, pmu_ctx->pmu); + pmu_groups_sched_in(ctx, &ctx->pinned_groups, pmu_ctx->pmu, event_type); if (event_type & EVENT_FLEXIBLE) - pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu_ctx->pmu); + pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu_ctx->pmu, event_type= ); } =20 static void @@ -4063,9 +4175,11 @@ ctx_sched_in(struct perf_event_context *ctx, struct = pmu *pmu, enum event_type_t return; =20 if (!(is_active & EVENT_TIME)) { + /* EVENT_TIME should be active while the guest runs */ + WARN_ON_ONCE(event_type & EVENT_GUEST); /* start ctx time */ __update_context_time(ctx, false); - perf_cgroup_set_timestamp(cpuctx); + perf_cgroup_set_timestamp(cpuctx, false); /* * CPU-release for the below ->is_active store, * see __load_acquire() in perf_event_time_now() @@ -4081,7 +4195,23 @@ ctx_sched_in(struct perf_event_context *ctx, struct = pmu *pmu, enum event_type_t WARN_ON_ONCE(cpuctx->task_ctx !=3D ctx); } =20 - is_active ^=3D ctx->is_active; /* changed bits */ + if (event_type & EVENT_GUEST) { + /* + * Schedule in the required exclude_guest events of PMU + * with PERF_PMU_CAP_MEDIATED_VPMU. + */ + is_active =3D event_type & EVENT_ALL; + + /* + * Update ctx time to set the new start time for + * the exclude_guest events. + */ + update_context_time(ctx); + update_cgrp_time_from_cpuctx(cpuctx, false); + barrier(); + } else { + is_active ^=3D ctx->is_active; /* changed bits */ + } =20 /* * First go through the list and put on any pinned groups @@ -4089,13 +4219,13 @@ ctx_sched_in(struct perf_event_context *ctx, struct= pmu *pmu, enum event_type_t */ if (is_active & EVENT_PINNED) { for_each_epc(pmu_ctx, ctx, pmu, event_type) - __pmu_ctx_sched_in(pmu_ctx, EVENT_PINNED); + __pmu_ctx_sched_in(pmu_ctx, EVENT_PINNED | (event_type & EVENT_GUEST)); } =20 /* Then walk through the lower prio flexible groups */ if (is_active & EVENT_FLEXIBLE) { for_each_epc(pmu_ctx, ctx, pmu, event_type) - __pmu_ctx_sched_in(pmu_ctx, EVENT_FLEXIBLE); + __pmu_ctx_sched_in(pmu_ctx, EVENT_FLEXIBLE | (event_type & EVENT_GUEST)= ); } } =20 @@ -6621,23 +6751,23 @@ void perf_event_update_userpage(struct perf_event *= event) if (!rb) goto unlock; =20 - /* - * compute total_time_enabled, total_time_running - * based on snapshot values taken when the event - * was last scheduled in. - * - * we cannot simply called update_context_time() - * because of locking issue as we can be called in - * NMI context - */ - calc_timer_values(event, &now, &enabled, &running); - - userpg =3D rb->user_page; /* * Disable preemption to guarantee consistent time stamps are stored to * the user page. */ preempt_disable(); + + /* + * Compute total_time_enabled, total_time_running based on snapshot + * values taken when the event was last scheduled in. + * + * We cannot simply call update_context_time() because doing so would + * lead to deadlock when called from NMI context. + */ + calc_timer_values(event, &now, &enabled, &running); + + userpg =3D rb->user_page; + ++userpg->lock; barrier(); userpg->index =3D perf_event_index(event); @@ -7902,13 +8032,11 @@ static void perf_output_read(struct perf_output_han= dle *handle, u64 read_format =3D event->attr.read_format; =20 /* - * compute total_time_enabled, total_time_running - * based on snapshot values taken when the event - * was last scheduled in. + * Compute total_time_enabled, total_time_running based on snapshot + * values taken when the event was last scheduled in. * - * we cannot simply called update_context_time() - * because of locking issue as we are called in - * NMI context + * We cannot simply call update_context_time() because doing so would + * lead to deadlock when called from NMI context. */ if (read_format & PERF_FORMAT_TOTAL_TIMES) calc_timer_values(event, &now, &enabled, &running); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B5C92C15BC for ; Wed, 6 Aug 2025 19:57:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510265; cv=none; b=pjKgON9kF71rzI7S4EnesCXkZfXfMwopLZ+soVMMzqy61TfDEgTMUiLWfkrtt/KHMw9DCoRjJ5dFKK0XW6+HgAQrwhH7l78uFizpsKMHp5aH6XNpDXDW3nvZmzYp41NF90kmskb7ynT83YTTv6uod2Sl+pXFPgfeXKNFLQ68LNE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510265; c=relaxed/simple; bh=p9dwRoIkAdNKqXS3ork0V3QcmE8t7/wykyfY7BDBXjw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HIZ2957u04JC14bm8Asq7476cIBac7ecjJHan6ZPocvdmVmdwgorC+4W7YHwAXQIVu8dWG8z1nRaSLpJp8OpjNlCC0ELcMiwfWzSVVa9x6D24eXjD9oIaSiRZC4CBFErSgZ1RJdTTSF5ynv3F7lJh4DjXxmPG+414sAcieu6U4c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=i8IBTCHs; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="i8IBTCHs" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-24249098fd0so2371955ad.0 for ; Wed, 06 Aug 2025 12:57:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510261; x=1755115061; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=1cgAGvW2Sa5tNwASy1Md3TT+4FZc1wpWNQfNLgFp1cE=; b=i8IBTCHsEt2DrcIcxVFbp7SFIHxRMBVKZRXNlv/Q/OQyKdNwpIdP2db8o7nwlILQTK QuBOlCA7EPm3sQ6zh+Ss1qlwCp5NrB9BPwMNlIRBQrUQSQ9ioC3tM1VaKbcXHyk6snqd hmXG7JGUCTvbHS6fpouMnHMYIHSPsl/9lmsPqc9U5Pft9REnjvnuhXAs/kVIzTydoauk ol6Gl5XL9veybgYrnhyj9jS/RgHO4kUfNU2+bp0KYPkkERRQ1/BWcXGfPWwVi9wjT41J QDtvF4djT+6eu1kP6aR7CzKHhr57d0b1TfkAMPhHLqV14Umg+NWAmNN/XmLEPeyXN8Un 2pNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510261; x=1755115061; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1cgAGvW2Sa5tNwASy1Md3TT+4FZc1wpWNQfNLgFp1cE=; b=JMHJ+A+NQ84k6E5+GHsCetqDMrZE3NE3epplBI0ivjoVpqJufexEN9R8bja6dhsEhh pL+OgMXoeUQDEW83vXi2zYTUkca9grgcmR6+maHhm9ezpPy6vxeAeqnXea1ff0bxuclV Y7SarwOIbuf3tvneJLJdu1njlkJaIU9w2DowXVGKGBaHalkqWZpEYCw/A67mCzu+/Xtr +FRhZbD+OLgSCW4XTl+lz3SK0b8lxeeXZxVVYiOcn14qhF5dedBMkUQK7RcMfzEEAist vn8man8k/AgT2TDGCUzsqYe3AwePF6i7pzZrg5X3Yr1kt3FIjbPzM5iP+X9GTrYKLMug PbtQ== X-Forwarded-Encrypted: i=1; AJvYcCV+v86zwmgru0hYSKalDnEht1w/DE/YocHU8ffu3YHcYWOMd1y6UQ5UjW70PBbvvfeiXvKO0rFVpr+v0u8=@vger.kernel.org X-Gm-Message-State: AOJu0YxRAqi32M3GwioTxXpdkOTzQeMx89kdotwB0grFRt2iL4hSJ8+5 e+M/gTbDbDW3rE8D4/GXJ7dh2DH8ULd9sTZu3pshwoTeohHwm0PXLBtwmpIXRWA4BpwmF3Nwz71 AomCRRg== X-Google-Smtp-Source: AGHT+IEQshj25YWfgh8jlPDalwm585npWoyFxUPXBG4oqUn1ilItXjqTi+CTUz913BcUQcsY19RFz/4rdM8= X-Received: from plrj9.prod.google.com ([2002:a17:903:289:b0:242:abd5:b3bf]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:c405:b0:240:b879:fed2 with SMTP id d9443c01a7336-2429f55daaamr65961135ad.16.1754510261485; Wed, 06 Aug 2025 12:57:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:29 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-8-seanjc@google.com> Subject: [PATCH v5 07/44] perf: Add APIs to load/put guest mediated PMU context From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang Add exported APIs to load/put a guest mediated PMU context. KVM will load the guest PMU shortly before VM-Enter, and put the guest PMU shortly after VM-Exit. On the perf side of things, schedule out all exclude_guest events when the guest context is loaded, and schedule them back in when the guest context is put. I.e. yield the hardware PMU resources to the guest, by way of KVM. Note, perf is only responsible for managing host context. KVM is responsible for loading/storing guest state to/from hardware. Suggested-by: Sean Christopherson Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang [sean: shuffle patches around, write changelog] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- include/linux/perf_event.h | 2 ++ kernel/events/core.c | 61 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 0958b6d0a61c..42d019d70b42 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1925,6 +1925,8 @@ extern u64 perf_event_pause(struct perf_event *event,= bool reset); #ifdef CONFIG_PERF_GUEST_MEDIATED_PMU int perf_create_mediated_pmu(void); void perf_release_mediated_pmu(void); +void perf_load_guest_context(unsigned long data); +void perf_put_guest_context(void); #endif =20 #else /* !CONFIG_PERF_EVENTS: */ diff --git a/kernel/events/core.c b/kernel/events/core.c index 6875b56ddd6b..77398b1ad4c5 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -469,10 +469,19 @@ static cpumask_var_t perf_online_pkg_mask; static cpumask_var_t perf_online_sys_mask; static struct kmem_cache *perf_event_cache; =20 +#ifdef CONFIG_PERF_GUEST_MEDIATED_PMU +static DEFINE_PER_CPU(bool, guest_ctx_loaded); + +static __always_inline bool is_guest_mediated_pmu_loaded(void) +{ + return __this_cpu_read(guest_ctx_loaded); +} +#else static __always_inline bool is_guest_mediated_pmu_loaded(void) { return false; } +#endif =20 /* * perf event paranoia level: @@ -6379,6 +6388,58 @@ void perf_release_mediated_pmu(void) atomic_dec(&nr_mediated_pmu_vms); } EXPORT_SYMBOL_GPL(perf_release_mediated_pmu); + +/* When loading a guest's mediated PMU, schedule out all exclude_guest eve= nts. */ +void perf_load_guest_context(unsigned long data) +{ + struct perf_cpu_context *cpuctx =3D this_cpu_ptr(&perf_cpu_context); + + lockdep_assert_irqs_disabled(); + + guard(perf_ctx_lock)(cpuctx, cpuctx->task_ctx); + + if (WARN_ON_ONCE(__this_cpu_read(guest_ctx_loaded))) + return; + + perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST); + ctx_sched_out(&cpuctx->ctx, NULL, EVENT_GUEST); + if (cpuctx->task_ctx) { + perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST); + task_ctx_sched_out(cpuctx->task_ctx, NULL, EVENT_GUEST); + } + + perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST); + if (cpuctx->task_ctx) + perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST); + + __this_cpu_write(guest_ctx_loaded, true); +} +EXPORT_SYMBOL_GPL(perf_load_guest_context); + +void perf_put_guest_context(void) +{ + struct perf_cpu_context *cpuctx =3D this_cpu_ptr(&perf_cpu_context); + + lockdep_assert_irqs_disabled(); + + guard(perf_ctx_lock)(cpuctx, cpuctx->task_ctx); + + if (WARN_ON_ONCE(!__this_cpu_read(guest_ctx_loaded))) + return; + + perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST); + if (cpuctx->task_ctx) + perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST); + + perf_event_sched_in(cpuctx, cpuctx->task_ctx, NULL, EVENT_GUEST); + + if (cpuctx->task_ctx) + perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST); + perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST); + + __this_cpu_write(guest_ctx_loaded, false); +} +EXPORT_SYMBOL_GPL(perf_put_guest_context); #else static int mediated_pmu_account_event(struct perf_event *event) { return 0= ; } static void mediated_pmu_unaccount_event(struct perf_event *event) {} --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D52F82C3254 for ; Wed, 6 Aug 2025 19:57:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510266; cv=none; b=ajXgFG+wDE7J32cuOQMSVkIhny3FdkBGsSeLcOpu02WRRVi2XXX9NXfOyEcSASZtIwd1oapzTRRbcKNKQLAnXsCcGEGjSxyiPgLq8RPfdsrhjdm6hIBh2M2Uzq86S3Zh+j+YAtufuyYETpbMKK6eZnY69I+de7C9gM5QtFf/D8I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510266; c=relaxed/simple; bh=x4daQrL7sPxb+dEOZVPTgJRHntO3qzWRLtRzDqkfDc8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Y6Z7Oaxx3m0S7FK1A6yeAAe01/ZB6g5r072tiVC0uw/JALOWwNu8Lmhr6a5za6ohkz8+rNTl9M2QxRLcTz5RiJyiyYRLEgInQyjqAW2+zwcw3mfO2mQbtG76oct0GmrBzObn3s0dKNQF5v9rm8btSV22xPQ+YsoPCc2SqencgaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=tbvXitN7; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tbvXitN7" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-76b6d359927so179076b3a.3 for ; Wed, 06 Aug 2025 12:57:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510263; x=1755115063; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Uj/oFQFNyss4LK4I6PYkReCFuQkjKCIa3E/WRuXwHJ0=; b=tbvXitN7QUfznltnTOID5SDg3+IBAKVh0RRf/MvRbDSZj6Td3VGdQZNJ4E0axKvuRs nenWJNo+3RVUTxak1a2kdjODcYoeZ9KRHZcTLJUuxa0OT5UZM5UN/8vuZ/4uBJi6itBw naH9ewvyWFwx49PCPFlMzcCKVFzejaJedXJQeFjXtRSP/6i0x9qQeYNgE3g6aJuS2KYg DurTw9D10lseCzotAXKpeXiYucG7+RuLpmC3VsXQ06bG9LZm4+j2erroVFcowfYfMN+6 +U0Lz/B2Qcbzr44FrZ66IsRwiL2u27HTXtok6AFLhHoBKDDcQ5LP2TYFnDtc9YL5b+Yw zsOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510263; x=1755115063; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Uj/oFQFNyss4LK4I6PYkReCFuQkjKCIa3E/WRuXwHJ0=; b=qXEjaNNSgMyjYpYq+NCuPNgpGxmSi4XSc48LrKajY4eG9I267XRZdhQilwP3AD3dtk HwwremqGWun3Iu+w90fRsvy/Ylr8TYLlp3PEJTvRFiqjcViKC8f71k9ommi0xDhsVGwY yLF9Yrnosr83/JxAl8DXU9qgR5jdfKy1U9vBl8GHabuEHCKTqjOBW70A1lBKRNpAKYeN iczGFqKor1B3yfH55xZvgr1Hn01pfcSbKJIYLblXdoHsPY5FeRceb3tUJm3KO50Qv8dL 1ovTeT6vCyB/ZN81BJ/M2Bt0pqWBg/PcZf2oPJzVu9bygAzeWZwE7L3HUKTtuYvUi4GO 3j8Q== X-Forwarded-Encrypted: i=1; AJvYcCUsgBVhglBZGrnQ42/wWhNn4qvLDXKe+Yb433Jh5hy3KxrInHH0PrHnXqAdNQaMrmcCCLdBuBvlrzA1FI4=@vger.kernel.org X-Gm-Message-State: AOJu0YxwT5a4LOloGv1Vmxc6krqeRcrX0Kx1RFgDcgn/2weK4tSmHVxb E6Jad5qBdY15nwI6xeLmUGQ7NMmo+zCPMfAQR+ZmP2F+4jmkdfVBDOjlNOilcws9Ivs5BLuqTjN GXTIoGQ== X-Google-Smtp-Source: AGHT+IGQB+bLhnxiIawVNn7HfRXPwF1XDhXBfLBplnIbWM9gXArdsZ1WUOqr9MNOZqSoFpP4ppFPeoHjojo= X-Received: from pfhh8.prod.google.com ([2002:a05:6a00:2308:b0:76b:c5af:cd3d]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2293:b0:76b:d363:4a3f with SMTP id d2e1a72fcca58-76c2af6ace2mr5675237b3a.3.1754510263031; Wed, 06 Aug 2025 12:57:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:30 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-9-seanjc@google.com> Subject: [PATCH v5 08/44] perf: core/x86: Register a new vector for handling mediated guest PMIs From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Wire up system vector 0xf5 for handling PMIs (i.e. interrupts delivered through the LVTPC) while running KVM guests with a mediated PMU. Perf currently delivers all PMIs as NMIs, e.g. so that events that trigger while IRQs are disabled aren't delayed and generate useless records, but due to the multiplexing of NMIs throughout the system, correctly identifying NMIs for a mediated PMU is practically infeasible. To (greatly) simplify identifying guest mediated PMU PMIs, perf will switch the CPU's LVTPC between PERF_GUEST_MEDIATED_PMI_VECTOR and NMI when guest PMU context is loaded/put. I.e. PMIs that are generated by the CPU while the guest is active will be identified purely based on the IRQ vector. Route the vector through perf, e.g. as opposed to letting KVM attach a handler directly a la posted interrupt notification vectors, as perf owns the LVTPC and thus is the rightful owner of PERF_GUEST_MEDIATED_PMI_VECTOR. Functionally, having KVM directly own the vector would be fine (both KVM and perf will be completely aware of when a mediated PMU is active), but would lead to an undesirable split in ownership: perf would be responsible for installing the vector, but not handling the resulting IRQs. Add a new perf_guest_info_callbacks hook (and static call) to allow KVM to register its handler with perf when running guests with mediated PMUs. Note, because KVM always runs guests with host IRQs enabled, there is no danger of a PMI being delayed from the guest's perspective due to using a regular IRQ instead of an NMI. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/entry/entry_fred.c | 1 + arch/x86/include/asm/hardirq.h | 3 +++ arch/x86/include/asm/idtentry.h | 6 ++++++ arch/x86/include/asm/irq_vectors.h | 4 +++- arch/x86/kernel/idt.c | 3 +++ arch/x86/kernel/irq.c | 19 +++++++++++++++++++ include/linux/perf_event.h | 8 ++++++++ kernel/events/core.c | 9 +++++++-- .../beauty/arch/x86/include/asm/irq_vectors.h | 3 ++- virt/kvm/kvm_main.c | 3 +++ 10 files changed, 55 insertions(+), 4 deletions(-) diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c index f004a4dc74c2..d80861a4cd00 100644 --- a/arch/x86/entry/entry_fred.c +++ b/arch/x86/entry/entry_fred.c @@ -114,6 +114,7 @@ static idtentry_t sysvec_table[NR_SYSTEM_VECTORS] __ro_= after_init =3D { =20 SYSVEC(IRQ_WORK_VECTOR, irq_work), =20 + SYSVEC(PERF_GUEST_MEDIATED_PMI_VECTOR, perf_guest_mediated_pmi_handler), SYSVEC(POSTED_INTR_VECTOR, kvm_posted_intr_ipi), SYSVEC(POSTED_INTR_WAKEUP_VECTOR, kvm_posted_intr_wakeup_ipi), SYSVEC(POSTED_INTR_NESTED_VECTOR, kvm_posted_intr_nested_ipi), diff --git a/arch/x86/include/asm/hardirq.h b/arch/x86/include/asm/hardirq.h index f00c09ffe6a9..f221d001e4ed 100644 --- a/arch/x86/include/asm/hardirq.h +++ b/arch/x86/include/asm/hardirq.h @@ -18,6 +18,9 @@ typedef struct { unsigned int kvm_posted_intr_ipis; unsigned int kvm_posted_intr_wakeup_ipis; unsigned int kvm_posted_intr_nested_ipis; +#endif +#ifdef CONFIG_GUEST_PERF_EVENTS + unsigned int perf_guest_mediated_pmis; #endif unsigned int x86_platform_ipis; /* arch dependent */ unsigned int apic_perf_irqs; diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentr= y.h index a4ec27c67988..5109c24b3c1c 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -751,6 +751,12 @@ DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR, sys= vec_kvm_posted_intr_nested # define fred_sysvec_kvm_posted_intr_nested_ipi NULL #endif =20 +# ifdef CONFIG_GUEST_PERF_EVENTS +DECLARE_IDTENTRY_SYSVEC(PERF_GUEST_MEDIATED_PMI_VECTOR, sysvec_perf_guest_= mediated_pmi_handler); +#else +# define fred_sysvec_perf_guest_mediated_pmi_handler NULL +#endif + # ifdef CONFIG_X86_POSTED_MSI DECLARE_IDTENTRY_SYSVEC(POSTED_MSI_NOTIFICATION_VECTOR, sysvec_posted_msi_= notification); #else diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_= vectors.h index 47051871b436..85253fc8e384 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -77,7 +77,9 @@ */ #define IRQ_WORK_VECTOR 0xf6 =20 -/* 0xf5 - unused, was UV_BAU_MESSAGE */ +/* IRQ vector for PMIs when running a guest with a mediated PMU. */ +#define PERF_GUEST_MEDIATED_PMI_VECTOR 0xf5 + #define DEFERRED_ERROR_VECTOR 0xf4 =20 /* Vector on which hypervisor callbacks will be delivered */ diff --git a/arch/x86/kernel/idt.c b/arch/x86/kernel/idt.c index f445bec516a0..260456588756 100644 --- a/arch/x86/kernel/idt.c +++ b/arch/x86/kernel/idt.c @@ -158,6 +158,9 @@ static const __initconst struct idt_data apic_idts[] = =3D { INTG(POSTED_INTR_WAKEUP_VECTOR, asm_sysvec_kvm_posted_intr_wakeup_ipi), INTG(POSTED_INTR_NESTED_VECTOR, asm_sysvec_kvm_posted_intr_nested_ipi), # endif +#ifdef CONFIG_GUEST_PERF_EVENTS + INTG(PERF_GUEST_MEDIATED_PMI_VECTOR, asm_sysvec_perf_guest_mediated_pmi_h= andler), +#endif # ifdef CONFIG_IRQ_WORK INTG(IRQ_WORK_VECTOR, asm_sysvec_irq_work), # endif diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index 9ed29ff10e59..70ab066aa4cb 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -191,6 +191,13 @@ int arch_show_interrupts(struct seq_file *p, int prec) irq_stats(j)->kvm_posted_intr_wakeup_ipis); seq_puts(p, " Posted-interrupt wakeup event\n"); #endif +#ifdef CONFIG_GUEST_PERF_EVENTS + seq_printf(p, "%*s: ", prec, "VPMI"); + for_each_online_cpu(j) + seq_printf(p, "%10u ", + irq_stats(j)->perf_guest_mediated_pmis); + seq_puts(p, " Perf Guest Mediated PMI\n"); +#endif #ifdef CONFIG_X86_POSTED_MSI seq_printf(p, "%*s: ", prec, "PMN"); for_each_online_cpu(j) @@ -315,6 +322,18 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_x86_platform_ipi) } #endif =20 +#ifdef CONFIG_GUEST_PERF_EVENTS +/* + * Handler for PERF_GUEST_MEDIATED_PMI_VECTOR. + */ +DEFINE_IDTENTRY_SYSVEC(sysvec_perf_guest_mediated_pmi_handler) +{ + apic_eoi(); + inc_irq_stat(perf_guest_mediated_pmis); + perf_guest_handle_mediated_pmi(); +} +#endif + #if IS_ENABLED(CONFIG_KVM) static void dummy_handler(void) {} static void (*kvm_posted_intr_wakeup_handler)(void) =3D dummy_handler; diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 42d019d70b42..0c529fbd97e6 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1677,6 +1677,8 @@ struct perf_guest_info_callbacks { unsigned int (*state)(void); unsigned long (*get_ip)(void); unsigned int (*handle_intel_pt_intr)(void); + + void (*handle_mediated_pmi)(void); }; =20 #ifdef CONFIG_GUEST_PERF_EVENTS @@ -1686,6 +1688,7 @@ extern struct perf_guest_info_callbacks __rcu *perf_g= uest_cbs; DECLARE_STATIC_CALL(__perf_guest_state, *perf_guest_cbs->state); DECLARE_STATIC_CALL(__perf_guest_get_ip, *perf_guest_cbs->get_ip); DECLARE_STATIC_CALL(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->ha= ndle_intel_pt_intr); +DECLARE_STATIC_CALL(__perf_guest_handle_mediated_pmi, *perf_guest_cbs->han= dle_mediated_pmi); =20 static inline unsigned int perf_guest_state(void) { @@ -1702,6 +1705,11 @@ static inline unsigned int perf_guest_handle_intel_p= t_intr(void) return static_call(__perf_guest_handle_intel_pt_intr)(); } =20 +static inline void perf_guest_handle_mediated_pmi(void) +{ + static_call(__perf_guest_handle_mediated_pmi)(); +} + extern void perf_register_guest_info_callbacks(struct perf_guest_info_call= backs *cbs); extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_ca= llbacks *cbs); =20 diff --git a/kernel/events/core.c b/kernel/events/core.c index 77398b1ad4c5..e1df3c3bfc0d 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7607,6 +7607,7 @@ struct perf_guest_info_callbacks __rcu *perf_guest_cb= s; DEFINE_STATIC_CALL_RET0(__perf_guest_state, *perf_guest_cbs->state); DEFINE_STATIC_CALL_RET0(__perf_guest_get_ip, *perf_guest_cbs->get_ip); DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs= ->handle_intel_pt_intr); +DEFINE_STATIC_CALL_RET0(__perf_guest_handle_mediated_pmi, *perf_guest_cbs-= >handle_mediated_pmi); =20 void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *= cbs) { @@ -7621,6 +7622,10 @@ void perf_register_guest_info_callbacks(struct perf_= guest_info_callbacks *cbs) if (cbs->handle_intel_pt_intr) static_call_update(__perf_guest_handle_intel_pt_intr, cbs->handle_intel_pt_intr); + + if (cbs->handle_mediated_pmi) + static_call_update(__perf_guest_handle_mediated_pmi, + cbs->handle_mediated_pmi); } EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks); =20 @@ -7632,8 +7637,8 @@ void perf_unregister_guest_info_callbacks(struct perf= _guest_info_callbacks *cbs) rcu_assign_pointer(perf_guest_cbs, NULL); static_call_update(__perf_guest_state, (void *)&__static_call_return0); static_call_update(__perf_guest_get_ip, (void *)&__static_call_return0); - static_call_update(__perf_guest_handle_intel_pt_intr, - (void *)&__static_call_return0); + static_call_update(__perf_guest_handle_intel_pt_intr, (void *)&__static_c= all_return0); + static_call_update(__perf_guest_handle_mediated_pmi, (void *)&__static_ca= ll_return0); synchronize_rcu(); } EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks); diff --git a/tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h b/t= ools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h index 47051871b436..6e1d5b955aae 100644 --- a/tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h +++ b/tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h @@ -77,7 +77,8 @@ */ #define IRQ_WORK_VECTOR 0xf6 =20 -/* 0xf5 - unused, was UV_BAU_MESSAGE */ +#define PERF_GUEST_MEDIATED_PMI_VECTOR 0xf5 + #define DEFERRED_ERROR_VECTOR 0xf4 =20 /* Vector on which hypervisor callbacks will be delivered */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6c07dd423458..ecafab2e17d9 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -6426,11 +6426,14 @@ static struct perf_guest_info_callbacks kvm_guest_c= bs =3D { .state =3D kvm_guest_state, .get_ip =3D kvm_guest_get_ip, .handle_intel_pt_intr =3D NULL, + .handle_mediated_pmi =3D NULL, }; =20 void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void)) { kvm_guest_cbs.handle_intel_pt_intr =3D pt_intr_handler; + kvm_guest_cbs.handle_mediated_pmi =3D NULL; + perf_register_guest_info_callbacks(&kvm_guest_cbs); } void kvm_unregister_perf_callbacks(void) --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2DFF2D028F for ; Wed, 6 Aug 2025 19:57:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510268; cv=none; b=siidC0JmxT69S7RCzm03gvbfSCx3NAWSxblM4GK2HhWeYS6bWx7KZDu/iJrAQl5TRUzP5iKXRB+4HtyHikR6Ct4NHgHWTck0n+P6Ck03y/kFRxeJdMCHZu25Ocx183y4iVV3Sgq8TcnHXR0ZyEDES/VyB5jxMEZQuIcl/jCIxz4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510268; c=relaxed/simple; bh=lPxtg3K6w22yYMFe3Rv5cpyOHm8QqrTeWqcW4ebUScU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mAvyBm6L+oFLMFm7VfP8yEO5pqoVWVwWHeyMLbdTQ1cH+/VRyOVNv3CTyaaVYgWjJh+j6vGgKNoxxoCGa2e2Z9iZ43f1TRhB3ZvCbyiIU2M1EE7/nmLwfUYGMIeVr/o0W+Sk8DZyWxpqyxorY/gS5t3SyrxKmqX/Z1CHGRnHE3o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=W9OsAinG; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="W9OsAinG" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b1ffc678adfso135600a12.0 for ; Wed, 06 Aug 2025 12:57:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510265; x=1755115065; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=jWazn7LbEKToZVBXfT81sYQ79Zs+usROHmWn+qgVcAw=; b=W9OsAinGjcMXAtBm76IFKsQZXxjqTBPqoyxyChEV3rz7VCofrp3pXNu32Ym22zydu3 Y9BfvloSB5xZ5sc0z3zoHyteK7N3gD70rNYxBdZinUznZu1Do48W3nfY5bOoOJZBZjAw dpgejWDf/FvLrmtm8HSidsIPaF2X6CM/dcUUAPZ/bhgSFTr8ETNZ/Jzzwbu6xL7j5fdt fuhoS8HkVm30yFkZgb9lt7lzclIILwH2++s+fuMy89u7LA50GiFa9lN+qIAi2wYtyIA7 IeYiQd0O27Zst//+GDreTFVXSnrDCHGa6J3wWnun9ifVy4xGxLS6JslZBo3IAtA9E9OQ /5Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510265; x=1755115065; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jWazn7LbEKToZVBXfT81sYQ79Zs+usROHmWn+qgVcAw=; b=HtGXd+noUSAeZx0YDhNds49loxY7jqsbBQ8sibgqzXzQiYCPpBloBg1RYYhWokmOCO rtgRYt47ShWXRTAdS+jRFdOVuOm9EuFxQmU7ykduHUeT77TLXrB/5T2W642+nzBGgPXf FG1RQhxixBgFyzDINNFXrOjiJBs9lzeQxGPFxjX/eOD5/fz7+jlH0taCnm3adU+d/7UJ YyovNVdYC7/yGlMP2Z6o262STDr6fMuAnkkpOuoxlFUuv3l5Ppl5ibfTzDr4wcIXlK+g DqTt9ljZ2G+4/lRap3UGRDV7rcb2U6ed2THUTNR5WNtn8FwEkWQa3BC1R0xZugGOGk7Q HqxQ== X-Forwarded-Encrypted: i=1; AJvYcCWxajWRlWZa/FemW5siXtgMQX7UYuWoYChIZLSOvMha7CfMNq/zJMjQu8Cpx49/t3pt7kL2WHEf6rRJs0g=@vger.kernel.org X-Gm-Message-State: AOJu0YyWOC5/oyygF8VfoZOoBCnYnC0dBKmDgasOOd1Xq/BVZMsybDxu DfyN7+wd0eefeAs0k7RUsjb9V0XPbxhnpvIBwnYyAn1WZ2kpePYNoRzyWHHt60LgGLK11L+thWe KlXFPZQ== X-Google-Smtp-Source: AGHT+IEOZHdE2/PbxBEmSs9IzcnvpAQDGgoykrCqSet5dWYTZm1GHUavvsVGL1zycY+fmJU68DVHh19KP/o= X-Received: from pgbcs10.prod.google.com ([2002:a05:6a02:418a:b0:b42:2ab5:b109]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6300:218d:b0:23d:f987:b033 with SMTP id adf61e73a8af0-240314c22f5mr6625238637.40.1754510264630; Wed, 06 Aug 2025 12:57:44 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:31 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-10-seanjc@google.com> Subject: [PATCH v5 09/44] perf/x86: Switch LVTPC to/from mediated PMI vector on guest load/put context From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add arch hooks to the mediated vPMU load/put APIs, and use the hooks to switch PMIs to the dedicated mediated PMU IRQ vector on load, and back to perf's standard NMI when the guest context is put. I.e. route PMIs to PERF_GUEST_MEDIATED_PMI_VECTOR when the guest context is active, and to NMIs while the host context is active. While running with guest context loaded, ignore all NMIs (in perf). Any NMI that arrives while the LVTPC points at the mediated PMU IRQ vector can't possibly be due to a host perf event. Signed-off-by: Xiong Zhang Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang [sean: use arch hook instead of per-PMU callback] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/events/core.c | 27 +++++++++++++++++++++++++++ include/linux/perf_event.h | 3 +++ kernel/events/core.c | 4 ++++ 3 files changed, 34 insertions(+) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 7610f26dfbd9..9b0525b252f1 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -55,6 +55,8 @@ DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events) =3D { .pmu =3D &pmu, }; =20 +static DEFINE_PER_CPU(bool, x86_guest_ctx_loaded); + DEFINE_STATIC_KEY_FALSE(rdpmc_never_available_key); DEFINE_STATIC_KEY_FALSE(rdpmc_always_available_key); DEFINE_STATIC_KEY_FALSE(perf_is_hybrid); @@ -1756,6 +1758,16 @@ perf_event_nmi_handler(unsigned int cmd, struct pt_r= egs *regs) u64 finish_clock; int ret; =20 + /* + * Ignore all NMIs when a guest's mediated PMU context is loaded. Any + * such NMI can't be due to a PMI as the CPU's LVTPC is switched to/from + * the dedicated mediated PMI IRQ vector while host events are quiesced. + * Attempting to handle a PMI while the guest's context is loaded will + * generate false positives and clobber guest state. + */ + if (this_cpu_read(x86_guest_ctx_loaded)) + return NMI_DONE; + /* * All PMUs/events that share this PMI handler should make sure to * increment active_events for their events. @@ -2727,6 +2739,21 @@ static struct pmu pmu =3D { .filter =3D x86_pmu_filter, }; =20 +void arch_perf_load_guest_context(unsigned long data) +{ + u32 masked =3D data & APIC_LVT_MASKED; + + apic_write(APIC_LVTPC, + APIC_DM_FIXED | PERF_GUEST_MEDIATED_PMI_VECTOR | masked); + this_cpu_write(x86_guest_ctx_loaded, true); +} + +void arch_perf_put_guest_context(void) +{ + this_cpu_write(x86_guest_ctx_loaded, false); + apic_write(APIC_LVTPC, APIC_DM_NMI); +} + void arch_perf_update_userpage(struct perf_event *event, struct perf_event_mmap_page *userpg, u64 now) { diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 0c529fbd97e6..3a9bd9c4c90e 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1846,6 +1846,9 @@ static inline unsigned long perf_arch_guest_misc_flag= s(struct pt_regs *regs) # define perf_arch_guest_misc_flags(regs) perf_arch_guest_misc_flags(regs) #endif =20 +extern void arch_perf_load_guest_context(unsigned long data); +extern void arch_perf_put_guest_context(void); + static inline bool needs_branch_stack(struct perf_event *event) { return event->attr.branch_sample_type !=3D 0; diff --git a/kernel/events/core.c b/kernel/events/core.c index e1df3c3bfc0d..ad22b182762e 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6408,6 +6408,8 @@ void perf_load_guest_context(unsigned long data) task_ctx_sched_out(cpuctx->task_ctx, NULL, EVENT_GUEST); } =20 + arch_perf_load_guest_context(data); + perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST); if (cpuctx->task_ctx) perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST); @@ -6433,6 +6435,8 @@ void perf_put_guest_context(void) =20 perf_event_sched_in(cpuctx, cpuctx->task_ctx, NULL, EVENT_GUEST); =20 + arch_perf_put_guest_context(); + if (cpuctx->task_ctx) perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST); perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD4FD2C08C0 for ; Wed, 6 Aug 2025 19:57:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510270; cv=none; b=lAfW/aPeiYgGfCnTbLIvCTm6mWLZoUoiJYF9FF1dNzw3P0GRzXrZP41Zfa2ec52w7mETytw3Rsl0luQ5NwJjtsC1uAXzidDNjnny1WeqSgxP04BvP3VSPbgu0mgiSlmeVTZiLZtE5/rpK7jUhw115fabf+mbi9/y/EIxGGGoNr0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510270; c=relaxed/simple; bh=5aUcvaWHp6TRUDOiSNC6KCm1Cyy80ztm2LX/UEyRYFs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ggSkvtLtHyDNIa8RMtQIbax2/Ga8a/TaZ0sfKaSMl8mj8gxB7mpSN6Upx2E2m56ZBvFBuOK8rWsD8KeoHRdtVs1dD1cxfAjNywwjd08cJkOjwsklRh9AkvNiwfoDYvXbGV5+lKgfaK7A83rRqHcuFByWjqUbFHtzhLcTMH90MlU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eMTwdtGF; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eMTwdtGF" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b3f38d3cabeso111803a12.3 for ; Wed, 06 Aug 2025 12:57:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510267; x=1755115067; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=V6GitVs58LgSs5T9j7jt8hclUfm0doY6DCHFmJ55L1k=; b=eMTwdtGFjbvVQoXhRX/iyjkk815eoFubGn5kVNq0OdKfwh8Krtaq8ybEe0Gxzu/O+y KIsvZKJt1zp1pLtuiSr0kP2cgPKDL+AUZ54lUBwztyui30nPWwS4LZKjT/nZdA8WwAhk VNzALLYK3IDoIunP6Qmc1d2mum6nH4+Jironf1gTMd5ex9ENVAVGks/Jt0DRzUjggpnb ny78NS0xfSgVp5cx0nd42nL+Uv4G3FMou6JzaJ8z+17elLLZNEV440l51x0TXcAfVjo0 zCtJyGPx/sIeN9oxibqxA6FfQjCMYe0q+DT6F0Ry643QWB+ORqn4rYBFWj91C/TSS2LA UVpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510267; x=1755115067; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=V6GitVs58LgSs5T9j7jt8hclUfm0doY6DCHFmJ55L1k=; b=UM7mO79mLXx/7UkzshdfpbKGtvAwAP7GPRssDPrmrv48theJpw5vCc6daU/1vE0gmG wGK2l3z3OwEB2LCjpF8vrbK7nd8JxkhBCl6oSG0lcNmlBV/Qn3Dq/F9qu4222osCRUnQ 9274vsy3hb0XPvJMEifCizu87L9KZJ079aM2qOu6jgHo9CqEEiHY/he/xcLrcXQo6jsy 5XYj9HkmdHEV6rTuEvPAfEg4wGVfPjZc9crI5tWCSslfrfeXYXLpzmIpyVvphWy5IY3w 50JTO7OaOgD7tQSD1jvagijKdg/xF0mjcOsQ8jSUnAUaODcMzrp0pZd3qJFlyyBWD27R dqOQ== X-Forwarded-Encrypted: i=1; AJvYcCWziCjAstRp2Gy1nUiUn7AtNGATxS3BBPhWlPMA0BtGje6I6kFB+oRQYBLDL36mD1ZqTJjqWC57+pefpaQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzzfK7Qhe84qOFh3CMFnQOU0A64uaZG0ejy9K78UDCeaB4T0Wr+ kVAsA5btOJQaodT/mm3AtR+C8H2a3/3K0dcDobdvVBpU3dHhWTKxD9HrZiuGCscvfopa94H/49x 2DRSvQw== X-Google-Smtp-Source: AGHT+IEe9aouosNHmFsjzcywZl+bDxrMa7gzbIrywT1O9OJ/U5YazLpsqSIEAJkZctqsm307cXJuaDMPuPY= X-Received: from pjbee13.prod.google.com ([2002:a17:90a:fc4d:b0:311:462d:cb60]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2743:b0:311:fde5:c4be with SMTP id 98e67ed59e1d1-32166d06ed8mr5169966a91.35.1754510266366; Wed, 06 Aug 2025 12:57:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:32 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-11-seanjc@google.com> Subject: [PATCH v5 10/44] perf/x86/core: Do not set bit width for unavailable counters From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sandipan Das Not all x86 processors have fixed counters. It may also be the case that a processor has only fixed counters and no general-purpose counters. Set the bit widths corresponding to each counter type only if such counters are available. Fixes: b3d9468a8bd2 ("perf, x86: Expose perf capability to other modules") Signed-off-by: Sandipan Das Co-developed-by: Dapeng Mi Signed-off-by: Dapeng Mi Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/events/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 9b0525b252f1..b8583a6962f1 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -3125,8 +3125,8 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capab= ility *cap) cap->version =3D x86_pmu.version; cap->num_counters_gp =3D x86_pmu_num_counters(NULL); cap->num_counters_fixed =3D x86_pmu_num_counters_fixed(NULL); - cap->bit_width_gp =3D x86_pmu.cntval_bits; - cap->bit_width_fixed =3D x86_pmu.cntval_bits; + cap->bit_width_gp =3D cap->num_counters_gp ? x86_pmu.cntval_bits : 0; + cap->bit_width_fixed =3D cap->num_counters_fixed ? x86_pmu.cntval_bits : = 0; cap->events_mask =3D (unsigned int)x86_pmu.events_maskl; cap->events_mask_len =3D x86_pmu.events_mask_len; cap->pebs_ept =3D x86_pmu.pebs_ept; --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBFDE2D2388 for ; Wed, 6 Aug 2025 19:57:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510272; cv=none; b=rGQutuBvCU6jUZVyKqPHQQ7Spm2AFKIjCmpS18GJuDYfDL3puRNSyydUS1TH7Y8Zzih6M5gAZLvmMcsqvuU2h265Hfv9cc3lxbRxW1PmILp0dhC1zv4NLPDruTwuBJdimiFJx38W+kvgKavJKCOleRCostN1v0w7+NneGGEKKHk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510272; c=relaxed/simple; bh=dsL7u0gCksSjkMYlnKFIZN+mhAdNQOloX5K9ddADbjw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=prP/hFRxBSmHWn2nC/B/xwuIqKyTMQDvPNDyxB9JQjo+AVVT4VWrHfAaYAlXMbAIzIYQnmh2QJ2tKMqIxemdxAQAQWY18kD9r46Fdm90/k2SAenqzfn3FuzBI9KhhRIy4OAFPWDpbn76YpWBjEp/M1yWUbH01bVHF0JdMV7sWJU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3mUGGDZ+; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3mUGGDZ+" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31ea430d543so310739a91.3 for ; Wed, 06 Aug 2025 12:57:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510269; x=1755115069; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=zcpMdowVJLAIFzpWUVXDuq4S/gKqPMBYO1OzyYmcx64=; b=3mUGGDZ+zv7uU59L8Ffhd//Jtbx25i3JD1bz2eGo6Z3VsU1hrc52wMguL5eBGBpd07 kvDkTu/02am6NGibACJdrCbahTNW0RfmdXOCjn1dkuKsTyTz1JZZ7FXCq+jsI53zSKEd xmRm5+ZTX9+Y/FVsOZaiz/UVmhqPc7kemgrGx2gmWvzAgMWJqU+F2/iKGz1j8nzUk21M Xa7p7wdmHoynPZahfgtmt0iTe+C7gbcarlY6Py1Y6m2OkkjAldVnkNYOIDhGBbJLmOZ6 lLYNcB1o6KuUNE4Acbs12URw9MLQiP3diNrz1QbIo0PtMR7sdVyeNA8CCUFWExuDBACP rDXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510269; x=1755115069; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zcpMdowVJLAIFzpWUVXDuq4S/gKqPMBYO1OzyYmcx64=; b=F+8F100FjeYeLoKCmaRzyDeBZD6IX1R9XTxDqlO9VnUbIXPmtUcL+SPEaAbFcPnJDu g10OVZ8+eMlhyQ5q04vevCwseJevHHi+mGIGlQt0TW/ndh0YL5annfQdSAnewWZOeXiY h4VblkGtExJD7aTdBUIWmsiXeRmv7VCYCTp4yKZyZIkxU5dKnK7JoMc3ao0Adg51ogrZ v7FJFmSacFSdcBcy34l/mBgy1Qgtrd4Y91tJQ6HgctBYw/0oqV7t8W31ZQMW/j6hhT5r Mn5ENGwiwKeUu7/f+dwfRgZp4oN3H1OmZr0Nju8zBHOVCBpHPGfTaz9oM7bJ7hfNsGm9 ktJA== X-Forwarded-Encrypted: i=1; AJvYcCW8D9F8i2h+10NkJahu+HBlcuZVnRIYl6t5Dc8A7p6srWT4golaBvErnX9n7TPkOQlVaI6u9x2tQIt2Qy8=@vger.kernel.org X-Gm-Message-State: AOJu0YyL1+6L3sWuJBxZCsCSg7LP+vhI7NbyKP11dk0+lPUMF1D4y7N3 6PqZLMhKcjiDAYzGoVZAJwSbFZ/ALWno6wIBthMFIfUg+SN5CDbJTGaVfhG9IzY5+YSt3fz3EJQ sN8QGVw== X-Google-Smtp-Source: AGHT+IEVLR7eG3e1/ODg1GEbzlPUzUDlLQzQgQfsonV4iFGNfX+Tq8xQvuZk8V1U7zslQQy51nOqc0FuQr8= X-Received: from pjtd2.prod.google.com ([2002:a17:90b:42:b0:321:76a2:947c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:380a:b0:31f:36da:3f85 with SMTP id 98e67ed59e1d1-32167520ee1mr6465429a91.17.1754510268743; Wed, 06 Aug 2025 12:57:48 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:33 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-12-seanjc@google.com> Subject: [PATCH v5 11/44] perf/x86/core: Plumb mediated PMU capability from x86_pmu to x86_pmu_cap From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mingwei Zhang Plumb mediated PMU capability to x86_pmu_cap in order to let any kernel entity such as KVM know that host PMU support mediated PMU mode and has the implementation. Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/events/core.c | 1 + arch/x86/include/asm/perf_event.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index b8583a6962f1..d58aa316b65a 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -3130,6 +3130,7 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capab= ility *cap) cap->events_mask =3D (unsigned int)x86_pmu.events_maskl; cap->events_mask_len =3D x86_pmu.events_mask_len; cap->pebs_ept =3D x86_pmu.pebs_ept; + cap->mediated =3D !!(pmu.capabilities & PERF_PMU_CAP_MEDIATED_VPMU); } EXPORT_SYMBOL_GPL(perf_get_x86_pmu_capability); =20 diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_= event.h index 70d1d94aca7e..74db361a53d3 100644 --- a/arch/x86/include/asm/perf_event.h +++ b/arch/x86/include/asm/perf_event.h @@ -292,6 +292,7 @@ struct x86_pmu_capability { unsigned int events_mask; int events_mask_len; unsigned int pebs_ept :1; + unsigned int mediated :1; }; =20 /* --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90E782D239D for ; Wed, 6 Aug 2025 19:57:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510274; cv=none; b=fI3Bo8y1gSKUWRT3uyx3k8jdXf8ivSahqAIZ78l9YksBzyRli6PpyxAGOulu3U10tDiJxd2WJnKdxwdoKBD81S1swwXl9x+qhE7QEeIavXmam4FSRosM4tCvRGk7XAPhVnA3F1wOXWHOmJKRDx9xwAp7Q8pq5JUQVoaq9B+kdOc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510274; c=relaxed/simple; bh=Bb1MCKlidS4HUVTcCu642tswWStbNqiXpa6NcxDtYbk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QSxj2CVBKd6mGFEPT9wTWvip6u/v7C8Hjczv0QfwX3p30F2wz0BKhDJyh4zVixh2pn1+dhlEfFg9MHFLiXi0dfGrUWiEG0w28jiTXACHbfiwc094GWrMQJqkn5RNh6Cm6h06vkXJlf3G+WQMKAk1R7NYmkpu1G8Mxy9jM79HExM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pGBJ+uSW; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pGBJ+uSW" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31f3b65ce07so464451a91.1 for ; Wed, 06 Aug 2025 12:57:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510270; x=1755115070; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=P+6kUSR5aBTgNcunERrWOcQMCZiG0moZeRQ0turCJ94=; b=pGBJ+uSWmY4u2nY77yBijaQ3pMQAq32EqtGrVokpellFCEw6GPtef9oej+go+VN3I3 Sq5svNwPiH82SYrZeJinnAa2YMkUdBujXyglUR2OImyeJBRhKpQQpG9py2/v+3G+ncOF 7Rf40w9xO1TFBSjh0FXPu0hKqPe2JyHd65kH9G131jwI3FhQBBCyM4hJ0Ta6e80/z12d fhLNiCtFwFINpHkrfbI9nghTiO+tUfM+UnARDMfORgb4jyveWWhsAOcuMIaFmVAk54QK /k9KAWqE/sn93EILPowyW2z9Ir91XNmlVpr+XHtFxbpxWP4gWBn6Ik31zh4M8hY1uDwi qRQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510270; x=1755115070; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=P+6kUSR5aBTgNcunERrWOcQMCZiG0moZeRQ0turCJ94=; b=fZDb4XfSKUckNPqtuyaC4LeD/4codAbMloqwNzQNeDTeNy+42hXkiy8zzPaY75KYyn SuXn+/tyh5CkD6dYdBLlLXqExND86UdCh63bd6YeMyUdfo1kSl/CU1RQJvTonzyUGoL9 4FbcM6TOl5JUA6Qcwm6qkmEC2FshZ+gyE0lhp1ndRFzvNIRWb1h4/dogA+0INhHMq1bu 9HN5JnVjXWi6UJtmWp0LM0IKphJ1o5jPBm/4FXzu/Yb9K/Dss3p9EpaSS6T99FNgdnKs 7Qav9bzTUNjPbNQSi/vUIyPtTOWIjNXoyjJ6lpHDYAJozadTt4SSCtUQhbGm1teXinFM 1Fdw== X-Forwarded-Encrypted: i=1; AJvYcCUsZta9VipxzHUq+aOqtoh783z2A7hlpcNEIWRYNXknclD7KttDetpz8gGtYt4wa9CeycqrFRwdsEu4Fdg=@vger.kernel.org X-Gm-Message-State: AOJu0Yyd7qTIhmmCXFrSLeqboRkzG1+j43fpUN/5S3H1yQq5YTDHHyv0 SrdvY6dqnv0sq4/WvVsjqWmyJrbWnA5W7grphygMJ/ramCrbQBqKOekG4cd8OqITnsG1vTRUyxR +fSDgSA== X-Google-Smtp-Source: AGHT+IF3NS6j+zPTC7HBpb/PhEo054/sULoDCrTNDdSbB3JBoD67HO9NJP/SyaYTvqhMXFbG0IGQUgBSmnM= X-Received: from pjbeu16.prod.google.com ([2002:a17:90a:f950:b0:31f:28cf:d340]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:e7d0:b0:312:ea46:3e66 with SMTP id 98e67ed59e1d1-32166ca3f0bmr5365514a91.21.1754510270478; Wed, 06 Aug 2025 12:57:50 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:34 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-13-seanjc@google.com> Subject: [PATCH v5 12/44] perf/x86/intel: Support PERF_PMU_CAP_MEDIATED_VPMU From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang Apply the PERF_PMU_CAP_MEDIATED_VPMU for Intel core PMU. It only indicates that the perf side of core PMU is ready to support the mediated vPMU. Besides the capability, the hypervisor, a.k.a. KVM, still needs to check the PMU version and other PMU features/capabilities to decide whether to enable support mediated vPMUs. Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang [sean: massage changelog] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/events/intel/core.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index c2fb729c270e..3d93fcf8b650 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -5322,6 +5322,8 @@ static void intel_pmu_check_hybrid_pmus(struct x86_hy= brid_pmu *pmu) else pmu->intel_ctrl &=3D ~(1ULL << GLOBAL_CTRL_EN_PERF_METRICS); =20 + pmu->pmu.capabilities |=3D PERF_PMU_CAP_MEDIATED_VPMU; + intel_pmu_check_event_constraints(pmu->event_constraints, pmu->cntr_mask64, pmu->fixed_cntr_mask64, @@ -6939,6 +6941,9 @@ __init int intel_pmu_init(void) pr_cont(" AnyThread deprecated, "); } =20 + /* The perf side of core PMU is ready to support the mediated vPMU. */ + x86_get_pmu(smp_processor_id())->capabilities |=3D PERF_PMU_CAP_MEDIATED_= VPMU; + /* * Many features on and after V6 require dynamic constraint, * e.g., Arch PEBS, ACR. --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DE532D1F69 for ; Wed, 6 Aug 2025 19:57:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510276; cv=none; b=qbCSmN07EqJ5p/mpxAn/DE+9DlDeWYf4QZsGY0pyqImCPtlCTApy5bVfDAqXopnQwK3Lu9ze3Fa3+RSIltKTxI9EcIpyxjhxhXAymjjtqvSmVtRcP1+mN6i00y8DcfIo5D5e0qmRHSVnqjM02ywOCu8xeZZl1JwSkJ47MOPjVsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510276; c=relaxed/simple; bh=jdq1crWcKn8y7XDSp99YavaEpnm3s9VMlFDhCc8MHB4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kBdXckLypCvE0BIOYN4V6SEyu7AK++Ue78GxErP3dCUAxhf8mmV0BxKFrRuz2JFwgCtUIBqBZ5KQt5Ce15PyLJmG35bqz/tQLKvNZnQqxViF0cqI2ACPlDDEcmXS3G+z2Eia+li/HdCIW2XoOeeRNH89Ifv/ktdTq+X4I4IFHlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=sKnxSDF6; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sKnxSDF6" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2425e41424cso2203705ad.1 for ; Wed, 06 Aug 2025 12:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510272; x=1755115072; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YZMeMFDLZr+yCjindHzAam67OkhMlNf3hlQ7J9N+n2w=; b=sKnxSDF6OzUh1O9m+DYJ84S2phrfAp9JUuB4ymZSub5iyiXZPwdD9+IbR2QFOZBYxD vhJbTii22o6Wo1vGPh4cm1O3YetTXyoXrKOP3k1HTndboNPVO909lBkA+/pq9vNsvZKo PgjMTdMNx9il0FdTnhxGhMUZcpNbFDfPjPAhOCj7WYSbGgMuzDooW4giPYpLxETuL5w3 /HtC4trCo/9jY6SSalI+/37npQ5ZXErfi5ahnGlWevemkZHGbaHH+YB6onilxP70llNQ lhtWVmA3bK4Qxxe/xQRV7LsbLGXqnnWRCRkBU1BC9Z3q43WdnWisiyieuThBh7Bo5WTb 1ugw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510272; x=1755115072; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YZMeMFDLZr+yCjindHzAam67OkhMlNf3hlQ7J9N+n2w=; b=B5BK0Fc5egzLP2h8zRjHA1lZuHkW44JKZXFdvVX+i4MhgkVUwSoii22c25Kwr9pYrD LLZDGwxu+U5azZsPjZlaFNe5jks1hvd1CAe9p5Q960B1+QebRBWQZKwe5fFDJsw4/f70 l2jdP3dAjtlfO9v0WZZHFAw5d7+ww/xEzsBVj+/ajnF0IJS2KghFfzL+3YvrDaLZitI+ S4hrnAyILSJ59J7HYtfumEs+eRg6+LQeZDLud/bWOh5ZL6sDqxUYudZP9USAz7x3fUqB WxoXbdvzS30Zlwj2PyWHKrVws0h9K7q7jY5cAFqWcr/7AO9VCSspqRu51JJmheFKBwZz grgQ== X-Forwarded-Encrypted: i=1; AJvYcCV1COx+OUVa2Veth4oYYDhhoF0IpfHLWpVBleqJ34f6HrEqgUlL8aSofDjY8cCJECpUcfgPUtwpi5ksw4Q=@vger.kernel.org X-Gm-Message-State: AOJu0YxssNLPq/ez1CRHpR15f7aUXP/pecYUoFMHOgqbflYVrFCZkm6k wbjHN4UZmFlLElrdBznx3KZbXrlm6chMHCVSiusAKzdflcsM0YnUsYFsXxynuwsmYM+Px+27yTZ UvuP11g== X-Google-Smtp-Source: AGHT+IFt32OpTUGU6us4Rz359zlOuJVL+j3ifqAbqQAET68s3resGqbQqEcv8vf6Ss5FatBMSRwumXjkA8M= X-Received: from pjbso3.prod.google.com ([2002:a17:90b:1f83:b0:311:1a09:11ff]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e784:b0:240:9f9:46a0 with SMTP id d9443c01a7336-242b1aa312cmr4180365ad.38.1754510272385; Wed, 06 Aug 2025 12:57:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:35 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-14-seanjc@google.com> Subject: [PATCH v5 13/44] perf/x86/amd: Support PERF_PMU_CAP_MEDIATED_VPMU for AMD host From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sandipan Das Apply the PERF_PMU_CAP_MEDIATED_VPMU flag for version 2 and later implementations of the core PMU. Aside from having Global Control and Status registers, virtualizing the PMU using the mediated model requires an interface to set or clear the overflow bits in the Global Status MSRs while restoring or saving the PMU context of a vCPU. PerfMonV2-capable hardware has additional MSRs for this purpose, namely PerfCntrGlobalStatusSet and PerfCntrGlobalStatusClr, thereby making it suitable for use with mediated vPMU. Signed-off-by: Sandipan Das Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/events/amd/core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index b20661b8621d..8179fb5f1ee3 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -1433,6 +1433,8 @@ static int __init amd_core_pmu_init(void) =20 amd_pmu_global_cntr_mask =3D x86_pmu.cntr_mask64; =20 + x86_get_pmu(smp_processor_id())->capabilities |=3D PERF_PMU_CAP_MEDIATED= _VPMU; + /* Update PMC handling functions */ x86_pmu.enable_all =3D amd_pmu_v2_enable_all; x86_pmu.disable_all =3D amd_pmu_v2_disable_all; --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 255102D3230 for ; Wed, 6 Aug 2025 19:57:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510277; cv=none; b=BXGey7YQ7j+RWSkytM1rRZW8rt0BUDXxmo035TkHHxGhrO/CaMsdkxwGQ0m6GgClbwQo1K725tIK01OHLJfg58Q8tFrC4C1SrVVien8AesYwdocjMVCqNyg3lq+UkH8zrlXdczu+506BDFAUIxi4JeCIBXXUlDOxiFjDwf5VWFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510277; c=relaxed/simple; bh=C4TXme1dQYfX3HBX2bRS9WhICMNyLXZykAhB0txM4zg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZMhU0Vw0QdfOvBdcs2avRdGNZCXz54juNeOFTJf/ods3iPoouc5bSGorRjCUggvUeYPWR8/i+efbfAUjswUg/TScTl6+ssEFK0MuRg1YxfZ0IbkysLoYms+K1MK0AtxnGCMabbCogoC5wMIFp3pAG1Xmr6GViWGVc0PYlwTsVGQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Pl9ZsfmU; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Pl9ZsfmU" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-23fe98c50daso1497435ad.3 for ; Wed, 06 Aug 2025 12:57:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510274; x=1755115074; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0uQwWV7lYMOxBil53zyDkn96gfX2tt2/DqWsDrl/9hs=; b=Pl9ZsfmUpZY8vSNBS2r2M3kPAS0csHRRtJ4ESTdtPDF45DJk87cCDv2U+GKR2CsIPZ hI7J/d8Z/XWOb4jxCsjS1esCSyaH4Roo6IYL/tYUG0cdm6NO8iWw6Il3t8M4kukamOs5 +cEY+VT2IyVElMw3Rx7+nUmJNpntPf643sujcpHTOPa3gr0kfvu/EIKUJizLABGGqXH/ SroNv0EN67ezVFNbOczU1n5JdXtGQQnA5I4aNJnblnabEDq2CHiru8RmLYku3r5PmImT 7oYQdkY0iCDteciU2cgemBYp5uPdGamFGlplyPLYPe38aOXzjWExn5UMMJG1EmfYgyR0 JFkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510274; x=1755115074; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0uQwWV7lYMOxBil53zyDkn96gfX2tt2/DqWsDrl/9hs=; b=iLt+I8JKajax+YLWq+u8+jLbZ2J5itqy2hCKPB02+i7qU7UAjZU54gbYrs8GBUCnJh JV6Dn2rVjePLKlWRdnDB/lYCf09mQw070PLCGLrJrKG83zaDD0TcuGQ5mOpXX5MZRf4U 4VC1XW5DWdi+kWp81owRc2ON6redqoFSctEM/sJb0j3ElhAcM4CENEAQ6wYhuBdiKk7C Vls2eLTHYADHx03e/97mYH03Z4oDVu/vyWOQ2ZsVr1MaqXse0jO3qDhuLqMBiX39Xa1g H9w0AwhfmeXWqjrYrq+fgJZFP4mWAX4wTNL697DVms9NFiErl5CA1o+1787GOhHHNdqU QtHA== X-Forwarded-Encrypted: i=1; AJvYcCWhe/UTod8nVqi26rh/NWrA3tI2PanhY7pjssqNrAZVgdeCe/0MCkjX6UZ25+K30CfJZbBRva2IxPVzu7Y=@vger.kernel.org X-Gm-Message-State: AOJu0Yw89ldQzdRikoUy8dz/54CCTTnsngSed7H+K0ClSd9QztycUHtx 065gpt2wiwA4KJvAI4sbvF3VTlRnP/Ushrs4BxWbYy3mlfAf/WdQ7Beea5AUoCii4UrMCM+PZ17 si9P6Sg== X-Google-Smtp-Source: AGHT+IGH2b2xUlwmCehGLeN80BLd5ICkd9hx9VxyJMjC6VJA8pJue1wqM8eYOcrMqHKsQHTGrwNwqHX4jyM= X-Received: from plkh12.prod.google.com ([2002:a17:903:19ec:b0:240:9ca:fc49]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ce05:b0:240:49d1:6347 with SMTP id d9443c01a7336-2429f30b512mr54270245ad.35.1754510274358; Wed, 06 Aug 2025 12:57:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:36 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-15-seanjc@google.com> Subject: [PATCH v5 14/44] KVM: VMX: Setup canonical VMCS config prior to kvm_x86_vendor_init() From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Setup the golden VMCS config during vmx_init(), before the call to kvm_x86_vendor_init(), instead of waiting until the callback to do hardware setup. setup_vmcs_config() only touches VMX state, i.e. doesn't poke anything in kvm.ko, and has no runtime dependencies beyond hv_init_evmcs(). Setting the VMCS config early on will allow referencing VMCS and VMX capabilities at any point during setup, e.g. to check for PERF_GLOBAL_CTRL save/load support during mediated PMU initialization. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/vmx/vmx.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 95765db52992..ed10013dac95 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8335,8 +8335,6 @@ __init int vmx_hardware_setup(void) =20 vmx_setup_user_return_msrs(); =20 - if (setup_vmcs_config(&vmcs_config, &vmx_capability) < 0) - return -EIO; =20 if (boot_cpu_has(X86_FEATURE_NX)) kvm_enable_efer_bits(EFER_NX); @@ -8560,11 +8558,18 @@ int __init vmx_init(void) return -EOPNOTSUPP; =20 /* - * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing - * to unwind if a later step fails. + * Note, VMCS and eVMCS configuration only touch VMX knobs/variables, + * i.e. there's nothing to unwind if a later step fails. */ hv_init_evmcs(); =20 + /* + * Parse the VMCS config and VMX capabilities before anything else, so + * that the information is available to all setup flows. + */ + if (setup_vmcs_config(&vmcs_config, &vmx_capability) < 0) + return -EIO; + r =3D kvm_x86_vendor_init(&vt_init_ops); if (r) return r; --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14CB52D3751 for ; Wed, 6 Aug 2025 19:57:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510279; cv=none; b=oUmC37h6ld5HyRISTbCepYH5+uPQVRrpEQY1fB8lOf5ybz3gMtoRDZxdT98wPy3kuKPfKWgKrn1SwVuDku8oUazUl8D4YeSOnTzMmFgBEYr+cNr6DgK+zBKLF27UeeFpetrvbQwYNyVwHpvCMiJcucYEulADpzuCwzvuUDKA2RA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510279; c=relaxed/simple; bh=Wn/70g/zhO0dUDFH5wZEa7RcDliEFiv6OnJtKAjielc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MY/WiGXUvKnOFAM/XZgEy4oETSkIVwwQD8VvhU3So7GQInoknUVksjiYLUI0GPl/MJJ6FdUIkoTetU02UFCZfWcaNVU7DxBp/9PtegXN+Q9oR5MD+t7tfax7uCKhljH4Sx7YPqFniyXasWerbU+rSwuLUwsTM8Ug5XmZK+zExgk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jtkmJc9c; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jtkmJc9c" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-23ffd934201so1340785ad.3 for ; Wed, 06 Aug 2025 12:57:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510276; x=1755115076; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=kcswufqwZeiVbOah5oR7VRWteUmksZlp7f05xG34s+k=; b=jtkmJc9cJ0fzxKszfmBkbNZX/tuaEHULFvmP05TZ18dN1rIKrRyrB5q7dtsBTrnONr UIp9is0G1CYORP6k0z1bb83xk5lmWUGBUFGLQjphTZlDLu4hmJroHvUDaK6STfruRInP y53fCsPu1TtFZS+20TDfjsYoxNaKyF3jNnqmNSYdf6gfKLylB04bUGqrbgRK8jQKz0YR Si0u74euyD1i0svnazPo8+LaJ++KvSw+3awEVvnFnG+Nykxkek8YK/WIZVDOTd0mJmid QIgIDq2RVp/hxU5RqkXeQFgV6Z6QXjeHyXkeq8b+fk3Jhk6CgoSI4cgunRPLS7fm2BKp OrZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510276; x=1755115076; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kcswufqwZeiVbOah5oR7VRWteUmksZlp7f05xG34s+k=; b=Wb71UMSlCRFRSWqnyWRe+e8xVP97ja39JJpjeSMYucGwW9vVz0r6v1a93m8AwhZo2C piOFxiqxWw5smumcGYi50VpHydT3svLrJ50wma2C5s7yldSAjbwlK8c/uaQ78Vqs1FnQ gjsE/yp7X4SE57GMGfFm/9xA8l01ykHRLcl7mvFAn46BIJJDD6Sjoaplw0xeqRyezqlK a72WM7n1dP/sjT6eIs/4j7oh82WSo8h0EsGtn3HWk/wTAEemjSDEMexap9VOIy6bW5am jn6JHrr/ADMI/K0orXjaJUYvwdOhBsSNKIZr+k35iRwgyzQ9jAb6BHg6HAiRtCm0cxgd eixg== X-Forwarded-Encrypted: i=1; AJvYcCWs7E1YBugvIqWUs0qp4TtYPmyi5PFVn+Evqvdo7AjRyI5rKZdfZl1rDv7fHNfPibaGh7s1wx5upgYUwK8=@vger.kernel.org X-Gm-Message-State: AOJu0YwrxYtKaC6q640/l43SiPko9Zku+nA5lDelWF/m43JTUADnMrXq jacsYihvjii/MrK/5jCJj7I1OqOkQPa3Z+OLayOZe5rlat+LGBhi6C8WZErXoEbh8av6AKM3hC4 nq0/lag== X-Google-Smtp-Source: AGHT+IE/UqkBVmIsYHo+3//+Db76eNFThoPapPb1lJatPNoq0GTt8I/qj7Zd+31YDzfBkj++w1pCOc8gls4= X-Received: from pjbnd14.prod.google.com ([2002:a17:90b:4cce:b0:31f:2a78:943]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ec8e:b0:235:ef67:b5a0 with SMTP id d9443c01a7336-242a0b3e6f7mr55163055ad.36.1754510276193; Wed, 06 Aug 2025 12:57:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:37 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-16-seanjc@google.com> Subject: [PATCH v5 15/44] KVM: SVM: Check pmu->version, not enable_pmu, when getting PMC MSRs From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Gate access to PMC MSRs based on pmu->version, not on kvm->arch.enable_pmu, to more accurately reflect KVM's behavior. This is a glorified nop, as pmu->version and pmu->nr_arch_gp_counters can only be non-zero if amd_pmu_refresh() is reached, kvm_pmu_refresh() invokes amd_pmu_refresh() if and only if kvm->arch.enable_pmu is true, and amd_pmu_refresh() forces pmu->version to be 1 or 2. I.e. the following holds true: !pmu->nr_arch_gp_counters || kvm->arch.enable_pmu =3D=3D (pmu->version > = 0) and so the only way for amd_pmu_get_pmc() to return a non-NULL value is if both kvm->arch.enable_pmu and pmu->version evaluate to true. No real functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Sandipan Das Tested-by: Xudong Hao --- arch/x86/kvm/svm/pmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index 288f7f2a46f2..7b8577f3c57a 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -41,7 +41,7 @@ static inline struct kvm_pmc *get_gp_pmc_amd(struct kvm_p= mu *pmu, u32 msr, struct kvm_vcpu *vcpu =3D pmu_to_vcpu(pmu); unsigned int idx; =20 - if (!vcpu->kvm->arch.enable_pmu) + if (!pmu->version) return NULL; =20 switch (msr) { --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D30152D3ED0 for ; Wed, 6 Aug 2025 19:57:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510282; cv=none; b=FkHXovilgs2Sz55RFHhP1dK93M4rM4hSYvh739ZcJezdqfx6WLdzrHKtmSiAynKdhGPsm5QAOJ90/eg7WMuib8ey2e3MM2vflgZEbngsTq1X3ZCW8qD9ZuXeMOnlTIKAMNN29RjKXIcHPVNqOVH0THh0LSeaphPagT59kmAj1Sg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510282; c=relaxed/simple; bh=S2PXduPc6MjaQTDHugYUZGj6hH4KjT2QfQWxoS/aD0c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rR0/Qi2sS4FZ5stkdP3DyDIcuokFueeMtqwclhHSXokSZavVmrQHOznCWJivcZCaWXk9kvkmiyAsYPxvKWPxX/6Ei4IZD3/bozLY6wl22q1SNdsuw5OX5JuLqf3LbbLxEeaYgUt2ghz4rjr1UbgaI9S6aws9ehYRhCfF1w3y3AI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RftfEKTN; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RftfEKTN" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31f2dd307d4so279794a91.0 for ; Wed, 06 Aug 2025 12:57:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510278; x=1755115078; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=JPhdbzRHNYwbPflarjN5FFRR+xgGKTfGx/VQdQ/Orqk=; b=RftfEKTNvfTY8Q1dM7kumzdRIgT/7PwiLqewU5flhj4i88I029T4mOwusCNgRsRM/C 3DC+Zom+mfGr0pw8qIJ7GkY3JK8TEayzHs+dRHTLtAp2ykPEs/SKvy+ZjK1/tJO+vrNF hJsVz6n0fZyRqtP/kN78Z9W/dcP9xZsIgJa4vA+ukrnnkEfVirucpGvl6IHrgWbTVchS 0qZYgVaJVn8/TrhJHAtxBDHp9ToYa8OQDoYRj9gH/GaH3j5ZM/om45iAIeO8w4rz1o5W MjNUxzir0SPCwmAbSPUOpRWZh+t4W2RU7eVF7ljOQ0rHDhkkAG+b5L1u+YJByKZIGab/ iFIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510278; x=1755115078; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JPhdbzRHNYwbPflarjN5FFRR+xgGKTfGx/VQdQ/Orqk=; b=UAhmMlXWzRZxHg+0IMSZWpfMPCz6+rc/JahkCW3VQfLa8MvgXyMeUu8OmrGycWT+Kj vhRnIAwZ38PX8s9JWtjoJtVXPg2npvWDqiBNWcaKGAiDl1vafWV7VQfPgsclMs9jAizN VvifJgERZdU17aynaABYEfhwTVOabcgBZcBXpBY6vBY2QFQwWK+Ow+gfQ1wJYRanRh3W 3tlymZnYNcR+UO6/CsPx0q38gCG/1h2lQb1G8gqhb1gWNCUuFSfVcICYAs45Xg/YBeOd ywJBeiOUulFdmxyxAoHaV8YLiQfRaS/qjHSKyGBDhsZbscUnuTGiDhm9lHz2mxMDTCWL uZVw== X-Forwarded-Encrypted: i=1; AJvYcCV09O7O+dncWuU/W728C2dm7s7tfINeo4JceBU56bRW2HMNuOLqs1vJlg7elJtdshUCNDLBYfAS/BXKOsE=@vger.kernel.org X-Gm-Message-State: AOJu0YyAjewqZlnU92lX6vchApybIXL75L2ExTX9l/WFc27HQ1sXnFdW xqRWOE49JBCLdL33lD9vAcqd94V+ewe2H4k6vFrIeaBrNwRW/Y1tMVAAlWAeOrqkvOuXkklVsdz s56nS6g== X-Google-Smtp-Source: AGHT+IFSe1FbjsND/XiMAlJbwSv08tGHE8p8b2tGPxUBNGb/T59D+jgsyqvY56UbSdKD1Cfbmzp8QhdLT3k= X-Received: from pjzj16.prod.google.com ([2002:a17:90a:eb10:b0:313:274d:3007]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2c86:b0:312:ffdc:42b2 with SMTP id 98e67ed59e1d1-32167541295mr4635514a91.23.1754510278028; Wed, 06 Aug 2025 12:57:58 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:38 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-17-seanjc@google.com> Subject: [PATCH v5 16/44] KVM: Add a simplified wrapper for registering perf callbacks From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a parameter-less API for registering perf callbacks in anticipation of introducing another x86-only parameter for handling mediated PMU PMIs. No functional change intended. Signed-off-by: Sean Christopherson Acked-by: Anup Patel Tested-by: Xudong Hao --- arch/arm64/kvm/arm.c | 2 +- arch/loongarch/kvm/main.c | 2 +- arch/riscv/kvm/main.c | 2 +- arch/x86/kvm/x86.c | 2 +- include/linux/kvm_host.h | 11 +++++++++-- virt/kvm/kvm_main.c | 5 +++-- 6 files changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 888f7c7abf54..6c604b5214f2 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -2328,7 +2328,7 @@ static int __init init_subsystems(void) if (err) goto out; =20 - kvm_register_perf_callbacks(NULL); + kvm_register_perf_callbacks(); =20 out: if (err) diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c index 80ea63d465b8..f62326fe29fa 100644 --- a/arch/loongarch/kvm/main.c +++ b/arch/loongarch/kvm/main.c @@ -394,7 +394,7 @@ static int kvm_loongarch_env_init(void) } =20 kvm_init_gcsr_flag(); - kvm_register_perf_callbacks(NULL); + kvm_register_perf_callbacks(); =20 /* Register LoongArch IPI interrupt controller interface. */ ret =3D kvm_loongarch_register_ipi_device(); diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c index 67c876de74ef..cbe842c2f615 100644 --- a/arch/riscv/kvm/main.c +++ b/arch/riscv/kvm/main.c @@ -159,7 +159,7 @@ static int __init riscv_kvm_init(void) kvm_info("AIA available with %d guest external interrupts\n", kvm_riscv_aia_nr_hgei); =20 - kvm_register_perf_callbacks(NULL); + kvm_register_perf_callbacks(); =20 rc =3D kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE); if (rc) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 5af2c5aed0f2..d80bbd5e0859 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9689,7 +9689,7 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) set_hv_tscchange_cb(kvm_hyperv_tsc_notifier); #endif =20 - kvm_register_perf_callbacks(ops->handle_intel_pt_intr); + __kvm_register_perf_callbacks(ops->handle_intel_pt_intr, NULL); =20 if (IS_ENABLED(CONFIG_KVM_SW_PROTECTED_VM) && tdp_mmu_enabled) kvm_caps.supported_vm_types |=3D BIT(KVM_X86_SW_PROTECTED_VM); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 15656b7fba6c..20c50eaa0089 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1731,10 +1731,17 @@ static inline bool kvm_arch_intc_initialized(struct= kvm *kvm) #ifdef CONFIG_GUEST_PERF_EVENTS unsigned long kvm_arch_vcpu_get_ip(struct kvm_vcpu *vcpu); =20 -void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void)); +void __kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void), + void (*mediated_pmi_handler)(void)); + +static inline void kvm_register_perf_callbacks(void) +{ + __kvm_register_perf_callbacks(NULL, NULL); +} + void kvm_unregister_perf_callbacks(void); #else -static inline void kvm_register_perf_callbacks(void *ign) {} +static inline void kvm_register_perf_callbacks(void) {} static inline void kvm_unregister_perf_callbacks(void) {} #endif /* CONFIG_GUEST_PERF_EVENTS */ =20 diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ecafab2e17d9..d477a7fda0ae 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -6429,10 +6429,11 @@ static struct perf_guest_info_callbacks kvm_guest_c= bs =3D { .handle_mediated_pmi =3D NULL, }; =20 -void kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void)) +void __kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void), + void (*mediated_pmi_handler)(void)) { kvm_guest_cbs.handle_intel_pt_intr =3D pt_intr_handler; - kvm_guest_cbs.handle_mediated_pmi =3D NULL; + kvm_guest_cbs.handle_mediated_pmi =3D mediated_pmi_handler; =20 perf_register_guest_info_callbacks(&kvm_guest_cbs); } --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7CFC2D4B44 for ; Wed, 6 Aug 2025 19:58:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510284; cv=none; b=NV706HKcx9neKk1jafenHVhphnPybu4MHQAB3+hClqbAHcP9PQo6P0BmM49/JxXNshK5U3r3N3z6OzTQoUTdQHwI6AGD5pz/PL14xzCZhizibLuTg+nDBiHBwk98jnVipD4fubuxJ+EDT643QJzXgRqC/cxfQvPiuPPmCXD78kQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510284; c=relaxed/simple; bh=9XrrLDv04mrzfeSThdad7tM9DkVcx6lRnARpaMqcOYI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sCNW6xhywMVEmeEUsJy09cVEoZGoQPURwKiaSbxKrgfgAZIb3dVE09RpAG+JMaCxgC3XIY1Ie6g+htaj+MTPNWW8bYC8stKvvlyWfY5JnPJCgBHWNO2FWpQ/lXnb47fu10JxGtVXdf1kl+36GPyDCgJHerynjoMS1f3QMfe58G4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wRkY+fOy; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wRkY+fOy" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-76bf2f8ba3aso969681b3a.1 for ; Wed, 06 Aug 2025 12:58:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510280; x=1755115080; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=1JVfLN/93q2WYEPFa3NA5hoBJW9RWjYsNWhQaVXoFO0=; b=wRkY+fOyj90sgFPKqK+HweBGF5mKFig72a1Pca5Dkf+rnCTNvYc4jxhFBWGUrB2q6n amFf6+aPpPi5X2pzFExIjzhwigyNmcHy5ezQeZHMlyuJbwPol778786qxbz+X8Q36nvg uPRuSKmMy8YOPPzr3p6/zz7gKi8YZOmtQxe53IXe81nYtwqveYWNNbFwIyhx4/gywL7x QWcjpv9DXFxSv5NVrtFO8AfWNpSoGHqVZHOT/8gliuqThmvQWpL/+muasgVs2W9vcS1C EM0b5lb1yYoKrokIcYeuDZZjMzdW9YRi1uwume6FNtEOlZ7XF18arZ3ZD7JeQg/1t1ns HcXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510280; x=1755115080; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1JVfLN/93q2WYEPFa3NA5hoBJW9RWjYsNWhQaVXoFO0=; b=Ex5Whf58Ly9F5rzhakF97pGEgFPArJujsPtwqcIwuY4QhNsXzvLaF/04MvRRfOAqAW u8eSpdQ3y0yKRVHo1qTZf7gHj5x5gU2nWT+AwrWHT9jFPoHpjalSXs8vHs96zjsuWYEj 9gf9vcSey05SlESSiOQAtTh1mZ95JmyrUDGAViD48xIFDM1MWAHSUwt1TfRAXqus/+ZH nQSW1Mf/CPRKeGSRQHdXbQLl0cq9mtFwgo/cghINp2eDnpD1/GKSYAVXkVBbWBUniCX1 /uUWoiAvwDHkM6e75xn/adJi5Y2eLey5Ote1c6prr7mqu4GXYpEDUR6faN+g7ITtI9YY +hmg== X-Forwarded-Encrypted: i=1; AJvYcCVjklTxhLcN/Ds7U9SdGzEP8hhpZLpVtg2AJwJ80VZr//msOHSnK2XCUBbbrPTcOz1bv9CsYdjcXSCgvuI=@vger.kernel.org X-Gm-Message-State: AOJu0Yxs2IoqXaZjb3S5jgRk7j9Ot8wsqqSgx+0F4raA7gWvzmzkdgF/ rwcl8hKGDXHdFKsKYQBc5ycga+OO/I8n0JtldIQHGuBHM8lx3rFnjd7KM3xzG2hw/0yuvKClase boOSYgw== X-Google-Smtp-Source: AGHT+IGEqoS6XNELOGxlbJeBvkZJOh3lrJxf4kRuSXEbOp8u6GEP+lPQ4e26oXCRK4Rwc//kWZSE9nj7nV8= X-Received: from pgbcr1.prod.google.com ([2002:a05:6a02:4101:b0:b42:25bb:216b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:7d9d:b0:240:1d69:9cc9 with SMTP id adf61e73a8af0-24041326731mr1241511637.16.1754510279868; Wed, 06 Aug 2025 12:57:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:39 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-18-seanjc@google.com> Subject: [PATCH v5 17/44] KVM: x86/pmu: Snapshot host (i.e. perf's) reported PMU capabilities From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Take a snapshot of the unadulterated PMU capabilities provided by perf so that KVM can compare guest vPMU capabilities against hardware capabilities when determining whether or not to intercept PMU MSRs (and RDPMC). Signed-off-by: Sean Christopherson Reviewed-by: Sandipan Das Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 3206412a35a1..0f3e011824ed 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -26,6 +26,10 @@ /* This is enough to filter the vast majority of currently defined events.= */ #define KVM_PMU_EVENT_FILTER_MAX_EVENTS 300 =20 +/* Unadultered PMU capabilities of the host, i.e. of hardware. */ +static struct x86_pmu_capability __read_mostly kvm_host_pmu; + +/* KVM's PMU capabilities, i.e. the intersection of KVM and hardware suppo= rt. */ struct x86_pmu_capability __read_mostly kvm_pmu_cap; EXPORT_SYMBOL_GPL(kvm_pmu_cap); =20 @@ -104,6 +108,8 @@ void kvm_init_pmu_capability(const struct kvm_pmu_ops *= pmu_ops) bool is_intel =3D boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL; int min_nr_gp_ctrs =3D pmu_ops->MIN_NR_GP_COUNTERS; =20 + perf_get_x86_pmu_capability(&kvm_host_pmu); + /* * Hybrid PMUs don't play nice with virtualization without careful * configuration by userspace, and KVM's APIs for reporting supported @@ -114,18 +120,16 @@ void kvm_init_pmu_capability(const struct kvm_pmu_ops= *pmu_ops) enable_pmu =3D false; =20 if (enable_pmu) { - perf_get_x86_pmu_capability(&kvm_pmu_cap); - /* * WARN if perf did NOT disable hardware PMU if the number of * architecturally required GP counters aren't present, i.e. if * there are a non-zero number of counters, but fewer than what * is architecturally required. */ - if (!kvm_pmu_cap.num_counters_gp || - WARN_ON_ONCE(kvm_pmu_cap.num_counters_gp < min_nr_gp_ctrs)) + if (!kvm_host_pmu.num_counters_gp || + WARN_ON_ONCE(kvm_host_pmu.num_counters_gp < min_nr_gp_ctrs)) enable_pmu =3D false; - else if (is_intel && !kvm_pmu_cap.version) + else if (is_intel && !kvm_host_pmu.version) enable_pmu =3D false; } =20 @@ -134,6 +138,7 @@ void kvm_init_pmu_capability(const struct kvm_pmu_ops *= pmu_ops) return; } =20 + memcpy(&kvm_pmu_cap, &kvm_host_pmu, sizeof(kvm_host_pmu)); kvm_pmu_cap.version =3D min(kvm_pmu_cap.version, 2); kvm_pmu_cap.num_counters_gp =3D min(kvm_pmu_cap.num_counters_gp, pmu_ops->MAX_NR_GP_COUNTERS); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4BE02D3EDF for ; Wed, 6 Aug 2025 19:58:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510286; cv=none; b=nTX2QRz+vKga+nvdjD906bLDwAjKwH0mp1ylnYFUdhpDRwLjcDO2OO9+pniJ3ukprTomfAwwQ76xFbjE8duTOTuBkWyF49iR/Rv6tTHiO9LcQhwuATGFSxashJJYktyI2HsgbjiBOeEE9svO6dNfZmzPPP5+XTvkyVhMTwwDkQk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510286; c=relaxed/simple; bh=PBnkLLFYZ4s5Yu2szvXDrhvmuZxi6mGoAiraC4ZgJBg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=UFDFiv5p4NBegxd+XJ3BoAEYM+7RC0Tc4kmwoc3QzGdw+eJazcXKPfcQwAbuV1O3MqB13yygFimCl7cYo+Rb6d65fqzmEa9HkrQfv9oWF8ZWu0qxO8qIUhIUnY0i+2KOlEoHt3RN2P6IQncBFSMnREykgCEYfABhzNbocq/sJb0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=i7queIt9; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="i7queIt9" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-3215b3229c7so289717a91.2 for ; Wed, 06 Aug 2025 12:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510282; x=1755115082; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=+6JQFfV5PUy/GrVBLvUiN2dICAbUeOItkyPrKIFKW8s=; b=i7queIt9V50ibqXGMFXWSTONrwyZMUxKjj8xt7+Re0R1BKVqAOlmnb2rNJFkecz/R4 8xgPGZtgK8xHwWWgDMV/SfnI9gkxDXLYeB4TQl7XmQ8lX6ihE46Fbq08V9ehfkbg21vC oSgaWInhXoIo4d2Ue/tV5Pf2x4EZWrmazm022wujtd5w+u5y8CUwetYo9Wifm9MRLRcv GXqqekyOntW7NJ2IKyct5cU7/3nYyUr3LhehOCmmu2nnLpEP6CnIWcLcTNLWmYBHkfXh QEjRUO1KCd65/WTWgBe3LUgdUpaXwiY8YNa+lPTU44ggEn3+5YYBSkkV4/1VRky5uVOx FoGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510282; x=1755115082; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+6JQFfV5PUy/GrVBLvUiN2dICAbUeOItkyPrKIFKW8s=; b=kUs7ph1l9nYMKPfBlwEKBCZSkrRl89TgD1GBc7YiT0jdNb8u2EmzW3hSHXlHufdJZN crrldOLxcXQQzcdtCM1b8EeBlN1G8d6/+mVuo3H3nsZIsnfACG9+IJNo8uijgRD0QZ+8 bt3IubpCvj6GO+0mVsXHrsC3KejKN2sZssviUGo8SnvzPLy9P1QQBWoJ3pp3F176EzPT +0vRERm8IzbKhDv97boNexa4H+vyjsNy9T9jQy81XLvZeGQNQcL2pobxzubDi8pUgJ3j Z5yq074h8w3FE/fuW4as3Nfl/cy7XnFI+gqzQulCiEp+UiNslV5BDHN0IFqXdIJ0wKBL wyMg== X-Forwarded-Encrypted: i=1; AJvYcCU8YwESRuvSOdSKf46JXHH8+RbFa+DyKc4M3y2RPDNhZOFBnTGxNsmXRiAcU0//BL5+H7SAvTkRJ2rs8tg=@vger.kernel.org X-Gm-Message-State: AOJu0YzbacEQKLQuwZNP5Cq/6QgIPFeqJ7NYdbzcqg4CUzRznvQNLwZ1 GSWsm7qZnARjsjkX8RdT/FHECC7POaTiV/+kxnUZ2g5z2K5LMJLkzz/noGKixJvKg5HkGiaJQJD iCoXPeA== X-Google-Smtp-Source: AGHT+IGZ7oKb0jL8lhNyyp9dHD+Kz4ijKlmhkeksSN6TGP0ouV8c6R8BFKEuTTGgS97308bG3SsY+cafJuk= X-Received: from pjbqi10.prod.google.com ([2002:a17:90b:274a:b0:311:c5d3:c7d0]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1812:b0:318:f0cb:5a50 with SMTP id 98e67ed59e1d1-32166cc88e8mr4877815a91.26.1754510281698; Wed, 06 Aug 2025 12:58:01 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:40 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-19-seanjc@google.com> Subject: [PATCH v5 18/44] KVM: x86/pmu: Start stubbing in mediated PMU support From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Introduce enable_mediated_pmu as a global variable, with the intent of exposing it to userspace a vendor module parameter, to control and reflect mediated vPMU support. Wire up the perf plumbing to create+release a mediated PMU, but defer exposing the parameter to userspace until KVM support for a mediated PMUs is fully landed. To (a) minimize compatibility issues, (b) to give userspace a chance to opt out of the restrictive side-effects of perf_create_mediated_pmu(), and (c) to avoid adding new dependencies between enabling an in-kernel irqchip and a mediated vPMU, defer "creating" a mediated PMU in perf until the first vCPU is created. Regarding userspace compatibility, an alternative solution would be to make the mediated PMU fully opt-in, e.g. to avoid unexpected failure due to perf_create_mediated_pmu() failing. Ironically, that approach creates an even bigger compatibility issue, as turning on enable_mediated_pmu would silently break VMMs that don't utilize KVM_CAP_PMU_CAPABILITY (well, silently until the guest tried to access PMU assets). Regarding an in-kernel irqchip, create a mediated PMU if and only if the VM has an in-kernel local APIC, as the mediated PMU will take a hard dependency on forwarding PMIs to the guest without bouncing through host userspace. Silently "drop" the PMU instead of rejecting KVM_CREATE_VCPU, as KVM's existing vPMU support doesn't function correctly if the local APIC is emulated by userspace, e.g. PMIs will never be delivered. I.e. it's far, far more likely that rejecting KVM_CREATE_VCPU would cause problems, e.g. for tests or userspace daemons that just want to probe basic KVM functionality. Note! Deliberately make mediated PMU creation "sticky", i.e. don't unwind it on failure to create a vCPU. Practically speaking, there's no harm to having a VM with a mediated PMU and no vCPUs. To avoid an "impossible" VM setup, reject KVM_CAP_PMU_CAPABILITY if a mediated PMU has been created, i.e. don't let userspace disable PMU support after failed vCPU creation (with PMU support enabled). Defer vendor specific requirements and constraints to the future. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/pmu.c | 4 ++++ arch/x86/kvm/pmu.h | 7 +++++++ arch/x86/kvm/x86.c | 37 +++++++++++++++++++++++++++++++-- arch/x86/kvm/x86.h | 1 + 5 files changed, 48 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index d7680612ba1e..ff0d753e2b07 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1471,6 +1471,7 @@ struct kvm_arch { =20 bool bus_lock_detection_enabled; bool enable_pmu; + bool created_mediated_pmu; =20 u32 notify_window; u32 notify_vmexit_flags; diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 0f3e011824ed..4d4bb9b17412 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -133,6 +133,10 @@ void kvm_init_pmu_capability(const struct kvm_pmu_ops = *pmu_ops) enable_pmu =3D false; } =20 + if (!enable_pmu || !enable_mediated_pmu || !kvm_host_pmu.mediated || + !pmu_ops->is_mediated_pmu_supported(&kvm_host_pmu)) + enable_mediated_pmu =3D false; + if (!enable_pmu) { memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap)); return; diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 08ae644db00e..f5b6181b772c 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -37,6 +37,8 @@ struct kvm_pmu_ops { void (*deliver_pmi)(struct kvm_vcpu *vcpu); void (*cleanup)(struct kvm_vcpu *vcpu); =20 + bool (*is_mediated_pmu_supported)(struct x86_pmu_capability *host_pmu); + const u64 EVENTSEL_EVENT; const int MAX_NR_GP_COUNTERS; const int MIN_NR_GP_COUNTERS; @@ -58,6 +60,11 @@ static inline bool kvm_pmu_has_perf_global_ctrl(struct k= vm_pmu *pmu) return pmu->version > 1; } =20 +static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu) +{ + return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version; +} + /* * KVM tracks all counters in 64-bit bitmaps, with general purpose counters * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed coun= ter 0 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d80bbd5e0859..396d1aa81732 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -187,6 +187,10 @@ bool __read_mostly enable_pmu =3D true; EXPORT_SYMBOL_GPL(enable_pmu); module_param(enable_pmu, bool, 0444); =20 +/* Enable/disabled mediated PMU virtualization. */ +bool __read_mostly enable_mediated_pmu; +EXPORT_SYMBOL_GPL(enable_mediated_pmu); + bool __read_mostly eager_page_split =3D true; module_param(eager_page_split, bool, 0644); =20 @@ -6542,7 +6546,7 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, break; =20 mutex_lock(&kvm->lock); - if (!kvm->created_vcpus) { + if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) { kvm->arch.enable_pmu =3D !(cap->args[0] & KVM_PMU_CAP_DISABLE); r =3D 0; } @@ -12174,8 +12178,13 @@ static int sync_regs(struct kvm_vcpu *vcpu) return 0; } =20 +#define PERF_MEDIATED_PMU_MSG \ + "Failed to enable mediated vPMU, try disabling system wide perf events an= d nmi_watchdog.\n" + int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) { + int r; + if (kvm_check_tsc_unstable() && kvm->created_vcpus) pr_warn_once("SMP vm created on host with unstable TSC; " "guest TSC will not be reliable\n"); @@ -12186,7 +12195,29 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsig= ned int id) if (id >=3D kvm->arch.max_vcpu_ids) return -EINVAL; =20 - return kvm_x86_call(vcpu_precreate)(kvm); + /* + * Note, any actions done by .vcpu_create() must be idempotent with + * respect to creating multiple vCPUs, and therefore are not undone if + * creating a vCPU fails (including failure during pre-create). + */ + r =3D kvm_x86_call(vcpu_precreate)(kvm); + if (r) + return r; + + if (enable_mediated_pmu && kvm->arch.enable_pmu && + !kvm->arch.created_mediated_pmu) { + if (irqchip_in_kernel(kvm)) { + r =3D perf_create_mediated_pmu(); + if (r) { + pr_warn_ratelimited(PERF_MEDIATED_PMU_MSG); + return r; + } + kvm->arch.created_mediated_pmu =3D true; + } else { + kvm->arch.enable_pmu =3D false; + } + } + return 0; } =20 int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) @@ -12818,6 +12849,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm) __x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); mutex_unlock(&kvm->slots_lock); } + if (kvm->arch.created_mediated_pmu) + perf_release_mediated_pmu(); kvm_destroy_vcpus(kvm); kvm_free_msr_filter(srcu_dereference_check(kvm->arch.msr_filter, &kvm->sr= cu, 1)); #ifdef CONFIG_KVM_IOAPIC diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 46220b04cdf2..bd1149768acc 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -445,6 +445,7 @@ extern struct kvm_caps kvm_caps; extern struct kvm_host_values kvm_host; =20 extern bool enable_pmu; +extern bool enable_mediated_pmu; =20 /* * Get a filtered version of KVM's supported XCR0 that strips out dynamic --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12A892D5426 for ; Wed, 6 Aug 2025 19:58:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510287; cv=none; b=OUrdwkB3m+tyysQSsDFJMmQJV2AlDNRPwfXTcmKPY941ebPp1mdBjLsR8q3flPU99/4t/6rfGXYaWKyXidBoZfF43qLOZ3hzFA3SOHZ8dgKQRe0KdXMmx+24qjvd9SlJ/Kt+9iHdmM6YPYrHpqi3Kmv5SY3FUGEengyJkWzRxFs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510287; c=relaxed/simple; bh=SrY28soq4e2umnXSFWFek5BM85G7hV4RrtHjWFxzyM0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dxVpvqNKOCmG2oMPlFn7SbBStsxZUHZhViuCjrjEhTBeKR9slVl112LKVP9oBFEHRgwadkhx/9g1nyMNxzFqekFdtHDgegKJ3Ui4Djzf0bafXH89QqWJswpUCSClk+ag6h5odDDZ0t3a8FIAL+nqtGziDlXcbrf9PCjlKwm97WY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lsTE+izp; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lsTE+izp" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-76bf3e05024so226811b3a.2 for ; Wed, 06 Aug 2025 12:58:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510284; x=1755115084; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=GWdUZrt+U2ZtCbYKTsMlrHRvg1+poTmQboTMZxCN7ww=; b=lsTE+izpqSy/cIW9NFPPrDqdyCJ7ANLnG342H3Se9/7tbKtiDqfQV8CdT72T8q1Dma f9unfBmSGccU/nw47APfm8o7eRDLKo71U70iD3/PLyfAgbs3xkXhCMtDU9fL8QrjulxJ slg5Gj4GZvHbZBURxZgWDKhDmYsy6PRzphwnOorj3vfXlueyg5ycqh0ylwDUg596FOqv TfZOB7E5WE/B3XeDjSo3KnqJN9ci1xlqzxG6nmGxz8nXnvPoFyLnNqhRzAVrnTYQftoX A28Hh09NkVtz21gMeLq+12F6eMFAs+Bmp7H+s2C2YwmM3QM3Z3vAyavudU9Up9m/2J+O QuMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510284; x=1755115084; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GWdUZrt+U2ZtCbYKTsMlrHRvg1+poTmQboTMZxCN7ww=; b=NIgk2uRbvEb/cCcfuTvd5kjdx8B6UFVz7lXFbMHyczzytffXvgaE4CEVFGy06WBYEJ LjOH8nqXa9noqi5rdaIDtw3HYOjydLH+hmjujin2NW0Fmf76UR00xYQ+nUS6jdlENJas vq+h5b8qmNU+7NsR7Y5zIpLTfkH/WDmKVuI56zJq3f/D3pej6sxwqIsUJetBwuIfV1/t 3cZPl60X/tY7B71U8f5jyzYK+xBQaRGRLwP38+/ZIkyhxBGRDinQHXJQEL5Mqx9tJcay eftzBPH6/xe3vv7Fj2ldy96jlip87n6Q6MIhx2HIWSDWlPpXAHNB5k0KXmN+5oKL6ZrB AO7A== X-Forwarded-Encrypted: i=1; AJvYcCVrpC/HrQ7DanbbhT9P1kwS5ubovkAAIPfUKducsmv1VCuvb01s/MaEkN9xiEsXYrzeD1ZeTDZOMCFEqDo=@vger.kernel.org X-Gm-Message-State: AOJu0YxcWsJq7FiYy//epHfe/VYWJxoLBM2xqoGVvu/hYBu+yg2R+qb8 pD3vbVzC9ZXXWneHFqYfW7ZyMTaufnDd75k1hNVnSbBTkz5iCIcqFPJ/ojBEDur0/01pa/glkn9 UAJj92A== X-Google-Smtp-Source: AGHT+IGuU+RWQRwCAfzWML148fgqdvdKW6alxUtwh3XwkGhhvD7y/3BjKg8jOpNbCUSNYHrZUCLfYMHRL8Y= X-Received: from pghd12.prod.google.com ([2002:a63:fd0c:0:b0:b42:2d66:cdc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6300:8045:b0:240:43e9:6cb7 with SMTP id adf61e73a8af0-24043e97001mr56266637.45.1754510284410; Wed, 06 Aug 2025 12:58:04 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:41 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-20-seanjc@google.com> Subject: [PATCH v5 19/44] KVM: x86/pmu: Implement Intel mediated PMU requirements and constraints From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Implement Intel PMU requirements and constraints for mediated PMU support. Require host PMU version 4+ so that PERF_GLOBAL_STATUS_SET can be used to precisely load the guest's status value into hardware, and require full- width writes so that KVM can precisely load guest counter values. Disable PEBS and LBRs if mediated PMU support is enabled, as they won't be supported in the initial implementation. Signed-off-by: Dapeng Mi Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang [sean: split to separate patch, add full-width writes dependency] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/vmx/capabilities.h | 3 ++- arch/x86/kvm/vmx/pmu_intel.c | 17 +++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 3 ++- 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilitie= s.h index 5316c27f6099..854e54c352f8 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -389,7 +389,8 @@ static inline bool vmx_pt_mode_is_host_guest(void) =20 static inline bool vmx_pebs_supported(void) { - return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept; + return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept && + !enable_mediated_pmu; } =20 static inline bool cpu_has_notify_vmexit(void) diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 07baff96300f..8df8d7b4f212 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -776,6 +776,20 @@ void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu) } } =20 +static bool intel_pmu_is_mediated_pmu_supported(struct x86_pmu_capability = *host_pmu) +{ + u64 host_perf_cap =3D 0; + + if (boot_cpu_has(X86_FEATURE_PDCM)) + rdmsrq(MSR_IA32_PERF_CAPABILITIES, host_perf_cap); + + /* + * Require v4+ for MSR_CORE_PERF_GLOBAL_STATUS_SET, and full-width + * writes so that KVM can precisely load guest counter values. + */ + return host_pmu->version >=3D 4 && host_perf_cap & PMU_CAP_FW_WRITES; +} + struct kvm_pmu_ops intel_pmu_ops __initdata =3D { .rdpmc_ecx_to_pmc =3D intel_rdpmc_ecx_to_pmc, .msr_idx_to_pmc =3D intel_msr_idx_to_pmc, @@ -787,6 +801,9 @@ struct kvm_pmu_ops intel_pmu_ops __initdata =3D { .reset =3D intel_pmu_reset, .deliver_pmi =3D intel_pmu_deliver_pmi, .cleanup =3D intel_pmu_cleanup, + + .is_mediated_pmu_supported =3D intel_pmu_is_mediated_pmu_supported, + .EVENTSEL_EVENT =3D ARCH_PERFMON_EVENTSEL_EVENT, .MAX_NR_GP_COUNTERS =3D KVM_MAX_NR_INTEL_GP_COUNTERS, .MIN_NR_GP_COUNTERS =3D 1, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index ed10013dac95..8c6343494e62 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7795,7 +7795,8 @@ static __init u64 vmx_get_perf_capabilities(void) if (boot_cpu_has(X86_FEATURE_PDCM)) rdmsrq(MSR_IA32_PERF_CAPABILITIES, host_perf_cap); =20 - if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) { + if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) && + !enable_mediated_pmu) { x86_perf_get_lbr(&vmx_lbr_caps); =20 /* --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3FCCC2D5C91 for ; Wed, 6 Aug 2025 19:58:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510289; cv=none; b=dHnc7hMSUkoLse6nV4qg/8zomPwXvRYvzW/MloRu+2gdaPgl4zf6/yccncJA0R/3MqRgDiLvHK0IsRXRhYZqtZUSSJ7TXi+bEmSFZ3YOqHE3Iyspsa2OoXRhvS9/kYf2fpFxu+23Lc2yNxw7xzQdKYtX2ONvWysyPJAbDnxxVxk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510289; c=relaxed/simple; bh=JJbKDILTxe6TmmJfqZTBwaNtNaEK5gYeJU99RMfSqbk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VHITAwI55voaD6QXbSsd5EigVBkoj8oZ5d7UGLRNKcZVMzexXbgE1vdiN1MWKv7BTlXlQBLDuvXrbbmdV9zr0wSiqXMAd5IMMbmMjj009/HvHzY6Xlz39R4OdGxB36kOhgLGpx+ZR+mRIyljRksgVlsQYOnzOTdeVfyYf+qOFT0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NG8S4guY; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NG8S4guY" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-24011ceafc8so1594785ad.3 for ; Wed, 06 Aug 2025 12:58:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510286; x=1755115086; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=UYs6XLsbq3wmUfEBezJ9021CIzqcfmK4yWnl0Vk41Lo=; b=NG8S4guYhFZftTktJx/3KraRGl4TJTLFugM4i7pwjurAOwU/SkdbsSkn90UKJtwpQI C//Co/NmazMiKi2lQ6RxGdwbd48IElHyj1pXD768S5sMAHnVIULfX94Q2R4IUP7+rijk P424HtPw7w6KfCG1M8s/xsS8LKQHogpgt1oUjPMKx1nPS2igK9gEneM5QoUtXBz/n5+Y bLP/CQ9CuWuZcE6n8/yaZRGicBj1TPg6i1S9CbRMM/4DNmK0AGk8/6e8NZcytVCVx2EC 8gq9KGpcwmpD5+zkOf22AHSgvp9JK0YLM9SI3xUPRGoWBgDsIluo+QTQWvsM2EKCOuyR Xlfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510286; x=1755115086; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UYs6XLsbq3wmUfEBezJ9021CIzqcfmK4yWnl0Vk41Lo=; b=oKo4KYge0TZBPwF+ivOC94AvS7AisgeomVLFbB8CzQ84+slAsilrh+73sfZzk5yZB7 Y/ZDyNmr11srm6VT3yvZxLaGD0GY4Jm8pFJPlMIlWi7qmhkHODJS0EYi+bs+WXLkslv7 aAhO9a1G0ve/8PVHuu3jwho+H2p2VHxeVDOLs8ItP2w3siy8Ff+S9KcjvPY+2ilvApmj T7TNLFuCfpQ0RpT6WZXrSe7RW8VyFc5mYr+MofMmA8Wul63b/q4D+Tb5qiUOxMxooN0/ cOC2QDIRBczzN61MHH+ifeOTwX9+/wuB/ww9VjgHwqeB9MaslAOGt9o2JNKgc2XY8P1s 9ryw== X-Forwarded-Encrypted: i=1; AJvYcCXHbE+LgRgkFmQAqCY2VKqveJ6V6UAaZ7LXLQAWSpHYucb0TeOJDm4BoebcnwtoZW0N7j5GESY2sEes1jI=@vger.kernel.org X-Gm-Message-State: AOJu0Yz6ZkUSJx85XjYVdJS/091P7PSXwNm0Tw4Y3K88G2aNIVeI9bIy NR/neQu8ChxkqtKQYJ/+zNaVL58EiOI2R5EdJapNvfufVqvgVbzXFJu/HZ8BMaVVdxq8l7EyxtH f0ciVFg== X-Google-Smtp-Source: AGHT+IEjtNNsSzvoH0/VB3MvaueTobKjhNGxenKGyVkg+/CWGpyESmYfNsRK9kQsugEnvTMUeRjISaSTO8s= X-Received: from pjov3.prod.google.com ([2002:a17:90a:9603:b0:31c:2fe4:33b6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1b44:b0:240:b28:22a3 with SMTP id d9443c01a7336-242b1a687dfmr5147985ad.29.1754510285965; Wed, 06 Aug 2025 12:58:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:42 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-21-seanjc@google.com> Subject: [PATCH v5 20/44] KVM: x86/pmu: Implement AMD mediated PMU requirements From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Require host PMU version 2+ for AMD mediated PMU support, as PERF_GLOBAL_CTRL and friends are hard requirements for the mediated PMU. Signed-off-by: Dapeng Mi Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang [sean: extract to separate patch, write changelog] Signed-off-by: Sean Christopherson Reviewed-by: Sandipan Das Tested-by: Xudong Hao --- arch/x86/kvm/svm/pmu.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index 7b8577f3c57a..96be2c3e0d65 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -227,6 +227,11 @@ static void amd_pmu_init(struct kvm_vcpu *vcpu) } } =20 +static bool amd_pmu_is_mediated_pmu_supported(struct x86_pmu_capability *h= ost_pmu) +{ + return host_pmu->version >=3D 2; +} + struct kvm_pmu_ops amd_pmu_ops __initdata =3D { .rdpmc_ecx_to_pmc =3D amd_rdpmc_ecx_to_pmc, .msr_idx_to_pmc =3D amd_msr_idx_to_pmc, @@ -236,6 +241,9 @@ struct kvm_pmu_ops amd_pmu_ops __initdata =3D { .set_msr =3D amd_pmu_set_msr, .refresh =3D amd_pmu_refresh, .init =3D amd_pmu_init, + + .is_mediated_pmu_supported =3D amd_pmu_is_mediated_pmu_supported, + .EVENTSEL_EVENT =3D AMD64_EVENTSEL_EVENT, .MAX_NR_GP_COUNTERS =3D KVM_MAX_NR_AMD_GP_COUNTERS, .MIN_NR_GP_COUNTERS =3D AMD64_NUM_COUNTERS, --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B91282D63FB for ; Wed, 6 Aug 2025 19:58:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510291; cv=none; b=NfHYnzESJEff5G7I4hCtxwqHe8e5KV7b2kocUxhyLtxwhZmfTPI765q7/HdYIJKx9zNPgDriL98qX8FwlSIAZDzRPELlD29dd0qazDMWvZp6shiQRV9U81lbGFW3LB8fxYpJd6BOHXYl9xMOqXd5YzYKqaaWeSSwveHYe12xbd8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510291; c=relaxed/simple; bh=C53ygBWjsMP+UcUorpUgoXrWlq5dJSRuSmIlt0jKd/U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OIm1TKFWPSDlzPhnY0T3q7alg8owyEsRKkbTXYfyTJfG7b2IZIi26qa51A2e6jihQHJeqhOnmWmvkSXOh2tvKz60XlMEK04cz0us6WgowyhUQkErP7OaArJiGJD+q+qXIO8PBix3NDeqzOzVDHkOiPVMYw8WUM91NhXkMz7F+LE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BGx0X2k/; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BGx0X2k/" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31eec17b5acso271632a91.2 for ; Wed, 06 Aug 2025 12:58:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510288; x=1755115088; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=x5qdlfMt7f/v/zSfpvn0pZJb5j/BJcw1RvQoZStHDYU=; b=BGx0X2k/F+R1x1yZosxwA7KSbUSa40GEAi1KWMmNGvwh8PHnwr3d4JNvHe3UdiH2nU dM9gT99CwSIMm7skf/4bX6VZMfvV2FKF/htnbu6/g0lbS2ppNryTGUgq8m6WDppUBMfm YIeezq2Jl2X/6iJHxMPxYpeB1UImS4ZeDYs+PRLIQ9YfhI7mt5EsXHfhHpOWF6LF3xts MoYVgo/46US2PizxKvwDrpE42zPapC2JVtGe0PylRPivgvy4Tn4xLfjY1IurVkKa6mh9 8xy0WGAwdgO+jB/dK5sLkVFBi9peZqSmZEbD0TUQCZKpOh+T3IK0j+3bDSqOvlwDlaX4 94CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510288; x=1755115088; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=x5qdlfMt7f/v/zSfpvn0pZJb5j/BJcw1RvQoZStHDYU=; b=j02+mNXOVcrlzQIxkeRFdMTqgV8FsiPOM7AwmUgC6mp3lDcbkvtzToIlqQcCa5vzE2 mgOXNF+UYynVTdVSjN7Gg13aZGRa897Pk6MUzTDT9fNav/cxhJYx7E8CTHNPG2FAsQqJ TqYvZ+clrzlFFPdpUslNw+UjwkesakMPfdjHhhvdyyR34TcZm1jmddl6GZ4pWwL6TxOZ AD1MEsyNAq74fX4VT+VLB1nIY9zcOvaiWoul+wrNr/oWmwfIde+J0R1mU5U1kUXGFrYW E6CUCBEbAMue5RT+cQKYshWMo/scga4YGYGzyprPbp/OeEcZIkxmRHs7CbxlYf4ifov2 uAWg== X-Forwarded-Encrypted: i=1; AJvYcCXWD47F0lLQDvdKw0Pv1wHN5lqEJ4vj5ax3H4+q173KTSfWHRbOuT9K/q1l0rLdHBKJQo1YP1cVM1KaI+M=@vger.kernel.org X-Gm-Message-State: AOJu0Yxc04uA1yOAf88zXIuWq1nHddbDGsc5tV4jjjUfMYlsX5gUJ6AE ocBaBGa0cmvI3N6cKdXN25re6Wi7+Z+q5tnXGH1BTeyezrZpu85fxrrA/I39kY7WDm9ut40fWus ac+xEwA== X-Google-Smtp-Source: AGHT+IFfF+d9RBweWfYhERWBY7pftrU4yG2unJrGI6enSH3CQtm8m01r3mZaRpes0DVZoNbRLzozGADRUKg= X-Received: from pjbeu16.prod.google.com ([2002:a17:90a:f950:b0:31e:dd67:e98]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4ac1:b0:31e:f193:1822 with SMTP id 98e67ed59e1d1-32166ccac82mr5178351a91.28.1754510287763; Wed, 06 Aug 2025 12:58:07 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:43 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-22-seanjc@google.com> Subject: [PATCH v5 21/44] KVM: x86/pmu: Register PMI handler for mediated vPMU From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Xiong Zhang Register a dedicated PMI handler with perf's callback when mediated PMU support is enabled. Perf routes PMIs that arrive while guest context is loaded to the provided callback, by modifying the CPU's LVTPC to point at a dedicated mediated PMI IRQ vector. WARN upon receipt of a mediated PMI if there is no active vCPU, or if the vCPU doesn't have a mediated PMU. Even if a PMI manages to skid past VM-Exit, it should never be delayed all the way beyond unloading the vCPU. And while running vCPUs without a mediated PMU, the LVTPC should never be wired up to the mediated PMI IRQ vector, i.e. should always be routed through perf's NMI handler. Signed-off-by: Xiong Zhang Signed-off-by: Mingwei Zhang Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 10 ++++++++++ arch/x86/kvm/pmu.h | 2 ++ arch/x86/kvm/x86.c | 3 ++- 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 4d4bb9b17412..680523e9d11e 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -155,6 +155,16 @@ void kvm_init_pmu_capability(const struct kvm_pmu_ops = *pmu_ops) perf_get_hw_event_config(PERF_COUNT_HW_BRANCH_INSTRUCTIONS); } =20 +void kvm_handle_guest_mediated_pmi(void) +{ + struct kvm_vcpu *vcpu =3D kvm_get_running_vcpu(); + + if (WARN_ON_ONCE(!vcpu || !kvm_vcpu_has_mediated_pmu(vcpu))) + return; + + kvm_make_request(KVM_REQ_PMI, vcpu); +} + static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) { struct kvm_pmu *pmu =3D pmc_to_pmu(pmc); diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index f5b6181b772c..e038bce76b9e 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -46,6 +46,8 @@ struct kvm_pmu_ops { =20 void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops); =20 +void kvm_handle_guest_mediated_pmi(void); + static inline bool kvm_pmu_has_perf_global_ctrl(struct kvm_pmu *pmu) { /* diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 396d1aa81732..2c34dd3f0222 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9693,7 +9693,8 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) set_hv_tscchange_cb(kvm_hyperv_tsc_notifier); #endif =20 - __kvm_register_perf_callbacks(ops->handle_intel_pt_intr, NULL); + __kvm_register_perf_callbacks(ops->handle_intel_pt_intr, + enable_mediated_pmu ? kvm_handle_guest_mediated_pmi : NULL); =20 if (IS_ENABLED(CONFIG_KVM_SW_PROTECTED_VM) && tdp_mmu_enabled) kvm_caps.supported_vm_types |=3D BIT(KVM_X86_SW_PROTECTED_VM); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 953CF2D660E for ; Wed, 6 Aug 2025 19:58:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510294; cv=none; b=t/iuIbxo9LwaU/8zI57aVu+Q0pBhYbkpRawsRZ83X27rfFng5pGojJZRF940PEoJkImTEluduvBBGVV1T/CtUAHOHq+KWW/33MBECyQvXHFIJZABrVUj8Tq1r7ul0Ke7TjSm8m+dY4vzNY3vCW/1Jqq79qRdWBvZF6RltiztAwc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510294; c=relaxed/simple; bh=C/4r70x0T0JzgUP1RnZOZXEA4d1zjXrhrDvqbtcZVy0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=E7NP3PkUqAOROuuwO3zDBY9r2VZqc3EfdpTNL03/qze5p8lXa6n3ZMBItasTXmZ/oZ8ZnrI4mp//WSRQVubUmsTBOg7PXmvKW9isnWX5tSczPUO0mIioZyminhetojPiQG+zGJ1K5M7GM2iGCBzG+oFMNeYNYpapop4XsR4Xdis= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pxaoaxds; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pxaoaxds" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-315af0857f2so334480a91.0 for ; Wed, 06 Aug 2025 12:58:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510290; x=1755115090; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ThGb/1cQlCkxXgvsmi9By0m3uFp5vZO1mPp34NlVQTI=; b=pxaoaxdsqM3qDq0p82vQdGesaY5K5k9yAhyF44ZVSVGwdpsY2ZgvIyP9/hdDPGgVSm 0dXmP+Fk0QqbkEgfVjWc9wtipY6138MYASUoLceKATrWeu7cgcEexAHk56mqT0o3bqmD 4UpxSoE7wXBkug9J/SIi742y1Sa9fO1qYLNOjPdiB5PtpNxQ1j7Ks3hp7MkVPTQ11t3H 18v7zv7E0N+Oxkweq7ad9i0RSOOvcJ+UshJVExVE2I/pV60Oqk4mxy5MasyU9+LEt+kM U9fISkY0guVtYVULf6jMdrnKa+ZXx6BpswpGZUEghC9Fvv6BpM9Nw9F7eEKfB7XbXbak e9hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510290; x=1755115090; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ThGb/1cQlCkxXgvsmi9By0m3uFp5vZO1mPp34NlVQTI=; b=AysMINKych60ES2jrr9lz7wp+Z+kTvsBKGOBodq2NiVIdUZp+o1WmV4Mvf//QuIzi/ gq1PbFMn3Frfwuil4CqEGr0xd6mhGBspVkGVto+CKWUa3mUEBly7kyooIFeLjFRwO9iC iPQBsWQLpJMmkHn9fOhnTjoO+ikVEkSV7PfsxMaRpcMefPga+Q5DeHzGBAAIL9B8Tkcm HHKv8GnmFAZBbB/u0FVThb/Io2yHgbkCv5u20xCP/DZnwF6VV69uQnTn6D98fa60UIqO F2SdHFS8F20MUOYlVyDvXH2DtOqMy6drPIZt0i2vT4/SlReXiI6+C/AZhoDUOZklhgqL n5Sg== X-Forwarded-Encrypted: i=1; AJvYcCUG42HQ5Vf3M5DB7jObSvUv85xSeTFyXcr6h502S3gCoiVAS1edop5gUMbUHdaGx5Xdy/b1WV1wwoutT1s=@vger.kernel.org X-Gm-Message-State: AOJu0YwbJTtt2/IShd3AHMyKdpUsX/C4dDePmdlG6Nr++HwD9gsgtq2j dLqpgqwhcXOdfX7ASXK8M0ks8MFpIhO8mYtwU7efkK/riWFlBN7PqvOsBxCG2NJrjaO+3DGVxAb KcsG2cA== X-Google-Smtp-Source: AGHT+IEDX7osU0iIbko+P3lNad2rdh21UdoQ7zV3q9ejnGRLDtg0+KAFySIgmFRbYCqEx1sz5hOKZCwZh1w= X-Received: from pjbci4.prod.google.com ([2002:a17:90a:fc84:b0:313:242b:1773]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1dce:b0:31f:5ebe:fa1c with SMTP id 98e67ed59e1d1-32166d77ff2mr6724469a91.0.1754510289629; Wed, 06 Aug 2025 12:58:09 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:44 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-23-seanjc@google.com> Subject: [PATCH v5 22/44] KVM: x86: Rename vmx_vmentry/vmexit_ctrl() helpers From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Rename the two helpers vmx_vmentry/vmexit_ctrl() to vmx_get_initial_vmentry/vmexit_ctrl() to represent their real meaning. No functional change intended. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/vmx/vmx.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 8c6343494e62..7b0b51809f0e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4304,7 +4304,7 @@ static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *v= mx) return pin_based_exec_ctrl; } =20 -static u32 vmx_vmentry_ctrl(void) +static u32 vmx_get_initial_vmentry_ctrl(void) { u32 vmentry_ctrl =3D vmcs_config.vmentry_ctrl; =20 @@ -4321,7 +4321,7 @@ static u32 vmx_vmentry_ctrl(void) return vmentry_ctrl; } =20 -static u32 vmx_vmexit_ctrl(void) +static u32 vmx_get_initial_vmexit_ctrl(void) { u32 vmexit_ctrl =3D vmcs_config.vmexit_ctrl; =20 @@ -4686,10 +4686,10 @@ static void init_vmcs(struct vcpu_vmx *vmx) if (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_PAT) vmcs_write64(GUEST_IA32_PAT, vmx->vcpu.arch.pat); =20 - vm_exit_controls_set(vmx, vmx_vmexit_ctrl()); + vm_exit_controls_set(vmx, vmx_get_initial_vmexit_ctrl()); =20 /* 22.2.1, 20.8.1 */ - vm_entry_controls_set(vmx, vmx_vmentry_ctrl()); + vm_entry_controls_set(vmx, vmx_get_initial_vmentry_ctrl()); =20 vmx->vcpu.arch.cr0_guest_owned_bits =3D vmx_l1_guest_owned_cr0_bits(); vmcs_writel(CR0_GUEST_HOST_MASK, ~vmx->vcpu.arch.cr0_guest_owned_bits); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 133012D77FF for ; Wed, 6 Aug 2025 19:58:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510294; cv=none; b=P6QHx5R50cLcoIqbkrJtHkfLBzT8RAX5x8ephGOv5ohyfMCriULuibK5+6OTy5MkepClfKzeikVvrYTwHm+LbaxSrhZLdA/hhU9Dn8EJR6++mUqJkws7eSi6DeLWC2/IqXdcXdGS5NNWFJk4CYI/XTvy1ac59hsmH1lTnJiHSfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510294; c=relaxed/simple; bh=HjF6VqOOZUHrkl/7iev8rEDWaKamWaXyDUM+ISWFM5M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WuQriyGw5CWZ6BipuZ9mWJAeTGNxDlgIRyTG6z4z3XWcAF8VT7Whd6WtdCu1RrK1J15+xFz3ZQvk3uWsg4fPLWL7aKs/a0FwaTPITVUVJ9sQ87ZwPy/lRTcsgQuy2caAu1XYOkwWfWaySKZgzTL3/tNYGvQ/7M2yL00YjGolwJM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mp40kJwt; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mp40kJwt" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-23fe98c50daso1499005ad.3 for ; Wed, 06 Aug 2025 12:58:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510291; x=1755115091; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=/FX2+DfXNwgRGe/WbgJvV/8z8g7IlIB9y4b49oQ5nSU=; b=mp40kJwttBkOnbdJqg+ZIXn87T5lnBzz+UDT//G0D1jdpLHYc1pMDbv5CIKfZUJz3n QYriK5s4sfB1ZFLX4YZvZmRhKKRnXe3Edi+wIaaHH1zhPpj6+97rPxWh3C0hmfBSM0KP YEQK2tikqxcgPjpDTFRXvVFB81LqhfngPbgHVakAoT205J5ygdHa/PJl6INv9Cvt9M0V Y76LOcH5/ik2zmA1BJ7NIMAvlGMXakmrHhUdIZJ+YtZt82RiCeW52xe2Uh4BPsdbOhj6 nbVqqRnp6osme26eR66rcqCzS1nt7bs/cwvuQnWt/cEFeNEqeMtrtkKW4uq4ikcdBSAh EPQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510291; x=1755115091; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/FX2+DfXNwgRGe/WbgJvV/8z8g7IlIB9y4b49oQ5nSU=; b=ufHL2JnBCkjIjApiGhZnutvZJviAQ6lMTwciJ0iNA6HZ1yG5igTjRjjJ1IcWkBjMPl wUUdFicpsxUf+IguaWdfZcSIKv/q4IZ6C4E5gUkEM9c+HSqunVuxfjSxajGO/aNkPENq Mukjzb0+XOQ5XpqYL8qjLG+UYniTShmJGDD4CKN/yVR9mOl15Dx8w069DqBufcVAP1U1 ZIWxuZjYYIxw/+dgSplgV3xv+H2NZAvZ3zjnWYeAAVd4FZzHOM+VfJB24B1zTFReMAqo U37WcIHJPDKVwzmeEQKJ99o++S0xHnenM2mQoKl1yiddzs46rsGpG/p8EsAb0JB4BkaR in/g== X-Forwarded-Encrypted: i=1; AJvYcCXme7UxMwrroEGAkSr99T+QjjY89Q9wo9vgZWM7QxTHvJlLSrV/lvT6b/0GnhSpv9GfPgNAOM4+l+J+yZ4=@vger.kernel.org X-Gm-Message-State: AOJu0YytfYkcrUYNAU+ZBvL6kalJKc+VMBSxqeDFPeZshyRUlnjK6wtI b0izdSs/ItJDSOhkR9DOjqoeuudUood4PUDezkTGW1cQoJMeSZSGZcus7VsYf1BzTEMyKYgAK9Z bCsbpuA== X-Google-Smtp-Source: AGHT+IGe4jBR2KHz8e2goINgF4EUoH0wWvJHG6ysYfktepmzpkGXYfj1vM33eEgiesUpU8kg3lcTp4Ph7gE= X-Received: from plbmv16.prod.google.com ([2002:a17:903:b90:b0:231:de34:f9f6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:98d:b0:240:417d:8115 with SMTP id d9443c01a7336-2429ee89eb4mr58062565ad.16.1754510291288; Wed, 06 Aug 2025 12:58:11 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:45 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-24-seanjc@google.com> Subject: [PATCH v5 23/44] KVM: x86/pmu: Move PMU_CAP_{FW_WRITES,LBR_FMT} into msr-index.h header From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Move PMU_CAP_{FW_WRITES,LBR_FMT} into msr-index.h and rename them with PERF_CAP prefix to keep consistent with other perf capabilities macros. No functional change intended. Signed-off-by: Dapeng Mi Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/include/asm/msr-index.h | 15 +++++++++------ arch/x86/kvm/vmx/capabilities.h | 3 --- arch/x86/kvm/vmx/pmu_intel.c | 6 +++--- arch/x86/kvm/vmx/vmx.c | 12 ++++++------ 4 files changed, 18 insertions(+), 18 deletions(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-in= dex.h index c29127ac626a..f19d1ee9a396 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -315,12 +315,15 @@ #define PERF_CAP_PT_IDX 16 =20 #define MSR_PEBS_LD_LAT_THRESHOLD 0x000003f6 -#define PERF_CAP_PEBS_TRAP BIT_ULL(6) -#define PERF_CAP_ARCH_REG BIT_ULL(7) -#define PERF_CAP_PEBS_FORMAT 0xf00 -#define PERF_CAP_PEBS_BASELINE BIT_ULL(14) -#define PERF_CAP_PEBS_MASK (PERF_CAP_PEBS_TRAP | PERF_CAP_ARCH_REG | \ - PERF_CAP_PEBS_FORMAT | PERF_CAP_PEBS_BASELINE) + +#define PERF_CAP_LBR_FMT 0x3f +#define PERF_CAP_PEBS_TRAP BIT_ULL(6) +#define PERF_CAP_ARCH_REG BIT_ULL(7) +#define PERF_CAP_PEBS_FORMAT 0xf00 +#define PERF_CAP_FW_WRITES BIT_ULL(13) +#define PERF_CAP_PEBS_BASELINE BIT_ULL(14) +#define PERF_CAP_PEBS_MASK (PERF_CAP_PEBS_TRAP | PERF_CAP_ARCH_REG | \ + PERF_CAP_PEBS_FORMAT | PERF_CAP_PEBS_BASELINE) =20 #define MSR_IA32_RTIT_CTL 0x00000570 #define RTIT_CTL_TRACEEN BIT(0) diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilitie= s.h index 854e54c352f8..26ff606ff139 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -20,9 +20,6 @@ extern int __read_mostly pt_mode; #define PT_MODE_SYSTEM 0 #define PT_MODE_HOST_GUEST 1 =20 -#define PMU_CAP_FW_WRITES (1ULL << 13) -#define PMU_CAP_LBR_FMT 0x3f - struct nested_vmx_msrs { /* * We only store the "true" versions of the VMX capability MSRs. We diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 8df8d7b4f212..7ab35ef4a3b1 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -138,7 +138,7 @@ static inline u64 vcpu_get_perf_capabilities(struct kvm= _vcpu *vcpu) =20 static inline bool fw_writes_is_enabled(struct kvm_vcpu *vcpu) { - return (vcpu_get_perf_capabilities(vcpu) & PMU_CAP_FW_WRITES) !=3D 0; + return (vcpu_get_perf_capabilities(vcpu) & PERF_CAP_FW_WRITES) !=3D 0; } =20 static inline struct kvm_pmc *get_fw_gp_pmc(struct kvm_pmu *pmu, u32 msr) @@ -588,7 +588,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) =20 perf_capabilities =3D vcpu_get_perf_capabilities(vcpu); if (intel_pmu_lbr_is_compatible(vcpu) && - (perf_capabilities & PMU_CAP_LBR_FMT)) + (perf_capabilities & PERF_CAP_LBR_FMT)) memcpy(&lbr_desc->records, &vmx_lbr_caps, sizeof(vmx_lbr_caps)); else lbr_desc->records.nr =3D 0; @@ -787,7 +787,7 @@ static bool intel_pmu_is_mediated_pmu_supported(struct = x86_pmu_capability *host_ * Require v4+ for MSR_CORE_PERF_GLOBAL_STATUS_SET, and full-width * writes so that KVM can precisely load guest counter values. */ - return host_pmu->version >=3D 4 && host_perf_cap & PMU_CAP_FW_WRITES; + return host_pmu->version >=3D 4 && host_perf_cap & PERF_CAP_FW_WRITES; } =20 struct kvm_pmu_ops intel_pmu_ops __initdata =3D { diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 7b0b51809f0e..93b87f9e6dfd 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2127,7 +2127,7 @@ u64 vmx_get_supported_debugctl(struct kvm_vcpu *vcpu,= bool host_initiated) (host_initiated || guest_cpu_cap_has(vcpu, X86_FEATURE_BUS_LOCK_DETEC= T))) debugctl |=3D DEBUGCTLMSR_BUS_LOCK_DETECT; =20 - if ((kvm_caps.supported_perf_cap & PMU_CAP_LBR_FMT) && + if ((kvm_caps.supported_perf_cap & PERF_CAP_LBR_FMT) && (host_initiated || intel_pmu_lbr_is_enabled(vcpu))) debugctl |=3D DEBUGCTLMSR_LBR | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI; =20 @@ -2412,9 +2412,9 @@ int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_dat= a *msr_info) vmx->pt_desc.guest.addr_a[index / 2] =3D data; break; case MSR_IA32_PERF_CAPABILITIES: - if (data & PMU_CAP_LBR_FMT) { - if ((data & PMU_CAP_LBR_FMT) !=3D - (kvm_caps.supported_perf_cap & PMU_CAP_LBR_FMT)) + if (data & PERF_CAP_LBR_FMT) { + if ((data & PERF_CAP_LBR_FMT) !=3D + (kvm_caps.supported_perf_cap & PERF_CAP_LBR_FMT)) return 1; if (!cpuid_model_is_consistent(vcpu)) return 1; @@ -7786,7 +7786,7 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) =20 static __init u64 vmx_get_perf_capabilities(void) { - u64 perf_cap =3D PMU_CAP_FW_WRITES; + u64 perf_cap =3D PERF_CAP_FW_WRITES; u64 host_perf_cap =3D 0; =20 if (!enable_pmu) @@ -7807,7 +7807,7 @@ static __init u64 vmx_get_perf_capabilities(void) if (!vmx_lbr_caps.has_callstack) memset(&vmx_lbr_caps, 0, sizeof(vmx_lbr_caps)); else if (vmx_lbr_caps.nr) - perf_cap |=3D host_perf_cap & PMU_CAP_LBR_FMT; + perf_cap |=3D host_perf_cap & PERF_CAP_LBR_FMT; } =20 if (vmx_pebs_supported()) { --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28D512D837D for ; Wed, 6 Aug 2025 19:58:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510299; cv=none; b=ssY57+HEYS7CnHW27szRhWeV3/EqjSmb1PmpUNOnKIri295h9yGDiCCh2vBLEfvpEUtu5HJDENiRauVtqS0jSgyEZgCW2MdNlX5LM2ksrWgGAekOynnh25M+yW2UVTTMEN0hRr7z16omMrWvNBHvDFsLp8nOpBZVw2J9dR/p67g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510299; c=relaxed/simple; bh=byG6xS28Uf7b/R4Aqls1ih01tm3g3UFbLNRST2TR/5c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=t8LSivJ4OiLPiHw9UjBTocxyhFLrmsz5wU3Z+zC5JCgBYRX9DlUSDY7pSOBTPlYGsa6rCIH3cD+xso6PJhUgoxcMLUUInKmQde4jYM1Zf6bYiV66QaV1OjdriPxyzAwoJ1/vPBgev9XCV+2wqERfif3WbYSkZm8pb0TKlbSz1Ic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XWXz4ill; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XWXz4ill" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b0e0c573531so122144a12.3 for ; Wed, 06 Aug 2025 12:58:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510293; x=1755115093; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Xgsu0fWGzXM1P0ieUy8M+NNgbe9X2+RcO7orAU+l0hA=; b=XWXz4illZG+jNmDDjzIFMe1K6RKAAsAOg4ZY+lniQAtDlFJDh5wZqhT+AGop63xHxy cl98U1Sr1vFAKdz98qWjmpsCLCdw6HPRoyjJaUZT02l0cqYWUFyjYnChifbLgBDIYL9C uBNM0e7n00Mcc+FE+TbAqd9XTdcWE8NkyYgZknJ+dmTJGHF0Xz/fN13nmLl0DqlQNL41 J7MmjjK6tWNWNGnX6u83pnyiPact62OIJfmHNu3JBn1hRy2v+xaNxizA7WLkngRkUfo5 VuKd93w6gNen7N+ClJO0CBk9TapSZJttQjgf+CGxFjbzGbkd4YCopETY72hIqWbATOI/ IIOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510293; x=1755115093; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Xgsu0fWGzXM1P0ieUy8M+NNgbe9X2+RcO7orAU+l0hA=; b=hhv5wsBFrlVr5wG8ePsXfQtB0bo+jWDcmrj4UcyllWVS8bfrEv0lBBOHGBJc2vXZNq RcPIKhQDRo0X1DfHmTuZC5iSeIUjNV9CJiVvSYgGoUpvSxzv/LrEKDyypyISU8UDZsEY OYs9Zym8deaXWJp7toLrg2V0EfS+X0+wOpDo1UBH7Az8OudrpEEogvJK8JLv5CAVSC80 8lPAtwnvu1v0xr2xF/oF+Ox2Mz5OmfBfCnAN8gfXgUzToM2Td/i1fcYcXcSeOd5ZdOdq q9dNmiDfDkZbAMCsyY+9IfKHWTOdF6HWbtn+FlxJi5S3Pm5qhQoL6Gr3Ob7Sv/wIPEBp 4Txg== X-Forwarded-Encrypted: i=1; AJvYcCUaE/0WyO5Lpw5SOGJmsWJHEnBXVd0VxfJsDX8MamrA5+ii6iQBSw+l+XK3Tk02LJreWl/H/e8oAcrKh8s=@vger.kernel.org X-Gm-Message-State: AOJu0Yy/DThzNyomSHoBMVRq9Uqw74wCktMQaw6qHRF6vQ+YCECrOC8X zY/jRZGf1nCa+NgxiHs9hPtQA2z4xWm8euf1tJ1Vfnc9Bs0PiLVs0OjDKLyVI5AIp9pfk0Jnck9 U5wnjaQ== X-Google-Smtp-Source: AGHT+IGjOVjMJgLz2fS/cn5FiFF0L2j0BQ8Kgspjt+cfMRIO/JKDaNYzPFPk3n1FGowkS6X5Jtr/S5JnoBk= X-Received: from pjbgf4.prod.google.com ([2002:a17:90a:c7c4:b0:312:ea08:fa64]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:3504:b0:240:3e72:efb3 with SMTP id d9443c01a7336-2429f42fa4cmr56513685ad.43.1754510293150; Wed, 06 Aug 2025 12:58:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:46 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-25-seanjc@google.com> Subject: [PATCH v5 24/44] KVM: x86: Rework KVM_REQ_MSR_FILTER_CHANGED into a generic RECALC_INTERCEPTS From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rework the MSR_FILTER_CHANGED request into a more generic RECALC_INTERCEPTS request, and expand the responsibilities of vendor code to recalculate all intercepts that vary based on userspace input, e.g. instruction intercepts that are tied to guest CPUID. Providing a generic recalc request will allow the upcoming mediated PMU support to trigger a recalc when PMU features, e.g. PERF_CAPABILITIES, are set by userspace, without having to make multiple calls to/from PMU code. As a bonus, using a request will effectively coalesce recalcs, e.g. will reduce the number of recalcs for normal usage from 3+ to 1 (vCPU create, set CPUID, set PERF_CAPABILITIES (Intel only), set filter). The downside is that MSR filter changes that are done in isolation will do a small amount of unnecessary work, but that's already a relatively slow path, and the cost of recalculating instruction intercepts is negligible. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/svm/svm.c | 8 ++++---- arch/x86/kvm/vmx/main.c | 14 +++++++------- arch/x86/kvm/vmx/vmx.c | 9 +++++++-- arch/x86/kvm/vmx/x86_ops.h | 2 +- arch/x86/kvm/x86.c | 15 +++++++-------- 7 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index 18a5c3119e1a..7c240e23bd52 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -138,7 +138,7 @@ KVM_X86_OP(check_emulate_instruction) KVM_X86_OP(apic_init_signal_blocked) KVM_X86_OP_OPTIONAL(enable_l2_tlb_flush) KVM_X86_OP_OPTIONAL(migrate_timers) -KVM_X86_OP(recalc_msr_intercepts) +KVM_X86_OP(recalc_intercepts) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index ff0d753e2b07..b891bd92fc83 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -120,7 +120,7 @@ #define KVM_REQ_TLB_FLUSH_GUEST \ KVM_ARCH_REQ_FLAGS(27, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_APF_READY KVM_ARCH_REQ(28) -#define KVM_REQ_MSR_FILTER_CHANGED KVM_ARCH_REQ(29) +#define KVM_REQ_RECALC_INTERCEPTS KVM_ARCH_REQ(29) #define KVM_REQ_UPDATE_CPU_DIRTY_LOGGING \ KVM_ARCH_REQ_FLAGS(30, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \ @@ -1912,7 +1912,7 @@ struct kvm_x86_ops { int (*enable_l2_tlb_flush)(struct kvm_vcpu *vcpu); =20 void (*migrate_timers)(struct kvm_vcpu *vcpu); - void (*recalc_msr_intercepts)(struct kvm_vcpu *vcpu); + void (*recalc_intercepts)(struct kvm_vcpu *vcpu); int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err); =20 void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index f7e1e665a826..3d9dcc66a407 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1077,7 +1077,7 @@ static void svm_recalc_instruction_intercepts(struct = kvm_vcpu *vcpu) } } =20 -static void svm_recalc_intercepts_after_set_cpuid(struct kvm_vcpu *vcpu) +static void svm_recalc_intercepts(struct kvm_vcpu *vcpu) { svm_recalc_instruction_intercepts(vcpu); svm_recalc_msr_intercepts(vcpu); @@ -1225,7 +1225,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu) =20 svm_hv_init_vmcb(vmcb); =20 - svm_recalc_intercepts_after_set_cpuid(vcpu); + svm_recalc_intercepts(vcpu); =20 vmcb_mark_all_dirty(vmcb); =20 @@ -4479,7 +4479,7 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu = *vcpu) if (sev_guest(vcpu->kvm)) sev_vcpu_after_set_cpuid(svm); =20 - svm_recalc_intercepts_after_set_cpuid(vcpu); + svm_recalc_intercepts(vcpu); } =20 static bool svm_has_wbinvd_exit(void) @@ -5181,7 +5181,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata =3D { =20 .apic_init_signal_blocked =3D svm_apic_init_signal_blocked, =20 - .recalc_msr_intercepts =3D svm_recalc_msr_intercepts, + .recalc_intercepts =3D svm_recalc_intercepts, .complete_emulated_msr =3D svm_complete_emulated_msr, =20 .vcpu_deliver_sipi_vector =3D svm_vcpu_deliver_sipi_vector, diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index dbab1c15b0cd..68dcafd177a8 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -188,18 +188,18 @@ static int vt_get_msr(struct kvm_vcpu *vcpu, struct m= sr_data *msr_info) return vmx_get_msr(vcpu, msr_info); } =20 -static void vt_recalc_msr_intercepts(struct kvm_vcpu *vcpu) +static void vt_recalc_intercepts(struct kvm_vcpu *vcpu) { /* - * TDX doesn't allow VMM to configure interception of MSR accesses. - * TDX guest requests MSR accesses by calling TDVMCALL. The MSR - * filters will be applied when handling the TDVMCALL for RDMSR/WRMSR - * if the userspace has set any. + * TDX doesn't allow VMM to configure interception of instructions or + * MSR accesses. TDX guest requests MSR accesses by calling TDVMCALL. + * The MSR filters will be applied when handling the TDVMCALL for + * RDMSR/WRMSR if the userspace has set any. */ if (is_td_vcpu(vcpu)) return; =20 - vmx_recalc_msr_intercepts(vcpu); + vmx_recalc_intercepts(vcpu); } =20 static int vt_complete_emulated_msr(struct kvm_vcpu *vcpu, int err) @@ -995,7 +995,7 @@ struct kvm_x86_ops vt_x86_ops __initdata =3D { .apic_init_signal_blocked =3D vt_op(apic_init_signal_blocked), .migrate_timers =3D vmx_migrate_timers, =20 - .recalc_msr_intercepts =3D vt_op(recalc_msr_intercepts), + .recalc_intercepts =3D vt_op(recalc_intercepts), .complete_emulated_msr =3D vt_op(complete_emulated_msr), =20 .vcpu_deliver_sipi_vector =3D kvm_vcpu_deliver_sipi_vector, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 93b87f9e6dfd..2244ca074e9d 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4068,7 +4068,7 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vcp= u) } } =20 -void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu) +static void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu) { if (!cpu_has_vmx_msr_bitmap()) return; @@ -4121,6 +4121,11 @@ void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu) */ } =20 +void vmx_recalc_intercepts(struct kvm_vcpu *vcpu) +{ + vmx_recalc_msr_intercepts(vcpu); +} + static int vmx_deliver_nested_posted_interrupt(struct kvm_vcpu *vcpu, int vector) { @@ -7778,7 +7783,7 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) ~FEAT_CTL_SGX_LC_ENABLED; =20 /* Recalc MSR interception to account for feature changes. */ - vmx_recalc_msr_intercepts(vcpu); + vmx_recalc_intercepts(vcpu); =20 /* Refresh #PF interception to account for MAXPHYADDR changes. */ vmx_update_exception_bitmap(vcpu); diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 2b3424f638db..2c590ff44ced 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -52,7 +52,7 @@ void vmx_deliver_interrupt(struct kvm_lapic *apic, int de= livery_mode, int trig_mode, int vector); void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu); bool vmx_has_emulated_msr(struct kvm *kvm, u32 index); -void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu); +void vmx_recalc_intercepts(struct kvm_vcpu *vcpu); void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu); void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu); int vmx_get_feature_msr(u32 msr, u64 *data); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2c34dd3f0222..69f5d9deb75f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6742,7 +6742,11 @@ static int kvm_vm_ioctl_set_msr_filter(struct kvm *k= vm, =20 kvm_free_msr_filter(old_filter); =20 - kvm_make_all_cpus_request(kvm, KVM_REQ_MSR_FILTER_CHANGED); + /* + * Recalc MSR intercepts as userspace may want to intercept accesses to + * MSRs that KVM would otherwise pass through to the guest. + */ + kvm_make_all_cpus_request(kvm, KVM_REQ_RECALC_INTERCEPTS); =20 return 0; } @@ -10765,13 +10769,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_APF_READY, vcpu)) kvm_check_async_pf_completion(vcpu); =20 - /* - * Recalc MSR intercepts as userspace may want to intercept - * accesses to MSRs that KVM would otherwise pass through to - * the guest. - */ - if (kvm_check_request(KVM_REQ_MSR_FILTER_CHANGED, vcpu)) - kvm_x86_call(recalc_msr_intercepts)(vcpu); + if (kvm_check_request(KVM_REQ_RECALC_INTERCEPTS, vcpu)) + kvm_x86_call(recalc_intercepts)(vcpu); =20 if (kvm_check_request(KVM_REQ_UPDATE_CPU_DIRTY_LOGGING, vcpu)) kvm_x86_call(update_cpu_dirty_logging)(vcpu); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 618322BE648 for ; Wed, 6 Aug 2025 19:58:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510299; cv=none; b=nZWPwcbJijaVxgoiGin8hQwnDXfz3r+LrwdIGyVzvEook0z22uzaET0IIyKk/rQLRgUOTURmgb81p77Jhh33sNiCNI9GL/9k9Nj1DrOiovKYrYvgEg1WCn7rMbt7TTSBgTVuiXXV0Rt7oPjJfUSoS5MggozAVDa0XDb0kn3TQRk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510299; c=relaxed/simple; bh=wHmlJSVtwVaQD0ZM8WmfI3XDA8yD245AfBXY7EV2M10=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gbO3bkmWuT8G0aA994K+jDi/EiP9/j6iQYT1LaTxTqZ1lOT2xvwFgEustFoi4OwS58dUMHl+v20HxCiEptc+Ul+rfocMHL6EZEHidoFUoQdVPeq5T0gJdKOwBQMIzAKAJEw2iSSy1dePZRbVP98kWsqepH9yb7IoXuFqQ3EdFUw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wlWkR4yx; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wlWkR4yx" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2424aaa9840so2089585ad.1 for ; Wed, 06 Aug 2025 12:58:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510295; x=1755115095; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=7n78nD110ztSdGmfKKJPHep+PzR1j4ORmKnKizGjxcI=; b=wlWkR4yxBCBKUd7FpN2J//jigQqdrrSk22OgMrzaJUZf60flb19Mo8kFL5B6Gwt/V+ xxH4hHX8vdn0y462uePOP8nIkxez+kkJLkMJTYNTaBkHzeO9RL1qAuqMyRic2szqeqjP TEvYgD09Zu0QbT7oWnXj3oP8+pQH2h5bHzxq2QEl0nYdJMD1BMoiZnLQo+Zvlc2JRiUt wy3kJuE1Qnv5L/2fuUf1EiNkqhdQaal/8hn3ee52feH0SsPlgH1wgx42q/TGuKnURMn2 sXfxEjqnW1xWmCYPbXeZKWV3jmIiJW9tFwmu6D/qUlO/sgwUvSLRhZN8Na5gxCqb377x nfZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510295; x=1755115095; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7n78nD110ztSdGmfKKJPHep+PzR1j4ORmKnKizGjxcI=; b=Z9LUJVvThanW+XTrLF0KgEvs9S+Kh6N7eD4776MNYmYtv4oQPn211iiRMsdmdtt4JX WuOxUutFcZj3Isb7Syvh3U0pxgdM3Q4w4zx8A8wPP9Tbxf332jjDhePxGS8FqnjHXWFl Ao7azlYDgQ744Pkb2hvY+BMS0BS9/ulYY+/d/Briz+Ety68pO8A1qZnj1lO81bAbooua dCrKbgCmQJQCeoZDZdJ9tq5jRBWzZrMDV8nt322sIbTeo3JBKG/95JpwIM5bLq5diPu7 tq1Yyjw1PYCwQ7nkh25vz8XMsNpSXl4/HYoEZl9aGsDKu+1Uyu41YsGvPGzW3jY4LScp wJVw== X-Forwarded-Encrypted: i=1; AJvYcCXxhx+iocMJav7hNmbT9/QqiA4HAOaQ00cCWLe4LoglwlQLBLxwW5ELJqMj289reCMb27seJRf9XtLGI38=@vger.kernel.org X-Gm-Message-State: AOJu0YxKqEiHIYRv8d7S/4LOedKmLnIFwI8L6jUuM2aZ6+bQCAxx9ayc XsMMQr+K6VA60fuoeqnupFkG3zBnVn7pd/g1eXAH54uqLhDc5NV/FOhawWQC6C6jJIiPqu1uEPz AT5aLwg== X-Google-Smtp-Source: AGHT+IFqf7ZAG29dPCj/MpatUq6hwXzWupLmSqVpNm34U7crTT1sTW/TFLrPZa6u05A4GwIlFGKGaClhUR8= X-Received: from pjpy17.prod.google.com ([2002:a17:90a:a411:b0:321:370d:cae5]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:cec8:b0:242:abac:30f0 with SMTP id d9443c01a7336-242abac3c63mr29143805ad.42.1754510294999; Wed, 06 Aug 2025 12:58:14 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:47 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-26-seanjc@google.com> Subject: [PATCH v5 25/44] KVM: x86: Use KVM_REQ_RECALC_INTERCEPTS to react to CPUID updates From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Defer recalculating MSR and instruction intercepts after a CPUID update via RECALC_INTERCEPTS to converge on RECALC_INTERCEPTS as the "official" mechanism for triggering recalcs. As a bonus, because KVM does a "recalc" during vCPU creation, and every functional VMM sets CPUID at least once, for all intents and purposes this saves at least one recalc. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/cpuid.c | 2 ++ arch/x86/kvm/svm/svm.c | 4 +--- arch/x86/kvm/vmx/vmx.c | 3 --- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index e2836a255b16..cc16e28bfab2 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -448,6 +448,8 @@ void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) * adjustments to the reserved GPA bits. */ kvm_mmu_after_set_cpuid(vcpu); + + kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu); } =20 int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 3d9dcc66a407..ef7dffc54dca 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1225,7 +1225,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu) =20 svm_hv_init_vmcb(vmcb); =20 - svm_recalc_intercepts(vcpu); + kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu); =20 vmcb_mark_all_dirty(vmcb); =20 @@ -4478,8 +4478,6 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu = *vcpu) =20 if (sev_guest(vcpu->kvm)) sev_vcpu_after_set_cpuid(svm); - - svm_recalc_intercepts(vcpu); } =20 static bool svm_has_wbinvd_exit(void) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 2244ca074e9d..6094de4855d6 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7782,9 +7782,6 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) vmx->msr_ia32_feature_control_valid_bits &=3D ~FEAT_CTL_SGX_LC_ENABLED; =20 - /* Recalc MSR interception to account for feature changes. */ - vmx_recalc_intercepts(vcpu); - /* Refresh #PF interception to account for MAXPHYADDR changes. */ vmx_update_exception_bitmap(vcpu); } --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04BE72D877C for ; Wed, 6 Aug 2025 19:58:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510305; cv=none; b=Ha2b4VDFo9fFXSG1Of7ulRmQDgf4KR4WL7qaHC39SgeHWwo1HtXLbSGxZjrmRQS5jpohxG4/tfLZq17VoGSHU+W6KUGq7ujwN48ddQQxZkd6Ar7Rk/im83m3fiT4bQws9iVVbagLQPSowEvfnq2HiFqF6jkZ3PD2/OlS9d5aAu4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510305; c=relaxed/simple; bh=jAkcPQi41ZRUiWgI9bcgpSonHu4bIB2ojvxbKVROfG0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dp+A70jOuEcLpmhBtdGs/Pogc8QhF9tuOPhJn2tgCUB7px1DfeNK3uxBdixBOL4qgWY3tFEUUFtaDBSiwtaD01EgSF987UZ8y84b0IN9YIUVUPck8QRpmZhAGxTSnpv6Gh2z1ZGyDKndI9uce9bJtQ/jHxg2+i50lH+5HJrjxOc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TeRbBywX; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TeRbBywX" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b3f76a5ba7fso115508a12.1 for ; Wed, 06 Aug 2025 12:58:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510297; x=1755115097; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=z6mRW3QO5+v30hMhQPo0nMlPZQjpAzxmFKtx3JiIrDs=; b=TeRbBywXy3/HckK5eGfsnHxSEh+9vQCZfspv9MpDXJ+LqCtFF9262mSffThUlNvyCZ MXeUSfvu3ECVvAy7YUXAM7HMOFML++5Jkn/Vkf3YU6xVzzx2szodilCsNapPiU7t5xXi Cv5T8NFMwzJGs4w3T0+dcpXb3EPeXX9BoL0L/jj5oi2dd2wBHMbG/5obSQtRRXJrSAdl JGGH9Umcl+Zxn8Ks1JY8aIZMTndtCqfCWbMbzwUmCw6dTf7Ra0VO4sIlM5Cbk/AZfkeb 68LxFjb7P3XeYDsZZWi9zxJMxRN3oBMj7Po5EOnXFmYiMe4+N9PgJsqcnFVurEUynlei iWBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510297; x=1755115097; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=z6mRW3QO5+v30hMhQPo0nMlPZQjpAzxmFKtx3JiIrDs=; b=KV0EOqHBrsxALQlqGmJSTQe3dP+E2d1bPtHjDbg0jmm6n3rx6sNVRT7mjFGoe6WBPo UZayOj1b7PKL8BOzM7SrdfwtMbSNZ70tRoC5iN0+JDihr9y0TA/dy4LHbFA/KAXVZ3NR nnJKHEg90N0/42xsKhb8Eld6wRiaYJQi6m80OHu5yKGccVVs5gSaIWAKsbGscVL9/AE3 eWJoicHYogck7ocJsoQxx2LwnyBhUEn3rgyGup84ctTKBJ9VJNAPN00BP7feEh8qhNDo y8dO1jrGeq4KnfrFU4HLgvcAJL2rkx0S2z14Y69xiFOwjnoB/xQcZ4dBJxvSZXptr1ci 4Wfw== X-Forwarded-Encrypted: i=1; AJvYcCU2ySWzbA9il+KVcLcFQLIwhKS80Wg3Pya5oKFG4tGAveeaiWtci++TVqm70MNhTFjjDIbiAVXUMlJBiY8=@vger.kernel.org X-Gm-Message-State: AOJu0YzKNINmHvOQJviDLCBMRoYfwVqzw5lpj6rqckMI4e4khPec0eQH IMh/JG+Gk0xxV4/DRztxLBzpwphAhBdxgCgmuyZ+ZP+VYUwCW/xMx/b+G5x0qaaKnb2pjhTfXA+ t6Fx17g== X-Google-Smtp-Source: AGHT+IF1L5jDO3EJUjPLOXu/s6M1eVdU0dJDtzEbPUmRbyNFn2oUqefjt3C97kWpA7IvjzuPBYgNpNpIe1I= X-Received: from pgbcv5.prod.google.com ([2002:a05:6a02:4205:b0:b42:49c8:5488]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:548d:b0:23f:faa0:2c27 with SMTP id adf61e73a8af0-240330367c3mr5937028637.20.1754510296766; Wed, 06 Aug 2025 12:58:16 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:48 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-27-seanjc@google.com> Subject: [PATCH v5 26/44] KVM: VMX: Add helpers to toggle/change a bit in VMCS execution controls From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Expand the VMCS controls builder macros to generate helpers to change a bit to the desired value, and use the new helpers when toggling APICv related controls. No functional change intended. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Signed-off-by: Mingwei Zhang [sean: rewrite changelog] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/vmx/vmx.c | 20 +++++++------------- arch/x86/kvm/vmx/vmx.h | 8 ++++++++ 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 6094de4855d6..baea4a9cf74f 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4356,19 +4356,13 @@ void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *v= cpu) =20 pin_controls_set(vmx, vmx_pin_based_exec_ctrl(vmx)); =20 - if (kvm_vcpu_apicv_active(vcpu)) { - secondary_exec_controls_setbit(vmx, - SECONDARY_EXEC_APIC_REGISTER_VIRT | - SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY); - if (enable_ipiv) - tertiary_exec_controls_setbit(vmx, TERTIARY_EXEC_IPI_VIRT); - } else { - secondary_exec_controls_clearbit(vmx, - SECONDARY_EXEC_APIC_REGISTER_VIRT | - SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY); - if (enable_ipiv) - tertiary_exec_controls_clearbit(vmx, TERTIARY_EXEC_IPI_VIRT); - } + secondary_exec_controls_changebit(vmx, + SECONDARY_EXEC_APIC_REGISTER_VIRT | + SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY, + kvm_vcpu_apicv_active(vcpu)); + if (enable_ipiv) + tertiary_exec_controls_changebit(vmx, TERTIARY_EXEC_IPI_VIRT, + kvm_vcpu_apicv_active(vcpu)); =20 vmx_update_msr_bitmap_x2apic(vcpu); } diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index d3389baf3ab3..a4e5bcd1d023 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -608,6 +608,14 @@ static __always_inline void lname##_controls_clearbit(= struct vcpu_vmx *vmx, u##b { \ BUILD_BUG_ON(!(val & (KVM_REQUIRED_VMX_##uname | KVM_OPTIONAL_VMX_##uname= ))); \ lname##_controls_set(vmx, lname##_controls_get(vmx) & ~val); \ +} \ +static __always_inline void lname##_controls_changebit(struct vcpu_vmx *vm= x, u##bits val, \ + bool set) \ +{ \ + if (set) \ + lname##_controls_setbit(vmx, val); \ + else \ + lname##_controls_clearbit(vmx, val); \ } BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 522DB2BE64A for ; Wed, 6 Aug 2025 19:58:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510307; cv=none; b=GWUZUj4cjFxLiVx463YZ7deRptIZkujAzOyiuomnBaVqCnCTW4VJkgFc78uii8kq9Bng7So5T6KVM5EVvjbkaatV4ZMoX0XTHUvFSB1R792yIMMsve3vsFDo9hzu8EV29FluiUWbt6MEZGSDf/5ENFVT+8E3W9c2t/cqKXyUe/U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510307; c=relaxed/simple; bh=meRueoJoecSOLM6U35RVopVJ4WkffubPgWqswgDoiXc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lj82jX7WLnHTKgmGkErP8pxQCEjWELOPGnWPmKT0h+s6MKer2BlhLZ09VRFJQnBX5fS7JRqwpuYUSQvICw8MUMdHlD0JsrOpTGEbGfnNoJNoFXe8FFQwElLWo1DjplkPveqwbRvUS/c3xRLpbUS6M7nJhOaGGDRN1lSF0ytXa/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=blGjs/vG; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="blGjs/vG" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31f32271a1fso373195a91.0 for ; Wed, 06 Aug 2025 12:58:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510299; x=1755115099; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YQRjFPIG4PmR+HjiezlmfcX2mvV34MtTdW3Bam7UEwE=; b=blGjs/vGrtFyGaNGCgcw6j3Om8nu7dp87YUljZdXgYGnDrrfsrjeGhlME6O9d1xhsR cuZLYbrGK/Gsmw+nk5NmyMx2TQ9K5ahCg/FbsJMVsUk1HGQX0AHU0nIeP9Cqwd1Y1wuP 7LHpsGJHO0VV8iXkOeIPN/LuuPtPh9U9SRA57DlGi+rquymwFx3fxqVnQkQ1sReSJx35 QAJAaOViJsSXfmzD9gIJHUEyb8l65yd69XsK3zZz/dTUXLxGfH2sIGZwgLB0u+CUoZ0e OCsGwpJFpbxN/1CYI07Kl1zmgocHtHBefEBM61GTq/P0AnhUVPRqCkamBXR7LC5yCq6A 0etQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510299; x=1755115099; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YQRjFPIG4PmR+HjiezlmfcX2mvV34MtTdW3Bam7UEwE=; b=VOPYQciPwC8dJDr3DNn9JcYyNBDps5y1a+ejj4/qqMLdH1scXC61B5ruhuTfP3T/i4 kkwq1EI2+OHYLCurA32SbOcy8Eky00xnyD6gHuO7BducPOu0VJFRAb4crf2g4+6kVqaD lnVBxaKKTwcC0m04cVlBspRcrGAgwtx7wtxp5AzxpTt8ioZobSTiDJqTR+nftCeXvCK+ IQ3wNietzQqQGhvuz11hOUgDOo/jAbmhgmPKg6Vg45//kKg6xEOqAA2RriREMGqaTXj3 LEYmtXHgW0cEmQppvB6s+fZMk/YDsfvfKylO/EqOskaTannFMX53UwJlqlSPWqo+A5uN W3Gg== X-Forwarded-Encrypted: i=1; AJvYcCXkujuEAC0Ysvhd7CaW4NUUpA06FU7pjJPX4ilgYtIYeOHMVFtPWK51zroR2hyZCFwoqWqzmkAxKVcJL70=@vger.kernel.org X-Gm-Message-State: AOJu0Yxpx1zajEjD78pMbtlBQzOv+TEWYPeqX7YMTSfgMNVO5eh0s7bQ 53L+NlgEbBqH8OMoHWElDfjj9Wyso1+kDmPgUvI1LN4+grzYqnyESmoVxQQAamRtCOAyoe931Mp 2lKBUZQ== X-Google-Smtp-Source: AGHT+IEEL8MMMXuPjd0UTQ7B+xCdzayxBbFfQWod4bTn8hh+ljNhk+I3PdaJAFaKAxfSi5B+qkz2qkDbG+c= X-Received: from pjbqb13.prod.google.com ([2002:a17:90b:280d:b0:31f:61fc:b283]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:390f:b0:314:2cd2:595d with SMTP id 98e67ed59e1d1-321755de480mr997552a91.8.1754510298596; Wed, 06 Aug 2025 12:58:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:49 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-28-seanjc@google.com> Subject: [PATCH v5 27/44] KVM: x86/pmu: Disable RDPMC interception for compatible mediated vPMU From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Disable RDPMC interception for vCPUs with a mediated vPMU that is compatible with the host PMU, i.e. that doesn't require KVM emulation of RDPMC to honor the guest's vCPU model. With a mediated vPMU, all guest state accessible via RDPMC is loaded into hardware while the guest is running. Adust RDPMC interception only for non-TDX guests, as the TDX module is responsible for managing RDPMC intercepts based on the TD configuration. Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang Co-developed-by: Sandipan Das Signed-off-by: Sandipan Das Signed-off-by: Dapeng Mi Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 26 ++++++++++++++++++++++++++ arch/x86/kvm/pmu.h | 1 + arch/x86/kvm/svm/svm.c | 5 +++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 1 + 5 files changed, 40 insertions(+) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 680523e9d11e..674f42d083a9 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -712,6 +712,32 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx,= u64 *data) return 0; } =20 +bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + + if (!kvm_vcpu_has_mediated_pmu(vcpu)) + return true; + + /* + * VMware allows access to these Pseduo-PMCs even when read via RDPMC + * in Ring3 when CR4.PCE=3D0. + */ + if (enable_vmware_backdoor) + return true; + + /* + * Note! Check *host* PMU capabilities, not KVM's PMU capabilities, as + * KVM's capabilities are constrained based on KVM support, i.e. KVM's + * capabilities themselves may be a subset of hardware capabilities. + */ + return pmu->nr_arch_gp_counters !=3D kvm_host_pmu.num_counters_gp || + pmu->nr_arch_fixed_counters !=3D kvm_host_pmu.num_counters_fixed || + pmu->counter_bitmask[KVM_PMC_GP] !=3D (BIT_ULL(kvm_host_pmu.bit_wi= dth_gp) - 1) || + pmu->counter_bitmask[KVM_PMC_FIXED] !=3D (BIT_ULL(kvm_host_pmu.bit= _width_fixed) - 1); +} +EXPORT_SYMBOL_GPL(kvm_need_rdpmc_intercept); + void kvm_pmu_deliver_pmi(struct kvm_vcpu *vcpu) { if (lapic_in_kernel(vcpu)) { diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index e038bce76b9e..6b95e81c078c 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -238,6 +238,7 @@ void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu); void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu); =20 bool is_vmware_backdoor_pmc(u32 pmc_idx); +bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu); =20 extern struct kvm_pmu_ops intel_pmu_ops; extern struct kvm_pmu_ops amd_pmu_ops; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index ef7dffc54dca..2d42962b47aa 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1075,6 +1075,11 @@ static void svm_recalc_instruction_intercepts(struct= kvm_vcpu *vcpu) svm->vmcb->control.virt_ext |=3D VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK; } } + + if (kvm_need_rdpmc_intercept(vcpu)) + svm_set_intercept(svm, INTERCEPT_RDPMC); + else + svm_clr_intercept(svm, INTERCEPT_RDPMC); } =20 static void svm_recalc_intercepts(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index baea4a9cf74f..2f7db32710e3 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4121,8 +4121,15 @@ static void vmx_recalc_msr_intercepts(struct kvm_vcp= u *vcpu) */ } =20 +static void vmx_recalc_instruction_intercepts(struct kvm_vcpu *vcpu) +{ + exec_controls_changebit(to_vmx(vcpu), CPU_BASED_RDPMC_EXITING, + kvm_need_rdpmc_intercept(vcpu)); +} + void vmx_recalc_intercepts(struct kvm_vcpu *vcpu) { + vmx_recalc_instruction_intercepts(vcpu); vmx_recalc_msr_intercepts(vcpu); } =20 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 69f5d9deb75f..b8014435c988 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3793,6 +3793,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct = msr_data *msr_info) =20 vcpu->arch.perf_capabilities =3D data; kvm_pmu_refresh(vcpu); + kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu); break; case MSR_IA32_PRED_CMD: { u64 reserved_bits =3D ~(PRED_CMD_IBPB | PRED_CMD_SBPB); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 570822D8760 for ; Wed, 6 Aug 2025 19:58:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510305; cv=none; b=OBBN37EhT2l/vvj8hHEsOsngm0jUk9NhLQhzzl/DDt/hmpUEH76vtAtXfHc2MDTydt7CSTGW7jAjFKLqZo3z3tkGuVHuoUz+l7ZcF6GT4sB4VAuDNM8De+qKvzqB7i9oRK4W5QHAdDl3SyYCOTVPFzrmxx/YotDQmGE4HDed5io= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510305; c=relaxed/simple; bh=R67Joq41yriILvZEu+rt+LOTNRmRT1/PlQ171+4+0JY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=I4yMJQSnS4+66owYbP3BDZ/E3KNTFW+7QC85Yk3BTCALntK0Q7ITcZEcYmHyo4VOoUJ4SC8m8dc0Hu6WXIt9iUtDYkfnkm7fXWcXtgPacGK5mX2nfFxY6ErdA/M3FU1VEngNRYkQGkyOCRwHZYHTzG4v18PH1v1HuVe97X9jQpY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=daQZ6GZh; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="daQZ6GZh" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-242abac2cb4so2082365ad.0 for ; Wed, 06 Aug 2025 12:58:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510300; x=1755115100; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=gVyp6tgdczT9WqGB/7OEU1lR+4+s9dRJCcTxlTSolzo=; b=daQZ6GZhot32S9xdqNMdg9ssTJh+NuuzwZGMoH6fS/jFYMDAxrhRNZ48Zhq7IqXOVy Y9xud0IBLdpkLNOupRdM8e43YdYZ1+R6GeTzFEYxXDAe6DAQc4LwJ+xeMlZrgjRjZ2eA TfaN0PeZgr/y1mVqRUqWSz3MAgt0uLTVt+seXLj1Q0b5xj7yDgKaRA6J5OwZpZoub3T0 gnJ1tvSVlLok8y89dcEYKZ7iiNWKXyw1sYOL+B1HqzI0+O22c+LogPqeFex2n+VQ11BC YeAxUQWGSXwjMLeiqWJ2GqZNv7qWnxl8Oz9419OlIqWuz86ziwk6Uht7Koe7l+YQ2NjI Alhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510300; x=1755115100; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gVyp6tgdczT9WqGB/7OEU1lR+4+s9dRJCcTxlTSolzo=; b=wiNRuLdW7kK6Yvbnr2JnOx/Rbvog5RX3jbuNgj1stsmRkmEXsU6dP+NPN0D8ywXyXP S9qrpNX2zTf56vmA1gIrkm5L/JtJFbUhBbQgA9RBXXqdLywQ0mSKmadOBuLg7r6WEgLD HOWhgrHqnAGUjMUDMin2ZYSb9MhRWxbuOkUS3drbeYPVTTcNb5DwIY5ay9kxEUtEJ71z a43yekf++o03M3hMwjpX9Uh9vO2M8BHu+iT+G8Mwl8bCPKYq3In4iEQcEzvtRwrZlsEM NGb3mX5i+OQ9309hgAB/QGx+Mlgmlmrx60m4eHtPqOgkYTiiIxzYBbrk4SyFD9p2qtlG nIPg== X-Forwarded-Encrypted: i=1; AJvYcCX4/qpvADhqrblo0Gbh76hCjBYtK1j1/rDvqTAoo1pwFQVeShRPULopHoIZf9yAotyn629OLqz798Jb2hY=@vger.kernel.org X-Gm-Message-State: AOJu0Yzu9eqI5+kSy0jF/M+BFcZtMrvsg2gAKXDj16J72jandKO6X2xs t7AY9F9ya2L4pgPEPW0RM94769zeCoC6uLEWBDLTYtfzuAaiw5VonEs25b/lxwtjnEJ2M1oQhMz XkvtcBQ== X-Google-Smtp-Source: AGHT+IFl8pLHFeWL64GXKcFZRMj90gXiZ/jioteST4D9/w8WrcLO4Mu0ehBWI7xFV7mdsuh2Epecrrv18JA= X-Received: from pltf11.prod.google.com ([2002:a17:902:74cb:b0:234:d00c:b347]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ccc3:b0:23f:f983:5ca1 with SMTP id d9443c01a7336-242a0ae476bmr50691075ad.12.1754510300477; Wed, 06 Aug 2025 12:58:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:50 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-29-seanjc@google.com> Subject: [PATCH v5 28/44] KVM: x86/pmu: Load/save GLOBAL_CTRL via entry/exit fields for mediated PMU From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi When running a guest with a mediated PMU, context switch PERF_GLOBAL_CTRL via the dedicated VMCS fields for both host and guest. For the host, always zero GLOBAL_CTRL on exit as the guest's state will still be loaded in hardware (KVM will context switch the bulk of PMU state outside of the inner run loop). For the guest, use the dedicated fields to atomically load and save PERF_GLOBAL_CTRL on all entry/exits. Note, VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL was introduced by Sapphire Rapids, and is expected to be supported on all CPUs with PMU v4+. WARN if that expectation is not met. Alternatively, KVM could manually save PERF_GLOBAL_CTRL via the MSR save list, but the associated complexity and runtime overhead is unjustified given that the feature should always be available on relevant CPUs. To minimize VM-Entry latency, propagate IA32_PERF_GLOBAL_CTRL to the VMCS on-demand. But to minimize complexity, read IA32_PERF_GLOBAL_CTRL out of the VMCS on all non-failing VM-Exits. I.e. partially cache the MSR. KVM could track GLOBAL_CTRL as an EXREG and defer all reads, but writes are rare, i.e. the dirty tracking for an EXREG is unnecessary, and it's not obvious that shaving ~15-20 cycles per exit is meaningful given the total overhead associated with mediated PMU context switches. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/include/asm/kvm-x86-pmu-ops.h | 2 ++ arch/x86/include/asm/vmx.h | 1 + arch/x86/kvm/pmu.c | 13 +++++++++-- arch/x86/kvm/pmu.h | 3 ++- arch/x86/kvm/vmx/capabilities.h | 5 +++++ arch/x86/kvm/vmx/pmu_intel.c | 19 +++++++++++++++- arch/x86/kvm/vmx/vmx.c | 31 +++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 3 ++- 8 files changed, 71 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-pmu-ops.h b/arch/x86/include/asm/= kvm-x86-pmu-ops.h index 9159bf1a4730..ad2cc82abf79 100644 --- a/arch/x86/include/asm/kvm-x86-pmu-ops.h +++ b/arch/x86/include/asm/kvm-x86-pmu-ops.h @@ -23,5 +23,7 @@ KVM_X86_PMU_OP_OPTIONAL(reset) KVM_X86_PMU_OP_OPTIONAL(deliver_pmi) KVM_X86_PMU_OP_OPTIONAL(cleanup) =20 +KVM_X86_PMU_OP_OPTIONAL(write_global_ctrl) + #undef KVM_X86_PMU_OP #undef KVM_X86_PMU_OP_OPTIONAL diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index cca7d6641287..af71666c3a37 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -106,6 +106,7 @@ #define VM_EXIT_CLEAR_BNDCFGS 0x00800000 #define VM_EXIT_PT_CONCEAL_PIP 0x01000000 #define VM_EXIT_CLEAR_IA32_RTIT_CTL 0x02000000 +#define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL 0x40000000 =20 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR 0x00036dff =20 diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 674f42d083a9..a4fe0e76df79 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -103,7 +103,7 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_o= ps) #undef __KVM_X86_PMU_OP } =20 -void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops) +void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops) { bool is_intel =3D boot_cpu_data.x86_vendor =3D=3D X86_VENDOR_INTEL; int min_nr_gp_ctrs =3D pmu_ops->MIN_NR_GP_COUNTERS; @@ -137,6 +137,9 @@ void kvm_init_pmu_capability(const struct kvm_pmu_ops *= pmu_ops) !pmu_ops->is_mediated_pmu_supported(&kvm_host_pmu)) enable_mediated_pmu =3D false; =20 + if (!enable_mediated_pmu) + pmu_ops->write_global_ctrl =3D NULL; + if (!enable_pmu) { memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap)); return; @@ -831,6 +834,9 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_d= ata *msr_info) diff =3D pmu->global_ctrl ^ data; pmu->global_ctrl =3D data; reprogram_counters(pmu, diff); + + if (kvm_vcpu_has_mediated_pmu(vcpu)) + kvm_pmu_call(write_global_ctrl)(data); } break; case MSR_CORE_PERF_GLOBAL_OVF_CTRL: @@ -921,8 +927,11 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu) * in the global controls). Emulate that behavior when refreshing the * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL. */ - if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters) + if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters) { pmu->global_ctrl =3D GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0); + if (kvm_vcpu_has_mediated_pmu(vcpu)) + kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl); + } } =20 void kvm_pmu_init(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 6b95e81c078c..dcf4e2253875 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -38,6 +38,7 @@ struct kvm_pmu_ops { void (*cleanup)(struct kvm_vcpu *vcpu); =20 bool (*is_mediated_pmu_supported)(struct x86_pmu_capability *host_pmu); + void (*write_global_ctrl)(u64 global_ctrl); =20 const u64 EVENTSEL_EVENT; const int MAX_NR_GP_COUNTERS; @@ -183,7 +184,7 @@ static inline bool pmc_is_locally_enabled(struct kvm_pm= c *pmc) =20 extern struct x86_pmu_capability kvm_pmu_cap; =20 -void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops); +void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops); =20 void kvm_pmu_recalc_pmc_emulation(struct kvm_pmu *pmu, struct kvm_pmc *pmc= ); =20 diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilitie= s.h index 26ff606ff139..874c6dd34665 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -100,6 +100,11 @@ static inline bool cpu_has_load_perf_global_ctrl(void) return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL; } =20 +static inline bool cpu_has_save_perf_global_ctrl(void) +{ + return vmcs_config.vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL; +} + static inline bool cpu_has_vmx_mpx(void) { return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_BNDCFGS; diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 7ab35ef4a3b1..98f7b45ea391 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -787,7 +787,23 @@ static bool intel_pmu_is_mediated_pmu_supported(struct= x86_pmu_capability *host_ * Require v4+ for MSR_CORE_PERF_GLOBAL_STATUS_SET, and full-width * writes so that KVM can precisely load guest counter values. */ - return host_pmu->version >=3D 4 && host_perf_cap & PERF_CAP_FW_WRITES; + if (host_pmu->version < 4 || !(host_perf_cap & PERF_CAP_FW_WRITES)) + return false; + + /* + * All CPUs that support a mediated PMU are expected to support loading + * and saving PERF_GLOBAL_CTRL via dedicated VMCS fields. + */ + if (WARN_ON_ONCE(!cpu_has_load_perf_global_ctrl() || + !cpu_has_save_perf_global_ctrl())) + return false; + + return true; +} + +static void intel_pmu_write_global_ctrl(u64 global_ctrl) +{ + vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, global_ctrl); } =20 struct kvm_pmu_ops intel_pmu_ops __initdata =3D { @@ -803,6 +819,7 @@ struct kvm_pmu_ops intel_pmu_ops __initdata =3D { .cleanup =3D intel_pmu_cleanup, =20 .is_mediated_pmu_supported =3D intel_pmu_is_mediated_pmu_supported, + .write_global_ctrl =3D intel_pmu_write_global_ctrl, =20 .EVENTSEL_EVENT =3D ARCH_PERFMON_EVENTSEL_EVENT, .MAX_NR_GP_COUNTERS =3D KVM_MAX_NR_INTEL_GP_COUNTERS, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 2f7db32710e3..1233a0afb31e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4115,6 +4115,18 @@ static void vmx_recalc_msr_intercepts(struct kvm_vcp= u *vcpu) vmx_set_intercept_for_msr(vcpu, MSR_IA32_FLUSH_CMD, MSR_TYPE_W, !guest_cpu_cap_has(vcpu, X86_FEATURE_FLUSH_L1D)); =20 + if (enable_mediated_pmu) { + bool is_mediated_pmu =3D kvm_vcpu_has_mediated_pmu(vcpu); + struct vcpu_vmx *vmx =3D to_vmx(vcpu); + + vm_entry_controls_changebit(vmx, + VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL, is_mediated_pmu); + + vm_exit_controls_changebit(vmx, + VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | + VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL, is_mediated_pmu); + } + /* * x2APIC and LBR MSR intercepts are modified on-demand and cannot be * filtered by userspace. @@ -4282,6 +4294,16 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vm= x) =20 if (cpu_has_load_ia32_efer()) vmcs_write64(HOST_IA32_EFER, kvm_host.efer); + + /* + * When running a guest with a mediated PMU, guest state is resident in + * hardware after VM-Exit. Zero PERF_GLOBAL_CTRL on exit so that host + * activity doesn't bleed into the guest counters. When running with + * an emulated PMU, PERF_GLOBAL_CTRL is dynamically computed on every + * entry/exit to merge guest and host PMU usage. + */ + if (enable_mediated_pmu) + vmcs_write64(HOST_IA32_PERF_GLOBAL_CTRL, 0); } =20 void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) @@ -4349,7 +4371,8 @@ static u32 vmx_get_initial_vmexit_ctrl(void) VM_EXIT_CLEAR_IA32_RTIT_CTL); /* Loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically */ return vmexit_ctrl & - ~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | VM_EXIT_LOAD_IA32_EFER); + ~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | VM_EXIT_LOAD_IA32_EFER | + VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL); } =20 void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu) @@ -7087,6 +7110,9 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *= vmx) struct perf_guest_switch_msr *msrs; struct kvm_pmu *pmu =3D vcpu_to_pmu(&vmx->vcpu); =20 + if (kvm_vcpu_has_mediated_pmu(&vmx->vcpu)) + return; + pmu->host_cross_mapped_mask =3D 0; if (pmu->pebs_enable & pmu->global_ctrl) intel_pmu_cross_mapped_check(pmu); @@ -7407,6 +7433,9 @@ fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, u64 ru= n_flags) =20 vmx->loaded_vmcs->launched =3D 1; =20 + if (!msr_write_intercepted(vmx, MSR_CORE_PERF_GLOBAL_CTRL)) + vcpu_to_pmu(vcpu)->global_ctrl =3D vmcs_read64(GUEST_IA32_PERF_GLOBAL_CT= RL); + vmx_recover_nmi_blocking(vmx); vmx_complete_interrupts(vmx); =20 diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index a4e5bcd1d023..7eb57f5cb975 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -506,7 +506,8 @@ static inline u8 vmx_get_rvi(void) VM_EXIT_LOAD_IA32_EFER | \ VM_EXIT_CLEAR_BNDCFGS | \ VM_EXIT_PT_CONCEAL_PIP | \ - VM_EXIT_CLEAR_IA32_RTIT_CTL) + VM_EXIT_CLEAR_IA32_RTIT_CTL | \ + VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL) =20 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL \ (PIN_BASED_EXT_INTR_MASK | \ --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 423702D978D for ; Wed, 6 Aug 2025 19:58:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510307; cv=none; b=Y6gvZLn40phrVJfBDNmsoUahovkgtOfDca/soFEioL91UhtRMQg5E3BAkJ0Qg+Rue1qhWtHVA3UjUlyQZKgT5rfpMjfMlukHMqGZXxniI5yT12uS8NzZIdS0uzLnzZzRA1AJyvOFkJ5kCvGxVWNO8UZY02j5LQ+x3ZjTktpZDok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510307; c=relaxed/simple; bh=zkHffgBonSYg4vE9BMzEVoQYPuGz4X+vSu1d1oyeTYE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kbxJMMN8kDUK4bXs+XZU3EnJNJSpi1v4KxXCE+JPuV1/HNRCuoV/dxoM3uaaFggswUSlp0mxrxx0RBVr0pJEZ297kESU6fm0OwePe8Fb20KHWm7Pp+ZjqMLZLHqW/qM8bnlkdBZxvemJjSCREyoIoFp7n3V+3T24NHQfgOpGlQg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZuuBuR/K; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZuuBuR/K" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b42218e4a59so128085a12.0 for ; Wed, 06 Aug 2025 12:58:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510302; x=1755115102; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Jn8S/sdm+8bvzWJ6gbgYYHL6yp6bRqDxs3iW0RSe7uI=; b=ZuuBuR/K9xLC5JhgTwjiE14avrAtBxH2Jk+Y4+23C5SI0c/HDEI33Rw6jKG0UcP2Gy WQ7o9+WXKlH4pC5mgzr6AKEC90AswOTgV0qqIYA7dIG9GvNC04WJwb45a29LOcJ0MoC6 APNDsgpnxRW3Imh86119UgQEOiEJicwW38+2619PXt8+LQgJ0qmhclk6UZ4ryF33Rihk SEi9d4DTrZyo0m0ImmsCRn2yU0cySAoFnzaelaPns6QIkc3GEgmKaI3wVITdTlc7PGGk 066/lq9DYjtH5GDoQffWkphn/cX3tlAlhid4fs+W4g889WZxBIAltcyR7xtyFFqYhH6U cgZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510302; x=1755115102; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Jn8S/sdm+8bvzWJ6gbgYYHL6yp6bRqDxs3iW0RSe7uI=; b=k1YL8qdAiC25XbNCCS0SjOCKTr/7GFujIBxtoMmVgk6VAPOf2LWPWJGc3cpwY/xlLa Fpo8VSZprPPpjQU87BkwROSvpnbmLHY8rwVCqWaTjAaDCFeNnbt7o5Ddo5DVwnPAnu8+ GllAqr6iEMSICREnA/BEaiV4aFMXE31sKSuVjmMNf74s/WT2blLOVo9BbdzjGp3mg3QW /cve1caodCkH4xcxJ32t4SQvEimT1Omc/Ht7GxT2wvaRKZIERxHD51nZ7zPkGfEF+Wbv 9eUbGInaAg35vknHb720WZKQKzKeaYftuuKBuPOKgEtWohJWcu2Zv4Jvdc16cGJqD9Ax O86Q== X-Forwarded-Encrypted: i=1; AJvYcCVBgitIzPioJ9U8l5IKtgrSBqn6PGT+/F8+GPekEELQhXO9+JYHY3IcI/x2wcv8V1TT/d0YZTPFXgq4zio=@vger.kernel.org X-Gm-Message-State: AOJu0Yx89DaD4ylWFhH35jboav4W6CtkMlHDcg+ZzuDSt0PZZrqVrigT z1CDAyjWdggmrXFA/eRyiCyFxO6OKDEE1PNjoRc8bPFOYWthKrPG0IqjqDHI1MY62PNVJ1yFPw1 2MrSHLA== X-Google-Smtp-Source: AGHT+IFo65jEdjZGUxY/vUUE9hB8u0/Ehm65nxno4lYpqINC12VaTwhwaABlYMWVK9zS/nrWWsDoA8ThjT4= X-Received: from plha17.prod.google.com ([2002:a17:902:ecd1:b0:23f:d0e0:7e93]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:2cf:b0:240:71ad:a454 with SMTP id d9443c01a7336-2429f2d9e92mr70108355ad.1.1754510302251; Wed, 06 Aug 2025 12:58:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:51 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-30-seanjc@google.com> Subject: [PATCH v5 29/44] KVM: x86/pmu: Use BIT_ULL() instead of open coded equivalents From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Replace a variety of "1ull << N" and "(u64)1 << N" snippets with BIT_ULL() in the PMU code. No functional change intended. Signed-off-by: Dapeng Mi [sean: split to separate patch, write changelog] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/svm/pmu.c | 4 ++-- arch/x86/kvm/vmx/pmu_intel.c | 15 ++++++--------- 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index 96be2c3e0d65..b777c3743304 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -199,11 +199,11 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu) kvm_pmu_cap.num_counters_gp); =20 if (pmu->version > 1) { - pmu->global_ctrl_rsvd =3D ~((1ull << pmu->nr_arch_gp_counters) - 1); + pmu->global_ctrl_rsvd =3D ~(BIT_ULL(pmu->nr_arch_gp_counters) - 1); pmu->global_status_rsvd =3D pmu->global_ctrl_rsvd; } =20 - pmu->counter_bitmask[KVM_PMC_GP] =3D ((u64)1 << 48) - 1; + pmu->counter_bitmask[KVM_PMC_GP] =3D BIT_ULL(48) - 1; pmu->reserved_bits =3D 0xfffffff000280000ull; pmu->raw_event_mask =3D AMD64_RAW_EVENT_MASK; /* not applicable to AMD; but clean them to prevent any fall out */ diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 98f7b45ea391..6352d029298c 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -536,11 +536,10 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) kvm_pmu_cap.num_counters_gp); eax.split.bit_width =3D min_t(int, eax.split.bit_width, kvm_pmu_cap.bit_width_gp); - pmu->counter_bitmask[KVM_PMC_GP] =3D ((u64)1 << eax.split.bit_width) - 1; + pmu->counter_bitmask[KVM_PMC_GP] =3D BIT_ULL(eax.split.bit_width) - 1; eax.split.mask_length =3D min_t(int, eax.split.mask_length, kvm_pmu_cap.events_mask_len); - pmu->available_event_types =3D ~entry->ebx & - ((1ull << eax.split.mask_length) - 1); + pmu->available_event_types =3D ~entry->ebx & (BIT_ULL(eax.split.mask_leng= th) - 1); =20 if (pmu->version =3D=3D 1) { pmu->nr_arch_fixed_counters =3D 0; @@ -549,16 +548,15 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) kvm_pmu_cap.num_counters_fixed); edx.split.bit_width_fixed =3D min_t(int, edx.split.bit_width_fixed, kvm_pmu_cap.bit_width_fixed); - pmu->counter_bitmask[KVM_PMC_FIXED] =3D - ((u64)1 << edx.split.bit_width_fixed) - 1; + pmu->counter_bitmask[KVM_PMC_FIXED] =3D BIT_ULL(edx.split.bit_width_fixe= d) - 1; } =20 intel_pmu_enable_fixed_counter_bits(pmu, INTEL_FIXED_0_KERNEL | INTEL_FIXED_0_USER | INTEL_FIXED_0_ENABLE_PMI); =20 - counter_rsvd =3D ~(((1ull << pmu->nr_arch_gp_counters) - 1) | - (((1ull << pmu->nr_arch_fixed_counters) - 1) << KVM_FIXED_PMC_BASE_IDX)); + counter_rsvd =3D ~((BIT_ULL(pmu->nr_arch_gp_counters) - 1) | + ((BIT_ULL(pmu->nr_arch_fixed_counters) - 1) << KVM_FIXED_PMC_BASE_IDX)= ); pmu->global_ctrl_rsvd =3D counter_rsvd; =20 /* @@ -603,8 +601,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) pmu->pebs_data_cfg_rsvd =3D ~0xff00000full; intel_pmu_enable_fixed_counter_bits(pmu, ICL_FIXED_0_ADAPTIVE); } else { - pmu->pebs_enable_rsvd =3D - ~((1ull << pmu->nr_arch_gp_counters) - 1); + pmu->pebs_enable_rsvd =3D ~(BIT_ULL(pmu->nr_arch_gp_counters) - 1); } } } --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C51F02D9ED1 for ; Wed, 6 Aug 2025 19:58:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510309; cv=none; b=IArXL1WQhauSUc2ifOqd0kAq5KQQd+o3Xs8hqMSnZ9TubH6HdROyL4t1Vj5BMtVWuXxa1as7LwCwMImMfOZYf27GAo2hPz8Kq5uMjS6DXysq2b2Nibcvj2dIcJssGz1AmFaYt/ZCoI51H7YRmuAVu9lVHYiPcV0XIznHsV6DtbY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510309; c=relaxed/simple; bh=ukS/hePPonycrg9r313f+paqDEVzP34VnPUrCOh2p4I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bYXpP/uHmXTV59ZgSjoVIIDfqGAulx4ezz5j0rBYO9T4yam8VPnhkPYBi7fqHHg/X5nsSYb7SHOtgL/L2a55nJNdsOxY089LTilD4uCpB8zAzCLijIckoSlpSfEf3APIogmebmXMpoohF/Tt1dCRVN+SdZihQOLf0xWs3BBGKu0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UZQ82rAB; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UZQ82rAB" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-76bcf2cac36so317096b3a.0 for ; Wed, 06 Aug 2025 12:58:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510304; x=1755115104; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=aHvZdAUvNaaZFmbhTqFilDj1JXSRU8dLYlLgEZc/oCw=; b=UZQ82rAB1aW4wkzuRy+h0nrLTn6mywb7qRTYMXVmXs0Lw14scF1TgenREk4buW9P7V 3BI8YRbSMnxk5iu0066XjcYPmnapZifV6BOxUrPm12WCvJj9BgtzRkOKBYhj5BntbB9l RktHo5YDSGZYH4+LCxJ2Lv6TPm93RblA9g87h+W+gm4LMgGVMght3Y1CRYT/QZCkQQ5g HUogfMR/aqU0KuUg7rw/F7csjvHk/JXJ47/7FFFu9XTsy8L4XvQ2sqBK7VdJWgOjwOqa eZMuKAZybT0ET+ZRXf+8ZJ8oCDX+wm+fyRa5jm+pyuYsT+AMTDq7DPP5sES4cDpFiUfz 1+mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510304; x=1755115104; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=aHvZdAUvNaaZFmbhTqFilDj1JXSRU8dLYlLgEZc/oCw=; b=PU+cmxdGV2gN6ljixIgOzYO4ki2UmuxC6f7yV/BwSLz3p+VoiR93RKTciqZJByYpkb 9snQ/f+KDyOqq9rUqTV8RcYEhVxfUg/J6xy1VRg5kNTV0CFy1dc2bvKmbWWNJO2nLb3G NODlf8zAFY4lwVZhZdNasF0cW9Y/fbKgLgEV4jLSn2boZvD9XuGDpgxGogfcpcJ9Aipq gtWxm4+ZR8/6J3h+HZ0rJtj1ZEyk4YHroQ9P/G4i3BfxezsYdrrof8IT0axJD9f+Dmxx ajIieFl1cWQQ7fLbDQWxt6fuiNSDOeYfNcKTtIa4ADjFvHAWfNvkmPZoJ+NjcEQkx+IG eIrg== X-Forwarded-Encrypted: i=1; AJvYcCXPUhwuC7BThlwcB1ODvGSCfrf9vr5wlo/M8kws6B8jxtGICw+WtXop/HFkC+JRMzJr/zIZ3W6LbKyhzQg=@vger.kernel.org X-Gm-Message-State: AOJu0YwlKah/sE0thavT+reNxp/9Xu24m2vMTuqxdDRC6/iLWYO5YyHR Ra+66IsYataXWndkR3PRSNIuRL05/qlWtKhBGU7/Yrwrt0zPLC4F1Ow8FSc/PVX/Mso3nqkQP0t GuwXUUw== X-Google-Smtp-Source: AGHT+IGfcxFcqE+WO40f0G1jJJ785vOiC3NUhX/6TGZEyq4CvV1J0auJAX6NhvanreP79iKSuzsragVBkJw= X-Received: from pfvf15.prod.google.com ([2002:a05:6a00:1acf:b0:746:3162:8be1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:999f:b0:220:78b9:f849 with SMTP id adf61e73a8af0-240331272admr5902393637.24.1754510303927; Wed, 06 Aug 2025 12:58:23 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:52 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-31-seanjc@google.com> Subject: [PATCH v5 30/44] KVM: x86/pmu: Move initialization of valid PMCs bitmask to common x86 From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move all initialization of all_valid_pmc_idx to common code, as the logic is 100% common to Intel and AMD, and KVM heavily relies on Intel and AMD having the same semantics. E.g. the fact that AMD doesn't support fixed counters doesn't allow KVM to use all_valid_pmc_idx[63:32] for other purposes. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 4 ++++ arch/x86/kvm/svm/pmu.c | 1 - arch/x86/kvm/vmx/pmu_intel.c | 5 ----- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index a4fe0e76df79..4246e1d2cfcc 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -932,6 +932,10 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu) if (kvm_vcpu_has_mediated_pmu(vcpu)) kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl); } + + bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters); + bitmap_set(pmu->all_valid_pmc_idx, KVM_FIXED_PMC_BASE_IDX, + pmu->nr_arch_fixed_counters); } =20 void kvm_pmu_init(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index b777c3743304..9ffd44a5d474 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -209,7 +209,6 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu) /* not applicable to AMD; but clean them to prevent any fall out */ pmu->counter_bitmask[KVM_PMC_FIXED] =3D 0; pmu->nr_arch_fixed_counters =3D 0; - bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters); } =20 static void amd_pmu_init(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 6352d029298c..1de94a39ca18 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -579,11 +579,6 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) pmu->raw_event_mask |=3D (HSW_IN_TX|HSW_IN_TX_CHECKPOINTED); } =20 - bitmap_set(pmu->all_valid_pmc_idx, - 0, pmu->nr_arch_gp_counters); - bitmap_set(pmu->all_valid_pmc_idx, - INTEL_PMC_MAX_GENERIC, pmu->nr_arch_fixed_counters); - perf_capabilities =3D vcpu_get_perf_capabilities(vcpu); if (intel_pmu_lbr_is_compatible(vcpu) && (perf_capabilities & PERF_CAP_LBR_FMT)) --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F8A92DA757 for ; Wed, 6 Aug 2025 19:58:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510308; cv=none; b=M8JlONb9uK822CgyuAxD4/lroOb1UUx3tAcjREB1XkkmIGnYVM1U3FR4NQ5MQ1WsWPH6yR6GEdhaN8grmPFzF2zKeR8HjKvIkwj9yTN/UM0Zf6jzGG6JGLLtOyPRHro8bzJbYZZx7UzWOwl0RaGvFseqKFwEkhqfYALYbwSo+iE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510308; c=relaxed/simple; bh=BfJD/1Qs0KzM5Cij/W0LAx626YykgL14nn6y0sZD13E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kAy+8dPHowNQUuVoCAB/rrmAbCFHlvKrzwz303GvZ2HYKYtxk8tCEhyCPvYzJMog0EQ3tHbbPpKbaeQjS7g67igOxFNMBM9jXdqRI9AObC4S7+TlNpkPr5VFLRA+AbiYyXRhVuJ6lAA4i2p3QHfVAuebsjqXsikC/ZR2e7hglhI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cajaEp2Z; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cajaEp2Z" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31eec17b5acso271839a91.2 for ; Wed, 06 Aug 2025 12:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510306; x=1755115106; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=I4KPqQMNJiob4xQ680shjgCKcMsOL7wxmLpd2hZ8prQ=; b=cajaEp2ZyaiCFe65XpbJ4eKD0j3itpxyib3cPse27OShwbbFSkbcUKTafq/XURJaMz 116z5NBLVC5eP7FWLJRdvFUCKPOC9xTLqqgwd7HFVJH+Fyez1vT+BpEGSZhTb0XWhNhp cUyThfx6M17auu0+zDetQuHZ4sEDH2Zmsi8mGAVsgKT2yGqpH7qcO4vcLFuR4z1LVvkc LUqnZqtD7WUgFzpoRiRXmUQCOv4drP5UTx2JCu8Ue6MFOjQ3xJci8jtnzfOCZHFTsvy5 ysn3hgCHDHw0CUyVq3ZTmqYrZxGQFgXo8QalTF+qsUsH+KBdYHPeVn68zwMtV26W/Ord mmuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510306; x=1755115106; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I4KPqQMNJiob4xQ680shjgCKcMsOL7wxmLpd2hZ8prQ=; b=GrqtcCpTybVogcJf0ITXiaDzsI8CIPsQ4LxO/tY0V9ZjbB0seoi5mxar61wUHURT+e NWziWIn0sM+8Iisd+ls9ZbXt/Z4Qk/uUokM20yeH9oe9lhAXh3kziKPSyY3G17YjsI24 wDGNYggVPMofRhwlhQNXv1D2k+lCNVfUpv46DKv2xLr1eRPWlLDySCoSsXUs/Zh7Y31+ 0Ty5HArMjzZGEM56fDDUi01d8YcNFUxU4AA8SjFv+61K1jF7MMxIdRlwPIPNMyMECK0X ut8mpluCU1r68C9RNUf17+UfJJzfLjCMBLB5sfNnrx+Xbz6WXQU+SnGNdWkDT7WLwZLa wzZg== X-Forwarded-Encrypted: i=1; AJvYcCWOazIBLbwkUt3wsx8h2sIo9aa7YLzLhZlEtUMPcTsFUW6TgoEnET+flPhukLkQZGrKi4VdowVXf+wxr0w=@vger.kernel.org X-Gm-Message-State: AOJu0YxnGv+kPBHrnm1Y+QOfUMh0ci5eMRDsz5/85RYgAhTp98QA7C39 mo5ea8RhngIub96Ag5VkUD9mXxbLZY257dJEbaxDMhz5z016x7ALXoXKG9/cr4J+uWs/aWo6mup oBlCvJg== X-Google-Smtp-Source: AGHT+IEv/lsCJTvAMTt82qYQ74YEmuDhqDy/AIKEkMbUihLXDbLqrOMm3UgmKM47ElROS0wMJkdtmjAP5S0= X-Received: from pjbsn13.prod.google.com ([2002:a17:90b:2e8d:b0:30a:31eb:ec8e]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:554e:b0:31e:ff97:75a1 with SMTP id 98e67ed59e1d1-32166c19241mr5583048a91.3.1754510305634; Wed, 06 Aug 2025 12:58:25 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:53 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-32-seanjc@google.com> Subject: [PATCH v5 31/44] KVM: x86/pmu: Restrict GLOBAL_{CTRL,STATUS}, fixed PMCs, and PEBS to PMU v2+ From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Restrict support for GLOBAL_CTRL, GLOBAL_STATUS, fixed PMCs, and PEBS to v2 or later vPMUs. The SDM explicitly states that GLOBAL_{CTRL,STATUS} and fixed counters were introduced with PMU v2, and PEBS has hard dependencies on fixed counters and the bitmap MSR layouts established by PMU v2. Reported-by: Dapeng Mi Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/vmx/pmu_intel.c | 51 ++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 1de94a39ca18..779b4e64acac 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -541,16 +541,33 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) kvm_pmu_cap.events_mask_len); pmu->available_event_types =3D ~entry->ebx & (BIT_ULL(eax.split.mask_leng= th) - 1); =20 - if (pmu->version =3D=3D 1) { - pmu->nr_arch_fixed_counters =3D 0; - } else { - pmu->nr_arch_fixed_counters =3D min_t(int, edx.split.num_counters_fixed, - kvm_pmu_cap.num_counters_fixed); - edx.split.bit_width_fixed =3D min_t(int, edx.split.bit_width_fixed, - kvm_pmu_cap.bit_width_fixed); - pmu->counter_bitmask[KVM_PMC_FIXED] =3D BIT_ULL(edx.split.bit_width_fixe= d) - 1; + entry =3D kvm_find_cpuid_entry_index(vcpu, 7, 0); + if (entry && + (boot_cpu_has(X86_FEATURE_HLE) || boot_cpu_has(X86_FEATURE_RTM)) && + (entry->ebx & (X86_FEATURE_HLE|X86_FEATURE_RTM))) { + pmu->reserved_bits ^=3D HSW_IN_TX; + pmu->raw_event_mask |=3D (HSW_IN_TX|HSW_IN_TX_CHECKPOINTED); } =20 + perf_capabilities =3D vcpu_get_perf_capabilities(vcpu); + if (intel_pmu_lbr_is_compatible(vcpu) && + (perf_capabilities & PERF_CAP_LBR_FMT)) + memcpy(&lbr_desc->records, &vmx_lbr_caps, sizeof(vmx_lbr_caps)); + else + lbr_desc->records.nr =3D 0; + + if (lbr_desc->records.nr) + bitmap_set(pmu->all_valid_pmc_idx, INTEL_PMC_IDX_FIXED_VLBR, 1); + + if (pmu->version =3D=3D 1) + return; + + pmu->nr_arch_fixed_counters =3D min_t(int, edx.split.num_counters_fixed, + kvm_pmu_cap.num_counters_fixed); + edx.split.bit_width_fixed =3D min_t(int, edx.split.bit_width_fixed, + kvm_pmu_cap.bit_width_fixed); + pmu->counter_bitmask[KVM_PMC_FIXED] =3D BIT_ULL(edx.split.bit_width_fixed= ) - 1; + intel_pmu_enable_fixed_counter_bits(pmu, INTEL_FIXED_0_KERNEL | INTEL_FIXED_0_USER | INTEL_FIXED_0_ENABLE_PMI); @@ -571,24 +588,6 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) pmu->global_status_rsvd &=3D ~MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI; =20 - entry =3D kvm_find_cpuid_entry_index(vcpu, 7, 0); - if (entry && - (boot_cpu_has(X86_FEATURE_HLE) || boot_cpu_has(X86_FEATURE_RTM)) && - (entry->ebx & (X86_FEATURE_HLE|X86_FEATURE_RTM))) { - pmu->reserved_bits ^=3D HSW_IN_TX; - pmu->raw_event_mask |=3D (HSW_IN_TX|HSW_IN_TX_CHECKPOINTED); - } - - perf_capabilities =3D vcpu_get_perf_capabilities(vcpu); - if (intel_pmu_lbr_is_compatible(vcpu) && - (perf_capabilities & PERF_CAP_LBR_FMT)) - memcpy(&lbr_desc->records, &vmx_lbr_caps, sizeof(vmx_lbr_caps)); - else - lbr_desc->records.nr =3D 0; - - if (lbr_desc->records.nr) - bitmap_set(pmu->all_valid_pmc_idx, INTEL_PMC_IDX_FIXED_VLBR, 1); - if (perf_capabilities & PERF_CAP_PEBS_FORMAT) { if (perf_capabilities & PERF_CAP_PEBS_BASELINE) { pmu->pebs_enable_rsvd =3D counter_rsvd; --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA0E22DAFA4 for ; Wed, 6 Aug 2025 19:58:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510310; cv=none; b=N9LJagfbpIsHIp1G9hiLsXfFPRrXEBuiGTQJ8JLGT+RgKnRIkSkZuLHP9BoNIHjJsX3tCprLwhET/772RzoJSTq16NHhUstrhXTInkAc21gRF7as3rf4dD0W9L3v2ZqVcsGfzQvWalpdKxKOh2s/IOS+r+5PcaSkixvRM7Q1MDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510310; c=relaxed/simple; bh=0nGwxtmjwtpfQ3TN4W9/t/0LCqh4Gk+u4uvrKvrw+k4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LqAkkKdFs6h4lD0Scl3ChGmSCUurPxfV05Y/QI9tt0koyxbjRCnmElg5jjPW6kQJmkWWZqy+85GYJmeWdC9JxGe7uMqeqCmTcc+oy02ecDEHJXkpvIcHW36ha7/B/COY1zmpUUb9HWijdqBkgnfx+4acVXx1reouhHOQo0Rv9gI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=01T4XjTZ; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="01T4XjTZ" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2425364e0b2so1992675ad.1 for ; Wed, 06 Aug 2025 12:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510307; x=1755115107; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OFlIqQp7LfOXLtzFAGq5nR1orhSSuVevPR/iGUx1pag=; b=01T4XjTZ78YEkDSbYU6EV/t+XZf4XKThEseOBLxvIWbMBWtXuRmORFUNizCcg5osFY 06b8Uj4RJnry55cr4isNnV972paya22BmykCaHJZHBlLZduBhW7E4HjHWygiNXxNIuXD Jc6uDkD+/VknGUwrf91XclnpUiUWq5BZ87bfAGxItolQbT+ixFL2qUzqxBo+uqhaHqaV cA4zKwQcxASjF1TdRPDJk6/tPDDPOicNDCJNgw8LKBrQF6HDqxdxXo3F7uBdDAOUhmXC MVoR0gfhQiH1ywNnTXP76EqAMRiYemwY1xedz71atMC5WjMDuaLdw7GU1T0A/tFcWDfv r6Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510307; x=1755115107; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OFlIqQp7LfOXLtzFAGq5nR1orhSSuVevPR/iGUx1pag=; b=rOkDi/JKkAWQGd6QvFWMzXYzH1/fB7aswe4Sttl9GtIuq7ICy2yNmp6hb9k5+v2dHS K1eRpr8/OGYwWXAj+GvjSeR4v02sELjxdaZtvQ9LckvISB2XTgGizMHlVI4FG1MkOAkO FPRO7Bdx8J2Tqm/VTCom/j5DE77BeCP2pzRgaQKweUaSAqMY8CjhgObSb8OTzANlpUMX i+Y9LMgfESVrXjJh+PkrXd8SaodJWuEPTvgbeJGy9ac/BKRe1yviloZZJpx3MZRebwjn bMWRTISVEM0OObPUlIPvxpAfwW+D5Rh5UET52S/T78x4PRLa1Nx7UG398PJkjfZZtC70 o5Fw== X-Forwarded-Encrypted: i=1; AJvYcCWPOUiT0FbXtHH1oXcF20GCsO0Ph2lPhiBN8VQc8r2Uya9wXWSDoEkB1fksnpeTlUsdwEFUPD7ClYzTln0=@vger.kernel.org X-Gm-Message-State: AOJu0YzJC9sCbtJsVsf7mTnLspkH++GZHTgJ3pKPyUybVdIrJrEP8Um0 wAyO6BkOBCMgYQnbr3beiBNnnCsz4Jpl0LY4bnJH0nAZdaShphCcR6i4ORWRHFU5CXaCrELzFYp W2Xhz1A== X-Google-Smtp-Source: AGHT+IExoXGbX1n3oux0+ebgASacivEUZvDFKwtmFYuGqpCTA7y0Fiel45h6jynxmc5ok7SFueltFvmebss= X-Received: from pjbqb13.prod.google.com ([2002:a17:90b:280d:b0:31c:160d:e3be]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e810:b0:240:10dc:b7c9 with SMTP id d9443c01a7336-2429f1e410amr61901265ad.9.1754510307282; Wed, 06 Aug 2025 12:58:27 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:54 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-33-seanjc@google.com> Subject: [PATCH v5 32/44] KVM: x86/pmu: Disable interception of select PMU MSRs for mediated vPMUs From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi For vCPUs with a mediated vPMU, disable interception of counter MSRs for PMCs that are exposed to the guest, and for GLOBAL_CTRL and related MSRs if they are fully supported according to the vCPU model, i.e. if the MSRs and all bits supported by hardware exist from the guest's point of view. Do NOT passthrough event selector or fixed counter control MSRs, so that KVM can enforce userspace-defined event filters, e.g. to prevent use of AnyThread events (which is unfortunately a setting in the fixed counter control MSR). Defer support for nested passthrough of mediated PMU MSRs to the future, as the logic for nested MSR interception is unfortunately vendor specific. Suggested-by: Sean Christopherson Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang Co-developed-by: Sandipan Das Signed-off-by: Sandipan Das Signed-off-by: Dapeng Mi [sean: squash patches, massage changelog, refresh VMX MSRs on filter change] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kvm/pmu.c | 34 ++++++++++++------ arch/x86/kvm/pmu.h | 1 + arch/x86/kvm/svm/svm.c | 36 +++++++++++++++++++ arch/x86/kvm/vmx/pmu_intel.c | 13 ------- arch/x86/kvm/vmx/pmu_intel.h | 15 ++++++++ arch/x86/kvm/vmx/vmx.c | 59 ++++++++++++++++++++++++++------ 7 files changed, 124 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-in= dex.h index f19d1ee9a396..06bd7aaae931 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -736,6 +736,7 @@ #define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS 0xc0000300 #define MSR_AMD64_PERF_CNTR_GLOBAL_CTL 0xc0000301 #define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR 0xc0000302 +#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET 0xc0000303 =20 /* AMD Last Branch Record MSRs */ #define MSR_AMD64_LBR_SELECT 0xc000010e diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 4246e1d2cfcc..817ef852bdf9 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -715,18 +715,14 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx= , u64 *data) return 0; } =20 -bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu) +bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu) { struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); =20 if (!kvm_vcpu_has_mediated_pmu(vcpu)) return true; =20 - /* - * VMware allows access to these Pseduo-PMCs even when read via RDPMC - * in Ring3 when CR4.PCE=3D0. - */ - if (enable_vmware_backdoor) + if (!kvm_pmu_has_perf_global_ctrl(pmu)) return true; =20 /* @@ -735,7 +731,22 @@ bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu) * capabilities themselves may be a subset of hardware capabilities. */ return pmu->nr_arch_gp_counters !=3D kvm_host_pmu.num_counters_gp || - pmu->nr_arch_fixed_counters !=3D kvm_host_pmu.num_counters_fixed || + pmu->nr_arch_fixed_counters !=3D kvm_host_pmu.num_counters_fixed; +} +EXPORT_SYMBOL_GPL(kvm_need_perf_global_ctrl_intercept); + +bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + + /* + * VMware allows access to these Pseduo-PMCs even when read via RDPMC + * in Ring3 when CR4.PCE=3D0. + */ + if (enable_vmware_backdoor) + return true; + + return kvm_need_perf_global_ctrl_intercept(vcpu) || pmu->counter_bitmask[KVM_PMC_GP] !=3D (BIT_ULL(kvm_host_pmu.bit_wi= dth_gp) - 1) || pmu->counter_bitmask[KVM_PMC_FIXED] !=3D (BIT_ULL(kvm_host_pmu.bit= _width_fixed) - 1); } @@ -927,11 +938,12 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu) * in the global controls). Emulate that behavior when refreshing the * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL. */ - if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters) { + if (pmu->nr_arch_gp_counters && + (kvm_pmu_has_perf_global_ctrl(pmu) || kvm_vcpu_has_mediated_pmu(vcpu)= )) pmu->global_ctrl =3D GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0); - if (kvm_vcpu_has_mediated_pmu(vcpu)) - kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl); - } + + if (kvm_vcpu_has_mediated_pmu(vcpu)) + kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl); =20 bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters); bitmap_set(pmu->all_valid_pmc_idx, KVM_FIXED_PMC_BASE_IDX, diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index dcf4e2253875..51963a3a167a 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -239,6 +239,7 @@ void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu); void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu); =20 bool is_vmware_backdoor_pmc(u32 pmc_idx); +bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu); bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu); =20 extern struct kvm_pmu_ops intel_pmu_ops; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 2d42962b47aa..add50b64256c 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -790,6 +790,40 @@ void svm_vcpu_free_msrpm(void *msrpm) __free_pages(virt_to_page(msrpm), get_order(MSRPM_SIZE)); } =20 +static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu) +{ + bool intercept =3D !kvm_vcpu_has_mediated_pmu(vcpu); + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + int i; + + if (!enable_mediated_pmu) + return; + + /* Legacy counters are always available for AMD CPUs with a PMU. */ + for (i =3D 0; i < min(pmu->nr_arch_gp_counters, AMD64_NUM_COUNTERS); i++) + svm_set_intercept_for_msr(vcpu, MSR_K7_PERFCTR0 + i, + MSR_TYPE_RW, intercept); + + intercept |=3D !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE); + for (i =3D 0; i < pmu->nr_arch_gp_counters; i++) + svm_set_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i, + MSR_TYPE_RW, intercept); + + for ( ; i < kvm_pmu_cap.num_counters_gp; i++) + svm_enable_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i, + MSR_TYPE_RW); + + intercept =3D kvm_need_perf_global_ctrl_intercept(vcpu); + svm_set_intercept_for_msr(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_CTL, + MSR_TYPE_RW, intercept); + svm_set_intercept_for_msr(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, + MSR_TYPE_RW, intercept); + svm_set_intercept_for_msr(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, + MSR_TYPE_RW, intercept); + svm_set_intercept_for_msr(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, + MSR_TYPE_RW, intercept); +} + static void svm_recalc_msr_intercepts(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm =3D to_svm(vcpu); @@ -847,6 +881,8 @@ static void svm_recalc_msr_intercepts(struct kvm_vcpu *= vcpu) if (sev_es_guest(vcpu->kvm)) sev_es_recalc_msr_intercepts(vcpu); =20 + svm_recalc_pmu_msr_intercepts(vcpu); + /* * x2APIC intercepts are modified on-demand and cannot be filtered by * userspace. diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 779b4e64acac..2bdddb95816e 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -128,19 +128,6 @@ static struct kvm_pmc *intel_rdpmc_ecx_to_pmc(struct k= vm_vcpu *vcpu, return &counters[array_index_nospec(idx, num_counters)]; } =20 -static inline u64 vcpu_get_perf_capabilities(struct kvm_vcpu *vcpu) -{ - if (!guest_cpu_cap_has(vcpu, X86_FEATURE_PDCM)) - return 0; - - return vcpu->arch.perf_capabilities; -} - -static inline bool fw_writes_is_enabled(struct kvm_vcpu *vcpu) -{ - return (vcpu_get_perf_capabilities(vcpu) & PERF_CAP_FW_WRITES) !=3D 0; -} - static inline struct kvm_pmc *get_fw_gp_pmc(struct kvm_pmu *pmu, u32 msr) { if (!fw_writes_is_enabled(pmu_to_vcpu(pmu))) diff --git a/arch/x86/kvm/vmx/pmu_intel.h b/arch/x86/kvm/vmx/pmu_intel.h index 5620d0882cdc..5d9357640aa1 100644 --- a/arch/x86/kvm/vmx/pmu_intel.h +++ b/arch/x86/kvm/vmx/pmu_intel.h @@ -4,6 +4,21 @@ =20 #include =20 +#include "cpuid.h" + +static inline u64 vcpu_get_perf_capabilities(struct kvm_vcpu *vcpu) +{ + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_PDCM)) + return 0; + + return vcpu->arch.perf_capabilities; +} + +static inline bool fw_writes_is_enabled(struct kvm_vcpu *vcpu) +{ + return (vcpu_get_perf_capabilities(vcpu) & PERF_CAP_FW_WRITES) !=3D 0; +} + bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu); int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu); =20 diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1233a0afb31e..85bd82d41f94 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4068,6 +4068,53 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vc= pu) } } =20 +static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu) +{ + bool has_mediated_pmu =3D kvm_vcpu_has_mediated_pmu(vcpu); + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + struct vcpu_vmx *vmx =3D to_vmx(vcpu); + bool intercept =3D !has_mediated_pmu; + int i; + + if (!enable_mediated_pmu) + return; + + vm_entry_controls_changebit(vmx, VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL, + has_mediated_pmu); + + vm_exit_controls_changebit(vmx, VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | + VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL, + has_mediated_pmu); + + for (i =3D 0; i < pmu->nr_arch_gp_counters; i++) { + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, + MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW, + intercept || !fw_writes_is_enabled(vcpu)); + } + for ( ; i < kvm_pmu_cap.num_counters_gp; i++) { + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, + MSR_TYPE_RW, true); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, + MSR_TYPE_RW, true); + } + + for (i =3D 0; i < pmu->nr_arch_fixed_counters; i++) + vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, + MSR_TYPE_RW, intercept); + for ( ; i < kvm_pmu_cap.num_counters_fixed; i++) + vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, + MSR_TYPE_RW, true); + + intercept =3D kvm_need_perf_global_ctrl_intercept(vcpu); + vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_STATUS, + MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, + MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, + MSR_TYPE_RW, intercept); +} + static void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu) { if (!cpu_has_vmx_msr_bitmap()) @@ -4115,17 +4162,7 @@ static void vmx_recalc_msr_intercepts(struct kvm_vcp= u *vcpu) vmx_set_intercept_for_msr(vcpu, MSR_IA32_FLUSH_CMD, MSR_TYPE_W, !guest_cpu_cap_has(vcpu, X86_FEATURE_FLUSH_L1D)); =20 - if (enable_mediated_pmu) { - bool is_mediated_pmu =3D kvm_vcpu_has_mediated_pmu(vcpu); - struct vcpu_vmx *vmx =3D to_vmx(vcpu); - - vm_entry_controls_changebit(vmx, - VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL, is_mediated_pmu); - - vm_exit_controls_changebit(vmx, - VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | - VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL, is_mediated_pmu); - } + vmx_recalc_pmu_msr_intercepts(vcpu); =20 /* * x2APIC and LBR MSR intercepts are modified on-demand and cannot be --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3EBF2DBF47 for ; Wed, 6 Aug 2025 19:58:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510312; cv=none; b=nuzUZVmo3GO480JjIAmpSQAbUImnjFE3EgXXREj8u0+BcEoQ6/MTS4TjlXXLHXJ4SHHJFivk0dgbuULxH3AeKjxZ+Z2WUNvgcMpQm0X23xIAk7c+YHmd6uIzYP1X1bN1wyTtL6TMDXPU6m+xAExOKwliS5DUpg19LUFvOAAZ7Ns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510312; c=relaxed/simple; bh=bLdrcgmYm6mqxmhXKdA5ykWfWtLQ9xTXCUeboAWobh8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=L47k+GZ+AkYtTaL33Dgc1/Zsdc8cZFR00UcLRHUQbJ7bPyh3VQaCm/uMCo2e8MckZ2mjJ+heSMbjK7vH5bTMCpAyMk5j80aTbyldLoaEgsIFZk9SNUDjxC6l6opcyLYySkdd/mv3p5ZwSW/F/tI/BwnGcl/6qDKZUbl8M9y64C0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iSXJIUU6; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iSXJIUU6" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-23fe28867b7so2375515ad.2 for ; Wed, 06 Aug 2025 12:58:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510309; x=1755115109; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=h+R0RK6o+FBkoODCsg4/sPruy7k1VkLsFBuytBNeaEE=; b=iSXJIUU6wdXDWE+TME9js/BZQbuHpAdlhA44UPPJ47gUH4lbOdS6vM2+6qDFAdQybE rbCpHs76ChOdhN6Ppd2d/JwAoVR08Eblv1MFP/7zcCJjQnEh22ngC+act2j8dyiO+hWI HKnzBwDicXtq+0NjnSee5M8vOwjYBzD6J7uMdQBTZRCzWhtnnCJ8AP0T7pO18F2w90bK BpwTyNaxEwIBOgOfq/EekyceNG0TMKmcroIucWUVN+XJ0hwLOjKuqXfWnspVBTMcZUOm xGf20/CPTp7f6hW06bV3OQgaM4JtIcnvE8mXEP58YQmi/gXxp6E8NiHbyt7R5hkpXg+U i7Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510309; x=1755115109; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=h+R0RK6o+FBkoODCsg4/sPruy7k1VkLsFBuytBNeaEE=; b=ewKRAFJ6VZe2qtlF6my2cayLd9WO6OQnRSL3hp8j64zWAPjUy/fzK15n3K5O/QLZ1k aRcNCgdB+NzHeyPYkWrNEf0qT6encB1mWJfdt06iPbKdAEXZyNLRsfT42Xbqbl9aOtwV ph2r2dov1rSd9E9SFz08+xluF0qUYWGwYk5WhwCnEtR/MUbZx2dJArV0XY5HRzcsVHgt SWOsMgLE5jldgeP++FhQY/li2EH4eOWvbsRZ7b3uJUY4I9i3dPTyNgaUG42a0KnWBwuP cCgM6ncdZMLPIUEo3DnRJwSL27oBMZANkgHzr7dXFp1f/WgvZu5RPaxbzmTKH0oFqjsT 6uPA== X-Forwarded-Encrypted: i=1; AJvYcCX351E4PSgWpS9hf3+DcoiM62nhVjNPCdAHbVFeI0CqnW7ZM53ovno1S51MjAi98U5Ja8EbTC1JXtrgx4Y=@vger.kernel.org X-Gm-Message-State: AOJu0YzJ8viuCR9k/LH9Jl8U8HOdWXr3Y4uA3VEi8R3EH/wVAzrzbP3a ZbuqMcpzD27YT3wjlxi80yvvdnU+l+kEqSoIDrTZWNyoQacvyuK8VakTR2P4kVMttB4KYcOrKnc SnYfuAw== X-Google-Smtp-Source: AGHT+IEfUJQHCKm2Ka63Vh1rz9TId3URvafpVHxlHcjND/kCO75vRaPMhhxSn7fVXTy0Nxb/+lRjTxRhOaU= X-Received: from pgbcz12.prod.google.com ([2002:a05:6a02:230c:b0:b42:1518:df18]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d510:b0:240:763d:e9a6 with SMTP id d9443c01a7336-242a0ae9d65mr50650645ad.25.1754510308841; Wed, 06 Aug 2025 12:58:28 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:55 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-34-seanjc@google.com> Subject: [PATCH v5 33/44] KVM: x86/pmu: Bypass perf checks when emulating mediated PMU counter accesses From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi When emulating a PMC counter read or write for a mediated PMU, bypass the perf checks and emulated_counter logic as the counters aren't proxied through perf, i.e. pmc->counter always holds the guest's up-to-date value, and thus there's no need to defer emulated overflow checks. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang [sean: split from event filtering change, write shortlog+changelog] Signed-off-by: Sean Christopherson Reviewed-by: Sandipan Das Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 5 +++++ arch/x86/kvm/pmu.h | 3 +++ 2 files changed, 8 insertions(+) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 817ef852bdf9..082d2905882b 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -377,6 +377,11 @@ static void pmc_update_sample_period(struct kvm_pmc *p= mc) =20 void pmc_write_counter(struct kvm_pmc *pmc, u64 val) { + if (kvm_vcpu_has_mediated_pmu(pmc->vcpu)) { + pmc->counter =3D val & pmc_bitmask(pmc); + return; + } + /* * Drop any unconsumed accumulated counts, the WRMSR is a write, not a * read-modify-write. Adjust the counter value so that its value is diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 51963a3a167a..1c9d26d60a60 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -111,6 +111,9 @@ static inline u64 pmc_read_counter(struct kvm_pmc *pmc) { u64 counter, enabled, running; =20 + if (kvm_vcpu_has_mediated_pmu(pmc->vcpu)) + return pmc->counter & pmc_bitmask(pmc); + counter =3D pmc->counter + pmc->emulated_counter; =20 if (pmc->perf_event && !pmc->is_paused) --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5168D2DC353 for ; Wed, 6 Aug 2025 19:58:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510314; cv=none; b=RfjCeuon65AQIo8YbnhywhmuGQox/BFq2pMrAwo9sriOb5LWJO6gJF5x00H92SJr6WDMd9dB+nFMOH5dFrcc7BxrBegymCYp6d43CdrWD8Sblz4Vppzn4tqqlAAwDmgCfa/MjaGoedJREx4YJT3c30EatgRsxrBvsGKV6bDieXY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510314; c=relaxed/simple; bh=1MpCULgJITNFp8BwtVBageH608sNes8/5DqfGMafEA8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bdclhOswKlEkbKXwnvGr3FkwZut/saZhWGCCQ1vgL1a6LRM4M8G0iifPpIxypJq5igJ/p8+1krY6nAJICkR9GuXWhNYYlSl0LplUP/QNdYxEVWEnBnk65DkrcjqW4SLP986Nje58+CQmWuGR31jhOfFdfHcZCG8akeXff0e2Kk8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zS05TrY2; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zS05TrY2" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-23ff6e5c131so1302255ad.2 for ; Wed, 06 Aug 2025 12:58:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510310; x=1755115110; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0x+D+og4ia0Wyj0NvWobUD8Pzlie97PfvMw2iXhJJfQ=; b=zS05TrY2p0uXrYUpTzHJmueTSc/L/n+Jhjl650tcOgDcU5hmfH0xY/VKREh4axhBCT 2XhiRuFcjd0uDfyWVYsWOkjk1m4ddNIbWJof1cwY+dxHEQmNN80sW0+6Az+5BEfQi4b4 9NR4k58V0ep2KcL5EPuu1YWTmHMnhPwG7RP9AtCdOR7QNNzCNy9P62kYa5xQeYtPVFqO beSL1xf2e5OKdXuPAJu8sF3WBttALQlDmt6iiEKXnSNzbT3uIc01a9QNziljX73nPQdc UsSiqQFZsKCiWIag0eDBvuzPY02avs1Tt0ZPwV/ugItajeDedRJ4nhVF0pJs02S7jkVo /u3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510310; x=1755115110; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0x+D+og4ia0Wyj0NvWobUD8Pzlie97PfvMw2iXhJJfQ=; b=lNq0dw58tzxgArlmsYprHK3OT4o9WPYjJ7SCp34TsbwjVL/Hz4mIHHK4ygXEp/EDl+ O+VaJuc7oEMJXshtRMJnqur2+iY9ppQtpP8otIkEknEmhXNz4a9tQdmxXmK9E393Hsjl Qt19yQPQvs9iztfHgxDR2SjASWlDDkkMNgLuByKc1eTDOyvGDaLUb15Pz7ksK+iLKq8e hNtLODD7OtbrNWk24bdlzXLpRsCkZ+8IQWSEyFOpPqADaIsT4LEBBh5JRslzTl9Oh7pI Mspw+i64uLW2prHMgArE9oRybTMbQ5MVEC+BnKtEUHyt6s+EZXE9MKh5QTTmY+erRMlX ml0w== X-Forwarded-Encrypted: i=1; AJvYcCVvB7nmvxkuG/B+xgFk8JDjDKtZbYFUJ6bxkCH4/T6iffEuPs7yx8LR3j29F5j8Ld1clpi5w1Ku7WlXEuE=@vger.kernel.org X-Gm-Message-State: AOJu0Yx9GRDpZoRyyLvCSboSg80HpG+rpm8j5B84tlmQopfxY1u5WgjJ Z/XbBSN/hc0VZPSsUcrFVPHQBwloGnCH0Ep/MngPlt+9c4WsnvEaC48sw8yKI3f5GML7qtXZq33 63dUV2A== X-Google-Smtp-Source: AGHT+IHACZSLEKXNmtK1WJ2A5Z4mKbWmXm0xD8HleVhgcyjfcmK2gM+EKW1OlQ2W16ylb8rulTtPFwBYIyI= X-Received: from pjoa3.prod.google.com ([2002:a17:90a:8c03:b0:312:14e5:174b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:98d:b0:225:abd2:5e4b with SMTP id d9443c01a7336-242b19acfe4mr5266535ad.16.1754510310549; Wed, 06 Aug 2025 12:58:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:56 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-35-seanjc@google.com> Subject: [PATCH v5 34/44] KVM: x86/pmu: Introduce eventsel_hw to prepare for pmu event filtering From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mingwei Zhang Introduce eventsel_hw and fixed_ctr_ctrl_hw to store the actual HW value in PMU event selector MSRs. In mediated PMU checks events before allowing the event values written to the PMU MSRs. However, to match the HW behavior, when PMU event checks fails, KVM should allow guest to read the value back. This essentially requires an extra variable to separate the guest requested value from actual PMU MSR value. Note this only applies to event selectors. Signed-off-by: Mingwei Zhang Co-developed-by: Dapeng Mi Signed-off-by: Dapeng Mi Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/pmu.c | 7 +++++-- arch/x86/kvm/svm/pmu.c | 1 + arch/x86/kvm/vmx/pmu_intel.c | 2 ++ 4 files changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index b891bd92fc83..5512e33db14a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -528,6 +528,7 @@ struct kvm_pmc { */ u64 emulated_counter; u64 eventsel; + u64 eventsel_hw; struct perf_event *perf_event; struct kvm_vcpu *vcpu; /* @@ -556,6 +557,7 @@ struct kvm_pmu { unsigned nr_arch_fixed_counters; unsigned available_event_types; u64 fixed_ctr_ctrl; + u64 fixed_ctr_ctrl_hw; u64 fixed_ctr_ctrl_rsvd; u64 global_ctrl; u64 global_status; diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 082d2905882b..e39ae37f0280 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -890,11 +890,14 @@ static void kvm_pmu_reset(struct kvm_vcpu *vcpu) pmc->counter =3D 0; pmc->emulated_counter =3D 0; =20 - if (pmc_is_gp(pmc)) + if (pmc_is_gp(pmc)) { pmc->eventsel =3D 0; + pmc->eventsel_hw =3D 0; + } } =20 - pmu->fixed_ctr_ctrl =3D pmu->global_ctrl =3D pmu->global_status =3D 0; + pmu->fixed_ctr_ctrl =3D pmu->fixed_ctr_ctrl_hw =3D 0; + pmu->global_ctrl =3D pmu->global_status =3D 0; =20 kvm_pmu_call(reset)(vcpu); } diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index 9ffd44a5d474..9641ef5d0dd7 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -165,6 +165,7 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struc= t msr_data *msr_info) data &=3D ~pmu->reserved_bits; if (data !=3D pmc->eventsel) { pmc->eventsel =3D data; + pmc->eventsel_hw =3D data; kvm_pmu_request_counter_reprogram(pmc); } return 0; diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 2bdddb95816e..6874c522577e 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -61,6 +61,7 @@ static void reprogram_fixed_counters(struct kvm_pmu *pmu,= u64 data) int i; =20 pmu->fixed_ctr_ctrl =3D data; + pmu->fixed_ctr_ctrl_hw =3D data; for (i =3D 0; i < pmu->nr_arch_fixed_counters; i++) { u8 new_ctrl =3D fixed_ctrl_field(data, i); u8 old_ctrl =3D fixed_ctrl_field(old_fixed_ctr_ctrl, i); @@ -430,6 +431,7 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, str= uct msr_data *msr_info) =20 if (data !=3D pmc->eventsel) { pmc->eventsel =3D data; + pmc->eventsel_hw =3D data; kvm_pmu_request_counter_reprogram(pmc); } break; --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B9B42DC359 for ; Wed, 6 Aug 2025 19:58:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510315; cv=none; b=s//E/TG0gunvuClKJQ/6jZBYRZ0yXyAczhdUGTVevDqliRzBNSiN7cPWFP3nBy9Ea3bYxkGx1JwLqTeLDtT/TooOujQsnZKSBP1OqxFiuvO0Y0Vj1n9hPffPPAyq61BQopGMLsURixYuq0NDAFtP7yufcjpEtFuawiZZXAOy9nY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510315; c=relaxed/simple; bh=4kQYudPfi5zx6gImzIgwc5QaKlQ7/lxrNTieFP93BO4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=vAbWcjGF5QXEFJ/p2kPGQBwVxuDToyPCmo8sqAsT8XJK5Xk3JzPOLDrOb65Ld/XDAXE6AFxtUsf1a3t9S8oQht4k4eJR76IiAh25iUfTsrfQLqqxvrrGrVmqwy+8YGcCIGHst3gAdk8YoaHRN7n2CJdalZe0F66l34tglfukKjc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dQ5Dy7OP; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dQ5Dy7OP" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31ea14cc097so281957a91.3 for ; Wed, 06 Aug 2025 12:58:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510312; x=1755115112; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=xm7g2YivXo6aQ0SA/h1DnHmIX017hVekqEmykFx3Fp8=; b=dQ5Dy7OPBZKOeslU30/Ty2nevmSlbyuAVWGlGPSo8iRtTTdIllWsmpyMaXEiR107Ka 5DauRKz0J/vblo45Z7vRb8BjqxivKxH1i+droia/sGoL6swt6gFx4QPi9oudJr0SNldl Dm+njJy/t/t3sLIEhwJn6qYSmq7DNytq3MdfiEwHfpL4no+QiYg/fpm8xm6cZjE/BBLE 4AeQVI2mvCtk40pE4TC4ctCmQMdAldCWh8Kl8UX3DTYPPAWKo+qbC927+uKkUF6LEutA I9j0EAp1T/yojeL/Pb1GiPohuhh4WA90MM3CeUaptCKfDqS19xJ1Eb/+ssWnovfSM+Aa lvug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510312; x=1755115112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xm7g2YivXo6aQ0SA/h1DnHmIX017hVekqEmykFx3Fp8=; b=nM0TI+uYFqlCjHjIAeskRoSOz/J8E8116XMKmP+iuyvaHNl8L/zADzNi0Jb1pvOMYY TLtoCJN7dxjjTp2gjg2c2fpytXbs4MQYh82Dno4oQc2hQt8D4FrQODg6gQTH1+512cic z+6BmAeA7G5PHLmUFovhocwghbsvs7EhEgznkHqiPPZJqsHp3SUbsciB7J8pe1ZETYfP hypg1wrjVWbhRACQAfgcrIC/DwVeSesfuLxNbUzapbCuXCOWChjsB1xB3RdZ2BFkX1zt GHuO72Ip+ywYdQcOnfULy8UBi2I0ex2Lg8a5olz5Mas9KJBqfMV7pNLFtLHIATLFuPVu dF3g== X-Forwarded-Encrypted: i=1; AJvYcCUhnHfYkkXzsCz9xCvTy2wNXqhyY5zSi1OI7UJD6Ax5RsIWU3e85oww3xNCnjTdjJxjThjtz6iDFeQzCA0=@vger.kernel.org X-Gm-Message-State: AOJu0Yy/GsTYrvG189vN3bJoIW/vB3MDpIAS4Dkrkf7Yn7KyAA5fqov/ 6+gmb7zL21ZJLG6pJtJXf6Qi44nDZNsKkDL9kGZlOxuGcFfac1ml45gsV7D4oaYfM5bAa82AyRw GPUOWGA== X-Google-Smtp-Source: AGHT+IEYZIIJpt/ZXd2reg+ZYe1ccOM3EaPSbjMXbuV7EDQKePHF+PpZ3A2kJS4xCp7zKjQzYF60hKA/y+o= X-Received: from pjbok13.prod.google.com ([2002:a17:90b:1d4d:b0:31c:2fe4:33bc]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2fc4:b0:31f:867:d6b4 with SMTP id 98e67ed59e1d1-321694b7c0dmr5279932a91.10.1754510312321; Wed, 06 Aug 2025 12:58:32 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:57 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-36-seanjc@google.com> Subject: [PATCH v5 35/44] KVM: x86/pmu: Reprogram mediated PMU event selectors on event filter updates From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Refresh the event selectors that are programmed into hardware when a PMC is "reprogrammed" for a mediated PMU, i.e. if userspace changes the PMU event filters Note, KVM doesn't utilize the reprogramming infrastructure to handle counter overflow for mediated PMUs, as there's no need to reprogram a non-existent perf event. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang [sean: add a helper to document behavior, split patch and rewrite changelog] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index e39ae37f0280..b4c6a7704a01 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -518,6 +518,25 @@ static bool pmc_is_event_allowed(struct kvm_pmc *pmc) return is_fixed_event_allowed(filter, pmc->idx); } =20 +static void kvm_mediated_pmu_refresh_event_filter(struct kvm_pmc *pmc) +{ + bool allowed =3D pmc_is_event_allowed(pmc); + struct kvm_pmu *pmu =3D pmc_to_pmu(pmc); + + if (pmc_is_gp(pmc)) { + pmc->eventsel_hw &=3D ~ARCH_PERFMON_EVENTSEL_ENABLE; + if (allowed) + pmc->eventsel_hw |=3D pmc->eventsel & + ARCH_PERFMON_EVENTSEL_ENABLE; + } else { + u64 mask =3D intel_fixed_bits_by_idx(pmc->idx - KVM_FIXED_PMC_BASE_IDX, = 0xf); + + pmu->fixed_ctr_ctrl_hw &=3D ~mask; + if (allowed) + pmu->fixed_ctr_ctrl_hw |=3D pmu->fixed_ctr_ctrl & mask; + } +} + static int reprogram_counter(struct kvm_pmc *pmc) { struct kvm_pmu *pmu =3D pmc_to_pmu(pmc); @@ -526,6 +545,11 @@ static int reprogram_counter(struct kvm_pmc *pmc) bool emulate_overflow; u8 fixed_ctr_ctrl; =20 + if (kvm_vcpu_has_mediated_pmu(pmu_to_vcpu(pmu))) { + kvm_mediated_pmu_refresh_event_filter(pmc); + return 0; + } + emulate_overflow =3D pmc_pause_counter(pmc); =20 if (!pmc_is_globally_enabled(pmc) || !pmc_is_locally_enabled(pmc) || --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14D0E2DE719 for ; Wed, 6 Aug 2025 19:58:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510317; cv=none; b=MLAYonbFAAWwC+QlDGeonsvToMmds30oEkPcuv0r2aGBIfuXsutbBDC+6XhtcRV8XJmfDNr12cTWxvmR407k1Eg9bL7OJAxy7TRdTriXhGtbJz+2jWNttHU+XCj6MFVV2Rgif0PiS/ebfR/RgGRtMjydk4wHtG/yg/LDNrhlv/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510317; c=relaxed/simple; bh=QVIn0+kukbkiW0o85fHutqEBFnL/lbFRNdNnmVngsXM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=nfKOcEyMNOAxpWmdaz41ntW6onv5KoFRaDnXR1mFJyY+u26wIPlTHNBUkhuq8NkGBlqRWQlhtdzTlRAoYgiGk3OmWXPJYHAhtSGVkltuSRk4bx6iQD5Pb5WWb4ywmz3EwWHjWLDZnrgvnhQaq25ctqPI45SRmJmJSIfF67Kp70o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RiTWC4TC; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RiTWC4TC" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2400a932e59so2356685ad.1 for ; Wed, 06 Aug 2025 12:58:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510314; x=1755115114; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Pi5hsDnuks7nSFD8RayInQ9oz1eBhk4hTX7Cx/64BFI=; b=RiTWC4TCtRE/O8Sj808HlsXbva7/dbqhOWn5kIqyoN6RBax1pExeHUe5uPG6yHZmxX eKTC7+9tGMIqoJJrl9zLgUDu53zHWWna8iIrOB1DYm4bLuc6n4v4efo6ggGDvOkE5fwP BmGmMD5drl77+rv5Oz2LloWEfxE3xz5ed8Q6NqOnh5yLmZEtbehAiAYdjAyTnIjIZPXY 2ojQYxSwjkKj4tjJ1d9nwUdjTqNIxXpttRRY1oGwW2wprtgyS41ZgPUHrCs+J9wLvJ1C X6jZVGyXcoAwi3+GbOTWFDtrxelsHsn/I/jwNt66fXiAz8C3rLM/RuDtWBslJnAZj4At DF2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510314; x=1755115114; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Pi5hsDnuks7nSFD8RayInQ9oz1eBhk4hTX7Cx/64BFI=; b=SvpKyq00F87u5GR6rju22s2TUcOsrc15mSDdA7LTpbCWmxEDHPT7P/oneziymIfc4P DqPbJPNip64LDojaXtUsFMpO89A61Ba/jG9I6CwguVUlsI3boxra/5f0P6TYH9aU+065 yQASUsCERkYkFx+a2JaV8oHpjkMYP9pYQSiMxQiGdL0TAm32m9jxKC1muTG2rQEcSJ5P nC6WE2aRHUPOqG0vDVCGdLE0wfFsqangDP5kB1I6dQM5LexwUf+sJxWAe6jWrhr6CPzh 8xb8eXwwWV/mLf8LA9EAtZ2tQ4Asp4W31hvfanZRCwmiuaGpZ8JBHK6kGPJg7hgWn/jp 8ByA== X-Forwarded-Encrypted: i=1; AJvYcCUgas79CdekzMcbIBLcIzDI2i547xtihDa/maV/9PdFYem70uDWPW33pNQSdsIA8eYWzXJKK5Zt4x/b370=@vger.kernel.org X-Gm-Message-State: AOJu0YxGRzQauB/ny+kunkYHxf/JuG0pUGLDh59Ioc4d/uCSZsJ0LZ2R bzqWMYQJSwoC0FHWRQHWfL0/YDFOzdt2Jm3HZUUylCS3srPGpSpIxOFjxoJKaYHVnQHPaNuCRBm +h1ppAA== X-Google-Smtp-Source: AGHT+IEJD5GH7UI5TAAlffejmtsOUdJu9B+mpPPWAxLwTCJvZJKLbNbFAds3hHNJ28h7v90ap7LjckKJrbQ= X-Received: from plhl7.prod.google.com ([2002:a17:903:1207:b0:235:f4e3:ba29]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d581:b0:234:d7b2:2ac3 with SMTP id d9443c01a7336-2429f2eea48mr59065095ad.20.1754510314065; Wed, 06 Aug 2025 12:58:34 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:58 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-37-seanjc@google.com> Subject: [PATCH v5 36/44] KVM: x86/pmu: Always stuff GuestOnly=1,HostOnly=0 for mediated PMCs on AMD From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sandipan Das On AMD platforms, there is no way to restore PerfCntrGlobalCtl at VM-Entry or clear it at VM-Exit. Since the register states will be restored before entering and saved after exiting guest context, the counters can keep ticking and even overflow leading to chaos while still in host context. To avoid this, intecept event selectors, which is already done by mediated PMU. In addition, always set the GuestOnly bit and clear the HostOnly bit for PMU selectors on AMD. Doing so allows the counters run only in guest context even if their enable bits are still set after VM exit and before host/guest PMU context switch. Signed-off-by: Sandipan Das Signed-off-by: Mingwei Zhang [sean: massage shortlog] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/svm/pmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index 9641ef5d0dd7..a5e70a4e7647 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -165,7 +165,8 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struc= t msr_data *msr_info) data &=3D ~pmu->reserved_bits; if (data !=3D pmc->eventsel) { pmc->eventsel =3D data; - pmc->eventsel_hw =3D data; + pmc->eventsel_hw =3D (data & ~AMD64_EVENTSEL_HOSTONLY) | + AMD64_EVENTSEL_GUESTONLY; kvm_pmu_request_counter_reprogram(pmc); } return 0; --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3CC72DECCC for ; Wed, 6 Aug 2025 19:58:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510319; cv=none; b=fmPSgq7LimnojZE5G67NeqyfdbtLgmI/IuGwGmLNMdJkdaPm6A9VuFXwgzaDH1dcR6Jx9fiP6uKdGdtoKusFQ6vKVWAAkHuFBAZECvuyd1TmxoYG+P8h2uLj4cy8VMbmBPUERV+J2qpXkXQOJxrY+znz/ACC7TAsVdK8Y/XOL9U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510319; c=relaxed/simple; bh=BkM6KVwWoNh4dFmNtuvLOgPAK4SVj00cIniNA7xy18I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GH+3a0pcT/rLlFDCzkN5zUfqAtRXGojTnEPUaPKYJ6EnvcMb8Icp+sJjfGtrw28lbveOwuoi3rsWyA5PTI/klR21bX6zY93Sdd7kpBqyrRLyykGBm/lhPBBH1rkNZcm53VapMCq0slQwACIPVBnAzrEC8EQ3ayYfg+zvamqLyIg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bM1v9mlk; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bM1v9mlk" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31f729bf733so471782a91.1 for ; Wed, 06 Aug 2025 12:58:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510316; x=1755115116; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=zkAAwl8vElfQMpdapHv3m96nWKKm7bgpMF9fVPIUZ/g=; b=bM1v9mlkCeouq545OKuhbAEFG8yyzcgjTwAJ2tADtTlIlukqSIRXyhGliAwJyN4OSp VfLrQM1ilcY5SVQZ2B23FZf7/I0M4+ASjo3+yjYicu1lKMAl1O6kZHsa4s02tUYX4CoX L0rbAAeYbgPPKsbWg8oNFR1cjpZKjCpi9usxg1165wHDssRotaK4lVu0T3U6xkqHOwNr aWC2u7V7IUtxGaeg8kHDa9ozqpSTaK9mvc2QPrzO0yyD2ORQnihbJYM7lvLIAmpaxvwK noJB8oOqGMEAuOornQx8nSUYrbwd/o9+e3ND2myIJcrsRf9spvryQasb/U/ZUWUczxxW ImhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510316; x=1755115116; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zkAAwl8vElfQMpdapHv3m96nWKKm7bgpMF9fVPIUZ/g=; b=xQeF9ireuLIpbxrl5sP9k7lg26pk79ch9bd6zoEPPhpChnQHG5fltaiTGuBqW1DUir k8v2k6SHAqQ6cGYIu3YDKMveaJq+wtrez40Kql5g3on0eBS4IDwdqiy1ZWpFgkYfE1px g+RmaQNIXpvzxvPQeZC24WhKGnb/NHoe/PE/xlHBYSpPKkoOicOiI8Lk4Td8F2rVaW42 qBNAyBfuNlQ+AgS1+aus8lM6mB8gmzjlVT52Kr4+Rg/wwXgQ/gIgwW+T2QyWO+OoEwcf ZBx1ZaoAtQq8pMHG54g8ElewEz6ze2YPQlZhHeEkuXTwx1QIS818NrHqt6xa2f/yNoGT 7SBg== X-Forwarded-Encrypted: i=1; AJvYcCVdephJg+BjoZkFsp69MvXHsFjPxyZoUe/P06PuxwbNj0/dCEpeIxaaR8gJHaX3jTGr5vtc8chLi56rREk=@vger.kernel.org X-Gm-Message-State: AOJu0YwNYEyLfEUQj8CIEuZ0XdF73zubeyMKT20NWOFNLq4oGSkbNaUR uoqYYgIGEvlLTNL2Ses/c2nuDswvCHdM58c1F2Kdg3KHaAw/cxUupg14y5VcNYGQtPgU8vEASAe sSOcgWQ== X-Google-Smtp-Source: AGHT+IFaZiYR5zJC7+pYfASpXTxuG1epwB6mNpav4wNliQpQcOBcA3rZjhVGTTG4ScmGlrhUBGt34kaYMPo= X-Received: from pjbqb13.prod.google.com ([2002:a17:90b:280d:b0:31f:61fc:b283]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2d87:b0:31e:c630:ec93 with SMTP id 98e67ed59e1d1-32166ca773bmr5198285a91.16.1754510316148; Wed, 06 Aug 2025 12:58:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:59 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-38-seanjc@google.com> Subject: [PATCH v5 37/44] KVM: x86/pmu: Load/put mediated PMU context when entering/exiting guest From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Implement the PMU "world switch" between host perf and guest mediated PMU. When loading guest state, call into perf to switch from host to guest, and then load guest state into hardware, and then reverse those actions when putting guest state. On the KVM side, when loading guest state, zero PERF_GLOBAL_CTRL to ensure all counters are disabled, then load selectors and counters, and finally call into vendor code to load control/status information. While VMX and SVM use different mechanisms to avoid counting host activity while guest controls are loaded, both implementations require PERF_GLOBAL_CTRL to be zeroed when the event selectors are in flux. When putting guest state, reverse the order, and save and zero controls and status prior to saving+zeroing selectors and counters. Defer clearing PERF_GLOBAL_CTRL to vendor code, as only SVM needs to manually clear the MSR; VMX configures PERF_GLOBAL_CTRL to be atomically cleared by the CPU on VM-Exit. Handle the difference in MSR layouts between Intel and AMD by communicating the bases and stride via kvm_pmu_ops. Because KVM requires Intel v4 (and full-width writes) and AMD v2, the MSRs to load/save are constant for a given vendor, i.e. do not vary based on the guest PMU, and do not vary based on host PMU (because KVM will simply disable mediated PMU support if the necessary MSRs are unsupported). Except for retrieving the guest's PERF_GLOBAL_CTRL, which needs to be read before invoking any fastpath handler (spoiler alert), perform the context switch around KVM's inner run loop. State only needs to be synchronized from hardware before KVM can access the software "caches". Note, VMX already grabs the guest's PERF_GLOBAL_CTRL immediately after VM-Exit, as hardware saves value into the VMCS. Co-developed-by: Mingwei Zhang Signed-off-by: Mingwei Zhang Co-developed-by: Sandipan Das Signed-off-by: Sandipan Das Signed-off-by: Dapeng Mi Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/include/asm/kvm-x86-pmu-ops.h | 2 + arch/x86/include/asm/msr-index.h | 1 + arch/x86/kvm/pmu.c | 111 +++++++++++++++++++++++++ arch/x86/kvm/pmu.h | 10 +++ arch/x86/kvm/svm/pmu.c | 34 ++++++++ arch/x86/kvm/svm/svm.c | 3 + arch/x86/kvm/vmx/pmu_intel.c | 45 ++++++++++ arch/x86/kvm/x86.c | 4 + 8 files changed, 210 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-pmu-ops.h b/arch/x86/include/asm/= kvm-x86-pmu-ops.h index ad2cc82abf79..f0aa6996811f 100644 --- a/arch/x86/include/asm/kvm-x86-pmu-ops.h +++ b/arch/x86/include/asm/kvm-x86-pmu-ops.h @@ -24,6 +24,8 @@ KVM_X86_PMU_OP_OPTIONAL(deliver_pmi) KVM_X86_PMU_OP_OPTIONAL(cleanup) =20 KVM_X86_PMU_OP_OPTIONAL(write_global_ctrl) +KVM_X86_PMU_OP(mediated_load) +KVM_X86_PMU_OP(mediated_put) =20 #undef KVM_X86_PMU_OP #undef KVM_X86_PMU_OP_OPTIONAL diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-in= dex.h index 06bd7aaae931..0f50a74797a4 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1171,6 +1171,7 @@ #define MSR_CORE_PERF_GLOBAL_STATUS 0x0000038e #define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f #define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x00000390 +#define MSR_CORE_PERF_GLOBAL_STATUS_SET 0x00000391 =20 #define MSR_PERF_METRICS 0x00000329 =20 diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index b4c6a7704a01..77042cad3155 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -1234,3 +1234,114 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *k= vm, void __user *argp) kfree(filter); return r; } + +static __always_inline u32 fixed_counter_msr(u32 idx) +{ + return kvm_pmu_ops.FIXED_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE; +} + +static __always_inline u32 gp_counter_msr(u32 idx) +{ + return kvm_pmu_ops.GP_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE; +} + +static __always_inline u32 gp_eventsel_msr(u32 idx) +{ + return kvm_pmu_ops.GP_EVENTSEL_BASE + idx * kvm_pmu_ops.MSR_STRIDE; +} + +static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + struct kvm_pmc *pmc; + u32 i; + + /* + * No need to zero out unexposed GP/fixed counters/selectors since RDPMC + * is intercepted if hardware has counters that aren't visible to the + * guest (KVM will inject #GP as appropriate). + */ + for (i =3D 0; i < pmu->nr_arch_gp_counters; i++) { + pmc =3D &pmu->gp_counters[i]; + + wrmsrl(gp_counter_msr(i), pmc->counter); + wrmsrl(gp_eventsel_msr(i), pmc->eventsel_hw); + } + for (i =3D 0; i < pmu->nr_arch_fixed_counters; i++) { + pmc =3D &pmu->fixed_counters[i]; + + wrmsrl(fixed_counter_msr(i), pmc->counter); + } +} + +void kvm_mediated_pmu_load(struct kvm_vcpu *vcpu) +{ + if (!kvm_vcpu_has_mediated_pmu(vcpu) || + KVM_BUG_ON(!lapic_in_kernel(vcpu), vcpu->kvm)) + return; + + lockdep_assert_irqs_disabled(); + + perf_load_guest_context(kvm_lapic_get_reg(vcpu->arch.apic, APIC_LVTPC)); + + /* + * Disable all counters before loading event selectors and PMCs so that + * KVM doesn't enable or load guest counters while host events are + * active. VMX will enable/disabled counters at VM-Enter/VM-Exit by + * atomically loading PERF_GLOBAL_CONTROL. SVM effectively performs + * the switch by configuring all events to be GUEST_ONLY. + */ + wrmsrl(kvm_pmu_ops.PERF_GLOBAL_CTRL, 0); + + kvm_pmu_load_guest_pmcs(vcpu); + + kvm_pmu_call(mediated_load)(vcpu); +} + +static void kvm_pmu_put_guest_pmcs(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + struct kvm_pmc *pmc; + u32 i; + + /* + * Clear selectors and counters to ensure hardware doesn't count using + * guest controls when the host (perf) restores its state. + */ + for (i =3D 0; i < pmu->nr_arch_gp_counters; i++) { + pmc =3D &pmu->gp_counters[i]; + + pmc->counter =3D rdpmc(i); + if (pmc->counter) + wrmsrl(gp_counter_msr(i), 0); + if (pmc->eventsel_hw) + wrmsrl(gp_eventsel_msr(i), 0); + } + + for (i =3D 0; i < pmu->nr_arch_fixed_counters; i++) { + pmc =3D &pmu->fixed_counters[i]; + + pmc->counter =3D rdpmc(INTEL_PMC_FIXED_RDPMC_BASE | i); + if (pmc->counter) + wrmsrl(fixed_counter_msr(i), 0); + } +} + +void kvm_mediated_pmu_put(struct kvm_vcpu *vcpu) +{ + if (!kvm_vcpu_has_mediated_pmu(vcpu) || + KVM_BUG_ON(!lapic_in_kernel(vcpu), vcpu->kvm)) + return; + + lockdep_assert_irqs_disabled(); + + /* + * Defer handling of PERF_GLOBAL_CTRL to vendor code. On Intel, it's + * atomically cleared on VM-Exit, i.e. doesn't need to be clear here. + */ + kvm_pmu_call(mediated_put)(vcpu); + + kvm_pmu_put_guest_pmcs(vcpu); + + perf_put_guest_context(); +} diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 1c9d26d60a60..e2e2d8476a3f 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -38,11 +38,19 @@ struct kvm_pmu_ops { void (*cleanup)(struct kvm_vcpu *vcpu); =20 bool (*is_mediated_pmu_supported)(struct x86_pmu_capability *host_pmu); + void (*mediated_load)(struct kvm_vcpu *vcpu); + void (*mediated_put)(struct kvm_vcpu *vcpu); void (*write_global_ctrl)(u64 global_ctrl); =20 const u64 EVENTSEL_EVENT; const int MAX_NR_GP_COUNTERS; const int MIN_NR_GP_COUNTERS; + + const u32 PERF_GLOBAL_CTRL; + const u32 GP_EVENTSEL_BASE; + const u32 GP_COUNTER_BASE; + const u32 FIXED_COUNTER_BASE; + const u32 MSR_STRIDE; }; =20 void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops); @@ -240,6 +248,8 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu); int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp); void kvm_pmu_instruction_retired(struct kvm_vcpu *vcpu); void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu); +void kvm_mediated_pmu_load(struct kvm_vcpu *vcpu); +void kvm_mediated_pmu_put(struct kvm_vcpu *vcpu); =20 bool is_vmware_backdoor_pmc(u32 pmc_idx); bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index a5e70a4e7647..c03720b30785 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -233,6 +233,32 @@ static bool amd_pmu_is_mediated_pmu_supported(struct x= 86_pmu_capability *host_pm return host_pmu->version >=3D 2; } =20 +static void amd_mediated_pmu_load(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + u64 global_status; + + rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, global_status); + /* Clear host global_status MSR if non-zero. */ + if (global_status) + wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, global_status); + + wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, pmu->global_status); + wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, pmu->global_ctrl); +} + +static void amd_mediated_pmu_put(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + + wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0); + rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, pmu->global_status); + + /* Clear global status bits if non-zero */ + if (pmu->global_status) + wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, pmu->global_status); +} + struct kvm_pmu_ops amd_pmu_ops __initdata =3D { .rdpmc_ecx_to_pmc =3D amd_rdpmc_ecx_to_pmc, .msr_idx_to_pmc =3D amd_msr_idx_to_pmc, @@ -244,8 +270,16 @@ struct kvm_pmu_ops amd_pmu_ops __initdata =3D { .init =3D amd_pmu_init, =20 .is_mediated_pmu_supported =3D amd_pmu_is_mediated_pmu_supported, + .mediated_load =3D amd_mediated_pmu_load, + .mediated_put =3D amd_mediated_pmu_put, =20 .EVENTSEL_EVENT =3D AMD64_EVENTSEL_EVENT, .MAX_NR_GP_COUNTERS =3D KVM_MAX_NR_AMD_GP_COUNTERS, .MIN_NR_GP_COUNTERS =3D AMD64_NUM_COUNTERS, + + .PERF_GLOBAL_CTRL =3D MSR_AMD64_PERF_CNTR_GLOBAL_CTL, + .GP_EVENTSEL_BASE =3D MSR_F15H_PERF_CTL0, + .GP_COUNTER_BASE =3D MSR_F15H_PERF_CTR0, + .FIXED_COUNTER_BASE =3D 0, + .MSR_STRIDE =3D 2, }; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index add50b64256c..ca6f453cc160 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4416,6 +4416,9 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_= vcpu *vcpu, u64 run_flags) =20 vcpu->arch.regs_avail &=3D ~SVM_REGS_LAZY_LOAD_SET; =20 + if (!msr_write_intercepted(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_CTL)) + rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, vcpu_to_pmu(vcpu)->global_ctrl); + /* * We need to handle MC intercepts here before the vcpu has a chance to * change the physical cpu diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 6874c522577e..41a845de789e 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -786,6 +786,43 @@ static void intel_pmu_write_global_ctrl(u64 global_ctr= l) vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, global_ctrl); } =20 + +static void intel_mediated_pmu_load(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + u64 global_status, toggle; + + rdmsrq(MSR_CORE_PERF_GLOBAL_STATUS, global_status); + toggle =3D pmu->global_status ^ global_status; + if (global_status & toggle) + wrmsrq(MSR_CORE_PERF_GLOBAL_OVF_CTRL, global_status & toggle); + if (pmu->global_status & toggle) + wrmsrq(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle); + + wrmsrq(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw); +} + +static void intel_mediated_pmu_put(struct kvm_vcpu *vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + + /* MSR_CORE_PERF_GLOBAL_CTRL is already saved at VM-exit. */ + rdmsrq(MSR_CORE_PERF_GLOBAL_STATUS, pmu->global_status); + + /* Clear hardware MSR_CORE_PERF_GLOBAL_STATUS MSR, if non-zero. */ + if (pmu->global_status) + wrmsrq(MSR_CORE_PERF_GLOBAL_OVF_CTRL, pmu->global_status); + + /* + * Clear hardware FIXED_CTR_CTRL MSR to avoid information leakage and + * also to avoid accidentally enabling fixed counters (based on guest + * state) while running in the host, e.g. when setting global ctrl. + */ + if (pmu->fixed_ctr_ctrl_hw) + wrmsrq(MSR_CORE_PERF_FIXED_CTR_CTRL, 0); +} + + struct kvm_pmu_ops intel_pmu_ops __initdata =3D { .rdpmc_ecx_to_pmc =3D intel_rdpmc_ecx_to_pmc, .msr_idx_to_pmc =3D intel_msr_idx_to_pmc, @@ -799,9 +836,17 @@ struct kvm_pmu_ops intel_pmu_ops __initdata =3D { .cleanup =3D intel_pmu_cleanup, =20 .is_mediated_pmu_supported =3D intel_pmu_is_mediated_pmu_supported, + .mediated_load =3D intel_mediated_pmu_load, + .mediated_put =3D intel_mediated_pmu_put, .write_global_ctrl =3D intel_pmu_write_global_ctrl, =20 .EVENTSEL_EVENT =3D ARCH_PERFMON_EVENTSEL_EVENT, .MAX_NR_GP_COUNTERS =3D KVM_MAX_NR_INTEL_GP_COUNTERS, .MIN_NR_GP_COUNTERS =3D 1, + + .PERF_GLOBAL_CTRL =3D MSR_CORE_PERF_GLOBAL_CTRL, + .GP_EVENTSEL_BASE =3D MSR_P6_EVNTSEL0, + .GP_COUNTER_BASE =3D MSR_IA32_PMC0, + .FIXED_COUNTER_BASE =3D MSR_CORE_PERF_FIXED_CTR0, + .MSR_STRIDE =3D 1, }; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b8014435c988..7fb94ef64e18 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10906,6 +10906,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) run_flags |=3D KVM_RUN_LOAD_DEBUGCTL; vcpu->arch.host_debugctl =3D debug_ctl; =20 + kvm_mediated_pmu_load(vcpu); + guest_timing_enter_irqoff(); =20 for (;;) { @@ -10936,6 +10938,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) ++vcpu->stat.exits; } =20 + kvm_mediated_pmu_put(vcpu); + /* * Do this here before restoring debug registers on the host. And * since we do this before handling the vmexit, a DR access vmexit --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B48E2E03E0 for ; Wed, 6 Aug 2025 19:58:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510320; cv=none; b=nyZjWN10rJEycpXsF62H1Su49x1VA4PQTQDbEYC5oKLpSGfSZyZKg1tXV83PCHzzzGWkuE7FEr0gUXLMIlKEnHZTfhftY1QRlIz9rb1MqQX5sAROlDl/8BKiNyFUzhldWjVx91G1HyNBbbUe5TGtG64lVxq/FREY9XvSsSDt2cE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510320; c=relaxed/simple; bh=Wpc8Jd9c5FTviX8Nxdqz9AYHAb0HsNCDdNNI+vxMlVA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TWif7FIGHcnWgSTLJSnKupvsw6wpHxVPfDL93LbMK8JJmy7G/6cTASa26dSpvlrFlWzK+L8Z2orXIoxVXwa+j1d4RmHfZACVBvuhrE6Vqgyr9HhzGfalHjCk9wzvDAIAEvDtyl3sJKWbBIIaELiT4PLwQwEgibY6WL/3Q88JFxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=F4R2l5U3; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="F4R2l5U3" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-31ea430d543so311223a91.3 for ; Wed, 06 Aug 2025 12:58:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510318; x=1755115118; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=mbLaYrBpG9GN/ccdjbZmNVq0IkY2pKSaCp1hoLzaUb8=; b=F4R2l5U3Zs/Tt1JRAvtsqAzEd9cVdolHZvuVEYwjvFHDj7e9UdVCbAuSpfKH7fOeI2 Hnkd8wM4df8l+1paSBz7Jv/A5LdbxsF6fkl7ih7TaROhtUTeOFneZ09iMLuMc1+kDCIP GV8qQtFBIgk2JaLdE6nqRu50lWIO/bGxwknzIMFDsw8/xluZYWkop2VVjYSkCX+KwrYv YvRQ26PqRgUTxXHETnTovhgmAgZZPBaFs8ckBBcIpvsck9VZfkPNTGlPndKHOk2fhGuh LZOzBM+tNBiUlGXGB++gMaAIlRYKE6JPjFJDiDgDxnbhb6by7KOVj2eFzkUcFpr/1CGH lX8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510318; x=1755115118; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mbLaYrBpG9GN/ccdjbZmNVq0IkY2pKSaCp1hoLzaUb8=; b=qHKsMG1UQTCNmxhNaTeJKhYy6KyQOLoFq0K3c3GtliG9FYsS+5MDIn/DpCHIOzVxN0 2GwjpzsEdJGndJvm8hTF+j+NgRxw+l83wTSGNlDumZMp6vFc7lOjNVTCnbDIiXXYoTDz cQ7RT6WqFMTTQHM1MM5TH+C4QiOSOJvs7ue8GSA+uIfIHS5Y/6YN54PtV7zPev9ARvw1 J33RvTRLOzZAejJi7pfMQMzbLYWn8bS5UfApdYwomq2gcwdZ2sxU8Kz3Q0Ii1vPX5Td4 Et9b57odhxGyftVFOfV1n9z8UCJ62mCKzSmLCLnURFJkytIC1toX+U7c9A6ysE+K9/Ik 6IvQ== X-Forwarded-Encrypted: i=1; AJvYcCWun354g3iM4UnlYPJJXbWfcqNW3gvyIne52x47OT0n5NGOe2q8TkU2619iDchTQu6RqHGaqNP8a4x6Tcs=@vger.kernel.org X-Gm-Message-State: AOJu0Ywqy7FnSskHt6it5bFab8KrLGM/e/JvXZ4zn5wwMvrY/A/BTU4Z IdKTcgO7mG3ZblKB4pqPFA8CmcU5yuJPV3bQQwGgzskzbL9RZbST6emRN38OFzngBrtVMSDWRUC UHpxUpA== X-Google-Smtp-Source: AGHT+IG1RDTXadNPV+CIxuchWnyR7MvCJdJx9iRkBiPVNpyV8SP3VNbt+vP90Pa+IEkbzbifDn/Htni5q5g= X-Received: from pjzd15.prod.google.com ([2002:a17:90a:e28f:b0:312:151d:c818]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4a04:b0:31e:6f0a:6a1a with SMTP id 98e67ed59e1d1-32166dfac71mr5462707a91.3.1754510317778; Wed, 06 Aug 2025 12:58:37 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:57:00 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-39-seanjc@google.com> Subject: [PATCH v5 38/44] KVM: x86/pmu: Disallow emulation in the fastpath if mediated PMCs are active From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't handle exits in the fastpath if emulation is required, i.e. if an instruction needs to be skipped, the mediated PMU is enabled, and one or more PMCs is counting instructions. With the mediated PMU, KVM's cache of PMU state is inconsistent with respect to hardware until KVM exits the inner run loop (when the mediated PMU is "put"). Signed-off-by: Sean Christopherson Reviewed-by: Sandipan Das Tested-by: Xudong Hao --- arch/x86/kvm/pmu.h | 10 ++++++++++ arch/x86/kvm/x86.c | 9 +++++++++ 2 files changed, 19 insertions(+) diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index e2e2d8476a3f..a0cd42cbea9d 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -234,6 +234,16 @@ static inline bool pmc_is_globally_enabled(struct kvm_= pmc *pmc) return test_bit(pmc->idx, (unsigned long *)&pmu->global_ctrl); } =20 +static inline bool kvm_pmu_is_fastpath_emulation_allowed(struct kvm_vcpu *= vcpu) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + + return !kvm_vcpu_has_mediated_pmu(vcpu) || + !bitmap_intersects(pmu->pmc_counting_instructions, + (unsigned long *)&pmu->global_ctrl, + X86_PMC_IDX_MAX); +} + void kvm_pmu_deliver_pmi(struct kvm_vcpu *vcpu); void kvm_pmu_handle_event(struct kvm_vcpu *vcpu); int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned pmc, u64 *data); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 7fb94ef64e18..6bdf7ef0b535 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2092,6 +2092,9 @@ EXPORT_SYMBOL_GPL(kvm_emulate_invd); =20 fastpath_t handle_fastpath_invd(struct kvm_vcpu *vcpu) { + if (!kvm_pmu_is_fastpath_emulation_allowed(vcpu)) + return EXIT_FASTPATH_NONE; + if (!kvm_emulate_invd(vcpu)) return EXIT_FASTPATH_EXIT_USERSPACE; =20 @@ -2151,6 +2154,9 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_= vcpu *vcpu) u64 data =3D kvm_read_edx_eax(vcpu); u32 msr =3D kvm_rcx_read(vcpu); =20 + if (!kvm_pmu_is_fastpath_emulation_allowed(vcpu)) + return EXIT_FASTPATH_NONE; + switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): if (!lapic_in_kernel(vcpu) || !apic_x2apic_mode(vcpu->arch.apic) || @@ -11267,6 +11273,9 @@ EXPORT_SYMBOL_GPL(kvm_emulate_halt); =20 fastpath_t handle_fastpath_hlt(struct kvm_vcpu *vcpu) { + if (!kvm_pmu_is_fastpath_emulation_allowed(vcpu)) + return EXIT_FASTPATH_NONE; + if (!kvm_emulate_halt(vcpu)) return EXIT_FASTPATH_EXIT_USERSPACE; =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31B2C2E0415 for ; Wed, 6 Aug 2025 19:58:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510322; cv=none; b=qyFjnhJiPFa+T9HZS+qZaUOBSvaF3qRVpbWfWnRHC4Lok7rZqVMWC1YHN3i7OTZIHDcrfpL3AlOnq1QlQj2xYGW4fDTRevLXQWR/FmiA79VvO1YG4JOf+Jueec1w7W+4foMqJUR1rgsYYV5CM8el8hDfbPIY1fQMSo1J6N30hLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510322; c=relaxed/simple; bh=eowlj8R0vSVTi/YxsD+P1ORI9OIx1rmOow0uWDRXpQc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KBysBp5EA31Qihztjxj2wnfO1ApkfQm6jo+NUwMc3siAB55w0cDvGPTp4w4hj5k/KPNEIerm/V/gDbvR2aL/bwkER1w3omjHxw78YAZIxLjVlesNO0EaciXmcjvd8hJ78AyUJ4ZyHMs4U/pI3iAXbhWj4YqYt0OIeVtZME0pMQ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OlE6BvYM; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OlE6BvYM" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31f32271a1fso373507a91.0 for ; Wed, 06 Aug 2025 12:58:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510319; x=1755115119; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=AlJpQpYzAXA8yQUaTEjcLu5rmW1zUT5mjcWEtsvn+qY=; b=OlE6BvYMbK2ZPWrZUD9pGLXrsiqhSrga2a6dSdgOddGyNn90KL5UA6HYAP0Ttr7K4B tNTx5s5HBl/2k/jm83UC+6gGbgPXyM0SG8HzuXMCGcBa2WB9Dc91JgXi/2eDxHqCLSn/ aevWZ+Obqfw6yn84zCdjTV2ntOUIfnU0zpnSWC+XFt+xTOSm/ovxGpoOxehdvpUNhwCS 2f8W3yWuxSNkAS+CH0Q16sh0k8LU9Su0cjpVl3Bhmlx6mWmIgnLfos7i/0rHghMv0787 chIb9RgtUE+mJPs0UdmjJa+a1uOSEc75aqwJwz25Z6iyavLDIKJO8YZrYUtiSA/iLoOK gx6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510319; x=1755115119; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AlJpQpYzAXA8yQUaTEjcLu5rmW1zUT5mjcWEtsvn+qY=; b=OdNpIVNSHyJTdVHzuxkznO7eJXdAoeWAesCf5TXisyS7EpoyGmGo1hj5cv0iD8WV6z bgyIr9l2BxpXXRPVyZ3gocsYB5TIhMOhkCCi9jZKlKJHyx3/6mO875HrUL/fQ4aVbI0W 9zULle0y1SZqKg4eD9+hVJU3cN3gzz/339+Q0V1FNpJRJlMo3eNFTM2S12sf6ZmE2eg+ uHfCluqhAyA7LIwSXm683Os3yMgEwDnzORVn/Lwj3bYGd0VGt+fQ6CRayBFH75YJglYL ej6+8gC72Ek7A7DZiov0oZjLjAQq3uQM4qLkHsUWBn+p2hzHQeaXLFkvXYwBKv6FJip4 RClA== X-Forwarded-Encrypted: i=1; AJvYcCWnIF/XMD357qCoGsqYvQ1C4pHTdm+u4GPLq7Vou+f7NteF4wCGIzARV6yh8VJktKjsalKG/rn+M1mwf0I=@vger.kernel.org X-Gm-Message-State: AOJu0YwSBtQrk9ZXKqVOSjxPHIr+gU0zG8htnn2W+UvX+vCZ8gYgAtT6 qDIy/ncbK35x0xzXOZrkXG/hD9sMrAmrnvEc8L1PlhG1UVNADrNnZY6E+OlgZUSm1P3YsI+cViA qrUM8ng== X-Google-Smtp-Source: AGHT+IFAT6PWSjUL0hLa4l6orw0CuXbesqbqOENgS6V/LUSfRrl7M4gZLDmaQmAh4Zg14qO5rj6Zp5FZNJo= X-Received: from pjx15.prod.google.com ([2002:a17:90b:568f:b0:31c:32f8:3f88]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5110:b0:31f:4e9b:7c6a with SMTP id 98e67ed59e1d1-3217562accemr1061143a91.15.1754510319397; Wed, 06 Aug 2025 12:58:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:57:01 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-40-seanjc@google.com> Subject: [PATCH v5 39/44] KVM: x86/pmu: Handle emulated instruction for mediated vPMU From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Mediated vPMU needs to accumulate the emulated instructions into counter and load the counter into HW at vm-entry. Moreover, if the accumulation leads to counter overflow, KVM needs to update GLOBAL_STATUS and inject PMI into guest as well. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 77042cad3155..ddab1630a978 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -1018,10 +1018,45 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu) kvm_pmu_reset(vcpu); } =20 +static bool pmc_is_pmi_enabled(struct kvm_pmc *pmc) +{ + u8 fixed_ctr_ctrl; + + if (pmc_is_gp(pmc)) + return pmc->eventsel & ARCH_PERFMON_EVENTSEL_INT; + + fixed_ctr_ctrl =3D fixed_ctrl_field(pmc_to_pmu(pmc)->fixed_ctr_ctrl, + pmc->idx - KVM_FIXED_PMC_BASE_IDX); + return fixed_ctr_ctrl & INTEL_FIXED_0_ENABLE_PMI; +} + static void kvm_pmu_incr_counter(struct kvm_pmc *pmc) { - pmc->emulated_counter++; - kvm_pmu_request_counter_reprogram(pmc); + struct kvm_vcpu *vcpu =3D pmc->vcpu; + + /* + * For perf-based PMUs, accumulate software-emulated events separately + * from pmc->counter, as pmc->counter is offset by the count of the + * associated perf event. Request reprogramming, which will consult + * both emulated and hardware-generated events to detect overflow. + */ + if (!kvm_vcpu_has_mediated_pmu(vcpu)) { + pmc->emulated_counter++; + kvm_pmu_request_counter_reprogram(pmc); + return; + } + + /* + * For mediated PMUs, pmc->counter is updated when the vCPU's PMU is + * put, and will be loaded into hardware when the PMU is loaded. Simply + * increment the counter and signal overflow if it wraps to zero. + */ + pmc->counter =3D (pmc->counter + 1) & pmc_bitmask(pmc); + if (!pmc->counter) { + pmc_to_pmu(pmc)->global_status |=3D BIT_ULL(pmc->idx); + if (pmc_is_pmi_enabled(pmc)) + kvm_make_request(KVM_REQ_PMI, vcpu); + } } =20 static inline bool cpl_is_matched(struct kvm_pmc *pmc) --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 055422BEC37 for ; Wed, 6 Aug 2025 19:58:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510324; cv=none; b=O2HlnSdOtu9rtaxF+KZJ/3Y2DDjLdPRvH27h6w4klZSi8U/CMHWpSldtQKICJRJLTmqS53CjSRTj1XoXdKigsUzh3Fs86ote5fz+w5MYN6c89jL+rxurHFK9WoIrD6Ijd2/j8knLvs/EgBW++Xjwlcj5PPiCXbUL8uMNphswIow= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510324; c=relaxed/simple; bh=pnnkC1bAXhozYLn1+A5GeTVXCux318fw49cZ8cqaQ5s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gkmlmEBSXMr1b7f6ImX1gavrM84Rp2rixZfHLlR6RapkSsVKkPMJ5MxigfOqJyCrKlTyudGfQdDWgS33w4f0nyn1q0BJXn6NE2INVoJgGB9C+Fd6PHa0FJK65Ad3/nrvVVvLZZCvwj9Uk4twEfgbP5HSPVsVD+cKAzXAagIYUMI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IYnVlROR; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IYnVlROR" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31ed4a4a05bso1549524a91.1 for ; Wed, 06 Aug 2025 12:58:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510321; x=1755115121; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RsFIN1Au3Dl4zaCmP13PW3DAgvYNBzVpKCYeyT5lYGI=; b=IYnVlRORAOh1paTIK2Lm6a8cAUBbjU6mYFNK46LfNaO1qKv1SZBsHSBBwZr3wrPEI1 5pktWiRCNbCJrs8AOBxCOjMhGE3//vWjdJA9Ypd6FdVC7yJR+zALuPIIUYEy0qvUpTqC Qm2wPWFU/aKVi2xwFE4P/ntTogfYPhM9kIMlv2Hp3TOQGui9gLeaHBQUhi4VxF7aQjd4 HTTnLb2dHPP9NNwaoCc4amb0JNEskMzjI0UnJQQNzjZAaZfuPgC4+DQyIAGjvQfiCADQ qqs0KUGp7RymYMbO8ErKcYwAHz8aP89ZPnbMp8rapq6g4gqrvnrqtg2Br/8KjfAwkgcp x2XQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510321; x=1755115121; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RsFIN1Au3Dl4zaCmP13PW3DAgvYNBzVpKCYeyT5lYGI=; b=eIOexld0gi8/+XCYaHIgvLemNqx/mO6tox6Mvt1FmJnmoLhTwNUCE0PHD1SyD4TxAU wkWXIIM0B++T3ldaZHfaFyg845AhqZWp1dSGibmDAfE6RxMzacOpukXX9b8wkpzURVc/ NSEjNFPETwbHXaE8NTl9AKesMdeDnKmynqJkqxbrjeG86Jop9+b2wHJFkSpkA36Nxzj1 ZixjSSoo0ID1xut+qoYLsqPGaC0mmzz9IW00Kdx8vDM0MfpgKy1ORldI1l/Q6aMjxhFP ZswZJldiI4MEELTFnTQj5PMYWJKyC+br4NLXYhTV7RlOMo7J2xfAkxscqxkCQnC/bvr0 Lyqw== X-Forwarded-Encrypted: i=1; AJvYcCXfNrtPf9wtk+IANpUXsV44wf+eyCMlSa6t6fSifVClf2ZYWODAM63fsSvHNz+80Ao+jRrBLb+AqgbRAJY=@vger.kernel.org X-Gm-Message-State: AOJu0YxcP/LIIffH0LO/ocaF8D+rJs3orOs6jEJztiPemCMDIDj4GGrb tj0UV6hi8ekTG2AszB+tormjpzEFt6crqRr8sM/tUfM1cmQN13qEYw61ihTiXJSLiFDmaJNcZIG 4DGTS2w== X-Google-Smtp-Source: AGHT+IFbNifUFnJxYDSjClZPGiWnx/CS0ykT/Hb3U03iCPY3aeaisiXn0lmHgRNeXCjIQ2AkJAWL4xb8X68= X-Received: from pjqh12.prod.google.com ([2002:a17:90a:a88c:b0:311:d79d:e432]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4c07:b0:313:aefa:b08 with SMTP id 98e67ed59e1d1-32175622f9emr956977a91.16.1754510321187; Wed, 06 Aug 2025 12:58:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:57:02 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-41-seanjc@google.com> Subject: [PATCH v5 40/44] KVM: nVMX: Add macros to simplify nested MSR interception setting From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Add macros nested_vmx_merge_msr_bitmaps_xxx() to simplify nested MSR interception setting. No function change intended. Suggested-by: Sean Christopherson Signed-off-by: Dapeng Mi Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/vmx/nested.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index db2fd4eedc90..47f1f0c7d3a7 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -614,6 +614,19 @@ static inline void nested_vmx_set_intercept_for_msr(st= ruct vcpu_vmx *vmx, msr_bitmap_l0, msr); } =20 +#define nested_vmx_merge_msr_bitmaps(msr, type) \ + nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, \ + msr_bitmap_l0, msr, type) + +#define nested_vmx_merge_msr_bitmaps_read(msr) \ + nested_vmx_merge_msr_bitmaps(msr, MSR_TYPE_R) + +#define nested_vmx_merge_msr_bitmaps_write(msr) \ + nested_vmx_merge_msr_bitmaps(msr, MSR_TYPE_W) + +#define nested_vmx_merge_msr_bitmaps_rw(msr) \ + nested_vmx_merge_msr_bitmaps(msr, MSR_TYPE_RW) + /* * Merge L0's and L1's MSR bitmap, return false to indicate that * we do not use the hardware. @@ -697,23 +710,13 @@ static inline bool nested_vmx_prepare_msr_bitmap(stru= ct kvm_vcpu *vcpu, * other runtime changes to vmcs01's bitmap, e.g. dynamic pass-through. */ #ifdef CONFIG_X86_64 - nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, - MSR_FS_BASE, MSR_TYPE_RW); - - nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, - MSR_GS_BASE, MSR_TYPE_RW); - - nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, - MSR_KERNEL_GS_BASE, MSR_TYPE_RW); + nested_vmx_merge_msr_bitmaps_rw(MSR_FS_BASE); + nested_vmx_merge_msr_bitmaps_rw(MSR_GS_BASE); + nested_vmx_merge_msr_bitmaps_rw(MSR_KERNEL_GS_BASE); #endif - nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, - MSR_IA32_SPEC_CTRL, MSR_TYPE_RW); - - nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, - MSR_IA32_PRED_CMD, MSR_TYPE_W); - - nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, - MSR_IA32_FLUSH_CMD, MSR_TYPE_W); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_SPEC_CTRL); + nested_vmx_merge_msr_bitmaps_write(MSR_IA32_PRED_CMD); + nested_vmx_merge_msr_bitmaps_write(MSR_IA32_FLUSH_CMD); =20 nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_APERF, MSR_TYPE_R); --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DAFB2E0B74 for ; Wed, 6 Aug 2025 19:58:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510326; cv=none; b=ChDmiFfkngI9Z3w00d6jnJLuiquwXuuP702tWUdTL1B8E2Wqq1n4wyaspvs7ikQe5nsOMN0l1eQKCqVLd+7OgeBkHpSCSDATBxzVHTH6kyRvI1cV+lz/Zw1BjKKnkHvagh7iqedXI145y9YSlsGtNXyxMWttFfFyMndEIOkrIW0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510326; c=relaxed/simple; bh=xxEMYMafoH1hCuYfyuUxc0eXfBZ4HMKS++5sGsaBrpM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rJgpU5t+5p8Emrlvf20LL0ci7OOeMypJnC1zDD9E+AAw6x79pwSjVHKp9dQFMg5mXptpGDsRi36h5GKG9qWc6axvbPxSXIt7fs3k9mBBn4Jg30EhL6DZ87ZOoJSNmVFCVGGTE6e3y/dWZcUTvyvlNtkYE5rWpaoDfXLj7wtcd/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DdMX0++t; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DdMX0++t" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-313d6d671ffso303261a91.2 for ; Wed, 06 Aug 2025 12:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510323; x=1755115123; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=AqSoz/ys0/nDwriVLA2HvcSasFXIo2nt11bZwmhw7W4=; b=DdMX0++t8TEps43r4L+BtI7kaHxLQ23a6EcGQADh64fVkngAr1srAI1De2hl8ngyEe 3JFAGghDRRDhlOAcvP9QpeZp3jcHVJbnVRuqa+ZNKHu5SWTgBI6TeEbI9Mx4vJqKE+qj lNKy41AZYlLdc7rC+TdxoxcLxexH+rTAvSdcLSshVJd+brdol6zZpIB9FkD3douITXXU dfBmyHBMkBvqtcLwXMPwU8dOixdNW81FvNoRfNABw4tecv3qPuj3ko3EoybKTrIdQdqs QzD2e5nYosvcuOZCWJ/SUPk8M9KqkXlTqkiaNBB3Z4CS+Y0jS/0Z3PhIWzugklE7aT/u Ar1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510323; x=1755115123; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AqSoz/ys0/nDwriVLA2HvcSasFXIo2nt11bZwmhw7W4=; b=smhv97MDyIdaDIYkqvkpo4DsQzCDsA+ENwpQYkiE/daSQFrDB75bzZoD/nL3yz5e+v IjJ89l7rNsHTO4mFv1D1BJZpfWuz/015I+wjfw54lBT3NchTnrhDniFDHuKgJNapkePV GwtXa+YaembthrC6voLl9mOt//BvGw5MIeo7iz/F3skltIXqLX54nadDnBkaVxb3W9N8 oqpK3/CJnSY/4I+M1XrMyq30dDBJerlueasGkqazdYHFhgj8kqe1AUyxwI1H3B699e1P bZ0+i6dVC+i7dgiJkZOEgqfYJ1Cu9N2BwedqCAHRfBNdVmu4bnANIeKdzNnznuF3IzWQ 1hwA== X-Forwarded-Encrypted: i=1; AJvYcCViTnrjPsOi3K6EpqlHi+sP8slLZEPsY/Vaq5zL2dhN+Abq/K7eXuJmv5ob6GubPhVVAJ02LsMutMtWBrw=@vger.kernel.org X-Gm-Message-State: AOJu0Yz8A9fb0idDY92Acm4nVeDEHy6/D+S8GU72KhLmwVJtpJKrRf6Z ngyK1iWrNEGeAegDmKzBiAXEkZ/JfOpEWbchx6rzKnJCjk07BORes5EFlqL1viS+cUh3tTYUQ9d IHLI0Cw== X-Google-Smtp-Source: AGHT+IFhegW1gGR7UHrSJevPOd3ITDk2k4F1jYjU8AgzTqj6JmCrtpCZa2eEdv34H9OVmz6MG3yGyWC8HfY= X-Received: from pjbsx14.prod.google.com ([2002:a17:90b:2cce:b0:311:c197:70a4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1c09:b0:31e:3bbc:e9e6 with SMTP id 98e67ed59e1d1-3217638fd88mr438488a91.19.1754510322909; Wed, 06 Aug 2025 12:58:42 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:57:03 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-42-seanjc@google.com> Subject: [PATCH v5 41/44] KVM: nVMX: Disable PMU MSR interception as appropriate while running L2 From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mingwei Zhang Merge KVM's PMU MSR interception bitmaps with those of L1, i.e. merge the bitmaps of vmcs01 and vmcs12, e.g. so that KVM doesn't interpose on MSR accesses unnecessarily if L1 exposes a mediated PMU (or equivalent) to L2. Signed-off-by: Mingwei Zhang Co-developed-by: Dapeng Mi Signed-off-by: Dapeng Mi [sean: rewrite changelog and comment, omit MSRs that are always intercepted] Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/vmx/nested.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 47f1f0c7d3a7..b986a6fb684c 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -627,6 +627,34 @@ static inline void nested_vmx_set_intercept_for_msr(st= ruct vcpu_vmx *vmx, #define nested_vmx_merge_msr_bitmaps_rw(msr) \ nested_vmx_merge_msr_bitmaps(msr, MSR_TYPE_RW) =20 +static void nested_vmx_merge_pmu_msr_bitmaps(struct kvm_vcpu *vcpu, + unsigned long *msr_bitmap_l1, + unsigned long *msr_bitmap_l0) +{ + struct kvm_pmu *pmu =3D vcpu_to_pmu(vcpu); + struct vcpu_vmx *vmx =3D to_vmx(vcpu); + int i; + + /* + * Skip the merges if the vCPU doesn't have a mediated PMU MSR, i.e. if + * none of the MSRs can possibly be passed through to L1. + */ + if (!kvm_vcpu_has_mediated_pmu(vcpu)) + return; + + for (i =3D 0; i < pmu->nr_arch_gp_counters; i++) { + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_PERFCTR0 + i); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_PMC0 + i); + } + + for (i =3D 0; i < pmu->nr_arch_fixed_counters; i++) + nested_vmx_merge_msr_bitmaps_rw(MSR_CORE_PERF_FIXED_CTR0 + i); + + nested_vmx_merge_msr_bitmaps_rw(MSR_CORE_PERF_GLOBAL_CTRL); + nested_vmx_merge_msr_bitmaps_read(MSR_CORE_PERF_GLOBAL_STATUS); + nested_vmx_merge_msr_bitmaps_write(MSR_CORE_PERF_GLOBAL_OVF_CTRL); +} + /* * Merge L0's and L1's MSR bitmap, return false to indicate that * we do not use the hardware. @@ -724,6 +752,8 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct= kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_MPERF, MSR_TYPE_R); =20 + nested_vmx_merge_pmu_msr_bitmaps(vcpu, msr_bitmap_l1, msr_bitmap_l0); + kvm_vcpu_unmap(vcpu, &map); =20 vmx->nested.force_msr_bitmap_recalc =3D false; --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 286142E1754 for ; Wed, 6 Aug 2025 19:58:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510329; cv=none; b=BxpL0/Bta3BuDN2o6oC71mo+NmxgPvy2PkDXYUVGs9s5dr4EEV1ENWsOr8jAh76RouAYpGbZeFSxjM3hZxhL3vDxljgGkCLEEaRG7ZAGNLSh146o0miwbL2vQ3KSLahT9frxOLz++22nGGgOV4kS+bljXD+t2VBW0ba4MhorSro= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510329; c=relaxed/simple; bh=mktiJzkM34N6Zf3/nCj8gPSMPc22J/Q3PSgyAnIfDJg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BHcL3ThwU2M6A1q7IjhQFpLEtwQsnFnka4wkbAboRpWNdINiimyM5t1oT29Kr1ZhlQ+3oOpzjglymM91kNqgNphAFkTL98fiAohIK727wvgeUnUcCwejfqW35QVbmY/hU+NExQUcfzQ+cj9PSFAx2IVfKcXXrmXyA6Z0q3mYRr0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fXpA5j26; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fXpA5j26" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b427c287cdfso272212a12.1 for ; Wed, 06 Aug 2025 12:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510324; x=1755115124; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=aIwiS3b08lZbnfZBzBRHsRLYuS/hl5WgYmD0pyHBLVw=; b=fXpA5j26IVK6dc9rtP00kkCEtvN2bqVStiB/w8fOy12s9JAtL+3eaGIEN0E3zRCEpF PB3xnRRQ7u1/ZEdLfYH4/2FLCNwESlB2Cc++/dfTWBjt0Y1aDY/a6162vkv8DudCjTKy y9FKqOhFjiD5kjvq1gDrDG7gCAcoag/+U+L6PisuV457jMx9oKdwVhF0knKU2TMZnCZT aISWWeRj88t0BXWJ/gSngCEF9dChvgrgu8i3tKgdvqaHjc1FBADtS3qLsgemZByv1hMM Hb35zir3S358snxgHky/ncSHxxhbaG+ryusJJeANTjNNH5HYdVcVNu6KHjWnGGsWfrq8 cRlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510324; x=1755115124; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=aIwiS3b08lZbnfZBzBRHsRLYuS/hl5WgYmD0pyHBLVw=; b=LCskU5dOBxyU1LlwPNs+lsRl7bthkm3NirR6BankL+bh1uB9sEZWgYL+eSWCPSzHr/ I8nMZZCuuSKExG4wkK+8iRGthPrWXeZJLVhyJKoSd4q5ykI4zZHAkBD1ufsPD7YzNOsw lJvxNTu2Dytf3Sr27F3WblDADS2JL5xtjCo0Z2sQMkd6H0SCWggGElrHEOIdJru2neWL SjGJ6GuP/7hxjUP3lh9VmFOHvxIJWlT+jlA0As9c4b/JTzm+pB7M5N6sVGIGo4Z8x7J/ hZiiUOtqkYpmAhc+8dkRVH0rQBZx71Azjsos9ewuPhCZ1TNcAA0wxpXawPwcfqYItCAT n6sQ== X-Forwarded-Encrypted: i=1; AJvYcCW8TeX+iMOG9AxyyQhDbebQNHCM+P14U0LDyC3bwzQBlckMul7neBooDeWq7/uW/VKT/hC4Q/mtuPvpysU=@vger.kernel.org X-Gm-Message-State: AOJu0Ywbck8bXk7UDz3jSXTKxFJe2OMQo0TiRgum+tqOP1tEIs9ZuCdk OtAbMG8CzR2Qn8RORNnwzdiSNXNvbZOqk0vWGSCRY1vPdzZuLr/OYEso+PlCur434gWlDuB60G0 vLcuh3A== X-Google-Smtp-Source: AGHT+IEQR+T9dam/zfnLIV/i5izf3PHRRWb6ka2239rbCk7AqXY3VxMNYGaiVe7zhD28m3tT7rq7DHc261E= X-Received: from pjzd15.prod.google.com ([2002:a17:90a:e28f:b0:312:151d:c818]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:f645:b0:240:150e:57ba with SMTP id d9443c01a7336-242a0a90e7amr55806795ad.3.1754510324526; Wed, 06 Aug 2025 12:58:44 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:57:04 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-43-seanjc@google.com> Subject: [PATCH v5 42/44] KVM: nSVM: Disable PMU MSR interception as appropriate while running L2 From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add MSRs that might be passed through to L1 when running with a mediated PMU to the nested SVM's set of to-be-merged MSR indices, i.e. disable interception of PMU MSRs when running L2 if both KVM (L0) and L1 disable interception. There is no need for KVM to interpose on such MSR accesses, e.g. if L1 exposes a mediated PMU (or equivalent) to L2. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/svm/nested.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index b7fd2e869998..de2b9db2d0ba 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -194,7 +194,7 @@ void recalc_intercepts(struct vcpu_svm *svm) * Hardcode the capacity of the array based on the maximum number of _offs= ets_. * MSRs are batched together, so there are fewer offsets than MSRs. */ -static int nested_svm_msrpm_merge_offsets[7] __ro_after_init; +static int nested_svm_msrpm_merge_offsets[10] __ro_after_init; static int nested_svm_nr_msrpm_merge_offsets __ro_after_init; typedef unsigned long nsvm_msrpm_merge_t; =20 @@ -222,6 +222,22 @@ int __init nested_svm_init_msrpm_merge_offsets(void) MSR_IA32_LASTBRANCHTOIP, MSR_IA32_LASTINTFROMIP, MSR_IA32_LASTINTTOIP, + + MSR_K7_PERFCTR0, + MSR_K7_PERFCTR1, + MSR_K7_PERFCTR2, + MSR_K7_PERFCTR3, + MSR_F15H_PERF_CTR0, + MSR_F15H_PERF_CTR1, + MSR_F15H_PERF_CTR2, + MSR_F15H_PERF_CTR3, + MSR_F15H_PERF_CTR4, + MSR_F15H_PERF_CTR5, + + MSR_AMD64_PERF_CNTR_GLOBAL_CTL, + MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, + MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, + MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, }; int i, j; =20 --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E31BD2E264B for ; Wed, 6 Aug 2025 19:58:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510330; cv=none; b=entFx1Bw2SDEUJSx4vM31jIBpA+NkzAX+/xmjP6ZhtRjbcQ9rb84kTlJ2m3Kt5g5Po0mgdvhXttI3WR+20FsNETV+NwlrbNfJI67v6ra6stOOOowLymIaukBeEtQxRXyz9LYGUxY/PmWJ08LZp/OIwAvV9H8mjr5khJA36OCfxA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510330; c=relaxed/simple; bh=GASA5Z6faQfJd1EfEj5uOcxIVuCY+8w72a3NGzWBljs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=e789pIxEb1NTes8ZmJnJl/SCvGtA/3SBhCnVrbczmzdc+tQuPyXUwmltESfvFq4TlognMS+bDJ2DHSoTFlmpHvwn3G2/yhsp9kZhHQPov+9bVaPHFGX/OxYgIOdGASHfvxX9xwpf+Tnyb4N5MQfTqRm3teDhwxfKjuri9cH5W54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NA57kutG; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NA57kutG" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-31edd69d754so283761a91.1 for ; Wed, 06 Aug 2025 12:58:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510326; x=1755115126; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=w80ZA8rn05ntQihJcNwz1pv2hia2ecTvLpbk0Bz9WXc=; b=NA57kutGoHd7SQM2ydI8bzlKGOvvhHRUL1CzhW1r8+2CTM5hU3VKBSnTK0MF3AvQCe Ikq1OUIaG3Lp0L10C00RaIZBI7jjUhcQeQ/hQXRdg0PXWZpHP1t4JOkPIOWHnvthTo/i LecQ3PqsIkkrANu6JSarmxGeAEBySU95RU4LVZtcORr5Hlb46fS+FJWpJLjBYXK8imCC JHj6ebSr8Gr2JGGZkNr4jP1Bg+SlrXHIaDiji71g1jCY5Zx/nwp6ag2aBeOyXpjlO4Mu zZrAqfh//+e+c4vsQw4hk3NL5IAZo4DyyRITUEr3ZY0qxEtLuFN7pPHICQ0qSTZ2mlaC n0mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510326; x=1755115126; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=w80ZA8rn05ntQihJcNwz1pv2hia2ecTvLpbk0Bz9WXc=; b=pUOSTLX3bJH0ZH2mWQPth0Y8atAOVzyu9mIRjFBicKBc7w4G1zDJ7CHlv/vX37+vK9 ErJXvmUgJkl9oqvOM9NdhTB+fxD1hLw+mx9w7xXwIj2YnudfpXR6TXR1EYYNm/k1PSnG WzovjoqkNN3T11sPKm9YaMqVgqgxlec3jv1IGaSnoWIJ+lrI+AQis3gAwoxGzWLxEpgN 0j59XMPCDe48yy3RMH5jSkv18On+sFhMlGtkwQkOsKzUcutEcGGBhQP19mJxv6TZZ8he 3R3gM9R8MBP5HGTn7dNceWh6iZJOPWP4PQiGckhcYbLy8ISwiRUxpb5JJACtFaoWm9Hx h8AQ== X-Forwarded-Encrypted: i=1; AJvYcCXe5CC6Mk493G0B12x6rS11ABmguNYTT4x9fpjnfP75R5a/rBa0huOibvGcYWh5auAVojkx/KyU68A6PkE=@vger.kernel.org X-Gm-Message-State: AOJu0YwnIawNYg+eD42B7RIbsMAZSC7Sg3wSexE/PKvkFIoMMwczYrgb 5+JKuFG/JrYGug/VqDJb1jCsu9JHXr0i6C4Aoj3aLpbZBIQxd4DQ7ca9VjJPbG6go4iXcCc1pJX 9CHH3FA== X-Google-Smtp-Source: AGHT+IEfElow/kZ7u4L92aZnG23Q7FLbrDYX2HxfWcS+XU7qHvCNVpRcocJ/y0Y5Vl+7RC7W7jz29DD5/48= X-Received: from pjbdy8.prod.google.com ([2002:a17:90b:6c8:b0:321:6ddc:33a0]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2750:b0:321:157a:ee73 with SMTP id 98e67ed59e1d1-32166e27ab1mr4925727a91.6.1754510326195; Wed, 06 Aug 2025 12:58:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:57:05 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-44-seanjc@google.com> Subject: [PATCH v5 43/44] KVM: x86/pmu: Expose enable_mediated_pmu parameter to user space From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Dapeng Mi Expose enable_mediated_pmu parameter to user space, i.e. allow userspace to enable/disable mediated vPMU support. Document the mediated versus perf-based behavior as part of the kernel-parameters.txt entry, and opportunistically add an entry for the core enable_pmu param as well. Signed-off-by: Dapeng Mi Signed-off-by: Mingwei Zhang Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- .../admin-guide/kernel-parameters.txt | 49 +++++++++++++++++++ arch/x86/kvm/svm/svm.c | 2 + arch/x86/kvm/vmx/vmx.c | 2 + 3 files changed, 53 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 07e22ba5bfe3..12a96493de9a 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2840,6 +2840,26 @@ =20 Default is Y (on). =20 + kvm.enable_pmu=3D[KVM,X86] + If enabled, KVM will virtualize PMU functionality based + on the virtual CPU model defined by userspace. This + can be overridden on a per-VM basis via + KVM_CAP_PMU_CAPABILITY. + + If disabled, KVM will not virtualize PMU functionality, + e.g. MSRs, PMCs, PMIs, etc., even if userspace defines + a virtual CPU model that contains PMU assets. + + Note, KVM's vPMU support implicitly requires running + with an in-kernel local APIC, e.g. to deliver PMIs to + the guest. Running without an in-kernel local APIC is + not supported, though KVM will allow such a combination + (with severely degraded functionality). + + See also enable_mediated_pmu. + + Default is Y (on). + kvm.enable_virt_at_load=3D[KVM,ARM64,LOONGARCH,MIPS,RISCV,X86] If enabled, KVM will enable virtualization in hardware when KVM is loaded, and disable virtualization when KVM @@ -2886,6 +2906,35 @@ If the value is 0 (the default), KVM will pick a period based on the ratio, such that a page is zapped after 1 hour on average. =20 + kvm-{amd,intel}.enable_mediated_pmu=3D[KVM,AMD,INTEL] + If enabled, KVM will provide a mediated virtual PMU, + instead of the default perf-based virtual PMU (if + kvm.enable_pmu is true and PMU is enumerated via the + virtual CPU model). + + With a perf-based vPMU, KVM operates as a user of perf, + i.e. emulates guest PMU counters using perf events. + KVM-created perf events are managed by perf as regular + (guest-only) events, e.g. are scheduled in/out, contend + for hardware resources, etc. Using a perf-based vPMU + allows guest and host usage of the PMU to co-exist, but + incurs non-trivial overhead and can result in silently + dropped guest events (due to resource contention). + + With a mediated vPMU, hardware PMU state is context + switched around the world switch to/from the guest. + KVM mediates which events the guest can utilize, but + gives the guest direct access to all other PMU assets + when possible (KVM may intercept some accesses if the + virtual CPU model provides a subset of hardware PMU + functionality). Using a mediated vPMU significantly + reduces PMU virtualization overhead and eliminates lost + guest events, but is mutually exclusive with using perf + to profile KVM guests and adds latency to most VM-Exits + (to context switch PMU state). + + Default is N (off). + kvm-amd.nested=3D [KVM,AMD] Control nested virtualization feature in KVM/SVM. Default is 1 (enabled). =20 diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index ca6f453cc160..2797c3ab7854 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -178,6 +178,8 @@ module_param(intercept_smi, bool, 0444); bool vnmi =3D true; module_param(vnmi, bool, 0444); =20 +module_param(enable_mediated_pmu, bool, 0444); + static bool svm_gp_erratum_intercept =3D true; =20 static u8 rsm_ins_bytes[] =3D "\x0f\xaa"; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 85bd82d41f94..4a4691beba55 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -151,6 +151,8 @@ module_param_named(preemption_timer, enable_preemption_= timer, bool, S_IRUGO); extern bool __read_mostly allow_smaller_maxphyaddr; module_param(allow_smaller_maxphyaddr, bool, S_IRUGO); =20 +module_param(enable_mediated_pmu, bool, 0444); + #define KVM_VM_CR0_ALWAYS_OFF (X86_CR0_NW | X86_CR0_CD) #define KVM_VM_CR0_ALWAYS_ON_UNRESTRICTED_GUEST X86_CR0_NE #define KVM_VM_CR0_ALWAYS_ON \ --=20 2.50.1.565.gc32cd1483b-goog From nobody Sun Oct 5 07:21:23 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6346C2E2EEE for ; Wed, 6 Aug 2025 19:58:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510332; cv=none; b=UESrxTDbk58JRCUq8pkxj3ZKwlteVWBpMn8wbg1+ZVRDFOFVxd5Wa4MfgL/Cu8RB7XTfnt4puLjTHaTMRUKJhm3OhAWxGIMsHTMhJcWeB1DGtIClRL0n5yN/NWwHaHIAy7ufT2EoR1cL2fHxRzxMEIA9mXSou1dNDtaa7hNBuJc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510332; c=relaxed/simple; bh=UqKbQ/MEitFcIh6oJaDa5iQaYpCTlGUBLK908ME1g4Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oswrb/dj373n0EgM9K2Ay0fk7Ua/zqFUzqt+4oPM1FpNNI9ri6IM2dAI6uVY0l+F2g0ivIYQhnByfqNWVTX0p84pzhdgg30VzO46A4zZMtCz0e9nK6KDqb1BQo1adu83ULrwfZa1Dx9k2VA5OKwhQDSgCDUvbCtnyqOtAZA9+/g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4DFPLcrp; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4DFPLcrp" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-24283069a1cso1864885ad.1 for ; Wed, 06 Aug 2025 12:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510328; x=1755115128; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=FEfQdODxvc/WYDgKcBJNCqpRp5QcBVEYBtlnYDr8f+A=; b=4DFPLcrpXC8nzPtT2ABri2kiMNcEVcSAxV5rGGrD+ycZK7+G/xw3XhsrSAq1GUn+KV c3mpEz7/DDiuZDG7GsCIPbif7RO/cx41LAnNr0vTSFC6UyiRf85KJwYiHHN+6gMvD2x3 r/KZj9P6pt0zyfbT2R5csi8Lws+uVx2QP1ChHeoVC3qnHlqdtPqLRPMYbMDeSoA+untO RvTmh5AQsiEoV3GWBjr95m3BQ/5EUFHIiRvNkBOZmvg7CjaLz86K424skxFJa2S2WDzF 5659bUKO7cI22AYTqU4s+3QVJ2DpO+fanRhX7RIBIsPgxSskBdF4PvKFLbevBywWkkOk jgeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510328; x=1755115128; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=FEfQdODxvc/WYDgKcBJNCqpRp5QcBVEYBtlnYDr8f+A=; b=KSFQ6o0aAemBS7drydeJH3jP7w+nwiDPuxNTBTXzMuAiuBBfDqDsArk1U2n+GWcq9X 1fIjfGrUAkqIS3X6vnEhvZLfYd+dUJkUjxk07/foCaKfO7xGh4NU31HNLNGYjEwayWW7 3F+t9hru+a+6Zvfzkppb9vxaKItsEVBWsOwW3HICgttZpZRarzlcP9a6ZJZYUWh0qDW3 TNZu0w30OnZd2K75tXFsLbWCDlcRmqY5F6In733jAWhiDJLbCULciXL2vUEcrbxYfzb+ NmAwnEtdXEfDXjazFRBAVxYrrghI3iI/hv1jXQemMqQhwVh6UmpJrUYLNa9Wan8lRCFE R3UQ== X-Forwarded-Encrypted: i=1; AJvYcCXCsXZqiBHr6mwP5DZK2IiAZUZkBus6SLuoj8kD19gZ9Jrjyyj9tw4XVLMI/nhPg7swXqJkndxu/ZBVhNw=@vger.kernel.org X-Gm-Message-State: AOJu0Yz85KYIVJU/HLkSv1/1bArVL7jgLHDHb9uf4gEADhxxupAldy+S /Zi1acrG+jsBBk+JJIkkSKjmATIcKBKD/eSN3zXagq2iVsDKKdG6hyIgGimaUaDvDl53iH2z6tj 7obBwEA== X-Google-Smtp-Source: AGHT+IHKT98t7sh+NJ88sddK4YezK7lHLHfrTfL2AYLv+RO+SAkhLkiCccC37Go9T+seI3ZwHUra06wFxFw= X-Received: from plbmp11.prod.google.com ([2002:a17:902:fd0b:b0:23f:fa41:1de3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:3bcc:b0:234:c8ec:51b5 with SMTP id d9443c01a7336-242a0bffb31mr47813405ad.53.1754510327813; Wed, 06 Aug 2025 12:58:47 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:57:06 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-45-seanjc@google.com> Subject: [PATCH v5 44/44] KVM: x86/pmu: Elide WRMSRs when loading guest PMCs if values already match From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When loading a mediated PMU state, elide the WRMSRs to load PMCs with the guest's value if the value in hardware already matches the guest's value. For the relatively common case where neither the guest nor the host is actively using the PMU, i.e. when all/many counters are '0', eliding the WRMSRs reduces the latency of handling VM-Exit by a measurable amount (WRMSR is significantly more expensive than RDPMC). As measured by KVM-Unit-Tests' CPUID VM-Exit testcase, this provides a a ~25% reduction in latency (4k =3D> 3k cycles) on Intel Emerald Rapids, and a ~13% reduction (6.2k =3D> 5.3k cycles) on AMD Turing. Signed-off-by: Sean Christopherson Tested-by: Xudong Hao --- arch/x86/kvm/pmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index ddab1630a978..0e5048ae86fa 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -1299,13 +1299,15 @@ static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu= *vcpu) for (i =3D 0; i < pmu->nr_arch_gp_counters; i++) { pmc =3D &pmu->gp_counters[i]; =20 - wrmsrl(gp_counter_msr(i), pmc->counter); + if (pmc->counter !=3D rdpmc(i)) + wrmsrl(gp_counter_msr(i), pmc->counter); wrmsrl(gp_eventsel_msr(i), pmc->eventsel_hw); } for (i =3D 0; i < pmu->nr_arch_fixed_counters; i++) { pmc =3D &pmu->fixed_counters[i]; =20 - wrmsrl(fixed_counter_msr(i), pmc->counter); + if (pmc->counter !=3D rdpmc(INTEL_PMC_FIXED_RDPMC_BASE | i)) + wrmsrl(fixed_counter_msr(i), pmc->counter); } } =20 --=20 2.50.1.565.gc32cd1483b-goog