From nobody Tue Apr 8 07:27:07 2025
Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com
[209.85.128.73])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by smtp.subspace.kernel.org (Postfix) with ESMTPS id 070081F8F17
for <linux-kernel@vger.kernel.org>; Wed, 8 Jan 2025 11:46:26 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org;
arc=none smtp.client-ip=209.85.128.73
ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
t=1736336788; cv=none;
b=LwTjqT/+roPtYm1C8h0CceSAyEdxD9FRfG+UcAuYzJDn62Xzm9sDONj63LApdVvChmtt82M4Q6xjDFIz1LCC/5rUIJrv0SlnSoeLlOohL8Rrn2MAH3LK8hFPUTTLlBcuS2S253sz33dlayu+uUuyrCDh3+zCI+VFc6osSdX9cyM=
ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org;
s=arc-20240116; t=1736336788; c=relaxed/simple;
bh=8/RUDTnIgUf7aFh9NDhrAgl/Ki56JGCsnadyCJWwefM=;
h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From:
To:Cc:Content-Type;
b=klezh8TDFbLDfNxt7i6f25ywdkWGf8bxEVKetXaUvd3nPdXN9L/+rJul1AnqBY1r6p1uca13/RpCTwK0dOzCoj+i95aFRXPDXE5FutUZh+tKrtrzl4ptdWGym4KO8jap1KG8myFs5sbi3jVXllbwU3hNACeCK/pVyT7wF70C/2I=
ARC-Authentication-Results: i=1; smtp.subspace.kernel.org;
dmarc=pass (p=reject dis=none) header.from=google.com;
spf=pass smtp.mailfrom=flex--vdonnefort.bounces.google.com;
dkim=pass (2048-bit key) header.d=google.com header.i=@google.com
header.b=PpQ/oETD; arc=none smtp.client-ip=209.85.128.73
Authentication-Results: smtp.subspace.kernel.org;
dmarc=pass (p=reject dis=none) header.from=google.com
Authentication-Results: smtp.subspace.kernel.org;
spf=pass smtp.mailfrom=flex--vdonnefort.bounces.google.com
Authentication-Results: smtp.subspace.kernel.org;
dkim=pass (2048-bit key) header.d=google.com header.i=@google.com
header.b="PpQ/oETD"
Received: by mail-wm1-f73.google.com with SMTP id
5b1f17b1804b1-4362f893bfaso91487025e9.1
for <linux-kernel@vger.kernel.org>;
Wed, 08 Jan 2025 03:46:25 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=google.com; s=20230601; t=1736336784; x=1736941584;
darn=vger.kernel.org;
h=cc:to:from:subject:message-id:references:mime-version:in-reply-to
:date:from:to:cc:subject:date:message-id:reply-to;
bh=lH7dKXy2tl1VnisSF0iTM00IkcOBjA3TrFxfzzUtdog=;
b=PpQ/oETDPiDiRTErj81ykRtlJDd2aTRP8lNrMk+/NR/YCsvWhWRezZa1PNTVRKCs6E
BNhHQaeVkTndhdayLX0C6c7wIQqwImyEFGIFjNereK++0nRbhnuRjvjAOBIP3pNrkadZ
ep2AT3UQrc2kmxw/LcqdyPbbZTFnrQNFi5kDR/zmyZkTZF8i21NBZdQf56tDNEo93V0D
rf3bWcm3fTFsD9GJh0AOwzkL7g9sGnNIUEbDGKqdK5jw0S9uxZf1Q77jgF0+aQ5xqGbQ
n3sIzFxMngNtlw7MDihq1ZwnCt63JXym4tba4k8iIFd2AccQ25JhrrzmREHyrG+m/wDa
TLng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1736336784; x=1736941584;
h=cc:to:from:subject:message-id:references:mime-version:in-reply-to
:date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=lH7dKXy2tl1VnisSF0iTM00IkcOBjA3TrFxfzzUtdog=;
b=iOgt5tgzFDK5dXWGY00ejBjvIBuEm20/ydu94/UTwOh2OVrALK6MfaaDI6kZTUNcxK
HNui4Y79dSt5f2H5H9yIeSsR/5nPS0YjtxjWJpoITk9RmsGogIslKbCG+7y3EzixJgD6
hLw3qllWF3vPi5t2BNnn3+zmC+K+NtNVaraKEaBPwBZhzwVApsFz2Qf34BD9OrKv+MQ/
QUnJAZFMl3aQwzhy5ddzIR9R5xYVZFD0VYwn5qTW6E8pSl4pazVU728ILzOuYEWvTcKd
apHXzOQmUj5CJP+yVybs+R6OvYWU1Ca7c1zbWyNWsjJ+7bNSX3YaPy/aYLJ5h/1XqNQr
IeOg==
X-Forwarded-Encrypted: i=1;
AJvYcCVQQEnL5GefNPAPoDt2vM+liQ2KqH1N9kDBlHfnQUKF62dEdjZisPMpCBq4fpIYC6iitjCbcQKFEwXUq7k=@vger.kernel.org
X-Gm-Message-State: AOJu0Yx7suvFWbhbMo8sWHboqIakIJxy+OnjrwGmEIeuUtTr1foJB0hA
Fygaad74ysYemAg36Fr6DDBj0ISdYknmoW0iGj0Gm6MlgfV1m4ddj6E9JT9aZlRch0rK0iyLSbg
PUXRd8Zvf34pu/4SPLA==
X-Google-Smtp-Source:
AGHT+IGynp8ZWmkpLNs9CyFu8UPI5Zc3mCoHws1ZpBM/8hKa9L6CZ54/kdIHhTSwZhpwXtm0J2cX9SxOnQrTfCJt
X-Received: from wrbhb14.prod.google.com
([2002:a05:6000:490e:b0:385:e0ea:d4d1])
(user=vdonnefort job=prod-delivery.src-stubby-dispatcher) by
2002:a05:6000:1848:b0:388:c75c:e839 with SMTP id
ffacd0b85a97d-38a8733750bmr1392762f8f.42.1736336784716;
Wed, 08 Jan 2025 03:46:24 -0800 (PST)
Date: Wed, 8 Jan 2025 11:45:35 +0000
In-Reply-To: <20250108114536.627715-1-vdonnefort@google.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
Mime-Version: 1.0
References: <20250108114536.627715-1-vdonnefort@google.com>
X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog
Message-ID: <20250108114536.627715-12-vdonnefort@google.com>
Subject: [PATCH v2 11/12] KVM: arm64: Add support for hyp events
From: Vincent Donnefort <vdonnefort@google.com>
To: rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com,
linux-trace-kernel@vger.kernel.org, maz@kernel.org, oliver.upton@linux.dev,
joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com
Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
jstultz@google.com, qperret@google.com, will@kernel.org,
kernel-team@android.com, linux-kernel@vger.kernel.org,
Vincent Donnefort <vdonnefort@google.com>
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
Following the introduction of hyp tracing for pKVM, add the ability to
describe and emit events into the hypervisor ring-buffers.
Hypervisor events are declared into kvm_hypevents.h and can be called
with trace_<event_name>() in a similar fashion to the kernel tracefs
events.
hyp_enter and hyp_exit events are provided as an example.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_=
asm.h
index 08b6a021b84e..784f6b3ae892 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -85,6 +85,7 @@ enum __kvm_host_smccc_func {
__KVM_HOST_SMCCC_FUNC___pkvm_enable_tracing,
__KVM_HOST_SMCCC_FUNC___pkvm_reset_tracing,
__KVM_HOST_SMCCC_FUNC___pkvm_swap_reader_tracing,
+ __KVM_HOST_SMCCC_FUNC___pkvm_enable_event,
};
=20
#define DECLARE_KVM_VHE_SYM(sym) extern char sym[]
diff --git a/arch/arm64/include/asm/kvm_define_hypevents.h b/arch/arm64/inc=
lude/asm/kvm_define_hypevents.h
new file mode 100644
index 000000000000..cda7dc27dba7
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_define_hypevents.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#include <linux/trace_events.h>
+
+#include <asm/kvm_hyptrace.h>
+#include <asm/kvm_hypevents_defs.h>
+
+#ifndef HYP_EVENT_FILE
+# undef __ARM64_KVM_HYPEVENTS_H_
+# define __HYP_EVENT_FILE <asm/kvm_hypevents.h>
+#else
+# define __HYP_EVENT_FILE __stringify(HYP_EVENT_FILE)
+#endif
+
+#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \
+ HYP_EVENT_FORMAT(__name, __struct); \
+ static void hyp_event_trace_##__name(struct ht_iterator *iter) \
+ { \
+ struct trace_hyp_format_##__name __maybe_unused *__entry =3D \
+ (struct trace_hyp_format_##__name *)iter->ent; \
+ trace_seq_puts(&iter->seq, #__name); \
+ trace_seq_putc(&iter->seq, ' '); \
+ trace_seq_printf(&iter->seq, __printk); \
+ trace_seq_putc(&iter->seq, '\n'); \
+ }
+#define HYP_EVENT_MULTI_READ
+#include __HYP_EVENT_FILE
+
+#undef he_field
+#define he_field(_type, _item) \
+ { \
+ .type =3D #_type, .name =3D #_item, \
+ .size =3D sizeof(_type), .align =3D __alignof__(_type), \
+ .is_signed =3D is_signed_type(_type), \
+ },
+#undef HYP_EVENT
+#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \
+ static struct trace_event_fields hyp_event_fields_##__name[] =3D { \
+ __struct \
+ {} \
+ };
+#include __HYP_EVENT_FILE
+
+#undef HYP_EVENT
+#undef HE_PRINTK
+#define __entry REC
+#define HE_PRINTK(fmt, args...) "\"" fmt "\", " __stringify(args)
+#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \
+ static char hyp_event_print_fmt_##__name[] =3D __printk; \
+ static bool hyp_event_enabled_##__name; \
+ struct hyp_event __section("_hyp_events."#__name) \
+ hyp_event_##__name =3D { \
+ .name =3D #__name, \
+ .enabled =3D &hyp_event_enabled_##__name, \
+ .fields =3D hyp_event_fields_##__name, \
+ .print_fmt =3D hyp_event_print_fmt_##__name, \
+ .trace_func =3D hyp_event_trace_##__name, \
+ }
+#include __HYP_EVENT_FILE
+
+#undef HYP_EVENT_MULTI_READ
diff --git a/arch/arm64/include/asm/kvm_hypevents.h b/arch/arm64/include/as=
m/kvm_hypevents.h
new file mode 100644
index 000000000000..0b98a87a1250
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_hypevents.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#if !defined(__ARM64_KVM_HYPEVENTS_H_) || defined(HYP_EVENT_MULTI_READ)
+#define __ARM64_KVM_HYPEVENTS_H_
+
+#ifdef __KVM_NVHE_HYPERVISOR__
+#include <nvhe/trace.h>
+#endif
+
+/*
+ * Hypervisor events definitions.
+ */
+
+HYP_EVENT(hyp_enter,
+ HE_PROTO(void),
+ HE_STRUCT(
+ ),
+ HE_ASSIGN(
+ ),
+ HE_PRINTK(" ")
+);
+
+HYP_EVENT(hyp_exit,
+ HE_PROTO(void),
+ HE_STRUCT(
+ ),
+ HE_ASSIGN(
+ ),
+ HE_PRINTK(" ")
+);
+#endif
diff --git a/arch/arm64/include/asm/kvm_hypevents_defs.h b/arch/arm64/inclu=
de/asm/kvm_hypevents_defs.h
new file mode 100644
index 000000000000..473bf4363d82
--- /dev/null
+++ b/arch/arm64/include/asm/kvm_hypevents_defs.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ARM64_KVM_HYPEVENTS_DEFS_H
+#define __ARM64_KVM_HYPEVENTS_DEFS_H
+
+struct hyp_event_id {
+ unsigned short id;
+ void *data;
+};
+
+#define HYP_EVENT_NAME_MAX 32
+
+struct hyp_event {
+ char name[HYP_EVENT_NAME_MAX];
+ bool *enabled;
+ char *print_fmt;
+ struct trace_event_fields *fields;
+ void (*trace_func)(struct ht_iterator *iter);
+ int id;
+};
+
+struct hyp_entry_hdr {
+ unsigned short id;
+};
+
+/*
+ * Hyp events definitions common to the hyp and the host
+ */
+#define HYP_EVENT_FORMAT(__name, __struct) \
+ struct __packed trace_hyp_format_##__name { \
+ struct hyp_entry_hdr hdr; \
+ __struct \
+ }
+
+#define HE_PROTO(args...) args
+#define HE_STRUCT(args...) args
+#define HE_ASSIGN(args...) args
+#define HE_PRINTK(args...) args
+
+#define he_field(type, item) type item;
+#endif
diff --git a/arch/arm64/include/asm/kvm_hyptrace.h b/arch/arm64/include/asm=
/kvm_hyptrace.h
index 7da6a248c7fa..7b66bd06537f 100644
--- a/arch/arm64/include/asm/kvm_hyptrace.h
+++ b/arch/arm64/include/asm/kvm_hyptrace.h
@@ -4,6 +4,22 @@
#include <asm/kvm_hyp.h>
=20
#include <linux/ring_buffer.h>
+#include <linux/trace_seq.h>
+#include <linux/workqueue.h>
+
+struct ht_iterator {
+ struct trace_buffer *trace_buffer;
+ int cpu;
+ struct hyp_entry_hdr *ent;
+ unsigned long lost_events;
+ int ent_cpu;
+ size_t ent_size;
+ u64 ts;
+ void *spare;
+ size_t copy_leftover;
+ struct trace_seq seq;
+ struct delayed_work poll_work;
+};
=20
/*
* Host donations to the hypervisor to store the struct hyp_buffer_page.
diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
index 8f5422ed1b75..e60754cdbf33 100644
--- a/arch/arm64/kernel/image-vars.h
+++ b/arch/arm64/kernel/image-vars.h
@@ -134,6 +134,10 @@ KVM_NVHE_ALIAS(__hyp_bss_start);
KVM_NVHE_ALIAS(__hyp_bss_end);
KVM_NVHE_ALIAS(__hyp_rodata_start);
KVM_NVHE_ALIAS(__hyp_rodata_end);
+#ifdef CONFIG_TRACING
+KVM_NVHE_ALIAS(__hyp_event_ids_start);
+KVM_NVHE_ALIAS(__hyp_event_ids_end);
+#endif
=20
/* pKVM static key */
KVM_NVHE_ALIAS(kvm_protected_mode_initialized);
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld=
s.S
index f84c71f04d9e..cb87172fc1a7 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -13,12 +13,23 @@
*(__kvm_ex_table) \
__stop___kvm_ex_table =3D .;
=20
+#ifdef CONFIG_TRACING
+#define HYPERVISOR_EVENT_IDS \
+ . =3D ALIGN(PAGE_SIZE); \
+ __hyp_event_ids_start =3D .; \
+ *(HYP_SECTION_NAME(.event_ids)) \
+ __hyp_event_ids_end =3D .;
+#else
+#define HYPERVISOR_EVENT_IDS
+#endif
+
#define HYPERVISOR_DATA_SECTIONS \
HYP_SECTION_NAME(.rodata) : { \
. =3D ALIGN(PAGE_SIZE); \
__hyp_rodata_start =3D .; \
*(HYP_SECTION_NAME(.data..ro_after_init)) \
*(HYP_SECTION_NAME(.rodata)) \
+ HYPERVISOR_EVENT_IDS \
. =3D ALIGN(PAGE_SIZE); \
__hyp_rodata_end =3D .; \
}
@@ -200,6 +211,13 @@ SECTIONS
ASSERT(SIZEOF(.got.plt) =3D=3D 0 || SIZEOF(.got.plt) =3D=3D 0x18,
"Unexpected GOT/PLT entries detected!")
=20
+#ifdef CONFIG_TRACING
+ .rodata.hyp_events : {
+ __hyp_events_start =3D .;
+ *(SORT(_hyp_events.*))
+ __hyp_events_end =3D .;
+ }
+#endif
/* code sections that are never executed via the kernel mapping */
.rodata.text : {
TRAMP_TEXT
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 865971bb8905..f9e208273031 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -29,7 +29,7 @@ kvm-$(CONFIG_HW_PERF_EVENTS) +=3D pmu-emul.o pmu.o
kvm-$(CONFIG_ARM64_PTR_AUTH) +=3D pauth.o
kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) +=3D ptdump.o
=20
-kvm-$(CONFIG_TRACING) +=3D hyp_trace.o
+kvm-$(CONFIG_TRACING) +=3D hyp_events.o hyp_trace.o
=20
always-y :=3D hyp_constants.h hyp-constants.s
=20
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index cef933eb1cd4..2e609cb09e23 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -2642,6 +2642,8 @@ static int __init init_hyp_mode(void)
=20
kvm_hyp_init_symbols();
=20
+ hyp_trace_init_events();
+
if (is_protected_kvm_enabled()) {
if (IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL) &&
cpus_have_final_cap(ARM64_HAS_ADDRESS_AUTH))
diff --git a/arch/arm64/kvm/hyp/include/nvhe/arm-smccc.h b/arch/arm64/kvm/h=
yp/include/nvhe/arm-smccc.h
new file mode 100644
index 000000000000..4b69d33e4f2d
--- /dev/null
+++ b/arch/arm64/kvm/hyp/include/nvhe/arm-smccc.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#include <asm/kvm_hypevents.h>
+
+#include <linux/arm-smccc.h>
+
+#undef arm_smccc_1_1_smc
+#define arm_smccc_1_1_smc(...) \
+ do { \
+ trace_hyp_exit(); \
+ __arm_smccc_1_1(SMCCC_SMC_INST, __VA_ARGS__); \
+ trace_hyp_enter(); \
+ } while (0)
diff --git a/arch/arm64/kvm/hyp/include/nvhe/define_events.h b/arch/arm64/k=
vm/hyp/include/nvhe/define_events.h
new file mode 100644
index 000000000000..2298b49cb355
--- /dev/null
+++ b/arch/arm64/kvm/hyp/include/nvhe/define_events.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef HYP_EVENT_FILE
+# define __HYP_EVENT_FILE <asm/kvm_hypevents.h>
+#else
+# define __HYP_EVENT_FILE __stringify(HYP_EVENT_FILE)
+#endif
+
+#undef HYP_EVENT
+#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \
+ atomic_t __ro_after_init __name##_enabled =3D ATOMIC_INIT(0); \
+ struct hyp_event_id hyp_event_id_##__name \
+ __section(".hyp.event_ids."#__name) =3D { \
+ .data =3D (void *)&__name##_enabled, \
+ }
+
+#define HYP_EVENT_MULTI_READ
+#include __HYP_EVENT_FILE
+#undef HYP_EVENT_MULTI_READ
+
+#undef HYP_EVENT
diff --git a/arch/arm64/kvm/hyp/include/nvhe/trace.h b/arch/arm64/kvm/hyp/i=
nclude/nvhe/trace.h
index 1004e1edf24f..f29695b2df4c 100644
--- a/arch/arm64/kvm/hyp/include/nvhe/trace.h
+++ b/arch/arm64/kvm/hyp/include/nvhe/trace.h
@@ -2,6 +2,7 @@
#ifndef __ARM64_KVM_HYP_NVHE_TRACE_H
#define __ARM64_KVM_HYP_NVHE_TRACE_H
#include <asm/kvm_hyptrace.h>
+#include <asm/kvm_hypevents_defs.h>
=20
/* Internal struct that needs export for hyp-constants.c */
struct hyp_buffer_page {
@@ -15,6 +16,24 @@ struct hyp_buffer_page {
#ifdef CONFIG_TRACING
void *tracing_reserve_entry(unsigned long length);
void tracing_commit_entry(void);
+#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \
+ HYP_EVENT_FORMAT(__name, __struct); \
+ extern atomic_t __name##_enabled; \
+ extern struct hyp_event_id hyp_event_id_##__name; \
+ static __always_inline void trace_##__name(__proto) \
+ { \
+ size_t length =3D sizeof(struct trace_hyp_format_##__name); \
+ struct trace_hyp_format_##__name *__entry; \
+ \
+ if (!atomic_read(&__name##_enabled)) \
+ return; \
+ __entry =3D tracing_reserve_entry(length); \
+ if (!__entry) \
+ return; \
+ __entry->hdr.id =3D hyp_event_id_##__name.id; \
+ __assign \
+ tracing_commit_entry(); \
+ }
=20
void __pkvm_update_clock_tracing(u32 mult, u32 shift, u64 epoch_ns, u64 ep=
och_cyc);
int __pkvm_load_tracing(unsigned long desc_va, size_t desc_size);
@@ -22,9 +41,12 @@ void __pkvm_teardown_tracing(void);
int __pkvm_enable_tracing(bool enable);
int __pkvm_reset_tracing(unsigned int cpu);
int __pkvm_swap_reader_tracing(unsigned int cpu);
+int __pkvm_enable_event(unsigned short id, bool enable);
#else
static inline void *tracing_reserve_entry(unsigned long length) { return N=
ULL; }
static inline void tracing_commit_entry(void) { }
+#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \
+ static inline void trace_##__name(__proto) {}
=20
static inline
void __pkvm_update_clock_tracing(u32 mult, u32 shift, u64 epoch_ns, u64 ep=
och_cyc) { }
@@ -33,5 +55,6 @@ static inline void __pkvm_teardown_tracing(void) { }
static inline int __pkvm_enable_tracing(bool enable) { return -ENODEV; }
static inline int __pkvm_reset_tracing(unsigned int cpu) { return -ENODEV;=
}
static inline int __pkvm_swap_reader_tracing(unsigned int cpu) { return -E=
NODEV; }
+static inline int __pkvm_enable_event(unsigned short id, bool enable) { r=
eturn -ENODEV; }
#endif
#endif
diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Mak=
efile
index 40f243c44cf5..fc11e47a1e90 100644
--- a/arch/arm64/kvm/hyp/nvhe/Makefile
+++ b/arch/arm64/kvm/hyp/nvhe/Makefile
@@ -28,7 +28,7 @@ hyp-obj-y :=3D timer-sr.o sysreg-sr.o debug-sr.o switch.o=
tlb.o hyp-init.o host.o
hyp-obj-y +=3D ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../en=
try.o \
../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o
hyp-obj-$(CONFIG_LIST_HARDENED) +=3D list_debug.o
-hyp-obj-$(CONFIG_TRACING) +=3D clock.o trace.o
+hyp-obj-$(CONFIG_TRACING) +=3D clock.o events.o trace.o
hyp-obj-y +=3D $(lib-objs)
=20
##
diff --git a/arch/arm64/kvm/hyp/nvhe/events.c b/arch/arm64/kvm/hyp/nvhe/eve=
nts.c
new file mode 100644
index 000000000000..ad214f3f698c
--- /dev/null
+++ b/arch/arm64/kvm/hyp/nvhe/events.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023 Google LLC
+ */
+
+#include <nvhe/mm.h>
+#include <nvhe/trace.h>
+
+#include <nvhe/define_events.h>
+
+extern struct hyp_event_id __hyp_event_ids_start[];
+extern struct hyp_event_id __hyp_event_ids_end[];
+
+int __pkvm_enable_event(unsigned short id, bool enable)
+{
+ struct hyp_event_id *event_id =3D __hyp_event_ids_start;
+ atomic_t *enable_key;
+
+ for (; (unsigned long)event_id < (unsigned long)__hyp_event_ids_end;
+ event_id++) {
+ if (event_id->id !=3D id)
+ continue;
+
+ enable_key =3D (atomic_t *)event_id->data;
+ enable_key =3D hyp_fixmap_map(__hyp_pa(enable_key));
+
+ atomic_set(enable_key, enable);
+
+ hyp_fixmap_unmap();
+
+ return 0;
+ }
+
+ return -EINVAL;
+}
diff --git a/arch/arm64/kvm/hyp/nvhe/ffa.c b/arch/arm64/kvm/hyp/nvhe/ffa.c
index e433dfab882a..6c740f8dcf82 100644
--- a/arch/arm64/kvm/hyp/nvhe/ffa.c
+++ b/arch/arm64/kvm/hyp/nvhe/ffa.c
@@ -26,10 +26,10 @@
* the duration and are therefore serialised.
*/
=20
-#include <linux/arm-smccc.h>
#include <linux/arm_ffa.h>
#include <asm/kvm_pkvm.h>
=20
+#include <nvhe/arm-smccc.h>
#include <nvhe/ffa.h>
#include <nvhe/mem_protect.h>
#include <nvhe/memory.h>
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h=
yp-main.c
index a69c95e1f5e2..1920af496952 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
@@ -11,6 +11,7 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_host.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_hypevents.h>
#include <asm/kvm_mmu.h>
=20
#include <nvhe/ffa.h>
@@ -427,6 +428,14 @@ static void handle___pkvm_swap_reader_tracing(struct k=
vm_cpu_context *host_ctxt)
cpu_reg(host_ctxt, 1) =3D __pkvm_swap_reader_tracing(cpu);
}
=20
+static void handle___pkvm_enable_event(struct kvm_cpu_context *host_ctxt)
+{
+ DECLARE_REG(unsigned short, id, host_ctxt, 1);
+ DECLARE_REG(bool, enable, host_ctxt, 2);
+
+ cpu_reg(host_ctxt, 1) =3D __pkvm_enable_event(id, enable);
+}
+
typedef void (*hcall_t)(struct kvm_cpu_context *);
=20
#define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] =3D (hcall_t)handle_##x
@@ -464,6 +473,7 @@ static const hcall_t host_hcall[] =3D {
HANDLE_FUNC(__pkvm_enable_tracing),
HANDLE_FUNC(__pkvm_reset_tracing),
HANDLE_FUNC(__pkvm_swap_reader_tracing),
+ HANDLE_FUNC(__pkvm_enable_event),
};
=20
static void handle_host_hcall(struct kvm_cpu_context *host_ctxt)
@@ -504,7 +514,9 @@ static void handle_host_hcall(struct kvm_cpu_context *h=
ost_ctxt)
=20
static void default_host_smc_handler(struct kvm_cpu_context *host_ctxt)
{
+ trace_hyp_exit();
__kvm_hyp_host_forward_smc(host_ctxt);
+ trace_hyp_enter();
}
=20
static void handle_host_smc(struct kvm_cpu_context *host_ctxt)
@@ -528,6 +540,8 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)
{
u64 esr =3D read_sysreg_el2(SYS_ESR);
=20
+ trace_hyp_enter();
+
switch (ESR_ELx_EC(esr)) {
case ESR_ELx_EC_HVC64:
handle_host_hcall(host_ctxt);
@@ -548,4 +562,6 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)
default:
BUG();
}
+
+ trace_hyp_exit();
}
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hy=
p.lds.S
index f4562f417d3f..2f9262057bac 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S
+++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S
@@ -16,6 +16,12 @@ SECTIONS {
HYP_SECTION(.text)
HYP_SECTION(.data..ro_after_init)
HYP_SECTION(.rodata)
+#ifdef CONFIG_TRACING
+ . =3D ALIGN(PAGE_SIZE);
+ BEGIN_HYP_SECTION(.event_ids)
+ *(SORT(.hyp.event_ids.*))
+ END_HYP_SECTION
+#endif
=20
/*
* .hyp..data..percpu needs to be page aligned to maintain the same
diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe=
/psci-relay.c
index 9c2ce1e0e99a..00bc2ab94d59 100644
--- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c
+++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c
@@ -6,11 +6,12 @@
=20
#include <asm/kvm_asm.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_hypevents.h>
#include <asm/kvm_mmu.h>
-#include <linux/arm-smccc.h>
#include <linux/kvm_host.h>
#include <uapi/linux/psci.h>
=20
+#include <nvhe/arm-smccc.h>
#include <nvhe/memory.h>
#include <nvhe/trap_handler.h>
=20
@@ -153,6 +154,7 @@ static int psci_cpu_suspend(u64 func_id, struct kvm_cpu=
_context *host_ctxt)
DECLARE_REG(u64, power_state, host_ctxt, 1);
DECLARE_REG(unsigned long, pc, host_ctxt, 2);
DECLARE_REG(unsigned long, r0, host_ctxt, 3);
+ int ret;
=20
struct psci_boot_args *boot_args;
struct kvm_nvhe_init_params *init_params;
@@ -171,9 +173,11 @@ static int psci_cpu_suspend(u64 func_id, struct kvm_cp=
u_context *host_ctxt)
* Will either return if shallow sleep state, or wake up into the entry
* point if it is a deep sleep state.
*/
- return psci_call(func_id, power_state,
- __hyp_pa(&kvm_hyp_cpu_resume),
- __hyp_pa(init_params));
+ ret =3D psci_call(func_id, power_state,
+ __hyp_pa(&kvm_hyp_cpu_resume),
+ __hyp_pa(init_params));
+
+ return ret;
}
=20
static int psci_system_suspend(u64 func_id, struct kvm_cpu_context *host_c=
txt)
@@ -205,6 +209,7 @@ asmlinkage void __noreturn __kvm_host_psci_cpu_entry(bo=
ol is_cpu_on)
struct psci_boot_args *boot_args;
struct kvm_cpu_context *host_ctxt;
=20
+ trace_hyp_enter();
host_ctxt =3D host_data_ptr(host_ctxt);
=20
if (is_cpu_on)
@@ -218,6 +223,7 @@ asmlinkage void __noreturn __kvm_host_psci_cpu_entry(bo=
ol is_cpu_on)
if (is_cpu_on)
release_boot_args(boot_args);
=20
+ trace_hyp_exit();
__host_enter(host_ctxt);
}
=20
diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/swi=
tch.c
index cc69106734ca..6a778279d32f 100644
--- a/arch/arm64/kvm/hyp/nvhe/switch.c
+++ b/arch/arm64/kvm/hyp/nvhe/switch.c
@@ -7,7 +7,6 @@
#include <hyp/switch.h>
#include <hyp/sysreg-sr.h>
=20
-#include <linux/arm-smccc.h>
#include <linux/kvm_host.h>
#include <linux/types.h>
#include <linux/jump_label.h>
@@ -21,6 +20,7 @@
#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
+#include <asm/kvm_hypevents.h>
#include <asm/kvm_mmu.h>
#include <asm/fpsimd.h>
#include <asm/debug-monitors.h>
@@ -336,10 +336,13 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
__debug_switch_to_guest(vcpu);
=20
do {
+ trace_hyp_exit();
+
/* Jump in the fire! */
exit_code =3D __guest_enter(vcpu);
=20
/* And we're baaack! */
+ trace_hyp_enter();
} while (fixup_guest_exit(vcpu, &exit_code));
=20
__sysreg_save_state_nvhe(guest_ctxt);
diff --git a/arch/arm64/kvm/hyp_events.c b/arch/arm64/kvm/hyp_events.c
new file mode 100644
index 000000000000..ec56a63f3451
--- /dev/null
+++ b/arch/arm64/kvm/hyp_events.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023 Google LLC
+ */
+
+#include <linux/tracefs.h>
+
+#include <asm/kvm_host.h>
+#include <asm/kvm_define_hypevents.h>
+#include <asm/setup.h>
+
+#include "hyp_trace.h"
+
+extern struct hyp_event __hyp_events_start[];
+extern struct hyp_event __hyp_events_end[];
+
+/* hyp_event section used by the hypervisor */
+extern struct hyp_event_id __hyp_event_ids_start[];
+extern struct hyp_event_id __hyp_event_ids_end[];
+
+static ssize_t
+hyp_event_write(struct file *filp, const char __user *ubuf, size_t cnt, lo=
ff_t *ppos)
+{
+ struct seq_file *seq_file =3D (struct seq_file *)filp->private_data;
+ struct hyp_event *evt =3D (struct hyp_event *)seq_file->private;
+ unsigned short id =3D evt->id;
+ bool enabling;
+ int ret;
+ char c;
+
+ if (!cnt || cnt > 2)
+ return -EINVAL;
+
+ if (get_user(c, ubuf))
+ return -EFAULT;
+
+ switch (c) {
+ case '1':
+ enabling =3D true;
+ break;
+ case '0':
+ enabling =3D false;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (enabling !=3D *evt->enabled) {
+ ret =3D kvm_call_hyp_nvhe(__pkvm_enable_event, id, enabling);
+ if (ret)
+ return ret;
+ }
+
+ *evt->enabled =3D enabling;
+
+ return cnt;
+}
+
+static int hyp_event_show(struct seq_file *m, void *v)
+{
+ struct hyp_event *evt =3D (struct hyp_event *)m->private;
+
+ seq_printf(m, "%d\n", *evt->enabled);
+
+ return 0;
+}
+
+static int hyp_event_open(struct inode *inode, struct file *filp)
+{
+ return single_open(filp, hyp_event_show, inode->i_private);
+}
+
+static const struct file_operations hyp_event_fops =3D {
+ .open =3D hyp_event_open,
+ .write =3D hyp_event_write,
+ .read =3D seq_read,
+ .llseek =3D seq_lseek,
+ .release =3D single_release,
+};
+
+static int hyp_event_id_show(struct seq_file *m, void *v)
+{
+ struct hyp_event *evt =3D (struct hyp_event *)m->private;
+
+ seq_printf(m, "%d\n", evt->id);
+
+ return 0;
+}
+
+static int hyp_event_id_open(struct inode *inode, struct file *filp)
+{
+ return single_open(filp, hyp_event_id_show, inode->i_private);
+}
+
+static const struct file_operations hyp_event_id_fops =3D {
+ .open =3D hyp_event_id_open,
+ .read =3D seq_read,
+ .llseek =3D seq_lseek,
+ .release =3D single_release,
+};
+
+void hyp_trace_init_event_tracefs(struct dentry *parent)
+{
+ struct hyp_event *event =3D __hyp_events_start;
+
+ parent =3D tracefs_create_dir("events", parent);
+ if (!parent) {
+ pr_err("Failed to create tracefs folder for hyp events\n");
+ return;
+ }
+
+ parent =3D tracefs_create_dir("hypervisor", parent);
+ if (!parent) {
+ pr_err("Failed to create tracefs folder for hyp events\n");
+ return;
+ }
+
+ for (; (unsigned long)event < (unsigned long)__hyp_events_end; event++) {
+ struct dentry *event_dir =3D tracefs_create_dir(event->name, parent);
+
+ if (!event_dir) {
+ pr_err("Failed to create events/hypervisor/%s\n",
+ event->name);
+ continue;
+ }
+
+ tracefs_create_file("enable", 0640, event_dir, (void *)event,
+ &hyp_event_fops);
+ tracefs_create_file("id", 0440, event_dir, (void *)event,
+ &hyp_event_id_fops);
+ }
+}
+
+struct hyp_event *hyp_trace_find_event(int id)
+{
+ struct hyp_event *event =3D __hyp_events_start + id;
+
+ if ((unsigned long)event >=3D (unsigned long)__hyp_events_end)
+ return NULL;
+
+ return event;
+}
+
+/*
+ * Register hyp events and write their id into the hyp section _hyp_event_=
ids.
+ */
+int hyp_trace_init_events(void)
+{
+ struct hyp_event_id *hyp_event_id =3D __hyp_event_ids_start;
+ struct hyp_event *event =3D __hyp_events_start;
+ int id =3D 0;
+
+ /* Events on both sides hypervisor are sorted */
+ for (; (unsigned long)event < (unsigned long)__hyp_events_end;
+ event++, hyp_event_id++, id++)
+ event->id =3D hyp_event_id->id =3D id;
+
+ return 0;
+}
diff --git a/arch/arm64/kvm/hyp_trace.c b/arch/arm64/kvm/hyp_trace.c
index 7c9548bef8f8..fef082559a19 100644
--- a/arch/arm64/kvm/hyp_trace.c
+++ b/arch/arm64/kvm/hyp_trace.c
@@ -6,10 +6,12 @@
=20
#include <linux/arm-smccc.h>
#include <linux/percpu-defs.h>
+#include <linux/trace_events.h>
#include <linux/tracefs.h>
=20
#include <asm/kvm_host.h>
#include <asm/kvm_hyptrace.h>
+#include <asm/kvm_hypevents_defs.h>
=20
#include "hyp_constants.h"
#include "hyp_trace.h"
@@ -570,6 +572,8 @@ static void ht_print_trace_cpu(struct ht_iterator *iter)
=20
static int ht_print_trace_fmt(struct ht_iterator *iter)
{
+ struct hyp_event *e;
+
if (iter->lost_events)
trace_seq_printf(&iter->seq, "CPU:%d [LOST %lu EVENTS]\n",
iter->ent_cpu, iter->lost_events);
@@ -577,6 +581,12 @@ static int ht_print_trace_fmt(struct ht_iterator *iter)
ht_print_trace_cpu(iter);
ht_print_trace_time(iter);
=20
+ e =3D hyp_trace_find_event(iter->ent->id);
+ if (e)
+ e->trace_func(iter);
+ else
+ trace_seq_printf(&iter->seq, "Unknown event id %d\n", iter->ent->id);
+
return trace_seq_has_overflowed(&iter->seq) ? -EOVERFLOW : 0;
};
=20
@@ -942,5 +952,7 @@ int hyp_trace_init_tracefs(void)
(void *)cpu, &hyp_trace_fops);
}
=20
+ hyp_trace_init_event_tracefs(root);
+
return 0;
}
diff --git a/arch/arm64/kvm/hyp_trace.h b/arch/arm64/kvm/hyp_trace.h
index 14fc06c625a6..3ac648415bf9 100644
--- a/arch/arm64/kvm/hyp_trace.h
+++ b/arch/arm64/kvm/hyp_trace.h
@@ -3,26 +3,13 @@
#ifndef __ARM64_KVM_HYP_TRACE_H__
#define __ARM64_KVM_HYP_TRACE_H__
=20
-#include <linux/trace_seq.h>
-#include <linux/workqueue.h>
-
-struct ht_iterator {
- struct trace_buffer *trace_buffer;
- int cpu;
- struct hyp_entry_hdr *ent;
- unsigned long lost_events;
- int ent_cpu;
- size_t ent_size;
- u64 ts;
- void *spare;
- size_t copy_leftover;
- struct trace_seq seq;
- struct delayed_work poll_work;
-};
-
#ifdef CONFIG_TRACING
int hyp_trace_init_tracefs(void);
+int hyp_trace_init_events(void);
+struct hyp_event *hyp_trace_find_event(int id);
+void hyp_trace_init_event_tracefs(struct dentry *parent);
#else
static inline int hyp_trace_init_tracefs(void) { return 0; }
+static inline int hyp_trace_init_events(void) { return 0; }
#endif
#endif
--=20
2.47.1.613.gc27f4b7a9f-goog