From nobody Fri Dec 19 22:06:05 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 613A81F4CAC for ; Fri, 16 May 2025 13:41:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747402878; cv=none; b=rDb9wn8FFUCmB1wESb5yzA3XVgdA58IY9aahyn+PPeA1Mcc8begQJjPKb9Gy1UV37LqgXtK+g3bi/C1KFmhyrTLvNqr3SJTz0SdvZKca1v+0/DhXR17wLoTZIKQe46hnHUVZxULbY+2RWk9LZH3hs3H/j08h3RwMv5xpahKqG48= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747402878; c=relaxed/simple; bh=zf6QMv4zMw43Hm+hUkMwWqy2pT5lDG0u4FIIluK514k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aR3v8bfgCBr8PUezxdzus+rcnzhvNcd3vTUBrDehcl9YASAzRwlfBTkHFXFjYGzTV4bICXUqCqZFcGQQwA6UUY/a1GHbYoWrj8cgHnjUSWNFj2zj9o6lOKg+vjYlc7m71vMlI0VdJdzjDwJqPtUq1mR/u2bpFSxW4d83jJfLNqM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--vdonnefort.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LdNO4i55; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--vdonnefort.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LdNO4i55" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43d209dc2d3so13013205e9.3 for ; Fri, 16 May 2025 06:41:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747402875; x=1748007675; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=50cABer3HQxhzTzMQqFgXeF8wTbU9ovRGo1oyuYUZVI=; b=LdNO4i55VVOqP7G9jIBehfsBIIyq82KEQ7B5muanL+j5kIkOYu++gKGTsAOh1zLEDN oQMqPTg6NAeNFDHdtJe4EQb8rFbgQjgJ7J1xQIBjGS5ZT8SPXV2tz84cRTKEYwgq3xut 4r6Btn/z6ucK35js4VBFkMljaoyJwXqdxY14+CZRKVSt4AsUASoa2LjLUFQjKu8sHrMg t8UFb0qpu5UOqHJa8k8UZjJJNYFAkvD7vrFn015dhdpU0488vwhoaEfRseFNUwNlb5dF VEBGqgvtddAHtA5ckXu56qlVPVvqriJavJNgZVkJiUxdHQP1c8VDYxJJf4VkE/gY71tU wtLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747402875; x=1748007675; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=50cABer3HQxhzTzMQqFgXeF8wTbU9ovRGo1oyuYUZVI=; b=uxQeFkUNR3RycT/nmO+HSpJqdN6UmXJXGsoHs+tSZmNfbm4k2z5hYa2tMPQ45rajZC qmn7YhMQuFFUxEUBbot7VzRKpszLT7bmpjBR2URsSibA/RlJCGjXN9eNeU9t9G9BL0Tf RQWeZkx7lW1lTvn+X3M40ExkzL+inpFEo0Wb9Yq0Tvevo2Zm1j5ZOR+O+pbE1vGauadm IdBo2qEh59iROERmuVeWNQjJzQATJdz+3Z2xtvFDKQayafaMbe26+/RBjBIvVZrZYMmm aWbWr8ojejw6MWmHKxAd3MHdgFbzZZg19TP6B7Cx31xCwl+c3Q0879ij9Qlu3m0klGSl rDZg== X-Forwarded-Encrypted: i=1; AJvYcCUR63DGGwQm/0YGg7r5c9dQpyC0dcRf8ExyVrLMTkuIuLBRhx1+OGeXaxVWCUsfsk7Zf7JSx57OoUO6tRA=@vger.kernel.org X-Gm-Message-State: AOJu0Yx1Ba01WYIksryN+0CZp7L8K4aqCcpjDBKRNlOuKDrikK1vxI4U oImnR8HJSnrUzq+7P5vupurC2F5RW/NTL38c3i4TdAnz8QLZhGtvdiU58Er2LZnYwKTYeaDBnrP Bdri7hTNnVHwpZePEACx2xg== X-Google-Smtp-Source: AGHT+IG6Uf92JKTYznRARr1d5t7lnTskqcViFGfEmFi1DgDEWf/mJ4DngVzJZGrUCvznPwJh+xZg+WTyCQ8cOO3+ X-Received: from wmbbd17.prod.google.com ([2002:a05:600c:1f11:b0:442:ddf8:99dc]) (user=vdonnefort job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a41:b0:43d:526:e0ce with SMTP id 5b1f17b1804b1-442fd66f0c4mr25308955e9.21.1747402874816; Fri, 16 May 2025 06:41:14 -0700 (PDT) Date: Fri, 16 May 2025 14:40:28 +0100 In-Reply-To: <20250516134031.661124-1-vdonnefort@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250516134031.661124-1-vdonnefort@google.com> X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250516134031.661124-23-vdonnefort@google.com> Subject: [PATCH v5 22/25] KVM: arm64: Add event support to the pKVM hyp and trace remote From: Vincent Donnefort To: rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, linux-trace-kernel@vger.kernel.org, maz@kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, jstultz@google.com, qperret@google.com, will@kernel.org, kernel-team@android.com, linux-kernel@vger.kernel.org, Vincent Donnefort Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow the creation of hypervisor and trace remote events with a single macro HYP_EVENT(). That macro expands in the kernel side to add all the required declarations (based on REMOTE_EVENT()) as well as in the hypervisor side to create the trace_() function. Signed-off-by: Vincent Donnefort diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_= asm.h index c40820a4b049..79019e11f529 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -93,6 +93,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_enable_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_reset_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_swap_reader_tracing, + __KVM_HOST_SMCCC_FUNC___pkvm_enable_event, }; =20 #define DECLARE_KVM_VHE_SYM(sym) extern char sym[] diff --git a/arch/arm64/include/asm/kvm_define_hypevents.h b/arch/arm64/inc= lude/asm/kvm_define_hypevents.h new file mode 100644 index 000000000000..0ef5a9eefcbe --- /dev/null +++ b/arch/arm64/include/asm/kvm_define_hypevents.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef HYP_EVENT_FILE +# undef __ARM64_KVM_HYPEVENTS_H_ +# define REMOTE_EVENT_INCLUDE_FILE arch/arm64/include/asm/kvm_hypevents.h +#else +# define REMOTE_EVENT_INCLUDE_FILE HYP_EVENT_FILE +#endif + +#define REMOTE_EVENT_SECTION "_hyp_events" + +#define HE_STRUCT(__args) __args +#define HE_PRINTK(__args...) __args +#define he_field re_field + +#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \ + REMOTE_EVENT(__name, 0, RE_STRUCT(__struct), RE_PRINTK(__printk)) + +#define HYP_EVENT_MULTI_READ + +#include diff --git a/arch/arm64/include/asm/kvm_hypevents.h b/arch/arm64/include/as= m/kvm_hypevents.h new file mode 100644 index 000000000000..d6e033c96c52 --- /dev/null +++ b/arch/arm64/include/asm/kvm_hypevents.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#if !defined(__ARM64_KVM_HYPEVENTS_H_) || defined(HYP_EVENT_MULTI_READ) +#define __ARM64_KVM_HYPEVENTS_H_ + +#ifdef __KVM_NVHE_HYPERVISOR__ +#include +#endif + +#endif diff --git a/arch/arm64/include/asm/kvm_hyptrace.h b/arch/arm64/include/asm= /kvm_hyptrace.h index 9c30a479bc36..d6e0953a07d6 100644 --- a/arch/arm64/include/asm/kvm_hyptrace.h +++ b/arch/arm64/include/asm/kvm_hyptrace.h @@ -10,4 +10,17 @@ struct hyp_trace_desc { struct trace_buffer_desc trace_buffer_desc; =20 }; + +struct hyp_event_id { + unsigned short id; + void *data; +}; + +extern struct remote_event __hyp_events_start[]; +extern struct remote_event __hyp_events_end[]; + +/* hyp_event section used by the hypervisor */ +extern struct hyp_event_id __hyp_event_ids_start[]; +extern struct hyp_event_id __hyp_event_ids_end[]; + #endif diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 798405ec48a3..6369446a48b3 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -131,6 +131,10 @@ KVM_NVHE_ALIAS(__hyp_data_start); KVM_NVHE_ALIAS(__hyp_data_end); KVM_NVHE_ALIAS(__hyp_rodata_start); KVM_NVHE_ALIAS(__hyp_rodata_end); +#ifdef CONFIG_PKVM_TRACING +KVM_NVHE_ALIAS(__hyp_event_ids_start); +KVM_NVHE_ALIAS(__hyp_event_ids_end); +#endif =20 /* pKVM static key */ KVM_NVHE_ALIAS(kvm_protected_mode_initialized); diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.ld= s.S index 7c770053f072..c9baff9d6f3a 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -13,12 +13,23 @@ *(__kvm_ex_table) \ __stop___kvm_ex_table =3D .; =20 +#ifdef CONFIG_PKVM_TRACING +#define HYPERVISOR_EVENT_IDS \ + . =3D ALIGN(PAGE_SIZE); \ + __hyp_event_ids_start =3D .; \ + *(HYP_SECTION_NAME(.event_ids)) \ + __hyp_event_ids_end =3D .; +#else +#define HYPERVISOR_EVENT_IDS +#endif + #define HYPERVISOR_RODATA_SECTIONS \ HYP_SECTION_NAME(.rodata) : { \ . =3D ALIGN(PAGE_SIZE); \ __hyp_rodata_start =3D .; \ *(HYP_SECTION_NAME(.data..ro_after_init)) \ *(HYP_SECTION_NAME(.rodata)) \ + HYPERVISOR_EVENT_IDS \ . =3D ALIGN(PAGE_SIZE); \ __hyp_rodata_end =3D .; \ } @@ -307,6 +318,13 @@ SECTIONS =20 HYPERVISOR_DATA_SECTION =20 +#ifdef CONFIG_PKVM_TRACING + .data.hyp_events : { + __hyp_events_start =3D .; + *(SORT(_hyp_events.*)) + __hyp_events_end =3D .; + } +#endif /* * Data written with the MMU off but read with the MMU on requires * cache lines to be invalidated, discarding up to a Cache Writeback diff --git a/arch/arm64/kvm/hyp/include/nvhe/define_events.h b/arch/arm64/k= vm/hyp/include/nvhe/define_events.h new file mode 100644 index 000000000000..2298b49cb355 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/nvhe/define_events.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef HYP_EVENT_FILE +# define __HYP_EVENT_FILE +#else +# define __HYP_EVENT_FILE __stringify(HYP_EVENT_FILE) +#endif + +#undef HYP_EVENT +#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \ + atomic_t __ro_after_init __name##_enabled =3D ATOMIC_INIT(0); \ + struct hyp_event_id hyp_event_id_##__name \ + __section(".hyp.event_ids."#__name) =3D { \ + .data =3D (void *)&__name##_enabled, \ + } + +#define HYP_EVENT_MULTI_READ +#include __HYP_EVENT_FILE +#undef HYP_EVENT_MULTI_READ + +#undef HYP_EVENT diff --git a/arch/arm64/kvm/hyp/include/nvhe/trace.h b/arch/arm64/kvm/hyp/i= nclude/nvhe/trace.h index 0d2732f0d406..f7b286e92853 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/trace.h +++ b/arch/arm64/kvm/hyp/include/nvhe/trace.h @@ -1,21 +1,52 @@ /* SPDX-License-Identifier: GPL-2.0-only */ #ifndef __ARM64_KVM_HYP_NVHE_TRACE_H #define __ARM64_KVM_HYP_NVHE_TRACE_H + +#include + #include =20 +#define HE_PROTO(__args...) __args + #ifdef CONFIG_PKVM_TRACING void *tracing_reserve_entry(unsigned long length); void tracing_commit_entry(void); =20 +#define HE_ASSIGN(__args...) __args +#define HE_STRUCT RE_STRUCT +#define he_field re_field + +#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \ + REMOTE_EVENT_FORMAT(__name, __struct); \ + extern atomic_t __name##_enabled; \ + extern struct hyp_event_id hyp_event_id_##__name; \ + static __always_inline void trace_##__name(__proto) \ + { \ + struct remote_event_format_##__name *__entry; \ + size_t length =3D sizeof(*__entry); \ + \ + if (!atomic_read(&__name##_enabled)) \ + return; \ + __entry =3D tracing_reserve_entry(length); \ + if (!__entry) \ + return; \ + __entry->hdr.id =3D hyp_event_id_##__name.id; \ + __assign \ + tracing_commit_entry(); \ + } + void __pkvm_update_clock_tracing(u32 mult, u32 shift, u64 epoch_ns, u64 ep= och_cyc); int __pkvm_load_tracing(unsigned long desc_va, size_t desc_size); void __pkvm_unload_tracing(void); int __pkvm_enable_tracing(bool enable); int __pkvm_reset_tracing(unsigned int cpu); int __pkvm_swap_reader_tracing(unsigned int cpu); +int __pkvm_enable_event(unsigned short id, bool enable); #else static inline void *tracing_reserve_entry(unsigned long length) { return N= ULL; } static inline void tracing_commit_entry(void) { } +#define HYP_EVENT(__name, __proto, __struct, __assign, __printk) \ + static inline void trace_##__name(__proto) {} =20 static inline void __pkvm_update_clock_tracing(u32 mult, u32 shift, u64 epoch_ns, u64 ep= och_cyc) { } @@ -24,5 +55,6 @@ static inline void __pkvm_unload_tracing(void) { } static inline int __pkvm_enable_tracing(bool enable) { return -ENODEV; } static inline int __pkvm_reset_tracing(unsigned int cpu) { return -ENODEV;= } static inline int __pkvm_swap_reader_tracing(unsigned int cpu) { return -E= NODEV; } +static inline int __pkvm_enable_event(unsigned short id, bool enable) { r= eturn -ENODEV; } #endif #endif diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Mak= efile index 6d92f58aea7e..1467c9f28558 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -28,7 +28,7 @@ hyp-obj-y :=3D timer-sr.o sysreg-sr.o debug-sr.o switch.o= tlb.o hyp-init.o host.o hyp-obj-y +=3D ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../en= try.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o hyp-obj-$(CONFIG_LIST_HARDENED) +=3D list_debug.o -hyp-obj-$(CONFIG_PKVM_TRACING) +=3D clock.o trace.o ../../../../../kernel/= trace/simple_ring_buffer.o +hyp-obj-$(CONFIG_PKVM_TRACING) +=3D clock.o trace.o ../../../../../kernel/= trace/simple_ring_buffer.o events.o hyp-obj-y +=3D $(lib-objs) =20 ## diff --git a/arch/arm64/kvm/hyp/nvhe/events.c b/arch/arm64/kvm/hyp/nvhe/eve= nts.c new file mode 100644 index 000000000000..5905b42cb0d0 --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/events.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2025 Google LLC + * Author: Vincent Donnefort + */ + +#include +#include + +#include + +extern struct hyp_event_id __hyp_event_ids_start[]; +extern struct hyp_event_id __hyp_event_ids_end[]; + +int __pkvm_enable_event(unsigned short id, bool enable) +{ + struct hyp_event_id *event_id =3D __hyp_event_ids_start; + atomic_t *enable_key; + + for (; (unsigned long)event_id < (unsigned long)__hyp_event_ids_end; + event_id++) { + if (event_id->id !=3D id) + continue; + + enable_key =3D (atomic_t *)event_id->data; + enable_key =3D hyp_fixmap_map(__hyp_pa(enable_key)); + + atomic_set(enable_key, enable); + + hyp_fixmap_unmap(); + + return 0; + } + + return -EINVAL; +} diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 65260bfd33ac..3d36be6d94ca 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -619,6 +619,14 @@ static void handle___pkvm_swap_reader_tracing(struct k= vm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) =3D __pkvm_swap_reader_tracing(cpu); } =20 +static void handle___pkvm_enable_event(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(unsigned short, id, host_ctxt, 1); + DECLARE_REG(bool, enable, host_ctxt, 2); + + cpu_reg(host_ctxt, 1) =3D __pkvm_enable_event(id, enable); +} + typedef void (*hcall_t)(struct kvm_cpu_context *); =20 #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] =3D (hcall_t)handle_##x @@ -664,6 +672,7 @@ static const hcall_t host_hcall[] =3D { HANDLE_FUNC(__pkvm_enable_tracing), HANDLE_FUNC(__pkvm_reset_tracing), HANDLE_FUNC(__pkvm_swap_reader_tracing), + HANDLE_FUNC(__pkvm_enable_event), }; =20 static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hy= p.lds.S index d724f6d69302..a68411bf4bef 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -16,6 +16,12 @@ SECTIONS { HYP_SECTION(.text) HYP_SECTION(.data..ro_after_init) HYP_SECTION(.rodata) +#ifdef CONFIG_PKVM_TRACING + . =3D ALIGN(PAGE_SIZE); + BEGIN_HYP_SECTION(.event_ids) + *(SORT(.hyp.event_ids.*)) + END_HYP_SECTION +#endif =20 /* * .hyp..data..percpu needs to be page aligned to maintain the same diff --git a/arch/arm64/kvm/hyp_trace.c b/arch/arm64/kvm/hyp_trace.c index 5c115c3a26a1..70c0939ebc6a 100644 --- a/arch/arm64/kvm/hyp_trace.c +++ b/arch/arm64/kvm/hyp_trace.c @@ -306,7 +306,7 @@ static int hyp_trace_reset(unsigned int cpu, void *priv) =20 static int hyp_trace_enable_event(unsigned short id, bool enable, void *pr= iv) { - return 0; + return kvm_call_hyp_nvhe(__pkvm_enable_event, id, enable); } =20 static int hyp_trace_clock_show(struct seq_file *m, void *v) @@ -333,10 +333,27 @@ static struct trace_remote_callbacks trace_remote_cal= lbacks =3D { .enable_event =3D hyp_trace_enable_event, }; =20 +#include + +static void hyp_trace_init_events(void) +{ + struct hyp_event_id *hyp_event_id =3D __hyp_event_ids_start; + struct remote_event *event =3D __hyp_events_start; + int id =3D 0; + + /* Events on both sides hypervisor are sorted */ + for (; (unsigned long)event < (unsigned long)__hyp_events_end; + event++, hyp_event_id++, id++) + event->id =3D hyp_event_id->id =3D id; +} + int hyp_trace_init(void) { if (!is_protected_kvm_enabled()) return 0; =20 - return trace_remote_register("hypervisor", &trace_remote_callbacks, &trac= e_buffer, NULL, 0); + hyp_trace_init_events(); + + return trace_remote_register("hypervisor", &trace_remote_callbacks, &trac= e_buffer, + __hyp_events_start, __hyp_events_end - __hyp_events_start); } --=20 2.49.0.1101.gccaa498523-goog