From nobody Sat Nov 30 07:33:43 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 090B01AAE29 for ; Tue, 10 Sep 2024 20:04:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725998647; cv=none; b=GxBror42oKxNPJypHkNV04nQga1/4E/TtWvzlsDyRsMdOu5ingfJ1F+wlXSD++d9htPrxofIo1T0ZmZ0Iq1O3OSYz906zdFiM1ARjWzs1SsdJ6b8bg4yrAlel9QWPKBDqyQtrJLDmJBcOa9E/UNd6pehHpD65JGPJDm5I8fHmGI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725998647; c=relaxed/simple; bh=ayOCfJbbjbaGGU/QbtX3RE1/lptM6Fj5qu/dt1ZJ75Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KJ7rs7ug6wgGX710roFrIpffgSxmsvHAcUiC5JLHGn9ZDrnKU6xChFHxXMUMqyHEQIL1oW59oLNtTb+mLpLTMplQGPyQzPah80/OmNPmh3ldfowpiurtNrXckQBmBsRjdhdPhEwf+PKGHi35uzMCdSn8RJ8Ub5/Ra57P+a6+IeM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=M8pG/3Ip; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="M8pG/3Ip" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725998644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kPmYDwtae8csR+kX/Cb9MA+okEXSlun+9ETh+kgiZWA=; b=M8pG/3IpUn5k6ls81P+sB9GOO8IsAmYsghvKhZ/yfviA3ZeHj2oC6Th8YOVdq5msJBe4VV YdiVgWTwiHltZ1lugfW+c6nzwu6+KyKrBfQzgmllh9Z+cHTBtQ/aXHH5cW9AlVhLwyy1SI WivDypdpBcuTaD448Lx8P1gaDAMQf7w= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-504-NOmrHE1XPaGXsFW0ofK4mA-1; Tue, 10 Sep 2024 16:04:00 -0400 X-MC-Unique: NOmrHE1XPaGXsFW0ofK4mA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 531B019560B0; Tue, 10 Sep 2024 20:03:57 +0000 (UTC) Received: from starship.lan (unknown [10.22.64.235]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B93381956088; Tue, 10 Sep 2024 20:03:54 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: x86@kernel.org, Dave Hansen , Thomas Gleixner , Borislav Petkov , Paolo Bonzini , Ingo Molnar , Sean Christopherson , "H. Peter Anvin" , linux-kernel@vger.kernel.org, Maxim Levitsky Subject: [PATCH v5 1/3] KVM: x86: add more information to the kvm_entry tracepoint Date: Tue, 10 Sep 2024 16:03:48 -0400 Message-Id: <20240910200350.264245-2-mlevitsk@redhat.com> In-Reply-To: <20240910200350.264245-1-mlevitsk@redhat.com> References: <20240910200350.264245-1-mlevitsk@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Content-Type: text/plain; charset="utf-8" Add VMX/SVM specific interrupt injection info to vm entry tracepoint. Also add a flag showing that immediate vm exit is set to happen after the entry. Signed-off-by: Maxim Levitsky --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 5 ++++- arch/x86/kvm/svm/svm.c | 17 +++++++++++++++++ arch/x86/kvm/trace.h | 17 ++++++++++++++--- arch/x86/kvm/vmx/main.c | 1 + arch/x86/kvm/vmx/vmx.c | 11 +++++++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ 7 files changed, 52 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index 03b7e13f15bbd..af5c4d55d47bc 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -99,6 +99,7 @@ KVM_X86_OP(get_l2_tsc_multiplier) KVM_X86_OP(write_tsc_offset) KVM_X86_OP(write_tsc_multiplier) KVM_X86_OP(get_exit_info) +KVM_X86_OP(get_entry_info) KVM_X86_OP(check_intercept) KVM_X86_OP(handle_exit_irqoff) KVM_X86_OP_OPTIONAL(update_cpu_dirty_logging) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 52443ccda320f..8118f75a8a35d 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1756,13 +1756,16 @@ struct kvm_x86_ops { void (*write_tsc_multiplier)(struct kvm_vcpu *vcpu); =20 /* - * Retrieve somewhat arbitrary exit information. Intended to + * Retrieve somewhat arbitrary exit/entry information. Intended to * be used only from within tracepoints or error paths. */ void (*get_exit_info)(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, u64 *info2, u32 *exit_int_info, u32 *exit_int_info_err_code); =20 + void (*get_entry_info)(struct kvm_vcpu *vcpu, + u32 *inj_info, u32 *inj_info_error_code); + int (*check_intercept)(struct kvm_vcpu *vcpu, struct x86_instruction_info *info, enum x86_intercept_stage stage, diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9a0506ef87dfb..485f3c2826312 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -3525,6 +3525,22 @@ static void svm_get_exit_info(struct kvm_vcpu *vcpu,= u32 *reason, *error_code =3D 0; } =20 +static void svm_get_entry_info(struct kvm_vcpu *vcpu, + u32 *inj_info, + u32 *inj_info_error_code) +{ + struct vmcb_control_area *control =3D &to_svm(vcpu)->vmcb->control; + + *inj_info =3D control->event_inj; + + if ((*inj_info & SVM_EXITINTINFO_VALID) && + (*inj_info & SVM_EXITINTINFO_VALID_ERR)) + *inj_info_error_code =3D control->event_inj_err; + else + *inj_info_error_code =3D 0; + +} + static int svm_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) { struct vcpu_svm *svm =3D to_svm(vcpu); @@ -5057,6 +5073,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata =3D { .required_apicv_inhibits =3D AVIC_REQUIRED_APICV_INHIBITS, =20 .get_exit_info =3D svm_get_exit_info, + .get_entry_info =3D svm_get_entry_info, =20 .vcpu_after_set_cpuid =3D svm_vcpu_after_set_cpuid, =20 diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index d3aeffd6ae753..b4c014b1aa9a1 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -22,16 +22,27 @@ TRACE_EVENT(kvm_entry, __field( unsigned int, vcpu_id ) __field( unsigned long, rip ) __field( bool, immediate_exit ) + __field( u32, inj_info ) + __field( u32, inj_info_err ) + __field( bool, guest_mode ) ), =20 TP_fast_assign( __entry->vcpu_id =3D vcpu->vcpu_id; __entry->rip =3D kvm_rip_read(vcpu); - __entry->immediate_exit =3D force_immediate_exit; + __entry->immediate_exit =3D force_immediate_exit; + __entry->guest_mode =3D is_guest_mode(vcpu); + + static_call(kvm_x86_get_entry_info)(vcpu, + &__entry->inj_info, + &__entry->inj_info_err); ), =20 - TP_printk("vcpu %u, rip 0x%lx%s", __entry->vcpu_id, __entry->rip, - __entry->immediate_exit ? "[immediate exit]" : "") + TP_printk("vcpu %u, rip 0x%lx inj 0x%08x inj_error_code 0x%08x%s%s", + __entry->vcpu_id, __entry->rip, + __entry->inj_info, __entry->inj_info_err, + __entry->immediate_exit ? "[immediate exit]" : "", + __entry->guest_mode ? "[guest]" : "") ); =20 /* diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 7e2e78a142574..769a7ca06f59b 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -108,6 +108,7 @@ struct kvm_x86_ops vt_x86_ops __initdata =3D { .get_mt_mask =3D vmx_get_mt_mask, =20 .get_exit_info =3D vmx_get_exit_info, + .get_entry_info =3D vmx_get_entry_info, =20 .vcpu_after_set_cpuid =3D vmx_vcpu_after_set_cpuid, =20 diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 89682832dded7..af18174cc4a20 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6180,6 +6180,17 @@ void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *r= eason, } } =20 +void vmx_get_entry_info(struct kvm_vcpu *vcpu, + u32 *inj_info, + u32 *inj_info_error_code) +{ + *inj_info =3D vmcs_read32(VM_ENTRY_INTR_INFO_FIELD); + if (is_exception_with_error_code(*inj_info)) + *inj_info_error_code =3D vmcs_read32(VM_ENTRY_EXCEPTION_ERROR_CODE); + else + *inj_info_error_code =3D 0; +} + static void vmx_destroy_pml_buffer(struct vcpu_vmx *vmx) { if (vmx->pml_pg) { diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index b6a7cfc6ae317..124425997ec15 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -104,8 +104,12 @@ void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *= eoi_exit_bitmap); int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr); int vmx_set_identity_map_addr(struct kvm *kvm, u64 ident_addr); u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); + void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code); +void vmx_get_entry_info(struct kvm_vcpu *vcpu, u32 *inj_info, + u32 *inj_info_error_code); + u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu); u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu); void vmx_write_tsc_offset(struct kvm_vcpu *vcpu); --=20 2.26.3 From nobody Sat Nov 30 07:33:43 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 088BB1AAE3E for ; Tue, 10 Sep 2024 20:04:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725998648; cv=none; b=C6tY6F5R3ahgzUlZgJjz4pP+6rKMbXibBILn5SHynzbJ1LjabaUppjpqzgDMHiPMM+QFEllQuxBLmuNJeuWlHLOdkocc2GL3bhKpywHcUHFRX461XRcATzE3lTJQv4XlZqcS+QCb+eHB1kB/mzOqQPO/izL+yBaAOlOfyE8gS4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725998648; c=relaxed/simple; bh=LUyHOVPypjlR84RYyqJQSGlbhbD0ZpeGF01uBNTLrI0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=grzF2PguNm/eRe1M/00nUrV9JG1Q3pe0sB0D+rKJ008eUI7J63+ue5RrgQ8P5kD8DuHN1GpbkfkmG/DwmT5WQKm/tZNLla+kFV1Y+Dhxr3yaAfPaEAsPX+r56V6IFRgZr4iPqSMhwwWm5LEPPSQGtmEVca/EXb6Pf01n8+VMamo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=RdWo3IV8; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RdWo3IV8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725998645; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ODrX+ASCtmtGCl0uByVc2NPs4XHLuXOI/zGjtysfC8o=; b=RdWo3IV8YZNSqkwtgdnM0eLzhjQHJHt/KXiq5bB+mqQfvu3uVHkO0gwcJr2puhZoPvqazV p5XM/Xkln6qqzcSHCMBtDj/wUtSq7s9vu55cqFQ2Us0qzfzwPdNjCquoEtKyELP9yNreCb QolP3aESJHGjmeTZmFsI+0uJJ9DT1BA= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-141-OpUMxQ4EO-mLYVErOoghOg-1; Tue, 10 Sep 2024 16:04:02 -0400 X-MC-Unique: OpUMxQ4EO-mLYVErOoghOg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 06CA319560AB; Tue, 10 Sep 2024 20:04:00 +0000 (UTC) Received: from starship.lan (unknown [10.22.64.235]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7442F1956086; Tue, 10 Sep 2024 20:03:57 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: x86@kernel.org, Dave Hansen , Thomas Gleixner , Borislav Petkov , Paolo Bonzini , Ingo Molnar , Sean Christopherson , "H. Peter Anvin" , linux-kernel@vger.kernel.org, Maxim Levitsky Subject: [PATCH v5 2/3] KVM: x86: add information about pending requests to kvm_exit tracepoint Date: Tue, 10 Sep 2024 16:03:49 -0400 Message-Id: <20240910200350.264245-3-mlevitsk@redhat.com> In-Reply-To: <20240910200350.264245-1-mlevitsk@redhat.com> References: <20240910200350.264245-1-mlevitsk@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Content-Type: text/plain; charset="utf-8" This allows to gather information on how often kvm interrupts vCPUs due to specific requests. Signed-off-by: Maxim Levitsky --- arch/x86/kvm/trace.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index b4c014b1aa9a1..5a5b7757e8456 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -319,12 +319,14 @@ TRACE_EVENT(name, \ __field( u32, intr_info ) \ __field( u32, error_code ) \ __field( unsigned int, vcpu_id ) \ + __field( u64, requests ) \ ), \ \ TP_fast_assign( \ __entry->guest_rip =3D kvm_rip_read(vcpu); \ __entry->isa =3D isa; \ __entry->vcpu_id =3D vcpu->vcpu_id; \ + __entry->requests =3D READ_ONCE(vcpu->requests); \ kvm_x86_call(get_exit_info)(vcpu, \ &__entry->exit_reason, \ &__entry->info1, \ @@ -334,11 +336,13 @@ TRACE_EVENT(name, \ ), \ \ TP_printk("vcpu %u reason %s%s%s rip 0x%lx info1 0x%016llx " \ - "info2 0x%016llx intr_info 0x%08x error_code 0x%08x", \ + "info2 0x%016llx intr_info 0x%08x error_code 0x%08x " \ + "requests 0x%016llx", \ __entry->vcpu_id, \ kvm_print_exit_reason(__entry->exit_reason, __entry->isa), \ __entry->guest_rip, __entry->info1, __entry->info2, \ - __entry->intr_info, __entry->error_code) \ + __entry->intr_info, __entry->error_code, \ + __entry->requests) \ ) =20 /* --=20 2.26.3 From nobody Sat Nov 30 07:33:43 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32EFA1A7051 for ; Tue, 10 Sep 2024 20:04:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725998648; cv=none; b=NVDR4sNrgsiXjbXEUfB3wrTrvxZBNL42AcMulEWp4EfVzNsitT48NvgOSQ1dhTInTtrpALo+aSDYYF/vq6bLZ2ur9IaId3ujSw+vsaE4bWdJS15Kg/Ac6O9MRg0fm3yumD79TeuvuQ1/k2StILNT5Ak+1nicwxzSfAocIeqPe6o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725998648; c=relaxed/simple; bh=o1oABd9aLUXxcV1DQ/QsEHpcPb9GJF7E8NMrYpkXuro=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kI90w3we5EafkbJZjlqWVvC3TjqOTFP0ISxZvf7njpHprLp2qmJSymavxvOOJlu1LSKRAtg7yBIO60kGzjDB8XXml54+S0Gst1ZDBsDxZ2785zNuHTWA2Zr3irg8pJqQhinNG+EAsVyfPGcEEOXtWPfGjtYclJa3jfi+sQr/6ag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=XHHqWC4W; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XHHqWC4W" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725998646; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8/sD2LsSSzBO9zfwEwUI+IvEBIWqw3/8kbtVj9dPpIQ=; b=XHHqWC4WOwzGPypZgGT5N67jMzV2TbanSrKnCgGgs+1dm9xpgrO7BkwnIJGgJY+QE1+ie4 Ss7wbw20g693OofNqwKv0egx53aFSFOnrMn3y9MmRIvkti/xaVUigGFxMweW3F9u3mf+yR +plviToeKkscettaUUP80pyfMYa/qFM= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-301-K3cEogO0OiKYMtCAfRNfpQ-1; Tue, 10 Sep 2024 16:04:04 -0400 X-MC-Unique: K3cEogO0OiKYMtCAfRNfpQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 634A91955D4B; Tue, 10 Sep 2024 20:04:02 +0000 (UTC) Received: from starship.lan (unknown [10.22.64.235]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D85681956088; Tue, 10 Sep 2024 20:03:59 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: x86@kernel.org, Dave Hansen , Thomas Gleixner , Borislav Petkov , Paolo Bonzini , Ingo Molnar , Sean Christopherson , "H. Peter Anvin" , linux-kernel@vger.kernel.org, Maxim Levitsky Subject: [PATCH v5 3/3] KVM: x86: add new nested vmexit tracepoints Date: Tue, 10 Sep 2024 16:03:50 -0400 Message-Id: <20240910200350.264245-4-mlevitsk@redhat.com> In-Reply-To: <20240910200350.264245-1-mlevitsk@redhat.com> References: <20240910200350.264245-1-mlevitsk@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Content-Type: text/plain; charset="utf-8" Add 3 new tracepoints for nested VM exits which are intended to capture extra information to gain insights about the nested guest behavior. The new tracepoints are: - kvm_nested_msr - kvm_nested_hypercall These tracepoints capture extra register state to be able to know which MSR or which hypercall was done. - kvm_nested_page_fault This tracepoint allows to capture extra info about which host pagefault error code caused the nested page fault. Signed-off-by: Maxim Levitsky --- arch/x86/kvm/svm/nested.c | 22 +++++++++++ arch/x86/kvm/trace.h | 82 +++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/nested.c | 27 +++++++++++++ arch/x86/kvm/x86.c | 3 ++ 4 files changed, 131 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 6f704c1037e51..2020307481553 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -38,6 +38,8 @@ static void nested_svm_inject_npf_exit(struct kvm_vcpu *v= cpu, { struct vcpu_svm *svm =3D to_svm(vcpu); struct vmcb *vmcb =3D svm->vmcb; + u64 host_error_code =3D vmcb->control.exit_info_1; + =20 if (vmcb->control.exit_code !=3D SVM_EXIT_NPF) { /* @@ -48,11 +50,15 @@ static void nested_svm_inject_npf_exit(struct kvm_vcpu = *vcpu, vmcb->control.exit_code_hi =3D 0; vmcb->control.exit_info_1 =3D (1ULL << 32); vmcb->control.exit_info_2 =3D fault->address; + host_error_code =3D 0; } =20 vmcb->control.exit_info_1 &=3D ~0xffffffffULL; vmcb->control.exit_info_1 |=3D fault->error_code; =20 + trace_kvm_nested_page_fault(fault->address, host_error_code, + fault->error_code); + nested_svm_vmexit(svm); } =20 @@ -1126,6 +1132,22 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->control.exit_int_info_err, KVM_ISA_SVM); =20 + /* Collect some info about nested VM exits */ + switch (vmcb12->control.exit_code) { + case SVM_EXIT_MSR: + trace_kvm_nested_msr(vmcb12->control.exit_info_1 =3D=3D 1, + kvm_rcx_read(vcpu), + (vmcb12->save.rax & 0xFFFFFFFFull) | + (((u64)kvm_rdx_read(vcpu) << 32))); + break; + case SVM_EXIT_VMMCALL: + trace_kvm_nested_hypercall(vmcb12->save.rax, + kvm_rbx_read(vcpu), + kvm_rcx_read(vcpu), + kvm_rdx_read(vcpu)); + break; + } + kvm_vcpu_unmap(vcpu, &map, true); =20 nested_svm_transition_tlb_flush(vcpu); diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index 5a5b7757e8456..6074b4f85d5e2 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -613,7 +613,7 @@ TRACE_EVENT(kvm_pv_eoi, ); =20 /* - * Tracepoint for nested VMRUN + * Tracepoint for nested VMRUN/VMENTER */ TRACE_EVENT(kvm_nested_vmenter, TP_PROTO(__u64 rip, __u64 vmcb, __u64 nested_rip, __u32 int_ctl, @@ -746,8 +746,84 @@ TRACE_EVENT(kvm_nested_intr_vmexit, TP_printk("rip: 0x%016llx", __entry->rip) ); =20 + /* - * Tracepoint for nested #vmexit because of interrupt pending + * Tracepoint for nested guest MSR access. + */ +TRACE_EVENT(kvm_nested_msr, + TP_PROTO(bool write, u32 ecx, u64 data), + TP_ARGS(write, ecx, data), + + TP_STRUCT__entry( + __field( bool, write ) + __field( u32, ecx ) + __field( u64, data ) + ), + + TP_fast_assign( + __entry->write =3D write; + __entry->ecx =3D ecx; + __entry->data =3D data; + ), + + TP_printk("msr_%s %x =3D 0x%llx", + __entry->write ? "write" : "read", + __entry->ecx, __entry->data) +); + +/* + * Tracepoint for nested hypercalls, capturing generic info about the + * hypercall + */ + +TRACE_EVENT(kvm_nested_hypercall, + TP_PROTO(u64 rax, u64 rbx, u64 rcx, u64 rdx), + TP_ARGS(rax, rbx, rcx, rdx), + + TP_STRUCT__entry( + __field( u64, rax ) + __field( u64, rbx ) + __field( u64, rcx ) + __field( u64, rdx ) + ), + + TP_fast_assign( + __entry->rax =3D rax; + __entry->rbx =3D rbx; + __entry->rcx =3D rcx; + __entry->rdx =3D rdx; + ), + + TP_printk("rax 0x%llx rbx 0x%llx rcx 0x%llx rdx 0x%llx", + __entry->rax, __entry->rbx, __entry->rcx, __entry->rdx) +); + + +TRACE_EVENT(kvm_nested_page_fault, + TP_PROTO(u64 gpa, u64 host_error_code, u64 guest_error_code), + TP_ARGS(gpa, host_error_code, guest_error_code), + + TP_STRUCT__entry( + __field( u64, gpa ) + __field( u64, host_error_code ) + __field( u64, guest_errror_code ) + ), + + TP_fast_assign( + __entry->gpa =3D gpa; + __entry->host_error_code =3D host_error_code; + __entry->guest_errror_code =3D guest_error_code; + ), + + TP_printk("gpa 0x%llx host err 0x%llx guest err 0x%llx", + __entry->gpa, + __entry->host_error_code, + __entry->guest_errror_code) +); + + +/* + * Tracepoint for invlpga */ TRACE_EVENT(kvm_invlpga, TP_PROTO(__u64 rip, unsigned int asid, u64 address), @@ -770,7 +846,7 @@ TRACE_EVENT(kvm_invlpga, ); =20 /* - * Tracepoint for nested #vmexit because of interrupt pending + * Tracepoint for skinit */ TRACE_EVENT(kvm_skinit, TP_PROTO(__u64 rip, __u32 slb), diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 2392a7ef254df..3881a02694fc2 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -454,6 +454,16 @@ static void nested_ept_inject_page_fault(struct kvm_vc= pu *vcpu, */ nested_ept_invalidate_addr(vcpu, vmcs12->ept_pointer, fault->address); + + /* + * vmx_get_exit_qual() returns the original exit qualification, + * before it was overridden with exit qualification that + * is about to be injected to the guest. + */ + + trace_kvm_nested_page_fault(fault->address, + vmx_get_exit_qual(vcpu), + exit_qualification); } =20 nested_vmx_vmexit(vcpu, vm_exit_reason, 0, exit_qualification); @@ -4985,6 +4995,23 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm= _exit_reason, vmcs12->vm_exit_intr_error_code, KVM_ISA_VMX); =20 + switch ((u16)vmcs12->vm_exit_reason) { + case EXIT_REASON_MSR_READ: + case EXIT_REASON_MSR_WRITE: + trace_kvm_nested_msr(vmcs12->vm_exit_reason =3D=3D EXIT_REASON_MSR_WRIT= E, + kvm_rcx_read(vcpu), + (kvm_rax_read(vcpu) & 0xFFFFFFFFull) | + (((u64)kvm_rdx_read(vcpu)) << 32)); + break; + case EXIT_REASON_VMCALL: + trace_kvm_nested_hypercall(kvm_rax_read(vcpu), + kvm_rbx_read(vcpu), + kvm_rcx_read(vcpu), + kvm_rdx_read(vcpu)); + break; + + } + load_vmcs12_host_state(vcpu, vmcs12); =20 return; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f72e5d89e942d..cb01cf2ad6ac9 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -14032,6 +14032,9 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmenter); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit_inject); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_intr_vmexit); +EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_hypercall); +EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_page_fault); +EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_msr); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmenter_failed); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_invlpga); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_skinit); --=20 2.26.3