From nobody Mon Feb 9 14:38:02 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1611976980; cv=none; d=zohomail.com; s=zohoarc; b=B8g1FncCPe9Zx+gNfErFUDtEQlet1yDPVkmyfpuhMEVqp52TMUnqr6MPyGwz9IHGrfH/sffvL68xxR88wLa0riZX8g8zfzo9H6IojmrPw/roDedZeICK4DYZ/8jmF82VDZa7jPCMmvb0x7Laa3Ef/+6VI83Nj+/d6Nol16omFYw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611976980; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LhTFkky6+CGTSxL9+YWWFZVDPYPPU0SeZL+zIJjfq5c=; b=Ac/NuY1V1WPH/snGDnm22XeHkmfGaFj2Xy9MvdM6zl5UiIMDcijHsRCrF3IMXpr21tDi232ofZU7zd3jXRzXVei9yD2IQIOBFTsHa2Viz0DUCAZeZCWkSOGKpWrQWocSLCh+aXbGjs9jCDosur+jvFUut5IArSItIKIjjwYLLKA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611976980136450.450166901334; Fri, 29 Jan 2021 19:23:00 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.78685.143280 (Exim 4.92) (envelope-from ) id 1l5gqN-0004aN-PX; Sat, 30 Jan 2021 03:22:39 +0000 Received: by outflank-mailman (output) from mailman id 78685.143280; Sat, 30 Jan 2021 03:22:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l5gqN-0004aG-ME; Sat, 30 Jan 2021 03:22:39 +0000 Received: by outflank-mailman (input) for mailman id 78685; Sat, 30 Jan 2021 03:22:38 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l5gqM-0004Zu-LW for xen-devel@lists.xenproject.org; Sat, 30 Jan 2021 03:22:38 +0000 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id c6a0995e-e43d-4ee0-86f3-b69c2a95e844; Sat, 30 Jan 2021 03:22:36 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c6a0995e-e43d-4ee0-86f3-b69c2a95e844 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1611976956; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RvodxG/MydqCN+E4xvoiUIh25DplluKTEEVpMB71d0o=; b=TE7F7YHIzQvdPnrmsxJLWfW/57mLs9k8nZ1Zb+KRidDRlovXjCqBny1s vcNRDMTBbhpXutTQhQNPPZrFRqXNd3NK96LeSrQUc/AgI2g0fKgdVxQOp TgBVp3SuE5re9U3szdXo5Zpy9fGcyu5RmVlWv35jLtNucumnIWQ9SrZvo k=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: DrkrKzMAQa+Z0GdL1BLcbo7CJkrKLryk7+CGoketOR6//ROcYNrbD2WmRmlf7/f8amImlXtI9M zK866ocDCUFvlsdB9PP67dAdwbQmkKcgjsbQr10ZPQzbs9CfDq6jjuEnYMyW+wcUDNAblDQg8K C7+PKaa0VxIV8xj4Nrr7CN/1s7FzEyWE/8nqkhhXLKbgu9xOn9CwVRuSOcbxJmlkqvi659LxWu d8BkaLlpClP71LsyEGA/yXKIssYORG/4EmNNRaUtMG/Bxvk5GPwXRA5Dc84NuH/yQjbgrdek2I f4M= X-SBRS: 5.1 X-MesageID: 36195586 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.79,387,1602561600"; d="scan'208";a="36195586" From: Andrew Cooper To: Xen-devel CC: =?UTF-8?q?Micha=C5=82=20Leszczy=C5=84ski?= , Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu , Jun Nakajima , Kevin Tian , Tamas K Lengyel Subject: [PATCH v8 12/16] xen/domctl: Add XEN_DOMCTL_vmtrace_op Date: Sat, 30 Jan 2021 02:58:48 +0000 Message-ID: <20210130025852.12430-13-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210130025852.12430-1-andrew.cooper3@citrix.com> References: <20210130025852.12430-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) From: Micha=C5=82 Leszczy=C5=84ski Implement an interface to configure and control tracing operations. Reuse = the existing SETDEBUGGING flask vector rather than inventing a new one. Userspace using this interface is going to need platform specific knowledge anyway to interpret the contents of the trace buffer. While some operations (e.g. enable/disable) can reasonably be generic, others cannot. Provide an explicitly-platform specific pair of get/set operations to reduce API churn= as new options get added/enabled. For the VMX specific Processor Trace implementation, tolerate reading and modifying a safe subset of bits in CTL, STATUS and OUTPUT_MASK. This permi= ts userspace to control the content which gets logged, but prevents modificati= on of details such as the position/size of the output buffer. Signed-off-by: Micha=C5=82 Leszczy=C5=84ski Signed-off-by: Andrew Cooper Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Wei Liu CC: Jun Nakajima CC: Kevin Tian CC: Micha=C5=82 Leszczy=C5=84ski CC: Tamas K Lengyel v8: * Reposition mask constants. v7: * Major chop&change within the series. --- xen/arch/x86/domctl.c | 55 +++++++++++++++ xen/arch/x86/hvm/vmx/vmx.c | 155 ++++++++++++++++++++++++++++++++++++++= ++++ xen/include/asm-x86/hvm/hvm.h | 63 +++++++++++++++++ xen/include/public/domctl.h | 35 ++++++++++ xen/xsm/flask/hooks.c | 1 + 5 files changed, 309 insertions(+) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index b28cfe9817..b464465230 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -155,6 +155,55 @@ void arch_get_domain_info(const struct domain *d, info->arch_config.emulation_flags =3D d->arch.emulation_flags; } =20 +static int do_vmtrace_op(struct domain *d, struct xen_domctl_vmtrace_op *o= p, + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl) +{ + struct vcpu *v; + int rc; + + if ( !d->vmtrace_size || d =3D=3D current->domain /* No vcpu_pause() *= / ) + return -EINVAL; + + ASSERT(is_hvm_domain(d)); /* Restricted by domain creation logic. */ + + v =3D domain_vcpu(d, op->vcpu); + if ( !v ) + return -ENOENT; + + vcpu_pause(v); + switch ( op->cmd ) + { + case XEN_DOMCTL_vmtrace_enable: + case XEN_DOMCTL_vmtrace_disable: + case XEN_DOMCTL_vmtrace_reset_and_enable: + rc =3D hvm_vmtrace_control( + v, op->cmd !=3D XEN_DOMCTL_vmtrace_disable, + op->cmd =3D=3D XEN_DOMCTL_vmtrace_reset_and_enable); + break; + + case XEN_DOMCTL_vmtrace_output_position: + rc =3D hvm_vmtrace_output_position(v, &op->value); + if ( rc >=3D 0 ) + rc =3D 0; + break; + + case XEN_DOMCTL_vmtrace_get_option: + rc =3D hvm_vmtrace_get_option(v, op->key, &op->value); + break; + + case XEN_DOMCTL_vmtrace_set_option: + rc =3D hvm_vmtrace_set_option(v, op->key, op->value); + break; + + default: + rc =3D -EOPNOTSUPP; + break; + } + vcpu_unpause(v); + + return rc; +} + #define MAX_IOPORTS 0x10000 =20 long arch_do_domctl( @@ -1320,6 +1369,12 @@ long arch_do_domctl( domain_unpause(d); break; =20 + case XEN_DOMCTL_vmtrace_op: + ret =3D do_vmtrace_op(d, &domctl->u.vmtrace_op, u_domctl); + if ( !ret ) + copyback =3D true; + break; + default: ret =3D iommu_do_domctl(domctl, d, u_domctl); break; diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 12b961113e..a64c4e4177 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -2261,6 +2261,157 @@ static bool vmx_get_pending_event(struct vcpu *v, s= truct x86_event *info) return true; } =20 +/* + * We only let vmtrace agents see and modify a subset of bits in MSR_RTIT_= CTL. + * These all pertain to data-emitted into the trace buffer(s). Must not + * include controls pertaining to the structure/position of the trace + * buffer(s). + */ +#define RTIT_CTL_MASK \ + (RTIT_CTL_TRACE_EN | RTIT_CTL_OS | RTIT_CTL_USR | RTIT_CTL_TSC_EN | \ + RTIT_CTL_DIS_RETC | RTIT_CTL_BRANCH_EN) + +/* + * Status bits restricted to the first-gen subset (i.e. no further CPUID + * requirements.) + */ +#define RTIT_STATUS_MASK \ + (RTIT_STATUS_FILTER_EN | RTIT_STATUS_CONTEXT_EN | RTIT_STATUS_TRIGGER_= EN | \ + RTIT_STATUS_ERROR | RTIT_STATUS_STOPPED) + +static int vmtrace_get_option(struct vcpu *v, uint64_t key, uint64_t *outp= ut) +{ + const struct vcpu_msrs *msrs =3D v->arch.msrs; + + switch ( key ) + { + case MSR_RTIT_OUTPUT_MASK: + *output =3D msrs->rtit.output_mask; + break; + + case MSR_RTIT_CTL: + *output =3D msrs->rtit.ctl & RTIT_CTL_MASK; + break; + + case MSR_RTIT_STATUS: + *output =3D msrs->rtit.status & RTIT_STATUS_MASK; + break; + + default: + *output =3D 0; + return -EINVAL; + } + return 0; +} + +static int vmtrace_set_option(struct vcpu *v, uint64_t key, uint64_t value) +{ + struct vcpu_msrs *msrs =3D v->arch.msrs; + bool new_en, old_en =3D msrs->rtit.ctl & RTIT_CTL_TRACE_EN; + + switch ( key ) + { + case MSR_RTIT_OUTPUT_MASK: + /* + * MSR_RTIT_OUTPUT_MASK, when using Single Output mode, has a limit + * field in the lower 32 bits, and an offset in the upper 32 bits. + * + * Limit is fixed by the vmtrace buffer size and must not be + * controlled by userspace, while offset must be within the limit. + * + * Drop writes to the limit field to simply userspace wanting to r= eset + * the offset by writing 0. + */ + if ( (value >> 32) > msrs->rtit.output_limit ) + return -EINVAL; + msrs->rtit.output_offset =3D value >> 32; + break; + + case MSR_RTIT_CTL: + if ( value & ~RTIT_CTL_MASK ) + return -EINVAL; + + msrs->rtit.ctl &=3D ~RTIT_CTL_MASK; + msrs->rtit.ctl |=3D (value & RTIT_CTL_MASK); + break; + + case MSR_RTIT_STATUS: + if ( value & ~RTIT_STATUS_MASK ) + return -EINVAL; + + msrs->rtit.status &=3D ~RTIT_STATUS_MASK; + msrs->rtit.status |=3D (value & RTIT_STATUS_MASK); + break; + + default: + return -EINVAL; + } + + new_en =3D msrs->rtit.ctl & RTIT_CTL_TRACE_EN; + + /* ctl.trace_en changed =3D> update MSR load/save lists appropriately.= */ + if ( !old_en && new_en ) + { + if ( vmx_add_guest_msr(v, MSR_RTIT_CTL, msrs->rtit.ctl) || + vmx_add_host_load_msr(v, MSR_RTIT_CTL, 0) ) + { + /* + * The only failure cases here are failing the + * singleton-per-domain memory allocation, or exceeding the sp= ace + * in the allocation. We could unwind in principle, but there= is + * nothing userspace can usefully do to continue using this VM. + */ + domain_crash(v->domain); + return -ENXIO; + } + } + else if ( old_en && !new_en ) + { + vmx_del_msr(v, MSR_RTIT_CTL, VMX_MSR_GUEST); + vmx_del_msr(v, MSR_RTIT_CTL, VMX_MSR_HOST); + } + + return 0; +} + +static int vmtrace_control(struct vcpu *v, bool enable, bool reset) +{ + struct vcpu_msrs *msrs =3D v->arch.msrs; + uint64_t new_ctl; + int rc; + + /* + * Absolutely nothing good will come of Xen's and userspace's idea of + * whether ipt is enabled getting out of sync. + */ + if ( v->arch.hvm.vmx.ipt_active =3D=3D enable ) + return -EINVAL; + + if ( reset ) + { + msrs->rtit.status =3D 0; + msrs->rtit.output_offset =3D 0; + } + + new_ctl =3D msrs->rtit.ctl & ~RTIT_CTL_TRACE_EN; + if ( enable ) + new_ctl |=3D RTIT_CTL_TRACE_EN; + + rc =3D vmtrace_set_option(v, MSR_RTIT_CTL, new_ctl); + if ( rc ) + return rc; + + v->arch.hvm.vmx.ipt_active =3D enable; + + return 0; +} + +static int vmtrace_output_position(struct vcpu *v, uint64_t *pos) +{ + *pos =3D v->arch.msrs->rtit.output_offset; + return v->arch.hvm.vmx.ipt_active; +} + static struct hvm_function_table __initdata vmx_function_table =3D { .name =3D "VMX", .cpu_up_prepare =3D vmx_cpu_up_prepare, @@ -2316,6 +2467,10 @@ static struct hvm_function_table __initdata vmx_func= tion_table =3D { .altp2m_vcpu_update_vmfunc_ve =3D vmx_vcpu_update_vmfunc_ve, .altp2m_vcpu_emulate_ve =3D vmx_vcpu_emulate_ve, .altp2m_vcpu_emulate_vmfunc =3D vmx_vcpu_emulate_vmfunc, + .vmtrace_control =3D vmtrace_control, + .vmtrace_output_position =3D vmtrace_output_position, + .vmtrace_set_option =3D vmtrace_set_option, + .vmtrace_get_option =3D vmtrace_get_option, .tsc_scaling =3D { .max_ratio =3D VMX_TSC_MULTIPLIER_MAX, }, diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h index 334bd573b9..960ec03917 100644 --- a/xen/include/asm-x86/hvm/hvm.h +++ b/xen/include/asm-x86/hvm/hvm.h @@ -214,6 +214,12 @@ struct hvm_function_table { bool_t (*altp2m_vcpu_emulate_ve)(struct vcpu *v); int (*altp2m_vcpu_emulate_vmfunc)(const struct cpu_user_regs *regs); =20 + /* vmtrace */ + int (*vmtrace_control)(struct vcpu *v, bool enable, bool reset); + int (*vmtrace_output_position)(struct vcpu *v, uint64_t *pos); + int (*vmtrace_set_option)(struct vcpu *v, uint64_t key, uint64_t value= ); + int (*vmtrace_get_option)(struct vcpu *v, uint64_t key, uint64_t *valu= e); + /* * Parameters and callbacks for hardware-assisted TSC scaling, * which are valid only when the hardware feature is available. @@ -655,6 +661,41 @@ static inline bool altp2m_vcpu_emulate_ve(struct vcpu = *v) return false; } =20 +static inline int hvm_vmtrace_control(struct vcpu *v, bool enable, bool re= set) +{ + if ( hvm_funcs.vmtrace_control ) + return hvm_funcs.vmtrace_control(v, enable, reset); + + return -EOPNOTSUPP; +} + +/* Returns -errno, or a boolean of whether tracing is currently active. */ +static inline int hvm_vmtrace_output_position(struct vcpu *v, uint64_t *po= s) +{ + if ( hvm_funcs.vmtrace_output_position ) + return hvm_funcs.vmtrace_output_position(v, pos); + + return -EOPNOTSUPP; +} + +static inline int hvm_vmtrace_set_option( + struct vcpu *v, uint64_t key, uint64_t value) +{ + if ( hvm_funcs.vmtrace_set_option ) + return hvm_funcs.vmtrace_set_option(v, key, value); + + return -EOPNOTSUPP; +} + +static inline int hvm_vmtrace_get_option( + struct vcpu *v, uint64_t key, uint64_t *value) +{ + if ( hvm_funcs.vmtrace_get_option ) + return hvm_funcs.vmtrace_get_option(v, key, value); + + return -EOPNOTSUPP; +} + /* * This must be defined as a macro instead of an inline function, * because it uses 'struct vcpu' and 'struct domain' which have @@ -751,6 +792,28 @@ static inline bool hvm_has_set_descriptor_access_exiti= ng(void) return false; } =20 +static inline int hvm_vmtrace_control(struct vcpu *v, bool enable, bool re= set) +{ + return -EOPNOTSUPP; +} + +static inline int hvm_vmtrace_output_position(struct vcpu *v, uint64_t *po= s) +{ + return -EOPNOTSUPP; +} + +static inline int hvm_vmtrace_set_option( + struct vcpu *v, uint64_t key, uint64_t value) +{ + return -EOPNOTSUPP; +} + +static inline int hvm_vmtrace_get_option( + struct vcpu *v, uint64_t key, uint64_t *value) +{ + return -EOPNOTSUPP; +} + #define is_viridian_domain(d) ((void)(d), false) #define is_viridian_vcpu(v) ((void)(v), false) #define has_viridian_time_ref_count(d) ((void)(d), false) diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index 88a5b1ef5d..4dbf107785 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -1135,6 +1135,39 @@ struct xen_domctl_vuart_op { */ }; =20 +/* XEN_DOMCTL_vmtrace_op: Perform VM tracing operations. */ +struct xen_domctl_vmtrace_op { + uint32_t cmd; /* IN */ + uint32_t vcpu; /* IN */ + uint64_aligned_t key; /* IN - @cmd specific data. */ + uint64_aligned_t value; /* IN/OUT - @cmd specific data. */ + + /* + * General enable/disable of tracing. + * + * XEN_DOMCTL_vmtrace_reset_and_enable is provided as optimisation for + * common usecases, which want to reset status and position information + * when turning tracing back on. + */ +#define XEN_DOMCTL_vmtrace_enable 1 +#define XEN_DOMCTL_vmtrace_disable 2 +#define XEN_DOMCTL_vmtrace_reset_and_enable 3 + + /* Obtain the current output position within the buffer. Fills @value= . */ +#define XEN_DOMCTL_vmtrace_output_position 4 + + /* + * Get/Set platform specific configuration. + * + * For Intel Processor Trace, @key/@value are interpreted as MSR + * reads/writes to MSR_RTIT_*, filtered to a safe subset. + */ +#define XEN_DOMCTL_vmtrace_get_option 5 +#define XEN_DOMCTL_vmtrace_set_option 6 +}; +typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t; +DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t); + struct xen_domctl { uint32_t cmd; #define XEN_DOMCTL_createdomain 1 @@ -1219,6 +1252,7 @@ struct xen_domctl { #define XEN_DOMCTL_vuart_op 81 #define XEN_DOMCTL_get_cpu_policy 82 #define XEN_DOMCTL_set_cpu_policy 83 +#define XEN_DOMCTL_vmtrace_op 84 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -1279,6 +1313,7 @@ struct xen_domctl { struct xen_domctl_monitor_op monitor_op; struct xen_domctl_psr_alloc psr_alloc; struct xen_domctl_vuart_op vuart_op; + struct xen_domctl_vmtrace_op vmtrace_op; uint8_t pad[128]; } u; }; diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index 11784d7425..3b7313b949 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -703,6 +703,7 @@ static int flask_domctl(struct domain *d, int cmd) return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__VM_EVENT); =20 case XEN_DOMCTL_debug_op: + case XEN_DOMCTL_vmtrace_op: case XEN_DOMCTL_gdbsx_guestmemio: case XEN_DOMCTL_gdbsx_pausevcpu: case XEN_DOMCTL_gdbsx_unpausevcpu: --=20 2.11.0