From nobody Tue Apr 7 19:43:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37163ECAAD5 for ; Mon, 29 Aug 2022 07:49:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229695AbiH2Htf (ORCPT ); Mon, 29 Aug 2022 03:49:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229619AbiH2Htb (ORCPT ); Mon, 29 Aug 2022 03:49:31 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACDE64F64F; Mon, 29 Aug 2022 00:49:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661759370; x=1693295370; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=OPaFSyKKrEh5MFAOyjBx0rCTCEFrkxjeeiRiBH7zqYY=; b=IpfkKVg2aXp6UAGiTuA0+rrkeVN4eCY8AxEY/UJvHLpfIv7MNYLq153w 9TOWZQtDcik/cVoAWmXQ7PZvDAmpKXPSTdZQYmlYPqDajFCimcAhnn1i+ YH7FbBaAcyzzB+2AKncgwrtaAxupR+pUrPoj08c50pAvds8tWCb2ycQXy +DdZZRZjhJR4WhbM460mLvVdzGOLkeEE5+B2pkto+uP27V/cZJtFZqkzo ClxC4NK+uVIPBQ77Ft8/4E17jkDhNphS907WqTCaTRCKl14QEMoKkssAb 5fc2Xguf8VyRqeFUALMF2m6Tw1AAU5O4NZvreKCfUqhyVOfQ7kkffrbHq g==; X-IronPort-AV: E=McAfee;i="6500,9779,10453"; a="295612492" X-IronPort-AV: E=Sophos;i="5.93,272,1654585200"; d="scan'208";a="295612492" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2022 00:49:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,272,1654585200"; d="scan'208";a="672281195" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by fmsmga008.fm.intel.com with ESMTP; 29 Aug 2022 00:49:28 -0700 Received: from orsmsx608.amr.corp.intel.com (10.22.229.21) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Mon, 29 Aug 2022 00:49:28 -0700 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX608.amr.corp.intel.com (10.22.229.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Mon, 29 Aug 2022 00:49:27 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Mon, 29 Aug 2022 00:49:27 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.177) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Mon, 29 Aug 2022 00:49:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kuIOghl4xnZymRBcQwzaHtmvDze7EEoonEy+iDwHX5a2E/rAK0IwLbrZsA//VttAPZGUc7PuftURP0AJ/ClDlwhoRj3+z9GHUlWHOYRE2sewxvpL2pJDvCvdr3KQ9vWMfrZUNXbmBFWrsznz9qylHNwM11Bo8GT0zCj5TAa+rC7Eh/SEFSHdJDghIEhuBXvljVI97pZCWLoRjUSq9uv2KZScsKV2rO//5ib0VGCahBbjRPKM2Ljw0svMTjhCPgcu1yOxFoRhXR4cbonN6GXYanf0Y2TmjPLkIVMyLmX/LkQPnmzun2P8haRzr8bKJkos2yaDs/p8wWozmBFBpXWkcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ECvVhbwW8T60mYiei5o5JASXMgXqtRSlUa2FQzgT4LM=; b=mZMKzN2el74f2XTO8vF2+8Frxw9Xi2xjqWtwMF0/hWmv5FWe+GNHtqwTC5tZ6GNLULgPFhxfc/DmBExcGtDT7u+8f0UbzuypOH5JzDqGsgi5nbczuU6Q0K2TFJ7TiR1zVbfbBnJDEm1IEkZGrPTvOhwy1Se0m2dl+lZZJC05pnFk0WUf4nwKKF03rWc1DTDPbk7zSgjYyT7Hrq1FoD/6Q5NE2pbKoD8hzTkH2N4BmfQoi/q9Nu4r2OrtA6/5lQMo7hiWkyBKV4Lx9XCjmlNs9fXP+8osZTbctdAdOPy89SVOEP4bPuI5zMD2Z9UhFPjwMXAxsaxRYZSLG26jYQ9e1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from CY5PR11MB6365.namprd11.prod.outlook.com (2603:10b6:930:3b::5) by DM6PR11MB4057.namprd11.prod.outlook.com (2603:10b6:5:19d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.19; Mon, 29 Aug 2022 07:49:24 +0000 Received: from CY5PR11MB6365.namprd11.prod.outlook.com ([fe80::4016:8552:5fb1:e59]) by CY5PR11MB6365.namprd11.prod.outlook.com ([fe80::4016:8552:5fb1:e59%8]) with mapi id 15.20.5566.015; Mon, 29 Aug 2022 07:49:24 +0000 From: "Wang, Wei W" To: "Li, Xiaoyao" , Peter Zijlstra , Arnaldo Carvalho de Melo , "Mark Rutland" , Alexander Shishkin , Jiri Olsa , "Namhyung Kim" , "Christopherson,, Sean" , Paolo Bonzini CC: "Li, Xiaoyao" , "linux-perf-users@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" Subject: RE: [RFC PATCH 0/2] KVM: VMX: Fix VM entry failure on PT_MODE_HOST_GUEST while host is using PT Thread-Topic: [RFC PATCH 0/2] KVM: VMX: Fix VM entry failure on PT_MODE_HOST_GUEST while host is using PT Thread-Index: AQHYuGCfKwYxaXR4306din/WULaC4K3Fff5w Date: Mon, 29 Aug 2022 07:49:24 +0000 Message-ID: References: <20220825085625.867763-1-xiaoyao.li@intel.com> In-Reply-To: <20220825085625.867763-1-xiaoyao.li@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.500.17 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 70188c2f-438a-4905-3afb-08da8992fde2 x-ms-traffictypediagnostic: DM6PR11MB4057:EE_ x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1M9+tr2lw/FVohpKTK1p8yTK68ID2TqdHecaHHz3es0ELBSw3uVw+ZmCsK+ozZJVw+KGkwNkE/rV/q64FgyY3aCga6neFNBq+2hU+Nu9HZd9NeALmjt/uJmqWevEWun834PmofTFdZeVLUbedtwD1kKfQz+xxwUjUQVmVuFdgEL1MQbSMceN6jpeysOPt0gE+iGWRINxUA75p75dvCXMBLw8PSop+peTidOAkGYEjrrcUDFZywevDUhG2VPXd8Wy+WicWgaBwyASqeK1mLYuRO9Cu0qoi2qKfbehmZkE7PlTHxPIxWsSl2gWNQdVKzNbBiOIDQ3GyU4YiEA0EL+uzbWEKYNZ+KoABJi1odiOaWmL9GjtXHYpyM5N5Li5tFpXYSNWWkq22BRxNuw5l3ze03sh7yVUs3g0nF5X8WpdSv+ceNieGWvWqXG/fMJCPpAPoWmKB5M0rWnNaNrE8871s1ZakMQ9xE403a3jTjXR9yzL5AiTKaP0R/1+YOD9+PKDbPdXz8fj8FPMCkGVhr+Xq86Wj8KB8kRP82dzNM7SqoEe4XLyIZYwBI2w+lfRmfXQLoiKsRSq5X+nuJqyBEQnF8ohG9Vad5KFJLVyrXQFE4oiQ4G/+1jRJUWdyhGqd8SFNMXbSHY5a6V4CcITwPwqzGltAcJXK4w3uNW9EpYN4nmNUjSv5JdZe5CVSYaL7kiotCLW9hLu5Aznm5534eaJKWDVzowSP7LAPuWTHOwNhsFhXM+6c+U1sC4prg8fMc02U7x652mMNEm33UI9I+hZWA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY5PR11MB6365.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(366004)(346002)(396003)(376002)(39860400002)(136003)(82960400001)(6506007)(7696005)(86362001)(53546011)(9686003)(26005)(122000001)(33656002)(186003)(478600001)(41300700001)(71200400001)(83380400001)(38070700005)(7416002)(55016003)(66446008)(4326008)(64756008)(316002)(8676002)(54906003)(76116006)(8936002)(110136005)(66946007)(52536014)(66556008)(921005)(5660300002)(66476007)(38100700002)(2906002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?IUCllLkzy33gMmFITRE4Ecq0hANWpCFFfyS3fdj1QodIWuOgkkt5leWL4tlz?= =?us-ascii?Q?CZhP1dm9jOzP6d+BF7GX6BTQEWYFhA3Evf9RxjTA92+VaKUF95He4sYAf6SD?= =?us-ascii?Q?CytgECIFYtPyNWnvk98/8uWbsaa4QigwH+P9Ar/wPHaGd7ZpavYTAVeIeILg?= =?us-ascii?Q?vBMtbe9sbf+gsFTJA/N937DvcgTHCYUA/F2ic51QhSMWpMDiSvVp8G/gUauB?= =?us-ascii?Q?72rBU5HoXkdPm++mI792HA2Ekv+ggLIlqppGcnBYoeBQVTIvh9eTC5katmbU?= =?us-ascii?Q?boIUdaDG/lWkRrjF0bxPJH1S/CTQavayW/cxq+4UxJ1erPiRcpRCRtSrqvAf?= =?us-ascii?Q?PQ3nm+1JYnwMR385+7M/3/1OEtc1nh+xofq8FneuNo1qnaXuloB/NTt6/RS9?= =?us-ascii?Q?Jc4bWNTrb8gEtXBGKV24mmAuf4IMHVJRUibN1Jub2zfVjCZdgF+DaPKdwmby?= =?us-ascii?Q?jq2Z+4QL5ismMGue0tzxP9TwxO5TlYXXN0MBtydRnQMva3JWya226zBORfsz?= =?us-ascii?Q?bBs/0QhHPQ46/CYbCbfO7iv5O0kNR6GNpLYliDFYG1FEjRjtvbi8R97sxMo0?= =?us-ascii?Q?yZ/8rsfT8Ld6VsVaS5PkQXPUQRdKMIWq+FHNuHNoqahDHlTA/Je8bYo4rAE8?= =?us-ascii?Q?xAZTN+nHdYAnoqeuGX+vfX71ktl0dyT45zXCJwYGXO0kDbxNKVL6S6ijLbZn?= =?us-ascii?Q?YiB3U2IHlbkhqvtjXcptnbhAe1e3xehhmaKj5dLJTvlDeAhlm0zv17SK8l9U?= =?us-ascii?Q?7Znd1bGnzvnaIRF9DYTQVXU6MYJ1lFLuEEUWZL2LKH3Zt4t4g13Mc1+7un1U?= =?us-ascii?Q?dj7BvTbvoMzvYWbifUYJfOT6FHw+khQUXN7LRilms6dmGKDFoauuVid/PeD8?= =?us-ascii?Q?lRNXNHsNRaAJK0fFNKwNv7AAy6WxZwwWS6GV1SbJaX94nTrP3F6l4SXfB4zM?= =?us-ascii?Q?3goTrYyp7CK1UdjU/z4yzR0uxi0VhW+UMCSDuY0zlUTjN6UZYMJDHsGChmAM?= =?us-ascii?Q?klwctDYC3I2vRSQpB1uk8+Vdqc90kPfg4mKajfkqNvSi7XjuHOUJmjk2Yrz3?= =?us-ascii?Q?Jqf9UIUXE5ZnPz3OpGjIeG2eheAkZKtIDFnLF1vKPgzO3LPHZoeBpNW2RSp9?= =?us-ascii?Q?blzpH5nCnrCVwGWs0ikQ44kiN2KMMgvE3PIQ5BmkQIipRj81Rfl/kkhWqSUm?= =?us-ascii?Q?0Jq/jGvOP7Fku303/YwHrHFDywHRylX9baoWo6qHeiUCvpnTZxVBngpVS2mv?= =?us-ascii?Q?NoUX5iTJnrcl6ixP/Ku47JzCt0m9FHz319jqsG7mgptlnX4KQ/aGgDSdrcan?= =?us-ascii?Q?IPob9RT2/3zLCC9kMOKnzI92gYWTOwHBJOG0kUOqITVcX7CrNRYcA+LlFDH9?= =?us-ascii?Q?G57lbrHr3QM+3+H3ip346aIFIIwkLQnsEC6jM6jkvkTrTBfbDqD9xsRhI57P?= =?us-ascii?Q?eiLATuubqK2tpq2m0dsIJpbC26t/Z3N3qsDX+46jyDElneX0gfSQWwzt7qFL?= =?us-ascii?Q?gPCbPLv2ilnn/F1yZyC5fsd4keoJY5PDqsj7C1HftP7r+faHNR0wTT5DKEB/?= =?us-ascii?Q?9r8rJIojGFbPFcyh3FXyyTpU9x8Cfu2vabkDZ0Mn?= Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY5PR11MB6365.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 70188c2f-438a-4905-3afb-08da8992fde2 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Aug 2022 07:49:24.3414 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: mKjk4IdscBGuaX+DcvY9eyi5OSLXLtKyxrDvaUL5w2UqiFiu/jYyNhODxHBodcU9CHfB9sRSZZmr7qA3y/IQ/w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4057 X-OriginatorOrg: intel.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" On Thursday, August 25, 2022 4:56 PM, Xiaoyao Li wrote: > There is one bug in KVM that can hit vm-entry failure 100% on platform > supporting PT_MODE_HOST_GUEST mode following below steps: >=20 > 1. #modprobe -r kvm_intel > 2. #modprobe kvm_intel pt_mode=3D1 > 3. start a VM with QEMU > 4. on host: #perf record -e intel_pt// >=20 > The vm-entry failure happens because it violates the requirement stated in > Intel SDM 26.2.1.1 VM-Execution Control Fields >=20 > If the logical processor is operating with Intel PT enabled (if > IA32_RTIT_CTL.TraceEn =3D 1) at the time of VM entry, the "load > IA32_RTIT_CTL" VM-entry control must be 0. >=20 > On PT_MODE_HOST_GUEST node, PT_MODE_HOST_GUEST is always set. Thus > KVM needs to ensure IA32_RTIT_CTL.TraceEn is 0 before VM-entry. Currently > KVM manually WRMSR(IA32_RTIT_CTL) to clear TraceEn bit. However, it > doesn't work everytime since there is a posibility that IA32_RTIT_CTL.Tra= ceEn > is re-enabled in PT PMI handler before vm-entry. This series tries to fix= the > issue by exposing two interfaces from Intel PT driver for the purose to s= top and > resume Intel PT on host. It prevents PT PMI handler from re-enabling PT. = By the > way, it also fixes another issue that PT PMI touches PT MSRs whihc leads = to > what KVM stores for host bemomes stale. I'm thinking about another approach to fixing it. I think we need to have t= he running host pt event disabled when we switch to guest and don't expect to receive the host pt interrupt at this point. Also, the host pt context can = be save/restored by host perf core (instead of KVM) when we disable/enable the event. diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c index 82ef87e9a897..1d3e03ecaf6a 100644 --- a/arch/x86/events/intel/pt.c +++ b/arch/x86/events/intel/pt.c @@ -1575,6 +1575,7 @@ static void pt_event_start(struct perf_event *event, = int mode) pt_config_buffer(buf); pt_config(event); + pt->event =3D event; return; @@ -1600,6 +1601,7 @@ static void pt_event_stop(struct perf_event *event, i= nt mode) return; event->hw.state =3D PERF_HES_STOPPED; + pt->event =3D NULL; if (mode & PERF_EF_UPDATE) { struct pt_buffer *buf =3D perf_get_aux(&pt->handle); @@ -1624,6 +1626,15 @@ static void pt_event_stop(struct perf_event *event, = int mode) } } + +struct perf_event *pt_get_curr_event(void) +{ + struct pt *pt =3D this_cpu_ptr(&pt_ctx); + + return pt->event; +} +EXPORT_SYMBOL_GPL(pt_get_curr_event); + static long pt_event_snapshot_aux(struct perf_event *event, struct perf_output_handle *handle, unsigned long size) diff --git a/arch/x86/events/intel/pt.h b/arch/x86/events/intel/pt.h index 96906a62aacd..d46a85bb06bb 100644 --- a/arch/x86/events/intel/pt.h +++ b/arch/x86/events/intel/pt.h @@ -121,6 +121,7 @@ struct pt_filters { * @output_mask: cached RTIT_OUTPUT_MASK MSR value */ struct pt { + struct perf_event *event; struct perf_output_handle handle; struct pt_filters filters; int handle_nmi; diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_= event.h index f6fc8dd51ef4..be8dd24922a7 100644 --- a/arch/x86/include/asm/perf_event.h +++ b/arch/x86/include/asm/perf_event.h @@ -553,11 +553,14 @@ static inline int x86_perf_get_lbr(struct x86_pmu_lbr= *lbr) #ifdef CONFIG_CPU_SUP_INTEL extern void intel_pt_handle_vmx(int on); + extern struct perf_event *pt_get_curr_event(void); #else static inline void intel_pt_handle_vmx(int on) { + } +struct perf_event *pt_get_curr_event(void) { } #endif #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_AMD) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index d7f8331d6f7e..195debc1bff1 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1125,37 +1125,29 @@ static inline void pt_save_msr(struct pt_ctx *ctx, = u32 addr_range) static void pt_guest_enter(struct vcpu_vmx *vmx) { - if (vmx_pt_mode_is_system()) + struct perf_event *event; + + if (vmx_pt_mode_is_system() || + !(vmx->pt_desc.guest.ctl & RTIT_CTL_TRACEEN)) return; - /* - * GUEST_IA32_RTIT_CTL is already set in the VMCS. - * Save host state before VM entry. - */ - rdmsrl(MSR_IA32_RTIT_CTL, vmx->pt_desc.host.ctl); - if (vmx->pt_desc.guest.ctl & RTIT_CTL_TRACEEN) { - wrmsrl(MSR_IA32_RTIT_CTL, 0); - pt_save_msr(&vmx->pt_desc.host, vmx->pt_desc.num_address_ra= nges); - pt_load_msr(&vmx->pt_desc.guest, vmx->pt_desc.num_address_r= anges); - } + event =3D pt_get_curr_event(); + perf_event_disable(event); + vmx->pt_desc.host_event =3D event; + pt_load_msr(&vmx->pt_desc.guest, vmx->pt_desc.num_address_ranges); } static void pt_guest_exit(struct vcpu_vmx *vmx) { - if (vmx_pt_mode_is_system()) - return; + struct perf_event *event =3D vmx->pt_desc.host_event; - if (vmx->pt_desc.guest.ctl & RTIT_CTL_TRACEEN) { - pt_save_msr(&vmx->pt_desc.guest, vmx->pt_desc.num_address_r= anges); - pt_load_msr(&vmx->pt_desc.host, vmx->pt_desc.num_address_ra= nges); - } + if (vmx_pt_mode_is_system() || + !(vmx->pt_desc.guest.ctl & RTIT_CTL_TRACEEN)) + return; - /* - * KVM requires VM_EXIT_CLEAR_IA32_RTIT_CTL to expose PT to the gue= st, - * i.e. RTIT_CTL is always cleared on VM-Exit. Restore it if neces= sary. - */ - if (vmx->pt_desc.host.ctl) - wrmsrl(MSR_IA32_RTIT_CTL, vmx->pt_desc.host.ctl); + pt_save_msr(&vmx->pt_desc.guest, vmx->pt_desc.num_address_ranges); + if (event) + perf_event_enable(event); } void vmx_set_host_fs_gs(struct vmcs_host_state *host, u16 fs_sel, u16 gs_s= el, diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 24d58c2ffaa3..4c20bdabc85b 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -66,7 +66,7 @@ struct pt_desc { u64 ctl_bitmask; u32 num_address_ranges; u32 caps[PT_CPUID_REGS_NUM * PT_CPUID_LEAVES]; - struct pt_ctx host; + struct perf_event *host_event; struct pt_ctx guest; }; From nobody Tue Apr 7 19:43:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19F96C04AA5 for ; Thu, 25 Aug 2022 08:56:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235051AbiHYI4q (ORCPT ); Thu, 25 Aug 2022 04:56:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233514AbiHYI4h (ORCPT ); Thu, 25 Aug 2022 04:56:37 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E11F4A8949; Thu, 25 Aug 2022 01:56:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661417794; x=1692953794; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xVtDh+LKAsKn8mzusCLTl5XWtgVfaTRuzuXZ6EiKo/U=; b=JEvzeOYSjqBvu4QQT4xekKZNQzm4Qmkt/2JP6B0wV2yXQUw0yVGmduq7 0b7BH/kIUpy8R4NZnM0ZmoWS84wHesvVHW35+0rpaJ3QC5uM8WJPl4h4O z3J5R7beqKXQtkgH2/BFu7nzrurZTwSs1sAkmSvIwAGIJ9OiAdcsrpoFh TOKsIYMYST4vZW8yBTnACvMSBwNkF6S4besoX62h85p3c3reDWjAmWP06 15csre5SY9VClweru025UkieIBIcU+0A64eGVuLpLQfzH3AgbQW7R3Rei YnvOCYbTh99E+DImw9fnvk9femivB+IQhxSYi9TfR16v/+90sBA/Y2Q1d A==; X-IronPort-AV: E=McAfee;i="6500,9779,10449"; a="291756602" X-IronPort-AV: E=Sophos;i="5.93,262,1654585200"; d="scan'208";a="291756602" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Aug 2022 01:56:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,262,1654585200"; d="scan'208";a="639505171" Received: from lxy-dell.sh.intel.com ([10.239.48.38]) by orsmga008.jf.intel.com with ESMTP; 25 Aug 2022 01:56:29 -0700 From: Xiaoyao Li To: Peter Zijlstra , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: Xiaoyao Li , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [RFC PATCH 1/2] perf/x86/intel/pt: Introduce intel_pt_{stop,resume}() Date: Thu, 25 Aug 2022 16:56:24 +0800 Message-Id: <20220825085625.867763-2-xiaoyao.li@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220825085625.867763-1-xiaoyao.li@intel.com> References: <20220825085625.867763-1-xiaoyao.li@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" KVM supports PT_MODE_HOST_GUEST mode for Intel PT that host and guest have separate Intel PT configurations and work independently. In that mdoe, KVM needs to context switch all the Intel PT configurations between host and guest on VM-entry and VM-exit. Before VM-entry, if Intel PT is enabled on host, KVM needs to disable it first so as to context switch the PT configurations. After VM exit, KVM needs to re-enable Intel PT for host. Currently, KVM achieves it by manually toggle MSR_IA32_RTIT_CTL.TRACEEN bit to en/dis-able Intel PT. However, PT PMI can be delivered after MSR_IA32_RTIT_CTL.TRACEEN bit is cleared. PT PMI handler changes PT MSRs and re-enable PT, that leads to 1) VM-entry failure of guest 2) KVM stores stale value of PT MSRs. To solve the problems, expose two interfaces for KVM to stop and resume the PT tracing. Signed-off-by: Xiaoyao Li --- arch/x86/events/intel/pt.c | 11 ++++++++++- arch/x86/include/asm/intel_pt.h | 6 ++++-- arch/x86/kernel/crash.c | 4 ++-- 3 files changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c index 82ef87e9a897..55fc02036ff1 100644 --- a/arch/x86/events/intel/pt.c +++ b/arch/x86/events/intel/pt.c @@ -1730,13 +1730,22 @@ static int pt_event_init(struct perf_event *event) return 0; } =20 -void cpu_emergency_stop_pt(void) +void intel_pt_stop(void) { struct pt *pt =3D this_cpu_ptr(&pt_ctx); =20 if (pt->handle.event) pt_event_stop(pt->handle.event, PERF_EF_UPDATE); } +EXPORT_SYMBOL_GPL(intel_pt_stop); + +void intel_pt_resume(void) { + struct pt *pt =3D this_cpu_ptr(&pt_ctx); + + if (pt->handle.event) + pt_event_start(pt->handle.event, 0); +} +EXPORT_SYMBOL_GPL(intel_pt_resume); =20 int is_intel_pt_event(struct perf_event *event) { diff --git a/arch/x86/include/asm/intel_pt.h b/arch/x86/include/asm/intel_p= t.h index c796e9bc98b6..fdfa4d31740c 100644 --- a/arch/x86/include/asm/intel_pt.h +++ b/arch/x86/include/asm/intel_pt.h @@ -27,12 +27,14 @@ enum pt_capabilities { }; =20 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL) -void cpu_emergency_stop_pt(void); +void intel_pt_stop(void); +void intel_pt_resume(void); extern u32 intel_pt_validate_hw_cap(enum pt_capabilities cap); extern u32 intel_pt_validate_cap(u32 *caps, enum pt_capabilities cap); extern int is_intel_pt_event(struct perf_event *event); #else -static inline void cpu_emergency_stop_pt(void) {} +static inline void intel_pt_stop(void) {} +static inline void intel_pt_resume(void) {} static inline u32 intel_pt_validate_hw_cap(enum pt_capabilities cap) { ret= urn 0; } static inline u32 intel_pt_validate_cap(u32 *caps, enum pt_capabilities ca= pability) { return 0; } static inline int is_intel_pt_event(struct perf_event *event) { return 0; } diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index 9730c88530fc..2f2f72a209c0 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -93,7 +93,7 @@ static void kdump_nmi_callback(int cpu, struct pt_regs *r= egs) /* * Disable Intel PT to stop its logging */ - cpu_emergency_stop_pt(); + intel_pt_stop(); =20 disable_local_APIC(); } @@ -158,7 +158,7 @@ void native_machine_crash_shutdown(struct pt_regs *regs) /* * Disable Intel PT to stop its logging */ - cpu_emergency_stop_pt(); + intel_pt_stop(); =20 #ifdef CONFIG_X86_IO_APIC /* Prevent crash_kexec() from deadlocking on ioapic_lock. */ --=20 2.27.0 From nobody Tue Apr 7 19:43:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 307A0C3F6B0 for ; Thu, 25 Aug 2022 08:56:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236319AbiHYI4v (ORCPT ); Thu, 25 Aug 2022 04:56:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235551AbiHYI4l (ORCPT ); Thu, 25 Aug 2022 04:56:41 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47EC6A895D; Thu, 25 Aug 2022 01:56:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661417797; x=1692953797; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zNKMrn0ocm4C3ekDPQvN7ou5Bn4K35RLPaE4whp4PAc=; b=KRBWBb11oFjgm/7CU7q7EV+2lOh6ezTBg/ZK/OS9s6SAMeqFgBwI1MK9 h5Uw6Ui8eSVcpybir5Wk+BQPdwF7d27WdsUv8CGHADYtIlgfJUAwQl0qD LhLb1BiRIlTizvRw/YXjfLOGQHU9pQId04gXR+mdDfpdX++1in5ds5Bjt CQHaJsDKk/AzK7RrR2BK049iu6LrgNcwX1FnQUPOoQJcawYp70mXNjB7h e7cKTap+Fwi0//8D8IhH3i2Z8CN2tj+CUF9JGJW1iKp4ri/IsM0t/b0TA GGZtGXyb8mTALtHGVqU6T2bbgOBCt5jibJFz7tJ50Vbr7i6xkv9hktLMu A==; X-IronPort-AV: E=McAfee;i="6500,9779,10449"; a="291756607" X-IronPort-AV: E=Sophos;i="5.93,262,1654585200"; d="scan'208";a="291756607" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Aug 2022 01:56:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,262,1654585200"; d="scan'208";a="639505184" Received: from lxy-dell.sh.intel.com ([10.239.48.38]) by orsmga008.jf.intel.com with ESMTP; 25 Aug 2022 01:56:32 -0700 From: Xiaoyao Li To: Peter Zijlstra , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: Xiaoyao Li , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [RFC PATCH 2/2] KVM: VMX: Stop/resume host PT before/after VM entry when PT_MODE_HOST_GUEST Date: Thu, 25 Aug 2022 16:56:25 +0800 Message-Id: <20220825085625.867763-3-xiaoyao.li@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220825085625.867763-1-xiaoyao.li@intel.com> References: <20220825085625.867763-1-xiaoyao.li@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Current implementation in pt_guest_enter() has two issues when pt mode is PT_MODE_HOST_GUEST. 1. It relies on VM_ENTRY_LOAD_IA32_RTIT_CTL to disable host's Intel PT for the case that host's RTIT_CTL_TRACEEN is 1 while guest's is 0. However, it causes VM entry failure due to violating the requirement stated in SDM "VM-Execution Control Fields" If the logical processor is operating with Intel PT enabled (if IA32_RTIT_CTL.TraceEn =3D 1) at the time of VM entry, the "load IA32_RTIT_CTL" VM-entry control must be 0. 2. In the case both host and guest enable Intel PT, it disables host's Intel PT by manually clearing MSR_IA32_RTIT_CTL for the purpose to context switch host and guest's PT configurations. However, PT PMI can be delivered later and before VM entry. In the PT PMI handler, it will a) update the host PT MSRs which leads to what KVM stores in vmx->pt_desc.host becomes stale, and b) re-enable Intel PT which leads to VM entry failure as #1. To fix the above two issues, call intel_pt_stop() exposed by Intel PT driver to disable Intel PT of host unconditionally, it can ensure MSR_IA32_RTIT_CTL.TraceEn is 0 and following PT PMI does nothing. As paired, call intel_pt_resume() after VM exit. Signed-off-by: Xiaoyao Li --- arch/x86/kvm/vmx/vmx.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index d7f8331d6f7e..3e9ce8f600d2 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -38,6 +38,7 @@ #include #include #include +#include #include #include #include @@ -1128,13 +1129,19 @@ static void pt_guest_enter(struct vcpu_vmx *vmx) if (vmx_pt_mode_is_system()) return; =20 + /* + * Stop Intel PT on host to avoid vm-entry failure since + * VM_ENTRY_LOAD_IA32_RTIT_CTL is set + */ + intel_pt_stop(); + /* * GUEST_IA32_RTIT_CTL is already set in the VMCS. * Save host state before VM entry. */ rdmsrl(MSR_IA32_RTIT_CTL, vmx->pt_desc.host.ctl); if (vmx->pt_desc.guest.ctl & RTIT_CTL_TRACEEN) { - wrmsrl(MSR_IA32_RTIT_CTL, 0); + /* intel_pt_stop() ensures RTIT_CTL.TraceEn is zero */ pt_save_msr(&vmx->pt_desc.host, vmx->pt_desc.num_address_ranges); pt_load_msr(&vmx->pt_desc.guest, vmx->pt_desc.num_address_ranges); } @@ -1156,6 +1163,8 @@ static void pt_guest_exit(struct vcpu_vmx *vmx) */ if (vmx->pt_desc.host.ctl) wrmsrl(MSR_IA32_RTIT_CTL, vmx->pt_desc.host.ctl); + + intel_pt_resume(); } =20 void vmx_set_host_fs_gs(struct vmcs_host_state *host, u16 fs_sel, u16 gs_s= el, --=20 2.27.0