From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34C92C19F2A for ; Thu, 11 Aug 2022 12:31:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235085AbiHKMbF (ORCPT ); Thu, 11 Aug 2022 08:31:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235009AbiHKMbA (ORCPT ); Thu, 11 Aug 2022 08:31:00 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2050.outbound.protection.outlook.com [40.107.100.50]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C029E72; Thu, 11 Aug 2022 05:30:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nLBE9YWJIjVfvZl/E7ukixR6y21OZ/HJg+gDXqbYXIC2OAHB6FM1zKOLmxG+PwCZJ5DsguBELeaaNr2tVAUR03YcLTlGDduslsVR+J0EmERDs7gDgtPIF60/UUhjBSZdrXZP7uOVOn+x+LXEHyu57ceZ9s2w8Qok3xvnXApuO/gwgweiaD/WBQuWP1lZhsiWOIR5aeiDYXdaICX4VKwcCo7lEDL1N/hlXI4Gr8Nj5GbeJmiQkn9TaWG79IxHBGUuic4tawReAobZPQHptCdFWTr9b6cNZkcwbXqtirVGRuiXeqwS2Ty1GVEXwVel9rXb2KsAz5o9Y3b1dlH5ZKtypA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7PDyvUjd85V4Oo0il6+awqRFkkUjo7vgpY27O48QY50=; b=mwf8R1CM3b2kcZl/7waMYkbtWdf+vGLAnjCXHf2LDWaLGTZ9G4F/SGrE8+pGRk/tBNi/KN51OfzL3iwR8IVx4rnvD91SNvF+5BTrIyvLHpbdNviLtmoD7m/25PD7z2yRTsm+LliCJEuVK059GvcINv+BvsTdXFvTvyzRCNmBEOuRIQ/qulI1TcB8/M+qtZATi6EPY0cKHq9LOVYj1LZPNtujUkHKR7O6jgBQ3OpNcl+pONqTp8j4pieBQMVTp4wndZP04L8UUCtGHOzD1lOTCNwpQNWgm+FPDyfV2heYNVuPR7vh221t9luR1QfnMs0oF++82yqllkV8P2QNco608w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7PDyvUjd85V4Oo0il6+awqRFkkUjo7vgpY27O48QY50=; b=YYzlRbltQI5vqzFRJL9PELUJ6RSTVI06q15Lpu78Qb5u+j6K3+twUMwIn9A13xey9OshuDWArwIpgd2t2n4JjyTB93ycV6Bdl7weALxtkY2otm0l971j9iuVbrJ4hsknF8wbrC1N1dfkRWNKtcfB1Q6Gyj2+3IJPY3VzZvTP4yQ= Received: from MW4PR04CA0053.namprd04.prod.outlook.com (2603:10b6:303:6a::28) by DS7PR12MB6007.namprd12.prod.outlook.com (2603:10b6:8:7e::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Thu, 11 Aug 2022 12:30:55 +0000 Received: from CO1NAM11FT033.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6a:cafe::85) by MW4PR04CA0053.outlook.office365.com (2603:10b6:303:6a::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.14 via Frontend Transport; Thu, 11 Aug 2022 12:30:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT033.mail.protection.outlook.com (10.13.174.247) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:30:55 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:30:48 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 01/13] perf/x86/amd/brs: Move feature-specific functions Date: Thu, 11 Aug 2022 17:59:49 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 95f2f9ba-a382-4816-d76f-08da7b95567d X-MS-TrafficTypeDiagnostic: DS7PR12MB6007:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LuGm2iycyH8nbhAtGRwEvAzVCV/ewxoaSf73E356AV4eC+Ht0dmTmqMD5Rdm9SY3s3UOQ4ssgpI0toAYekpm7A9bhL6//3ByvZzN+BC1NcIBjhc/KSl05ExFBEdnuab/PnXoMoKfOa8UlgXQj4WGNZZChnq3Fi7aU/sWm8BT5U/erQ/LkYtXHL38JALQjlgxtG8l87V/k5B22ZmrCQ+h8Hf/c0fnxvbMdHy0gbjmM+nN0Oqu8zTVeVWbzubeXQ/RsCFD+fpSv/YrCQqRHt7W+RLxcqh4PmNgZmMPVT+SrA3LaEGv7rkndki8k+NbW63dscLq04wmOBlG7qX3vsDvue+wf+vmGh0w410mU0456XAVhrZZu08CV2l/3O1pO6X60VziFnyJndYD2oHitVof0t5jdQ8qjHb9URr1MOi06cckqKdpn5Gb3aHi+mw/OtWNlSLsVtCt27aRqmRvPbAIQbttK+04EBfcr6h4GyOyUk+JCQZV/DQpFAfsoXh1z4Ot2KOqVuC2oePHMuXqWN4W43ngQrhJPiaCKa/246jaTUFJgUF8J3sV7PPHtC2mLYPaPn33EvTgubVY4++/zZK00W93L32FqWgLgba8bBr3wg5ow8BP8RfMW5+xmtDpqHILjQJIWkoRJsovsVe0bYlJLxh7JLWZcPiVci2NQVCnI0/S1Wz7GzbyY1kgUqK/PcSstpDoPka9phldNLCquiagp/OVUYW7G3koINGmSi0hmViOlL68jyKayk8sYNsd/wzSjxsZjTVR5AKAqG7RThFCHRVETwHbmwb1uRAK7NZ5Jn0Wo4e1mDbZhid4F/efT5SiQ8eXotXK5tqgbXgyNXjKjA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(376002)(346002)(39860400002)(136003)(396003)(40470700004)(36840700001)(46966006)(316002)(110136005)(54906003)(478600001)(41300700001)(7416002)(40460700003)(40480700001)(2906002)(82310400005)(70206006)(4326008)(8676002)(70586007)(5660300002)(8936002)(44832011)(36860700001)(82740400003)(36756003)(356005)(86362001)(81166007)(186003)(16526019)(336012)(2616005)(426003)(47076005)(7696005)(26005)(83380400001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:30:55.4496 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 95f2f9ba-a382-4816-d76f-08da7b95567d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT033.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6007 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Move some of the Branch Sampling (BRS) specific functions out of the Core events sources and into the BRS sources in preparation for adding other mechanisms to record branches. Signed-off-by: Sandipan Das --- arch/x86/events/amd/brs.c | 69 ++++++++++++++++++++++++++++++++++- arch/x86/events/amd/core.c | 70 ++---------------------------------- arch/x86/events/perf_event.h | 7 ++-- 3 files changed, 76 insertions(+), 70 deletions(-) diff --git a/arch/x86/events/amd/brs.c b/arch/x86/events/amd/brs.c index bee8765a1e9b..f1bff153d945 100644 --- a/arch/x86/events/amd/brs.c +++ b/arch/x86/events/amd/brs.c @@ -81,7 +81,7 @@ static bool __init amd_brs_detect(void) * a br_sel_map. Software filtering is not supported because it would not = correlate well * with a sampling period. */ -int amd_brs_setup_filter(struct perf_event *event) +static int amd_brs_setup_filter(struct perf_event *event) { u64 type =3D event->attr.branch_sample_type; =20 @@ -96,6 +96,73 @@ int amd_brs_setup_filter(struct perf_event *event) return 0; } =20 +static inline int amd_is_brs_event(struct perf_event *e) +{ + return (e->hw.config & AMD64_RAW_EVENT_MASK) =3D=3D AMD_FAM19H_BRS_EVENT; +} + +int amd_brs_hw_config(struct perf_event *event) +{ + int ret =3D 0; + + /* + * Due to interrupt holding, BRS is not recommended in + * counting mode. + */ + if (!is_sampling_event(event)) + return -EINVAL; + + /* + * Due to the way BRS operates by holding the interrupt until + * lbr_nr entries have been captured, it does not make sense + * to allow sampling on BRS with an event that does not match + * what BRS is capturing, i.e., retired taken branches. + * Otherwise the correlation with the event's period is even + * more loose: + * + * With retired taken branch: + * Effective P =3D P + 16 + X + * With any other event: + * Effective P =3D P + Y + X + * + * Where X is the number of taken branches due to interrupt + * skid. Skid is large. + * + * Where Y is the occurences of the event while BRS is + * capturing the lbr_nr entries. + * + * By using retired taken branches, we limit the impact on the + * Y variable. We know it cannot be more than the depth of + * BRS. + */ + if (!amd_is_brs_event(event)) + return -EINVAL; + + /* + * BRS implementation does not work with frequency mode + * reprogramming of the period. + */ + if (event->attr.freq) + return -EINVAL; + /* + * The kernel subtracts BRS depth from period, so it must + * be big enough. + */ + if (event->attr.sample_period <=3D x86_pmu.lbr_nr) + return -EINVAL; + + /* + * Check if we can allow PERF_SAMPLE_BRANCH_STACK + */ + ret =3D amd_brs_setup_filter(event); + + /* only set in case of success */ + if (!ret) + event->hw.flags |=3D PERF_X86_EVENT_AMD_BRS; + + return ret; +} + /* tos =3D top of stack, i.e., last valid entry written */ static inline int amd_brs_get_tos(union amd_debug_extn_cfg *cfg) { diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index 9ac3718410ce..e32a27899e11 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -330,16 +330,8 @@ static inline bool amd_is_pair_event_code(struct hw_pe= rf_event *hwc) } } =20 -#define AMD_FAM19H_BRS_EVENT 0xc4 /* RETIRED_TAKEN_BRANCH_INSTRUCTIONS */ -static inline int amd_is_brs_event(struct perf_event *e) -{ - return (e->hw.config & AMD64_RAW_EVENT_MASK) =3D=3D AMD_FAM19H_BRS_EVENT; -} - static int amd_core_hw_config(struct perf_event *event) { - int ret =3D 0; - if (event->attr.exclude_host && event->attr.exclude_guest) /* * When HO =3D=3D GO =3D=3D 1 the hardware treats that as GO =3D=3D HO = =3D=3D 0 @@ -356,66 +348,10 @@ static int amd_core_hw_config(struct perf_event *even= t) if ((x86_pmu.flags & PMU_FL_PAIR) && amd_is_pair_event_code(&event->hw)) event->hw.flags |=3D PERF_X86_EVENT_PAIR; =20 - /* - * if branch stack is requested - */ - if (has_branch_stack(event)) { - /* - * Due to interrupt holding, BRS is not recommended in - * counting mode. - */ - if (!is_sampling_event(event)) - return -EINVAL; - - /* - * Due to the way BRS operates by holding the interrupt until - * lbr_nr entries have been captured, it does not make sense - * to allow sampling on BRS with an event that does not match - * what BRS is capturing, i.e., retired taken branches. - * Otherwise the correlation with the event's period is even - * more loose: - * - * With retired taken branch: - * Effective P =3D P + 16 + X - * With any other event: - * Effective P =3D P + Y + X - * - * Where X is the number of taken branches due to interrupt - * skid. Skid is large. - * - * Where Y is the occurences of the event while BRS is - * capturing the lbr_nr entries. - * - * By using retired taken branches, we limit the impact on the - * Y variable. We know it cannot be more than the depth of - * BRS. - */ - if (!amd_is_brs_event(event)) - return -EINVAL; + if (has_branch_stack(event)) + return amd_brs_hw_config(event); =20 - /* - * BRS implementation does not work with frequency mode - * reprogramming of the period. - */ - if (event->attr.freq) - return -EINVAL; - /* - * The kernel subtracts BRS depth from period, so it must - * be big enough. - */ - if (event->attr.sample_period <=3D x86_pmu.lbr_nr) - return -EINVAL; - - /* - * Check if we can allow PERF_SAMPLE_BRANCH_STACK - */ - ret =3D amd_brs_setup_filter(event); - - /* only set in case of success */ - if (!ret) - event->hw.flags |=3D PERF_X86_EVENT_AMD_BRS; - } - return ret; + return 0; } =20 static inline int amd_is_nb_event(struct hw_perf_event *hwc) diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h index ca2f8bfe6ff1..6d23e88d232c 100644 --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -1231,6 +1231,9 @@ static inline bool fixed_counter_disabled(int i, stru= ct pmu *pmu) int amd_pmu_init(void); =20 #ifdef CONFIG_PERF_EVENTS_AMD_BRS + +#define AMD_FAM19H_BRS_EVENT 0xc4 /* RETIRED_TAKEN_BRANCH_INSTRUCTIONS */ + int amd_brs_init(void); void amd_brs_disable(void); void amd_brs_enable(void); @@ -1239,7 +1242,7 @@ void amd_brs_disable_all(void); void amd_brs_drain(void); void amd_brs_lopwr_init(void); void amd_brs_disable_all(void); -int amd_brs_setup_filter(struct perf_event *event); +int amd_brs_hw_config(struct perf_event *event); void amd_brs_reset(void); =20 static inline void amd_pmu_brs_add(struct perf_event *event) @@ -1275,7 +1278,7 @@ static inline void amd_brs_enable(void) {} static inline void amd_brs_drain(void) {} static inline void amd_brs_lopwr_init(void) {} static inline void amd_brs_disable_all(void) {} -static inline int amd_brs_setup_filter(struct perf_event *event) +static inline int amd_brs_hw_config(struct perf_event *event) { return 0; } --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBE20C19F2D for ; Thu, 11 Aug 2022 12:31:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235009AbiHKMbm (ORCPT ); Thu, 11 Aug 2022 08:31:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235211AbiHKMbd (ORCPT ); Thu, 11 Aug 2022 08:31:33 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2053.outbound.protection.outlook.com [40.107.243.53]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DF139FE3; Thu, 11 Aug 2022 05:31:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VkxYLeddU7edHk61lS4dTxAdtgsHaccz3i/MvzkcUHVTNSQ16RTsBvlhvKYKplDnceOrUvN1TR2vn01jem2j+PSy9M11m5Tv8I/CeSiKVDI0+zwViudlqjQ/wVsMxX9nlZ2oAUo9xN9uyO7pHVb26wv8GeHuEyMQc4q2PkANWeBIdfYoY/ajaYLqYafm1l8eEWuAZBYjqZvmDhcDgcmCm9EAhFbv5H5Ne2Lg4NZ7mCJ20ERA+9wl1JrqFb9qPsx7uNWYbFLiEHWxJtyxRBMWASdlzf94aIOaXDMJBcGWVu9jVwKLF1sxrrGPsxrIEfZFiBT+EwvxHpvhO9ubOpamTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sWjE9eE7KSWqewB/5GMjPZ2l/N78vitMp32r5+alLk0=; b=SdTQw5GLu+rGbVlqhmydPw1aoAdcsyg234+yMGy/3kmUlrDupkZSNq98ylFg+bG3FqUKy5wn3ERZy2ku5N8s3gusmGeya1iRiVFnBuKxqnCJYfh55j6TWk8YQFEufsjfXiZVKe/9VFw8fC1DV8r39oT0Y5FGhPzCSNfuLpUNvTV3eoVkG1AdwM2NnVKg4ts+3+9QS0MGV+FHK6pYB4TpZBOafxzMv2wZmEIhsMOC3hIcx/q9AO990URiIL2BxXFKBBCSQ5ij2qMWKtBCzphMwM59SQN0MBZE6tD/G0fUdPSXyUNs/+sO7MCzLzUcYSOTpaTRD4GA0QZB2Awr9rmEtg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sWjE9eE7KSWqewB/5GMjPZ2l/N78vitMp32r5+alLk0=; b=5IsK41KEpg5TfA2mw8KC+X0wLiv6AKzQz2xWpCalsijqP3/OrqRz0scD4NpTpjdNHn9cEAMY6FJcGHQRtKa/UBCiW01ksFMA+j5dOY8C3iv2XU8ZbIABX3y29QeeGocNSDrjmM4o7IOHfFQBFYKAG0knJTRRuAyg1+xhQbo196U= Received: from MW4PR04CA0335.namprd04.prod.outlook.com (2603:10b6:303:8a::10) by MN2PR12MB3536.namprd12.prod.outlook.com (2603:10b6:208:104::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.17; Thu, 11 Aug 2022 12:31:29 +0000 Received: from CO1NAM11FT027.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8a:cafe::16) by MW4PR04CA0335.outlook.office365.com (2603:10b6:303:8a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.20 via Frontend Transport; Thu, 11 Aug 2022 12:31:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT027.mail.protection.outlook.com (10.13.174.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:31:28 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:31:22 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 02/13] perf/x86/amd/core: Refactor branch attributes Date: Thu, 11 Aug 2022 17:59:50 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e56ebe07-d410-4d5b-b2fa-08da7b956a35 X-MS-TrafficTypeDiagnostic: MN2PR12MB3536:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: e4POl/dOAikrykOfyNkC43sfZdpW2bK2GeIDU4FUc3NcZaIedFo2/3ON9Lp/lMIgU7z1raaoI0gI4S+kE5wHbXjQfd+lCQjdVTj9UIv6B2QPWqtZiHYBsapv72mxL0jPqHc7ZHH2rGrR9wYWvxzw7T2qacEABuJ1Ocs2MCtBF3eDgAuVxXdGyueaVFuGTqsUW5UViMA6KebHV0HXnEAoUeF6abhfY3lBxKXvVNMVa7Uzjz8QgktSPrjn5uDQxnyFtkLc8rqcm+7BH4kH35LIOYkSgvJhSb1DVvtx9PDWiQZHxScO1/y+OdCUIk061X0jLoHtUCQGuXivrkoKlBLjyRx8GQ+3GygNhNfGKpeR87kmeVwK6oXUGEwusrjz0HQuy5WCUsfdC00/pPWcQovXgI2NHlSUiA77P62Cc5U3gIqWIgPne3ex6f+Hoep/7f6XVG4PcAntdP6KCVpMMaDbh0Hc7tQbbMmXFWN4toH99SXqTUSyOJ7P60jDeaU8cdpH0JG9yA0EfzcG0QhY3TlSpoT570DaBLxctFRXJwpnb5pMLlSIJ97/30T9h328uTyCh/fLxWo2c0yuqGPYkqIYU+npHvVCmli69Ld92PoO0iA3e514zBjnfDrZlCOlSJBg5wAcsYviSU4RMjZyDIzWY4XBDO/zOg9HMH9h0kOXIatI+8AzYSEZqHY3gYc6/Eb6E1cw+q+BaO5ZTUw23ez4BZZCfE3Xmy4c7YCR93hxteg++R0TSnA+PVqmlEbzCiap1s0IUY04LDA4gXH+7Ou0nw16gmULfoPLseeem9MNRZ+0YiwOhWKcwPvemxC1kSuWNp4bnnIB6qYHek+Ws1yaOg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(136003)(346002)(396003)(376002)(39860400002)(40470700004)(36840700001)(46966006)(110136005)(54906003)(316002)(41300700001)(478600001)(40480700001)(82310400005)(70586007)(7416002)(2906002)(4326008)(8676002)(44832011)(70206006)(5660300002)(8936002)(40460700003)(83380400001)(356005)(81166007)(86362001)(36860700001)(426003)(2616005)(336012)(186003)(82740400003)(16526019)(47076005)(6666004)(26005)(36756003)(7696005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:31:28.6875 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e56ebe07-d410-4d5b-b2fa-08da7b956a35 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT027.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3536 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" AMD processors that are capable of recording branches support either Branch Sampling (BRS) or Last Branch Record (LBR). In preparation for adding Last Branch Record Extension Version 2 (LbrExtV2) support, reuse the "branches" capability to advertise information about both BRS and LBR but make the "branch-brs" event exclusive to Family 19h processors that support BRS. Signed-off-by: Sandipan Das --- arch/x86/events/amd/core.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index e32a27899e11..2f524cf84528 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -1247,23 +1247,25 @@ static ssize_t branches_show(struct device *cdev, =20 static DEVICE_ATTR_RO(branches); =20 -static struct attribute *amd_pmu_brs_attrs[] =3D { +static struct attribute *amd_pmu_branches_attrs[] =3D { &dev_attr_branches.attr, NULL, }; =20 static umode_t -amd_brs_is_visible(struct kobject *kobj, struct attribute *attr, int i) +amd_branches_is_visible(struct kobject *kobj, struct attribute *attr, int = i) { return x86_pmu.lbr_nr ? attr->mode : 0; } =20 -static struct attribute_group group_caps_amd_brs =3D { +static struct attribute_group group_caps_amd_branches =3D { .name =3D "caps", - .attrs =3D amd_pmu_brs_attrs, - .is_visible =3D amd_brs_is_visible, + .attrs =3D amd_pmu_branches_attrs, + .is_visible =3D amd_branches_is_visible, }; =20 +#ifdef CONFIG_PERF_EVENTS_AMD_BRS + EVENT_ATTR_STR(branch-brs, amd_branch_brs, "event=3D" __stringify(AMD_FAM19H_BRS_EVENT)"\n"); =20 @@ -1272,15 +1274,26 @@ static struct attribute *amd_brs_events_attrs[] =3D= { NULL, }; =20 +static umode_t +amd_brs_is_visible(struct kobject *kobj, struct attribute *attr, int i) +{ + return static_cpu_has(X86_FEATURE_BRS) && x86_pmu.lbr_nr ? + attr->mode : 0; +} + static struct attribute_group group_events_amd_brs =3D { .name =3D "events", .attrs =3D amd_brs_events_attrs, .is_visible =3D amd_brs_is_visible, }; =20 +#endif /* CONFIG_PERF_EVENTS_AMD_BRS */ + static const struct attribute_group *amd_attr_update[] =3D { - &group_caps_amd_brs, + &group_caps_amd_branches, +#ifdef CONFIG_PERF_EVENTS_AMD_BRS &group_events_amd_brs, +#endif NULL, }; =20 --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFFE7C19F2D for ; Thu, 11 Aug 2022 12:32:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235197AbiHKMcY (ORCPT ); Thu, 11 Aug 2022 08:32:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234418AbiHKMcW (ORCPT ); Thu, 11 Aug 2022 08:32:22 -0400 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2085.outbound.protection.outlook.com [40.107.102.85]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9469BB491; Thu, 11 Aug 2022 05:32:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LIt3g4W+/oAtx0jkVSJK2L0VnZUHga+SpMvsCnPSTTvapFWZI2LXy+snZlNcJjI8GV3+/yYW7JZ9lba93erEBhVJ0mMV5jOUrMoyeY4MoY01eQVMXHeB/JcWkPzvL7M91h7pfcpuYGzRFC76A6SDDYLgWPkGDUmyIBpDC4ImMQ/b36MZZgeGeRL+/Vvo+sIeVqDol97jVHzNMzzmRU0rc95qH72/B52MRbVM8kr/fk1JezWSnU0bDeKnuJsyL5AcF2faB1kxtzuSuU5ApbwcM4FAT/w4r7WTggauFi+Qa5culek4A2Ov43B//52UKMfQPXcD6xwfmFI1Z/6s7au3pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=t0IEXx4iXNil1YsgW5Mqt4/KBc1p2ELClA/FeVZ0S9c=; b=ZJ/lHp5ihFm8DxW7pNcSITRN0wJGKMGC2OoGvoNyD1GLdSClKyn7zANa/YwA0euZHnQ4n7GEaYAgYhpjwzWGl5PekOe+vd9GY8auFEBT5aISLQ9NTZgIafzzMDs0s6XVXFKeaiyZQzedI1Ei+u9V9jDOZ4QwsbamX0zvcKuPcwVi7UPtfE8LOUK4VR262CLZSyuFAWQ6QVed3cvjK836pmlaacizkcmOVrf7+HO0LXv4/GbfJzxK7dohbhx0H0L6kKqX1VWx7CY3bIdEJbhFL5rfMrDgwbt9xF3gHI5lGCfUhwVL18gc0ZYEkMdi5Gl9Fl7oPHA9J72KSST8LcgkJQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t0IEXx4iXNil1YsgW5Mqt4/KBc1p2ELClA/FeVZ0S9c=; b=Os/YBDWCPRNCxsweXEf8En3/Kq3NbrzOY5x1i93qZ55kKs2RHcOH2axtlQQAVule38ST/o8scx+j9L/qanSK9pRBIpzoWjqAPXjRf4v8pLZN8G9QCesI8lcH7B5OqE4gdiv/vEoPU8dUGc6o0XXpipb7UcjXYKnphpMTkZ0YJQY= Received: from MW4PR04CA0233.namprd04.prod.outlook.com (2603:10b6:303:87::28) by SA1PR12MB6822.namprd12.prod.outlook.com (2603:10b6:806:25d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.15; Thu, 11 Aug 2022 12:32:15 +0000 Received: from CO1NAM11FT008.eop-nam11.prod.protection.outlook.com (2603:10b6:303:87:cafe::da) by MW4PR04CA0233.outlook.office365.com (2603:10b6:303:87::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.20 via Frontend Transport; Thu, 11 Aug 2022 12:32:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT008.mail.protection.outlook.com (10.13.175.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:32:14 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:31:41 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 03/13] perf/x86/amd/core: Add generic branch record interfaces Date: Thu, 11 Aug 2022 17:59:51 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e0c05e44-f7e8-46ad-bf6e-08da7b9585c9 X-MS-TrafficTypeDiagnostic: SA1PR12MB6822:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zwN7csurmqROEWHIKfi+5vfkjirlagh5CqyqmiD106JzmxT8ikDRSEeudrXTKn6LdxhOdY0+ptLwe65uWwR9+C3Jw1Rnwp7XMMY8kp5mpwpTscHKansLWrsAHaFJ2Mmzt8uWM0BMZb1gkewHIgsVwivK7bk3Xbue7YzLJp49vWyXt2j6zFLvSesnyCMuFoR6JwETVMWGaNZ41q5zSC+jtwt+K8mS/+0Ta93zy4mfham5MPTInNuMvaI/02iYTZmJNWx729sB/Z4s5xL75s7DoTBpBGlP5Tqmt5ojOA5QAvoC69siex+uGIgb8gbnHyeXHNJ3zxMkt1Uv2AG29fTGIE+wUnMQpP8mV/yRY9ANQ/cS9o+6gqYhRoI39zyV9R9Y4zAwVBW60zAjuhwo27VwEca71eAeyBNT0O4is9GhH2Lw1rIUu6i7XwKofZLiuxvefdpoUtK8aMjLnzGD1/KAvuDQ0RIW8Hbs+HxA2sSZ3LfA/RO26QYLzQp3KFNXeP/aPcn9c46VKvaGSlQcVUduroKLKtmSJm+JtzXp2hne7+a24KTF3hpvgYMOcqjXv9gNxaorts7/w4V1hJvrv4ZlePTNLzc0ph72YxI5SB9mJI2ty4U/DlOIi84EeCubsU94jx/t4G6r8gHo6pX00RpYioVchYQXK6R9wlukqDleEdHuia107nf6nWUJEL47vVCK4IOkn268kdPOQveBD+UwcJIstjk6qbgaW3t3BdvzALYi6rbxJPVlq2pkY2Xul/yxwNcwT1M51ZEVFUht/G//rxOQBnmmuyD54EQxRbp6d19QKLueRIA0+HabDJqQye9s+6VcBE+EIxRnrs/V4WWsrLJf9XQgG/iie8+wUZk7zlY= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(376002)(396003)(39860400002)(346002)(136003)(46966006)(36840700001)(40470700004)(86362001)(6666004)(426003)(336012)(26005)(41300700001)(83380400001)(47076005)(82310400005)(7696005)(40480700001)(2906002)(36860700001)(5660300002)(7416002)(54906003)(110136005)(316002)(356005)(4326008)(8676002)(70586007)(186003)(2616005)(478600001)(16526019)(36756003)(70206006)(81166007)(44832011)(40460700003)(8936002)(82740400003)(41533002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:32:14.9406 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e0c05e44-f7e8-46ad-bf6e-08da7b9585c9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT008.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6822 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" AMD processors that are capable of recording branches support either Branch Sampling (BRS) or Last Branch Record (LBR). In preparation for adding Last Branch Record Extension Version 2 (LbrExtV2) support, introduce new static calls which act as gateways to call into the feature-dependent functions based on what is available on the processor. Signed-off-by: Sandipan Das --- arch/x86/events/amd/core.c | 34 ++++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index 2f524cf84528..ef3520731a20 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -330,6 +330,8 @@ static inline bool amd_is_pair_event_code(struct hw_per= f_event *hwc) } } =20 +DEFINE_STATIC_CALL_RET0(amd_pmu_branch_hw_config, *x86_pmu.hw_config); + static int amd_core_hw_config(struct perf_event *event) { if (event->attr.exclude_host && event->attr.exclude_guest) @@ -349,7 +351,7 @@ static int amd_core_hw_config(struct perf_event *event) event->hw.flags |=3D PERF_X86_EVENT_PAIR; =20 if (has_branch_stack(event)) - return amd_brs_hw_config(event); + return static_call(amd_pmu_branch_hw_config)(event); =20 return 0; } @@ -518,8 +520,14 @@ static struct amd_nb *amd_alloc_nb(int cpu) return nb; } =20 +typedef void (amd_pmu_branch_reset_t)(void); +DEFINE_STATIC_CALL_NULL(amd_pmu_branch_reset, amd_pmu_branch_reset_t); + static void amd_pmu_cpu_reset(int cpu) { + if (x86_pmu.lbr_nr) + static_call(amd_pmu_branch_reset)(); + if (x86_pmu.version < 2) return; =20 @@ -576,7 +584,6 @@ static void amd_pmu_cpu_starting(int cpu) cpuc->amd_nb->nb_id =3D nb_id; cpuc->amd_nb->refcnt++; =20 - amd_brs_reset(); amd_pmu_cpu_reset(cpu); } =20 @@ -771,16 +778,20 @@ static void amd_pmu_v2_disable_all(void) amd_pmu_check_overflow(); } =20 +DEFINE_STATIC_CALL_NULL(amd_pmu_branch_add, *x86_pmu.add); + static void amd_pmu_add_event(struct perf_event *event) { if (needs_branch_stack(event)) - amd_pmu_brs_add(event); + static_call(amd_pmu_branch_add)(event); } =20 +DEFINE_STATIC_CALL_NULL(amd_pmu_branch_del, *x86_pmu.del); + static void amd_pmu_del_event(struct perf_event *event) { if (needs_branch_stack(event)) - amd_pmu_brs_del(event); + static_call(amd_pmu_branch_del)(event); } =20 /* @@ -1184,13 +1195,6 @@ static ssize_t amd_event_sysfs_show(char *page, u64 = config) return x86_event_sysfs_show(page, config, event); } =20 -static void amd_pmu_sched_task(struct perf_event_context *ctx, - bool sched_in) -{ - if (sched_in && x86_pmu.lbr_nr) - amd_pmu_brs_sched_task(ctx, sched_in); -} - static u64 amd_pmu_limit_period(struct perf_event *event, u64 left) { /* @@ -1375,8 +1379,14 @@ static int __init amd_core_pmu_init(void) */ if (boot_cpu_data.x86 >=3D 0x19 && !amd_brs_init()) { x86_pmu.get_event_constraints =3D amd_get_event_constraints_f19h; - x86_pmu.sched_task =3D amd_pmu_sched_task; + x86_pmu.sched_task =3D amd_pmu_brs_sched_task; x86_pmu.limit_period =3D amd_pmu_limit_period; + + static_call_update(amd_pmu_branch_hw_config, amd_brs_hw_config); + static_call_update(amd_pmu_branch_reset, amd_brs_reset); + static_call_update(amd_pmu_branch_add, amd_pmu_brs_add); + static_call_update(amd_pmu_branch_del, amd_pmu_brs_del); + /* * put_event_constraints callback same as Fam17h, set above */ --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADFF7C25B0C for ; Thu, 11 Aug 2022 12:33:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235207AbiHKMdU (ORCPT ); Thu, 11 Aug 2022 08:33:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234248AbiHKMdS (ORCPT ); Thu, 11 Aug 2022 08:33:18 -0400 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2068.outbound.protection.outlook.com [40.107.95.68]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED7AFDECF; Thu, 11 Aug 2022 05:33:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ClOvMSNaTdtk8HjtBVEgWCWbMr0hBb8CQaPsYIpU2VdiJwc/8/65VO1oeHzEz38X1vhtATwwOLXdKVd/qOlS4SfWD/RBnL5Ss1vq/prlevyaP0bomM32qfNMW1qSEhaykCTRf5wv5TdsjHcaTi8eWogAEHTC2c/XOwb8TPwZNnB6AcbDjS4XvKJ8eB2UJ9/BYL8FJiN607CmvU/OX+XHTBZJj8CEUc6rpk0CeoYHXLPVmTEaXRMEDLW2QAOQ08kmo8zP8F1dBWgkvQ4MRYyfKGxONsMEg7jV0KE0MChRi9byGvLh5Hj+xFsOp0AFiO8Gk/NHomQKGCap8mNBzCMKoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wvDmQPsFpUzzlzT6cpXiZvGIiABgfERnaavB60hctpY=; b=R+JiFXwFp1xHoOLb5F4TLPZeIOA3fHFuuOxPEFcThsQflr339QsvzfLKeu4b+hLPsQOW5HstydjpSjJ2onkU2flaX3w+B2HswmbUkrDa3lCm7/K+nto3cZ8QdoslSSHMysZCbmDIM6UDj/KWA1szz4Oxsa5vwbSsoJIK9VUQzL9ceKbHG1m8wPX8IXLgbxrNgPMJynAGggj4HCpbKcrIaYUQNFO9l8wevw1H2qIDA1Y3wNrTCD5NCdBJFBLS2XK1oB8gJKtobTyg9o05gEHli996mq4I66RfkngE5rR5d41+N0z3KsSzU7ENvugg30vhFa9YsRRUtF/Sl+zOL1yOAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wvDmQPsFpUzzlzT6cpXiZvGIiABgfERnaavB60hctpY=; b=qL7JYbMHlmLa6Lg7QSxLU5JxU1CjxsaYbz5gjGXZN0d7UvY1deh69DkiaM+QG707LDZGhlRS+uIhCI2DTYVwfPyRnSmawewwJoLt+Kr0eOe+YUmfMH67WcADsyQQxztmtJU8OX3NbAQ+92dWJcCj+aAUcjYFZnS7czo0VZTtoxQ= Received: from MW4PR04CA0323.namprd04.prod.outlook.com (2603:10b6:303:82::28) by DM6PR12MB2730.namprd12.prod.outlook.com (2603:10b6:5:41::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.16; Thu, 11 Aug 2022 12:33:14 +0000 Received: from CO1NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:303:82:cafe::fb) by MW4PR04CA0323.outlook.office365.com (2603:10b6:303:82::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.14 via Frontend Transport; Thu, 11 Aug 2022 12:33:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT063.mail.protection.outlook.com (10.13.175.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:33:14 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:33:08 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 04/13] x86/cpufeatures: Add LbrExtV2 feature bit Date: Thu, 11 Aug 2022 17:59:52 +0530 Message-ID: <172d2b0df39306ed77221c45ee1aa62e8ae0548d.1660211399.git.sandipan.das@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 69d00bd6-fda8-4713-93f1-08da7b95a912 X-MS-TrafficTypeDiagnostic: DM6PR12MB2730:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QCWZnRGtFhyRMzBhxXuTGYeIGT2jKAZHKJ3sVDd3Wte4HtD94Jyotd2ACvxM8LeH5Is5RpvRM9YkseB5icauhvZe2FYuzICnW8Webj/m69+YMmBXWxucKH1htxDVYQaWC5pkgrYnJ6YdrRDw/NSRTkRej5PKOy6rJvrxZ6qfviyDynsx2yFGqS/8WrnnbPFHB4VkBaTzNpxkUaWmC2ZFjjgz+/JL0y8XsVfvrU23+KGwOF9yeA8R1ZOiVEkvxVtGaZ75TgOXsrFvFAfOXKssFGUz2/9WGeMrofvn4KCoX2BVjJ/SfBUlAU71J5lE4DMalFRqxLeWep7nVM4Ybem5UMsVZZBeC/Al1prPJbVJvUiUxPo7Lb+H6JSWDgJdFwxCrbOeGG+Fu9Wl228ZVLCOKsqld4OwdQNO5VZbWLBPkiKljSwSsJBPRrEpHJ0kAJV8KkH9+ixFk3+1Z9Dse7zl3sp1CXmmRPxnLPpG1qqxGKqh81UEpN4prZUhf46q699m6uP0U+B8rLInorsI1DCKcra15HXmuRkXUhvNMpLZbPYwetEwUHxo82bb/clnVZnmidaPMLg/u9t/yw1Y17wMlojruYjeBLhMZkr7Zu1uM8iunUBj1WEzqR9vJ/iIz/e6oY70iQgWbu9d3lzZW+Bu3vPXB59oSjb+AcvPaTG23qHM1oNv0aVI/CTJPbF+mbCj2pzbs40RQTjVIQTtqqBh42kzQ4CIpxj2EI+0Cv79LfTPpCy6ClCoFKenNex/NjtzEvtRKPRyDUA/qBbvG4W/iEyYOZlIVF6XI8MT3FaksvAnFXueKwd4OFbHxzp9rOUov/3YD2CgeKhbsRQ/tkuAQA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(396003)(136003)(39860400002)(376002)(46966006)(36840700001)(40470700004)(426003)(16526019)(336012)(7416002)(5660300002)(47076005)(478600001)(186003)(86362001)(316002)(54906003)(110136005)(82740400003)(7696005)(40480700001)(26005)(2616005)(44832011)(41300700001)(6666004)(4326008)(36860700001)(81166007)(83380400001)(8936002)(70206006)(8676002)(70586007)(82310400005)(36756003)(40460700003)(356005)(2906002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:33:14.1382 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 69d00bd6-fda8-4713-93f1-08da7b95a912 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2730 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" CPUID leaf 0x80000022 i.e. ExtPerfMonAndDbg advertises some new performance monitoring features for AMD processors. Bit 1 of EAX indicates support for Last Branch Record Extension Version 2 (LbrExtV2) features. If found to be set during PMU initialization, the EBX bits of the same leaf can be used to determine the number of available LBR entries. For better utilization of feature words, LbrExtV2 is added as a scattered feature bit. Signed-off-by: Sandipan Das Acked-by: Borislav Petkov --- arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index 393f2bbb5e3a..e3fa476a24b0 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -96,7 +96,7 @@ #define X86_FEATURE_SYSCALL32 ( 3*32+14) /* "" syscall in IA32 userspace = */ #define X86_FEATURE_SYSENTER32 ( 3*32+15) /* "" sysenter in IA32 userspac= e */ #define X86_FEATURE_REP_GOOD ( 3*32+16) /* REP microcode works well */ -/* FREE! ( 3*32+17) */ +#define X86_FEATURE_LBREXT_V2 ( 3*32+17) /* AMD Last Branch Record Extens= ion Version 2 */ #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) /* "" LFENCE synchronizes RDTS= C */ #define X86_FEATURE_ACC_POWER ( 3*32+19) /* AMD Accumulated Power Mechani= sm */ #define X86_FEATURE_NOPL ( 3*32+20) /* The NOPL (0F 1F) instructions */ diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattere= d.c index dbaa8326d6f2..6be46dffddbf 100644 --- a/arch/x86/kernel/cpu/scattered.c +++ b/arch/x86/kernel/cpu/scattered.c @@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] =3D { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 }, + { X86_FEATURE_LBREXT_V2, CPUID_EAX, 1, 0x80000022, 0 }, { 0, 0, 0, 0, 0 } }; =20 --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C0A9C25B0C for ; Thu, 11 Aug 2022 12:34:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235225AbiHKMeK (ORCPT ); Thu, 11 Aug 2022 08:34:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229591AbiHKMeI (ORCPT ); Thu, 11 Aug 2022 08:34:08 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2072.outbound.protection.outlook.com [40.107.94.72]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A9ABDF38; Thu, 11 Aug 2022 05:34:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U1rvkpUnckP/tOIJ0/hmtTjpzrKyQCkJEHl9za6NxiYbVomgfV6ROa2tnVPEiGK/2r3C3BDML4kVjiBlegWLzl9JHvgeW6LQBT036A5S2A+m26Fokwji56waQ60Gcn5owMfVEcN/CNpywk2B6Eme6NcfYlS+JRPukCS/3E8bjeulW37OhTEc2PDfcB/ITarJ0oku6OUlmn7hMYgb8YW3jH7qErcwtmhoHXSJlquswabTZG1F9AstJT1dUugZ5JEhBjmnVNdu4T2t1ynv8XdRDuI94kdybd8RnLkpZuLupcxtr6gEEkJuImaaJOc81WaTePfKHaJgMDksxB5zmLrAug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MgSejP2fIr+1iG2vTY7E0mBP93zHzawB+panIjVatGE=; b=B9xVi7NTX+/F4ihBygGjdkMI3i83z5HRYyzctunKxxSZ8vozTScJYZDvmZbWyxc2MZKl3ptKF4pAa27upCl1IAjxEZoD23U4cZgYl32UHEm7TUVRSeoL3zBpjwUfSUL0Q1kyamsc0V2/6NhtOLf/uBqwv81j+PRrWIx24stL2pbrMW/9bIXiOb4yrpZap5XEQPcYB+goH94Yf2HuEYBIUWGLbjvFHPFyfKTvV/MwPJGQdYKKMPfJQMAnU3IjEMERUXJqrnyTSSFzkd0l8a1+u4VJP6yvJIXXszOFauB5z49Kiyh/emvBnMQqRiFMGEbHG7AoJoLYLkyMR3/5OGVZAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MgSejP2fIr+1iG2vTY7E0mBP93zHzawB+panIjVatGE=; b=Cy+0q4L0A/WRCxWGo6crPOzz13I/ALnWZ/3ChngtCMc0WpLYsPQaOozh2frCryBAOs9WJl1GMkY/0coupvyBCcvWlIAp/GtoEKDIhCBBHYD/z6DnY40jmZMx5JxMcYQF7r5/ntwU8E2/M7k8Jnw9Tqy5V8KvPNrWsxyxH3HGefg= Received: from MW2PR2101CA0028.namprd21.prod.outlook.com (2603:10b6:302:1::41) by MN2PR12MB4784.namprd12.prod.outlook.com (2603:10b6:208:39::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.16; Thu, 11 Aug 2022 12:34:04 +0000 Received: from CO1NAM11FT038.eop-nam11.prod.protection.outlook.com (2603:10b6:302:1:cafe::94) by MW2PR2101CA0028.outlook.office365.com (2603:10b6:302:1::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.6 via Frontend Transport; Thu, 11 Aug 2022 12:34:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT038.mail.protection.outlook.com (10.13.174.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:34:03 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:33:57 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 05/13] perf/x86/amd/lbr: Detect LbrExtV2 support Date: Thu, 11 Aug 2022 17:59:53 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0ade182f-4ce5-4574-79b8-08da7b95c673 X-MS-TrafficTypeDiagnostic: MN2PR12MB4784:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mWfdpy+fjOXE3mlVocaFdlPRBPX6Tpguq7ZFqxBWoXHH4E4zA18xnBlKt5MzawT3WIvVIPHm5w+JpmeCQ+ziyAL46bxyu3TIFf8QqWgjVkn6psxiNw3d+8PcgfDEhG19pJzX7erCiP7ou9C56u7mo3TAe3r5YLc98sf+EkoV9JGQSsk9QbMEkXxe29GOA92JOFnaYFgnJog404jRx06Z2gkrz+afhYkj38odQLyYbG9sZjcrMtya2+m1bp7yFsAKYTrHuta5xoIewILwOg/9TtIalpx335m8ibUJd5Wcfb5PRzxXG6DRuf04kS44NowsFgSjlpl/dmV+FztoZzyOF/ZyK0OtrFgXIBeMVJEAubG00ZlQGV8RF5dIuDazMHex7W/dg11lvgthIMxY2Ue+X3OjbPqaUW6Hm8o22wtOargAwfYOSMvWs3qDPl9NhizlCklvbhIJTC5yRpZ/GJR2vxi/UBsUv7kQq5VBSBXpdusdC1NKvxZcArA8aMq3JoMXRojPuCmaXBtZnnCVpZrjX/DjGfybKziRLareqUwDnGVRqL9Y5Vn+9+29jFeQNk/XCwLAx+hL/GBDqmxtxnrSjJbFIkQYkDlJYMR4UiEO5j+8qHi47r6EIk8M/byWyLWDqKb8kqtZ/NnmH0KqIFmR2PhDAPDcSPzAS8F3clm4BFgdjNqjT0jSPUqTBViHMd4Hp9KSjzieIXG/S39rB/6Y7i1OzCaxfsCbTLtQ8qV39SuqaIkmP+MYC/+pQyp6COhWY6w9oE3tj2PSjJieNYXeMprE7l7qMm+hp13F0CxD62nTp7ZjULCl7swphEo1I5aAUd1DS9j1Luts27IAbFN0MA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(396003)(39860400002)(376002)(346002)(136003)(46966006)(40470700004)(36840700001)(2906002)(2616005)(82310400005)(356005)(36860700001)(86362001)(40460700003)(82740400003)(40480700001)(83380400001)(41300700001)(5660300002)(110136005)(54906003)(81166007)(8676002)(44832011)(8936002)(7416002)(7696005)(70586007)(336012)(6666004)(70206006)(4326008)(478600001)(186003)(47076005)(426003)(316002)(16526019)(36756003)(26005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:34:03.4271 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0ade182f-4ce5-4574-79b8-08da7b95c673 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT038.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4784 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" AMD Last Branch Record Extension Version 2 (LbrExtV2) is driven by Core PMC overflows. It records recently taken branches up to the moment when the PMC overflow occurs. Detect the feature during PMU initialization and set the branch stack depth using CPUID leaf 0x80000022 EBX. Signed-off-by: Sandipan Das --- arch/x86/events/amd/Makefile | 2 +- arch/x86/events/amd/core.c | 9 +++++---- arch/x86/events/amd/lbr.c | 21 +++++++++++++++++++++ arch/x86/events/perf_event.h | 2 ++ arch/x86/include/asm/perf_event.h | 3 ++- 5 files changed, 31 insertions(+), 6 deletions(-) create mode 100644 arch/x86/events/amd/lbr.c diff --git a/arch/x86/events/amd/Makefile b/arch/x86/events/amd/Makefile index b9f5d4610256..527d947eb76b 100644 --- a/arch/x86/events/amd/Makefile +++ b/arch/x86/events/amd/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_CPU_SUP_AMD) +=3D core.o +obj-$(CONFIG_CPU_SUP_AMD) +=3D core.o lbr.o obj-$(CONFIG_PERF_EVENTS_AMD_BRS) +=3D brs.o obj-$(CONFIG_PERF_EVENTS_AMD_POWER) +=3D power.o obj-$(CONFIG_X86_LOCAL_APIC) +=3D ibs.o diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index ef3520731a20..a3aa67b4fd50 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -1374,10 +1374,11 @@ static int __init amd_core_pmu_init(void) x86_pmu.flags |=3D PMU_FL_PAIR; } =20 - /* - * BRS requires special event constraints and flushing on ctxsw. - */ - if (boot_cpu_data.x86 >=3D 0x19 && !amd_brs_init()) { + /* LBR and BRS are mutually exclusive features */ + if (amd_pmu_lbr_init() && !amd_brs_init()) { + /* + * BRS requires special event constraints and flushing on ctxsw. + */ x86_pmu.get_event_constraints =3D amd_get_event_constraints_f19h; x86_pmu.sched_task =3D amd_pmu_brs_sched_task; x86_pmu.limit_period =3D amd_pmu_limit_period; diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c new file mode 100644 index 000000000000..d240ff722b78 --- /dev/null +++ b/arch/x86/events/amd/lbr.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +#include "../perf_event.h" + +__init int amd_pmu_lbr_init(void) +{ + union cpuid_0x80000022_ebx ebx; + + if (x86_pmu.version < 2 || !boot_cpu_has(X86_FEATURE_LBREXT_V2)) + return -EOPNOTSUPP; + + /* Set number of entries */ + ebx.full =3D cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES); + x86_pmu.lbr_nr =3D ebx.split.lbr_v2_stack_sz; + + pr_cont("%d-deep LBR, ", x86_pmu.lbr_nr); + + return 0; +} diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h index 6d23e88d232c..a6cd83c57e8b 100644 --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -1230,6 +1230,8 @@ static inline bool fixed_counter_disabled(int i, stru= ct pmu *pmu) =20 int amd_pmu_init(void); =20 +int amd_pmu_lbr_init(void); + #ifdef CONFIG_PERF_EVENTS_AMD_BRS =20 #define AMD_FAM19H_BRS_EVENT 0xc4 /* RETIRED_TAKEN_BRANCH_INSTRUCTIONS */ diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_= event.h index 34348ae41cdb..07844424aaea 100644 --- a/arch/x86/include/asm/perf_event.h +++ b/arch/x86/include/asm/perf_event.h @@ -207,7 +207,8 @@ union cpuid_0x80000022_ebx { struct { /* Number of Core Performance Counters */ unsigned int num_core_pmc:4; - unsigned int reserved:6; + /* Number of available LBR Stack Entries */ + unsigned int lbr_v2_stack_sz:6; /* Number of Data Fabric Counters */ unsigned int num_df_pmc:6; } split; --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD72BC25B0C for ; Thu, 11 Aug 2022 12:34:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235239AbiHKMet (ORCPT ); Thu, 11 Aug 2022 08:34:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234669AbiHKMer (ORCPT ); Thu, 11 Aug 2022 08:34:47 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2073.outbound.protection.outlook.com [40.107.94.73]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38A7313EA2; Thu, 11 Aug 2022 05:34:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Oifu4e4yTYZ4H9X6b81kXeLj9tR7veYEJlv9AW0KIimK15Tqm/CDELYeS8k7aggdRoTghxV1Oom9jMzfsAGcyFPgFEzGIl5pJ6oi9PllwBb1uOe8BuhiS28A+rxHT/BqVSOFehM6tEfXLqWkfYfb5vPetSnKtFNnMHH96WsgnWE9005ZYLI+v1yMXHDEdRgBq3m2KY6CwERr3K5YnqfuQT6rJcg4Ii6oR7j6WyIPKOmu76INRHg2/WudVtmsjldpn7AbzUC1hKYT5QrnsOFdEYViTnrdvx4+eoqFxbF3F06eDLCbfEs0itJc1NULnnDoFEkjhouYQHmFAbpFKxDywg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w2nG0q17xE51uFBZ/DKe+LXh7BN3wXbQk5XH6Qe1Hpg=; b=eN3n9MauDVRu4MXIsbOhE35fLYEOoFQuV9uxMveo0b3sxS5pt3iBwRBHabbiAtKlcOvcAku3qji0H1SioolxIK97E2/Jr5v3ln5VErZytcHry8C9dfpu43zhgMMJm9aboYX2H2qV4fTG07VaMD/HZqDTFs7qXO73OsSz6Ul17M0SFF92wfZNWfkKUZPQj0UWYfN9ehU8RxjbdEvbIot47z2JlbQE03ogj3FrFabG6SBPVJgY+5vg0Chh9UyX7rHUyDlEj/b5uZn9sPEuQY6xtXypOsfzJ5PHELdtjiRqJr6fyP0QEt9t19/TSGle6ra+TbRHEh48yPRuNW3of7RTqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w2nG0q17xE51uFBZ/DKe+LXh7BN3wXbQk5XH6Qe1Hpg=; b=11uZvbxAgO2ypBKK6Or8bZqVkiRANkE8poSkUmd0jxwCg+T5Sp96D1Ehge+j9btuTmdDjqsN6Zi5jxzQ3Zbd9LtvH30zVOlG+CY08DuBBbiDxhg+iZwzNfvXOLf+uPHhXFR9N3+D2TX0Kjzwjs9geFpIhiDXQafwgDSOXGIv4Ao= Received: from MW4PR04CA0218.namprd04.prod.outlook.com (2603:10b6:303:87::13) by MN2PR12MB4335.namprd12.prod.outlook.com (2603:10b6:208:1d4::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.10; Thu, 11 Aug 2022 12:34:31 +0000 Received: from CO1NAM11FT008.eop-nam11.prod.protection.outlook.com (2603:10b6:303:87:cafe::a1) by MW4PR04CA0218.outlook.office365.com (2603:10b6:303:87::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.20 via Frontend Transport; Thu, 11 Aug 2022 12:34:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT008.mail.protection.outlook.com (10.13.175.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:34:30 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:34:24 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 06/13] perf/x86/amd/lbr: Add LbrExtV2 branch record support Date: Thu, 11 Aug 2022 17:59:54 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dc7e9389-a813-4b73-3990-08da7b95d66f X-MS-TrafficTypeDiagnostic: MN2PR12MB4335:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: r0K7wx0GJgrXx/WEKPA3pTK5Y9PKuoEBiDl8gtNMdmdD2YuXTWG59ZDarxBjr2oDPUqIvkcthjfh67I7sAikVFIlx36Ix6CXL2gt4LVaiWVleVWl9smaII7zeLSzSRZaQMM9Yt0fBqb6hRBcl7I+ZutBkcVrNItOpdTQOFVQI3lLSoVYbfcSYifkZJ74iG+gLNXguIVBmsmLgHO4Z03qNkBGkQW608zQG8+MhlXEccHGYXKPFpB/Ozk6qRyrbSdljKDx77xMrhApQR0jBqr+w99wRwSUKW4ng+CcHuPNDB+hBGQyUaNh5rvrEfKA/liIPgCMWVIIoLaJlC/IaJI5JTOUz1//8B9Z2ILbk7jOEqm66C5lGmZjNsK8mV80NdtGAwWiQON0pLz3U4TapUlWCK+oTACv6vJy/rMJXZ6DYjpJNsjrtLNbvoWjFuw2pzvmrxpGcZjetLLvdjXte0VGafLUCs1F/X/pb8mur2fa6Hu6pzzjW39fGRZI+uQbAv28yXMV310wmnnIwdMJXJeTgkMhXiFMiKxgev39XhAwiRLfnV1zAg7aek02HVXlwTvMY6kvxbKHGaWwHOESgUQuk7pT1gx4e1nUmMj9GPpc9lhGoiYbQCy92IOJ6FbMlk5dGK0VwsSf3o7PxkA6fqO8YjsbYuBzBUupHlIVhOt6yvvExsrr5h11CHGpDtGJUAIE/2kuEONP7oeLrMCdOO69gsTJZS9fgSRPcwgojJLVENnv/C3LRl8yWHcDqKD86zUhLoXp1ZxX94E5VCkubJqb6bYFJNZLuIPgWHAzCB8I3w5SeG85nCnwLs9y59eoj4kgQGfnmYwQbJEUlJdTAgPxXg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(346002)(396003)(136003)(376002)(40470700004)(46966006)(36840700001)(316002)(7416002)(356005)(41300700001)(70206006)(8676002)(2906002)(70586007)(7696005)(26005)(4326008)(30864003)(6666004)(82740400003)(82310400005)(81166007)(44832011)(5660300002)(16526019)(186003)(36756003)(47076005)(336012)(426003)(83380400001)(86362001)(478600001)(40460700003)(8936002)(40480700001)(54906003)(110136005)(36860700001)(2616005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:34:30.2428 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dc7e9389-a813-4b73-3990-08da7b95d66f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT008.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4335 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If AMD Last Branch Record Extension Version 2 (LbrExtV2) is detected, enable it alongside LBR Freeze on PMI when an event requests branch stack i.e. PERF_SAMPLE_BRANCH_STACK. Each branch record is represented by a pair of registers, LBR From and LBR To. The freeze feature prevents any updates to these registers once a PMC overflows. The contents remain unchanged until the freeze bit is cleared by the PMI handler. The branch records are read and copied to sample data before unfreezing. However, only valid entries are copied. There is no additional register to denote which of the register pairs represent the top of the stack (TOS) since internal register renaming always ensures that the first pair (i.e. index 0) is the one representing the most recent branch and so on. The LBR registers are per-thread resources and are cleared explicitly whenever a new task is scheduled in. There are no special implications on the contents of these registers when transitioning to deep C-states. Signed-off-by: Sandipan Das --- arch/x86/events/amd/core.c | 47 +++++-- arch/x86/events/amd/lbr.c | 203 +++++++++++++++++++++++++++++++ arch/x86/events/perf_event.h | 8 ++ arch/x86/include/asm/msr-index.h | 5 + 4 files changed, 252 insertions(+), 11 deletions(-) diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index a3aa67b4fd50..d799628016c8 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -620,7 +620,7 @@ static inline u64 amd_pmu_get_global_status(void) /* PerfCntrGlobalStatus is read-only */ rdmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, status); =20 - return status & amd_pmu_global_cntr_mask; + return status; } =20 static inline void amd_pmu_ack_global_status(u64 status) @@ -631,8 +631,6 @@ static inline void amd_pmu_ack_global_status(u64 status) * clears the same bit in PerfCntrGlobalStatus */ =20 - /* Only allow modifications to PerfCntrGlobalStatus.PerfCntrOvfl */ - status &=3D amd_pmu_global_cntr_mask; wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, status); } =20 @@ -742,11 +740,17 @@ static void amd_pmu_v2_enable_event(struct perf_event= *event) __x86_pmu_enable_event(hwc, ARCH_PERFMON_EVENTSEL_ENABLE); } =20 -static void amd_pmu_v2_enable_all(int added) +static __always_inline void amd_pmu_core_enable_all(void) { amd_pmu_set_global_ctl(amd_pmu_global_cntr_mask); } =20 +static void amd_pmu_v2_enable_all(int added) +{ + amd_pmu_lbr_enable_all(); + amd_pmu_core_enable_all(); +} + static void amd_pmu_disable_event(struct perf_event *event) { x86_pmu_disable_event(event); @@ -771,10 +775,15 @@ static void amd_pmu_disable_all(void) amd_pmu_check_overflow(); } =20 -static void amd_pmu_v2_disable_all(void) +static __always_inline void amd_pmu_core_disable_all(void) { - /* Disable all PMCs */ amd_pmu_set_global_ctl(0); +} + +static void amd_pmu_v2_disable_all(void) +{ + amd_pmu_core_disable_all(); + amd_pmu_lbr_disable_all(); amd_pmu_check_overflow(); } =20 @@ -877,8 +886,8 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) pmu_enabled =3D cpuc->enabled; cpuc->enabled =3D 0; =20 - /* Stop counting */ - amd_pmu_v2_disable_all(); + /* Stop counting but do not disable LBR */ + amd_pmu_core_disable_all(); =20 status =3D amd_pmu_get_global_status(); =20 @@ -886,6 +895,12 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) if (!status) goto done; =20 + /* Read branch records before unfreezing */ + if (status & GLOBAL_STATUS_LBRS_FROZEN) { + amd_pmu_lbr_read(); + status &=3D ~GLOBAL_STATUS_LBRS_FROZEN; + } + for (idx =3D 0; idx < x86_pmu.num_counters; idx++) { if (!test_bit(idx, cpuc->active_mask)) continue; @@ -905,6 +920,9 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) if (!x86_perf_event_set_period(event)) continue; =20 + if (has_branch_stack(event)) + data.br_stack =3D &cpuc->lbr_stack; + if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); =20 @@ -918,7 +936,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) */ WARN_ON(status > 0); =20 - /* Clear overflow bits */ + /* Clear overflow and freeze bits */ amd_pmu_ack_global_status(~status); =20 /* @@ -932,7 +950,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) =20 /* Resume counting only if PMU is active */ if (pmu_enabled) - amd_pmu_v2_enable_all(0); + amd_pmu_core_enable_all(); =20 return amd_pmu_adjust_nmi_window(handled); } @@ -1375,7 +1393,14 @@ static int __init amd_core_pmu_init(void) } =20 /* LBR and BRS are mutually exclusive features */ - if (amd_pmu_lbr_init() && !amd_brs_init()) { + if (!amd_pmu_lbr_init()) { + /* LBR requires flushing on context switch */ + x86_pmu.sched_task =3D amd_pmu_lbr_sched_task; + static_call_update(amd_pmu_branch_hw_config, amd_pmu_lbr_hw_config); + static_call_update(amd_pmu_branch_reset, amd_pmu_lbr_reset); + static_call_update(amd_pmu_branch_add, amd_pmu_lbr_add); + static_call_update(amd_pmu_branch_del, amd_pmu_lbr_del); + } else if (!amd_brs_init()) { /* * BRS requires special event constraints and flushing on ctxsw. */ diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c index d240ff722b78..39247e7be53c 100644 --- a/arch/x86/events/amd/lbr.c +++ b/arch/x86/events/amd/lbr.c @@ -4,6 +4,209 @@ =20 #include "../perf_event.h" =20 +struct branch_entry { + union { + struct { + u64 ip:58; + u64 ip_sign_ext:5; + u64 mispredict:1; + } split; + u64 full; + } from; + + union { + struct { + u64 ip:58; + u64 ip_sign_ext:3; + u64 reserved:1; + u64 spec:1; + u64 valid:1; + } split; + u64 full; + } to; +}; + +static __always_inline void amd_pmu_lbr_set_from(unsigned int idx, u64 val) +{ + wrmsrl(MSR_AMD_SAMP_BR_FROM + idx * 2, val); +} + +static __always_inline void amd_pmu_lbr_set_to(unsigned int idx, u64 val) +{ + wrmsrl(MSR_AMD_SAMP_BR_FROM + idx * 2 + 1, val); +} + +static __always_inline u64 amd_pmu_lbr_get_from(unsigned int idx) +{ + u64 val; + + rdmsrl(MSR_AMD_SAMP_BR_FROM + idx * 2, val); + + return val; +} + +static __always_inline u64 amd_pmu_lbr_get_to(unsigned int idx) +{ + u64 val; + + rdmsrl(MSR_AMD_SAMP_BR_FROM + idx * 2 + 1, val); + + return val; +} + +static __always_inline u64 sign_ext_branch_ip(u64 ip) +{ + u32 shift =3D 64 - boot_cpu_data.x86_virt_bits; + + return (u64)(((s64)ip << shift) >> shift); +} + +void amd_pmu_lbr_read(void) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + struct perf_branch_entry *br =3D cpuc->lbr_entries; + struct branch_entry entry; + int out =3D 0, i; + + if (!cpuc->lbr_users) + return; + + for (i =3D 0; i < x86_pmu.lbr_nr; i++) { + entry.from.full =3D amd_pmu_lbr_get_from(i); + entry.to.full =3D amd_pmu_lbr_get_to(i); + + /* Check if a branch has been logged */ + if (!entry.to.split.valid) + continue; + + perf_clear_branch_entry_bitfields(br + out); + + br[out].from =3D sign_ext_branch_ip(entry.from.split.ip); + br[out].to =3D sign_ext_branch_ip(entry.to.split.ip); + br[out].mispred =3D entry.from.split.mispredict; + br[out].predicted =3D !br[out].mispred; + out++; + } + + cpuc->lbr_stack.nr =3D out; + + /* + * Internal register renaming always ensures that LBR From[0] and + * LBR To[0] always represent the TOS + */ + cpuc->lbr_stack.hw_idx =3D 0; +} + +static int amd_pmu_lbr_setup_filter(struct perf_event *event) +{ + /* No LBR support */ + if (!x86_pmu.lbr_nr) + return -EOPNOTSUPP; + + return 0; +} + +int amd_pmu_lbr_hw_config(struct perf_event *event) +{ + int ret =3D 0; + + /* LBR is not recommended in counting mode */ + if (!is_sampling_event(event)) + return -EINVAL; + + ret =3D amd_pmu_lbr_setup_filter(event); + if (!ret) + event->attach_state |=3D PERF_ATTACH_SCHED_CB; + + return ret; +} + +void amd_pmu_lbr_reset(void) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + int i; + + if (!x86_pmu.lbr_nr) + return; + + /* Reset all branch records individually */ + for (i =3D 0; i < x86_pmu.lbr_nr; i++) { + amd_pmu_lbr_set_from(i, 0); + amd_pmu_lbr_set_to(i, 0); + } + + cpuc->last_task_ctx =3D NULL; + cpuc->last_log_id =3D 0; +} + +void amd_pmu_lbr_add(struct perf_event *event) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + + if (!x86_pmu.lbr_nr) + return; + + perf_sched_cb_inc(event->ctx->pmu); + + if (!cpuc->lbr_users++ && !event->total_time_running) + amd_pmu_lbr_reset(); +} + +void amd_pmu_lbr_del(struct perf_event *event) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + + if (!x86_pmu.lbr_nr) + return; + + cpuc->lbr_users--; + WARN_ON_ONCE(cpuc->lbr_users < 0); + perf_sched_cb_dec(event->ctx->pmu); +} + +void amd_pmu_lbr_sched_task(struct perf_event_context *ctx, bool sched_in) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + + /* + * A context switch can flip the address space and LBR entries are + * not tagged with an identifier. Hence, branches cannot be resolved + * from the old address space and the LBR records should be wiped. + */ + if (cpuc->lbr_users && sched_in) + amd_pmu_lbr_reset(); +} + +void amd_pmu_lbr_enable_all(void) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + u64 dbg_ctl, dbg_extn_cfg; + + if (!cpuc->lbr_users || !x86_pmu.lbr_nr) + return; + + rdmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl); + rdmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg); + + wrmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI); + wrmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg | DBG_EXTN_CFG_LBRV2EN); +} + +void amd_pmu_lbr_disable_all(void) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + u64 dbg_ctl, dbg_extn_cfg; + + if (!cpuc->lbr_users || !x86_pmu.lbr_nr) + return; + + rdmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg); + rdmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl); + + wrmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg & ~DBG_EXTN_CFG_LBRV2EN); + wrmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl & ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI); +} + __init int amd_pmu_lbr_init(void) { union cpuid_0x80000022_ebx ebx; diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h index a6cd83c57e8b..c8397290f388 100644 --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -1231,6 +1231,14 @@ static inline bool fixed_counter_disabled(int i, str= uct pmu *pmu) int amd_pmu_init(void); =20 int amd_pmu_lbr_init(void); +void amd_pmu_lbr_reset(void); +void amd_pmu_lbr_read(void); +void amd_pmu_lbr_add(struct perf_event *event); +void amd_pmu_lbr_del(struct perf_event *event); +void amd_pmu_lbr_sched_task(struct perf_event_context *ctx, bool sched_in); +void amd_pmu_lbr_enable_all(void); +void amd_pmu_lbr_disable_all(void); +int amd_pmu_lbr_hw_config(struct perf_event *event); =20 #ifdef CONFIG_PERF_EVENTS_AMD_BRS =20 diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-in= dex.h index 403e83b4adc8..440d0490f71e 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -539,6 +539,9 @@ #define MSR_AMD64_PERF_CNTR_GLOBAL_CTL 0xc0000301 #define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR 0xc0000302 =20 +/* AMD Last Branch Record MSRs */ +#define MSR_AMD64_LBR_SELECT 0xc000010e + /* Fam 17h MSRs */ #define MSR_F17H_IRPERF 0xc00000e9 =20 @@ -707,6 +710,8 @@ #define MSR_AMD_DBG_EXTN_CFG 0xc000010f #define MSR_AMD_SAMP_BR_FROM 0xc0010300 =20 +#define DBG_EXTN_CFG_LBRV2EN BIT_ULL(6) + #define MSR_IA32_MPERF 0x000000e7 #define MSR_IA32_APERF 0x000000e8 =20 --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7737C19F2D for ; Thu, 11 Aug 2022 12:35:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235261AbiHKMfy (ORCPT ); Thu, 11 Aug 2022 08:35:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235272AbiHKMfi (ORCPT ); Thu, 11 Aug 2022 08:35:38 -0400 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2066.outbound.protection.outlook.com [40.107.102.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE209B1F1; Thu, 11 Aug 2022 05:35:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JIIbsj/HIah9SecJbuBqX2sk4cs4zfpycmRG0OJZJJFuTzSA3PwgIjU6ZbkF+kWO/0HiHjdxGAZi3sB6vQO/vItSlkCqbtJOCJGPTSJk9c97+jVfRvmWzDZsl0qXcpt2J65KSWr/V0UJyjMwemsjbf2+64nInZI2R37IcHT8WKx0PLCYJ8U/53i6Ynp/55+HKwOcG0M6DEDaMa9DiCZEyUQdekAL5sktpDYIB16cp/PKBHs3lUj9vjbJQhteG2+YfaNLv2fAGBsGXumCvGzatHJK6i1z5fPr7wZZdvdQSfBCN7yvBxItbzMBZ8LQaW4COTM7bNkEUebx5cOnxZ6nYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ktCkq4RksL6s63xNn8wN9/EKhsCeFjP0+cY2Mcf2TZo=; b=JqdhM+kDIgJK3Cn/nAN7h8jkXOf3JcuG2SNkZKYQ5N5dr3KWfwY3o7jIdsx+NAYGAhr3AxxdT5FYtkkLp74l3e3BnqWBP2IClNleBSQJrDsUhtJDa3gJzKFststdT4QbNfhpHkJqpWi8ylwSWYy2GY1sOv2/IyvOE4RQnGBBkQ9ClinSgp0CpIwIlGt+UM3t+re53fNmVl/FwSaph44//xnRqoD2x7yZy4KkoFGh+rslFs094wpq/DFL62k/ZzK9CE/FIyyt9V4b9BQDTOU+TFD8JbjQS5dPZ2Fg2cGbjLmsRKvmiEO2/yYkUaQQPfQHK+b11ZqtvNZGfqqGqwcXvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ktCkq4RksL6s63xNn8wN9/EKhsCeFjP0+cY2Mcf2TZo=; b=hemKoF9yUBqo1HH7plk3mlyjtjlAR9bUawm8hpwKylXDF9kj7NQivuveOXKjOEFacJyeXpFIE3LpVNX+lm6vM0d9Pr/qhvYsK7OcIGbBm9JG0nhoiVW8wsEhrn8cDHHd1sYTz+RHti6ln5SeXZLlsJg9HczCcKg5McGftDX6JLo= Received: from MW4PR04CA0256.namprd04.prod.outlook.com (2603:10b6:303:88::21) by PH7PR12MB5594.namprd12.prod.outlook.com (2603:10b6:510:134::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.20; Thu, 11 Aug 2022 12:35:21 +0000 Received: from CO1NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:303:88:cafe::fc) by MW4PR04CA0256.outlook.office365.com (2603:10b6:303:88::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.16 via Frontend Transport; Thu, 11 Aug 2022 12:35:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT031.mail.protection.outlook.com (10.13.174.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:35:20 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:34:43 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 07/13] perf/x86/amd/lbr: Add LbrExtV2 hardware branch filter support Date: Thu, 11 Aug 2022 17:59:55 +0530 Message-ID: <9336af5c9785b8e14c62220fc0e6cfb10ab97de3.1660211399.git.sandipan.das@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ba06bbe7-b9e6-4c5b-2ee8-08da7b95f468 X-MS-TrafficTypeDiagnostic: PH7PR12MB5594:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wKfkXx5ENN6vfqaYdjxsGhVc61zDciL9gH3uwMEyjir2xzGKycUI9evd/fqZ4e3rx5st6Od9y6sI038+nZx6plj11BHyzSzrOgL6lroUW1T+a9t7ASs3k6Fo5xkiSCVAdPdqjC6XXEYyQ2e/0qtJf3qSg3xYNbgzMFURrMVIOcyuIsc5Cr0q7fFXuYf5QG80UJv33dgA3oDVBatZlX955KQ91yYCxHy2p6GqtAHmzUcBhbA81Eyc0rr07EfY5+eXfJI89WvUDXV1cvD7DHV2vOENM//wX8eCZnL9j8Sxynpe3STWPQRjyDWe00+y3N1kH/zWR3H/J5EMuRvsD5N3AZuU1uY9A+Y6rlYMvVwRg8N1TOi8ez261/z8pkJO7sx4/fkjJel/cJjdGorh+nHchGGdYT6OiWidv4Z8/pvXEgAdIv6wBQcnbS7vgIwONOa3KPn1XTsN9Hcys1VGZxK+BTB1KhdxtBKuINyVGSGhEAf5AtRol88ZNkB/GQh881dMzDxYdSyGV2WJdRuki7U2OHvt/owuUktE+lKCino2m7wskNhBU9i/gdgTJ1t+Pl2r1JA006xGKDGx1BNYGBWThEWRyiPI3EuMDlp3aYk9Vmn0UWjwxmDRkz7j3nVFSNuXYdek/sEqy+7IsEyLrG7MxT5u0pgXgo1uOrvQCalE7GkRT5qOI2ZbVx972z2B4LcB7YsIJQQFETI/LtukWh6/ruRgMIU+8aiqRaAfqJPlOL+jyzScrYvK0jBPIm0vprW0yNIDsGyPPmDjGhvZegXpZ33sXZu9jYMOp4emfRquZ9q8bLr5yxYvU2LQ9jlQobL7CkkEV63bRmOdzqaJS2ukOw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(396003)(346002)(136003)(39860400002)(376002)(46966006)(40470700004)(36840700001)(54906003)(70206006)(36860700001)(186003)(4326008)(82740400003)(5660300002)(7416002)(316002)(8676002)(82310400005)(40460700003)(44832011)(8936002)(36756003)(70586007)(110136005)(478600001)(26005)(16526019)(426003)(336012)(41300700001)(7696005)(6666004)(47076005)(2906002)(2616005)(356005)(81166007)(83380400001)(86362001)(40480700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:35:20.5283 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ba06bbe7-b9e6-4c5b-2ee8-08da7b95f468 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5594 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If AMD Last Branch Record Extension Version 2 (LbrExtV2) is detected, convert the requested branch filter (PERF_SAMPLE_BRANCH_* flags) to the corresponding hardware filter value and stash it in the event data when a branch stack is requested. The hardware filter value is also saved in per-CPU areas for use during event scheduling. Hardware filtering is provided by the LBR Branch Select register. It has bits which when set, suppress recording of the following types of branches: * CPL =3D 0 (Kernel only) * CPL > 0 (Userspace only) * Conditional Branches * Near Relative Calls * Near Indirect Calls * Near Returns * Near Indirect Jumps (excluding Near Indirect Calls and Near Returns) * Near Relative Jumps (excluding Near Relative Calls) * Far Branches Signed-off-by: Sandipan Das --- arch/x86/events/amd/core.c | 21 ++++++--- arch/x86/events/amd/lbr.c | 94 +++++++++++++++++++++++++++++++++++++- 2 files changed, 108 insertions(+), 7 deletions(-) diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index d799628016c8..36bede1d7b1e 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -542,16 +542,24 @@ static int amd_pmu_cpu_prepare(int cpu) { struct cpu_hw_events *cpuc =3D &per_cpu(cpu_hw_events, cpu); =20 + cpuc->lbr_sel =3D kzalloc_node(sizeof(struct er_account), GFP_KERNEL, + cpu_to_node(cpu)); + if (!cpuc->lbr_sel) + return -ENOMEM; + WARN_ON_ONCE(cpuc->amd_nb); =20 if (!x86_pmu.amd_nb_constraints) return 0; =20 cpuc->amd_nb =3D amd_alloc_nb(cpu); - if (!cpuc->amd_nb) - return -ENOMEM; + if (cpuc->amd_nb) + return 0; =20 - return 0; + kfree(cpuc->lbr_sel); + cpuc->lbr_sel =3D NULL; + + return -ENOMEM; } =20 static void amd_pmu_cpu_starting(int cpu) @@ -589,13 +597,14 @@ static void amd_pmu_cpu_starting(int cpu) =20 static void amd_pmu_cpu_dead(int cpu) { - struct cpu_hw_events *cpuhw; + struct cpu_hw_events *cpuhw =3D &per_cpu(cpu_hw_events, cpu); + + kfree(cpuhw->lbr_sel); + cpuhw->lbr_sel =3D NULL; =20 if (!x86_pmu.amd_nb_constraints) return; =20 - cpuhw =3D &per_cpu(cpu_hw_events, cpu); - if (cpuhw->amd_nb) { struct amd_nb *nb =3D cpuhw->amd_nb; =20 diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c index 39247e7be53c..064a6c8e1597 100644 --- a/arch/x86/events/amd/lbr.c +++ b/arch/x86/events/amd/lbr.c @@ -4,6 +4,39 @@ =20 #include "../perf_event.h" =20 +/* LBR Branch Select valid bits */ +#define LBR_SELECT_MASK 0x1ff + +/* + * LBR Branch Select filter bits which when set, ensures that the + * corresponding type of branches are not recorded + */ +#define LBR_SELECT_KERNEL 0 /* Branches ending in CPL =3D 0 */ +#define LBR_SELECT_USER 1 /* Branches ending in CPL > 0 */ +#define LBR_SELECT_JCC 2 /* Conditional branches */ +#define LBR_SELECT_CALL_NEAR_REL 3 /* Near relative calls */ +#define LBR_SELECT_CALL_NEAR_IND 4 /* Indirect relative calls */ +#define LBR_SELECT_RET_NEAR 5 /* Near returns */ +#define LBR_SELECT_JMP_NEAR_IND 6 /* Near indirect jumps (excl. calls and= returns) */ +#define LBR_SELECT_JMP_NEAR_REL 7 /* Near relative jumps (excl. calls) */ +#define LBR_SELECT_FAR_BRANCH 8 /* Far branches */ + +#define LBR_KERNEL BIT(LBR_SELECT_KERNEL) +#define LBR_USER BIT(LBR_SELECT_USER) +#define LBR_JCC BIT(LBR_SELECT_JCC) +#define LBR_REL_CALL BIT(LBR_SELECT_CALL_NEAR_REL) +#define LBR_IND_CALL BIT(LBR_SELECT_CALL_NEAR_IND) +#define LBR_RETURN BIT(LBR_SELECT_RET_NEAR) +#define LBR_REL_JMP BIT(LBR_SELECT_JMP_NEAR_REL) +#define LBR_IND_JMP BIT(LBR_SELECT_JMP_NEAR_IND) +#define LBR_FAR BIT(LBR_SELECT_FAR_BRANCH) +#define LBR_NOT_SUPP -1 /* unsupported filter */ +#define LBR_IGNORE 0 + +#define LBR_ANY \ + (LBR_JCC | LBR_REL_CALL | LBR_IND_CALL | LBR_RETURN | \ + LBR_REL_JMP | LBR_IND_JMP | LBR_FAR) + struct branch_entry { union { struct { @@ -97,12 +130,56 @@ void amd_pmu_lbr_read(void) cpuc->lbr_stack.hw_idx =3D 0; } =20 +static const int lbr_select_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] =3D { + [PERF_SAMPLE_BRANCH_USER_SHIFT] =3D LBR_USER, + [PERF_SAMPLE_BRANCH_KERNEL_SHIFT] =3D LBR_KERNEL, + [PERF_SAMPLE_BRANCH_HV_SHIFT] =3D LBR_IGNORE, + + [PERF_SAMPLE_BRANCH_ANY_SHIFT] =3D LBR_ANY, + [PERF_SAMPLE_BRANCH_ANY_CALL_SHIFT] =3D LBR_REL_CALL | LBR_IND_CALL, + [PERF_SAMPLE_BRANCH_ANY_RETURN_SHIFT] =3D LBR_RETURN, + [PERF_SAMPLE_BRANCH_IND_CALL_SHIFT] =3D LBR_IND_CALL, + [PERF_SAMPLE_BRANCH_ABORT_TX_SHIFT] =3D LBR_NOT_SUPP, + [PERF_SAMPLE_BRANCH_IN_TX_SHIFT] =3D LBR_NOT_SUPP, + [PERF_SAMPLE_BRANCH_NO_TX_SHIFT] =3D LBR_NOT_SUPP, + [PERF_SAMPLE_BRANCH_COND_SHIFT] =3D LBR_JCC, + + [PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT] =3D LBR_NOT_SUPP, + [PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT] =3D LBR_IND_JMP, + [PERF_SAMPLE_BRANCH_CALL_SHIFT] =3D LBR_REL_CALL, + + [PERF_SAMPLE_BRANCH_NO_FLAGS_SHIFT] =3D LBR_NOT_SUPP, + [PERF_SAMPLE_BRANCH_NO_CYCLES_SHIFT] =3D LBR_NOT_SUPP, + + [PERF_SAMPLE_BRANCH_TYPE_SAVE_SHIFT] =3D LBR_NOT_SUPP, +}; + static int amd_pmu_lbr_setup_filter(struct perf_event *event) { + struct hw_perf_event_extra *reg =3D &event->hw.branch_reg; + u64 br_type =3D event->attr.branch_sample_type; + u64 mask =3D 0, v; + int i; + /* No LBR support */ if (!x86_pmu.lbr_nr) return -EOPNOTSUPP; =20 + for (i =3D 0; i < PERF_SAMPLE_BRANCH_MAX_SHIFT; i++) { + if (!(br_type & BIT_ULL(i))) + continue; + + v =3D lbr_select_map[i]; + if (v =3D=3D LBR_NOT_SUPP) + return -EOPNOTSUPP; + + if (v !=3D LBR_IGNORE) + mask |=3D v; + } + + /* Filter bits operate in suppress mode */ + reg->config =3D mask ^ LBR_SELECT_MASK; + return 0; } =20 @@ -137,6 +214,7 @@ void amd_pmu_lbr_reset(void) =20 cpuc->last_task_ctx =3D NULL; cpuc->last_log_id =3D 0; + wrmsrl(MSR_AMD64_LBR_SELECT, 0); } =20 void amd_pmu_lbr_add(struct perf_event *event) @@ -146,6 +224,11 @@ void amd_pmu_lbr_add(struct perf_event *event) if (!x86_pmu.lbr_nr) return; =20 + if (has_branch_stack(event)) { + cpuc->lbr_select =3D 1; + cpuc->lbr_sel->config =3D event->hw.branch_reg.config; + } + perf_sched_cb_inc(event->ctx->pmu); =20 if (!cpuc->lbr_users++ && !event->total_time_running) @@ -159,6 +242,9 @@ void amd_pmu_lbr_del(struct perf_event *event) if (!x86_pmu.lbr_nr) return; =20 + if (has_branch_stack(event)) + cpuc->lbr_select =3D 0; + cpuc->lbr_users--; WARN_ON_ONCE(cpuc->lbr_users < 0); perf_sched_cb_dec(event->ctx->pmu); @@ -180,11 +266,17 @@ void amd_pmu_lbr_sched_task(struct perf_event_context= *ctx, bool sched_in) void amd_pmu_lbr_enable_all(void) { struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); - u64 dbg_ctl, dbg_extn_cfg; + u64 lbr_select, dbg_ctl, dbg_extn_cfg; =20 if (!cpuc->lbr_users || !x86_pmu.lbr_nr) return; =20 + /* Set hardware branch filter */ + if (cpuc->lbr_select) { + lbr_select =3D cpuc->lbr_sel->config & LBR_SELECT_MASK; + wrmsrl(MSR_AMD64_LBR_SELECT, lbr_select); + } + rdmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl); rdmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg); =20 --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27848C3F6B0 for ; Thu, 11 Aug 2022 12:36:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235288AbiHKMgc (ORCPT ); Thu, 11 Aug 2022 08:36:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235321AbiHKMgV (ORCPT ); Thu, 11 Aug 2022 08:36:21 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2066.outbound.protection.outlook.com [40.107.220.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B601817042; Thu, 11 Aug 2022 05:36:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cQ2ohoWLwCnKeXhZ3U8kEpRTnSsYxyHjK5nIW9xrTexNFZ9TEcE6fUTqwfik90Z+tip0m4hmhiLVNYCUqLU6PYPIedLXk3r8vSEBgW3FGMxTjdD58NW9WChMlJSFLaZTqUu46QuxB8jOdAHxGPrbTNHMq1edzG8BWon6lwLx+j4E85OczGjbmMOqY4BiEssCEAB1lmZh0WQx1goTC5gLZZdepdTEQWXXwycT2Zx0TlK/2zTs5YUiPl7CToK/8Sv9LEzH6PhirHjmpl2rIVoebEjOoLNEvruEO1NM5WwHqHtJKGLnOp+HCYWoDAVCJ6QPYmtfmpkO8H6v00W2ANlKMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7YAL/3Gb9taioYCVCWS/70p75KGTzItf8xr8PZPB4Nc=; b=JrKAH36BMBwQyLQg7NDoKKW3q00bYXVpVrhsG6JAPPl7XpD65QgJgfgFO07X5GHTLdlviHRY//FdOf96ayzI5Jy2hAwrWvB3g0OZcq8bP4n5QFDN5na1eHrZkZacLd9zPd5WCzF9XXFOxazFWkAgFjtqjuoHQYJG3NBquYJp62tswaj37oM1Quq0UhF0TWsMvD+ljSAX9HZgIsX42keBrQ+wxwZ+d1IUSQDVVFRjndiS0cQ+c6Xu+5KfI/j3Gxob12elqYP2Xo/iCZlcMt6br7quuGl2oQWG8JUDYT5aqJK0Qkhff2DL2WqHmeUz6oV6GOTLU2poTc6W38AdCd9oew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7YAL/3Gb9taioYCVCWS/70p75KGTzItf8xr8PZPB4Nc=; b=Y6CVxRF+vCiZlVXXRgQ72VBGEaeRGJkMvTSqU1VqCgr24Lgma1BjhQ+drdTiqPxPNUV9wDjeN3la5QfQ/ZjdC6wXlPKe29s53er/bxdWtHs984vMUStIs7QegKhm0dBN1OX2v5quoZX8ixkHyO0R8xTFbpl6WDTarznZorKJkCs= Received: from MW4PR04CA0077.namprd04.prod.outlook.com (2603:10b6:303:6b::22) by DM5PR12MB2422.namprd12.prod.outlook.com (2603:10b6:4:b9::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.17; Thu, 11 Aug 2022 12:36:14 +0000 Received: from CO1NAM11FT036.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6b:cafe::47) by MW4PR04CA0077.outlook.office365.com (2603:10b6:303:6b::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.20 via Frontend Transport; Thu, 11 Aug 2022 12:36:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT036.mail.protection.outlook.com (10.13.174.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:36:13 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:36:03 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 08/13] perf/x86: Move branch classifier Date: Thu, 11 Aug 2022 17:59:56 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 66be2a11-5b22-4c37-a169-08da7b961416 X-MS-TrafficTypeDiagnostic: DM5PR12MB2422:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fg2Xp3laCIVxIG2bjF/vuNgRO6QrdXr/TGmnUfQ+Bhi0nTHDdBYdSKj9dmIsoe7gxPSKuhKatBdbxyMHopUrwY1ok1AnpGf0wIWXHzCwmyM02CKAavn947HUghvSNwFJKtLIkfw5iEYtkuLzHDQr9l5229FWhF9o5YR2w4xAkMSvvCuM5FqtMNohi2ONkP76cedo/p+q4UW7nx4hr3esEKnPKoDBaIeRJdvrwkL1RQuGMsxr4u9RmSRqWMInVSkwevIiq9wVxTPp1BIisc/Plotc7dvjp6wd1tceaSXnJIWmze0opsuX3c2ZothFJhFVvLBxmBGRC5ah6Nm3zSkDMTVrI6aazcL8lQ1lpdfwuCNRTALYpj39GWxgJrZseqiyWBL9jiU3lF3GY6jER7Neyvvbl6T0NLW47NOcQbortxd2HGOr/hkWDR3Dg/W12YOF7PnW/YC8yjBuaPUSW4G+qLNc+EY7/YC1S3k0XysgF+9Nqk8A38nFBq79RqML3HtXKx8j9KqlwQM2ys96KzcBZpFdW+VVKcO9o+3NRmrSseuU7iWZa8fn07otXRa8fuWGCWY0ulgQpVnsX3E/B5JSXFjjC1Tx/QBY7UGitBItEdRxbp+vkCwkgkZv6ADhBBNbCTLDtD7q0Zv3qSZa8AYT4Un/Mg9g2eU7u1ZR3Kcs/lF0j899TFJClCW94HmzbtoZF9IROwVMp8nzp3lwWbPU0nE/5prX9uLac5yqb/kX0yXBUFzQpYsEJGk8kLCsJWOby6tKDNqPz4FkHH00J1jmYHjzByh63dmroOQsao0eVv6rLgH9lPchc4TpbrKbeJYmYMehzJoYrBX6uyUN6DRX0g== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(376002)(346002)(39860400002)(136003)(396003)(36840700001)(46966006)(40470700004)(356005)(47076005)(81166007)(82740400003)(186003)(16526019)(26005)(2616005)(336012)(6666004)(478600001)(41300700001)(8676002)(40480700001)(54906003)(110136005)(7696005)(86362001)(4326008)(316002)(5660300002)(40460700003)(82310400005)(70206006)(70586007)(426003)(36860700001)(83380400001)(36756003)(44832011)(2906002)(7416002)(8936002)(30864003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:36:13.6809 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 66be2a11-5b22-4c37-a169-08da7b961416 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT036.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2422 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Commit 3e702ff6d1ea ("perf/x86: Add LBR software filter support for Intel CPUs") introduces a software branch filter which complements the hardware branch filter and adds an x86 branch classifier. Move the branch classifier to arch/x86/events/ so that it can be utilized by other vendors for branch record filtering. Signed-off-by: Sandipan Das --- arch/x86/events/Makefile | 2 +- arch/x86/events/intel/lbr.c | 273 ----------------------------------- arch/x86/events/perf_event.h | 62 ++++++++ arch/x86/events/utils.c | 216 +++++++++++++++++++++++++++ 4 files changed, 279 insertions(+), 274 deletions(-) create mode 100644 arch/x86/events/utils.c diff --git a/arch/x86/events/Makefile b/arch/x86/events/Makefile index 9933c0e8e97a..86a76efa8bb6 100644 --- a/arch/x86/events/Makefile +++ b/arch/x86/events/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -obj-y +=3D core.o probe.o +obj-y +=3D core.o probe.o utils.o obj-$(CONFIG_PERF_EVENTS_INTEL_RAPL) +=3D rapl.o obj-y +=3D amd/ obj-$(CONFIG_X86_LOCAL_APIC) +=3D msr.o diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c index 13179f31fe10..d0dc1c358ec0 100644 --- a/arch/x86/events/intel/lbr.c +++ b/arch/x86/events/intel/lbr.c @@ -4,7 +4,6 @@ =20 #include #include -#include =20 #include "../perf_event.h" =20 @@ -65,65 +64,6 @@ =20 #define LBR_FROM_SIGNEXT_2MSB (BIT_ULL(60) | BIT_ULL(59)) =20 -/* - * x86control flow change classification - * x86control flow changes include branches, interrupts, traps, faults - */ -enum { - X86_BR_NONE =3D 0, /* unknown */ - - X86_BR_USER =3D 1 << 0, /* branch target is user */ - X86_BR_KERNEL =3D 1 << 1, /* branch target is kernel */ - - X86_BR_CALL =3D 1 << 2, /* call */ - X86_BR_RET =3D 1 << 3, /* return */ - X86_BR_SYSCALL =3D 1 << 4, /* syscall */ - X86_BR_SYSRET =3D 1 << 5, /* syscall return */ - X86_BR_INT =3D 1 << 6, /* sw interrupt */ - X86_BR_IRET =3D 1 << 7, /* return from interrupt */ - X86_BR_JCC =3D 1 << 8, /* conditional */ - X86_BR_JMP =3D 1 << 9, /* jump */ - X86_BR_IRQ =3D 1 << 10,/* hw interrupt or trap or fault */ - X86_BR_IND_CALL =3D 1 << 11,/* indirect calls */ - X86_BR_ABORT =3D 1 << 12,/* transaction abort */ - X86_BR_IN_TX =3D 1 << 13,/* in transaction */ - X86_BR_NO_TX =3D 1 << 14,/* not in transaction */ - X86_BR_ZERO_CALL =3D 1 << 15,/* zero length call */ - X86_BR_CALL_STACK =3D 1 << 16,/* call stack */ - X86_BR_IND_JMP =3D 1 << 17,/* indirect jump */ - - X86_BR_TYPE_SAVE =3D 1 << 18,/* indicate to save branch type */ - -}; - -#define X86_BR_PLM (X86_BR_USER | X86_BR_KERNEL) -#define X86_BR_ANYTX (X86_BR_NO_TX | X86_BR_IN_TX) - -#define X86_BR_ANY \ - (X86_BR_CALL |\ - X86_BR_RET |\ - X86_BR_SYSCALL |\ - X86_BR_SYSRET |\ - X86_BR_INT |\ - X86_BR_IRET |\ - X86_BR_JCC |\ - X86_BR_JMP |\ - X86_BR_IRQ |\ - X86_BR_ABORT |\ - X86_BR_IND_CALL |\ - X86_BR_IND_JMP |\ - X86_BR_ZERO_CALL) - -#define X86_BR_ALL (X86_BR_PLM | X86_BR_ANY) - -#define X86_BR_ANY_CALL \ - (X86_BR_CALL |\ - X86_BR_IND_CALL |\ - X86_BR_ZERO_CALL |\ - X86_BR_SYSCALL |\ - X86_BR_IRQ |\ - X86_BR_INT) - /* * Intel LBR_CTL bits * @@ -1143,219 +1083,6 @@ int intel_pmu_setup_lbr_filter(struct perf_event *e= vent) return ret; } =20 -/* - * return the type of control flow change at address "from" - * instruction is not necessarily a branch (in case of interrupt). - * - * The branch type returned also includes the priv level of the - * target of the control flow change (X86_BR_USER, X86_BR_KERNEL). - * - * If a branch type is unknown OR the instruction cannot be - * decoded (e.g., text page not present), then X86_BR_NONE is - * returned. - */ -static int branch_type(unsigned long from, unsigned long to, int abort) -{ - struct insn insn; - void *addr; - int bytes_read, bytes_left; - int ret =3D X86_BR_NONE; - int ext, to_plm, from_plm; - u8 buf[MAX_INSN_SIZE]; - int is64 =3D 0; - - to_plm =3D kernel_ip(to) ? X86_BR_KERNEL : X86_BR_USER; - from_plm =3D kernel_ip(from) ? X86_BR_KERNEL : X86_BR_USER; - - /* - * maybe zero if lbr did not fill up after a reset by the time - * we get a PMU interrupt - */ - if (from =3D=3D 0 || to =3D=3D 0) - return X86_BR_NONE; - - if (abort) - return X86_BR_ABORT | to_plm; - - if (from_plm =3D=3D X86_BR_USER) { - /* - * can happen if measuring at the user level only - * and we interrupt in a kernel thread, e.g., idle. - */ - if (!current->mm) - return X86_BR_NONE; - - /* may fail if text not present */ - bytes_left =3D copy_from_user_nmi(buf, (void __user *)from, - MAX_INSN_SIZE); - bytes_read =3D MAX_INSN_SIZE - bytes_left; - if (!bytes_read) - return X86_BR_NONE; - - addr =3D buf; - } else { - /* - * The LBR logs any address in the IP, even if the IP just - * faulted. This means userspace can control the from address. - * Ensure we don't blindly read any address by validating it is - * a known text address. - */ - if (kernel_text_address(from)) { - addr =3D (void *)from; - /* - * Assume we can get the maximum possible size - * when grabbing kernel data. This is not - * _strictly_ true since we could possibly be - * executing up next to a memory hole, but - * it is very unlikely to be a problem. - */ - bytes_read =3D MAX_INSN_SIZE; - } else { - return X86_BR_NONE; - } - } - - /* - * decoder needs to know the ABI especially - * on 64-bit systems running 32-bit apps - */ -#ifdef CONFIG_X86_64 - is64 =3D kernel_ip((unsigned long)addr) || any_64bit_mode(current_pt_regs= ()); -#endif - insn_init(&insn, addr, bytes_read, is64); - if (insn_get_opcode(&insn)) - return X86_BR_ABORT; - - switch (insn.opcode.bytes[0]) { - case 0xf: - switch (insn.opcode.bytes[1]) { - case 0x05: /* syscall */ - case 0x34: /* sysenter */ - ret =3D X86_BR_SYSCALL; - break; - case 0x07: /* sysret */ - case 0x35: /* sysexit */ - ret =3D X86_BR_SYSRET; - break; - case 0x80 ... 0x8f: /* conditional */ - ret =3D X86_BR_JCC; - break; - default: - ret =3D X86_BR_NONE; - } - break; - case 0x70 ... 0x7f: /* conditional */ - ret =3D X86_BR_JCC; - break; - case 0xc2: /* near ret */ - case 0xc3: /* near ret */ - case 0xca: /* far ret */ - case 0xcb: /* far ret */ - ret =3D X86_BR_RET; - break; - case 0xcf: /* iret */ - ret =3D X86_BR_IRET; - break; - case 0xcc ... 0xce: /* int */ - ret =3D X86_BR_INT; - break; - case 0xe8: /* call near rel */ - if (insn_get_immediate(&insn) || insn.immediate1.value =3D=3D 0) { - /* zero length call */ - ret =3D X86_BR_ZERO_CALL; - break; - } - fallthrough; - case 0x9a: /* call far absolute */ - ret =3D X86_BR_CALL; - break; - case 0xe0 ... 0xe3: /* loop jmp */ - ret =3D X86_BR_JCC; - break; - case 0xe9 ... 0xeb: /* jmp */ - ret =3D X86_BR_JMP; - break; - case 0xff: /* call near absolute, call far absolute ind */ - if (insn_get_modrm(&insn)) - return X86_BR_ABORT; - - ext =3D (insn.modrm.bytes[0] >> 3) & 0x7; - switch (ext) { - case 2: /* near ind call */ - case 3: /* far ind call */ - ret =3D X86_BR_IND_CALL; - break; - case 4: - case 5: - ret =3D X86_BR_IND_JMP; - break; - } - break; - default: - ret =3D X86_BR_NONE; - } - /* - * interrupts, traps, faults (and thus ring transition) may - * occur on any instructions. Thus, to classify them correctly, - * we need to first look at the from and to priv levels. If they - * are different and to is in the kernel, then it indicates - * a ring transition. If the from instruction is not a ring - * transition instr (syscall, systenter, int), then it means - * it was a irq, trap or fault. - * - * we have no way of detecting kernel to kernel faults. - */ - if (from_plm =3D=3D X86_BR_USER && to_plm =3D=3D X86_BR_KERNEL - && ret !=3D X86_BR_SYSCALL && ret !=3D X86_BR_INT) - ret =3D X86_BR_IRQ; - - /* - * branch priv level determined by target as - * is done by HW when LBR_SELECT is implemented - */ - if (ret !=3D X86_BR_NONE) - ret |=3D to_plm; - - return ret; -} - -#define X86_BR_TYPE_MAP_MAX 16 - -static int branch_map[X86_BR_TYPE_MAP_MAX] =3D { - PERF_BR_CALL, /* X86_BR_CALL */ - PERF_BR_RET, /* X86_BR_RET */ - PERF_BR_SYSCALL, /* X86_BR_SYSCALL */ - PERF_BR_SYSRET, /* X86_BR_SYSRET */ - PERF_BR_UNKNOWN, /* X86_BR_INT */ - PERF_BR_ERET, /* X86_BR_IRET */ - PERF_BR_COND, /* X86_BR_JCC */ - PERF_BR_UNCOND, /* X86_BR_JMP */ - PERF_BR_IRQ, /* X86_BR_IRQ */ - PERF_BR_IND_CALL, /* X86_BR_IND_CALL */ - PERF_BR_UNKNOWN, /* X86_BR_ABORT */ - PERF_BR_UNKNOWN, /* X86_BR_IN_TX */ - PERF_BR_UNKNOWN, /* X86_BR_NO_TX */ - PERF_BR_CALL, /* X86_BR_ZERO_CALL */ - PERF_BR_UNKNOWN, /* X86_BR_CALL_STACK */ - PERF_BR_IND, /* X86_BR_IND_JMP */ -}; - -static int -common_branch_type(int type) -{ - int i; - - type >>=3D 2; /* skip X86_BR_USER and X86_BR_KERNEL */ - - if (type) { - i =3D __ffs(type); - if (i < X86_BR_TYPE_MAP_MAX) - return branch_map[i]; - } - - return PERF_BR_UNKNOWN; -} - enum { ARCH_LBR_BR_TYPE_JCC =3D 0, ARCH_LBR_BR_TYPE_NEAR_IND_JMP =3D 1, diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h index c8397290f388..8ac8f9e6affb 100644 --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -1208,6 +1208,68 @@ static inline void set_linear_ip(struct pt_regs *reg= s, unsigned long ip) regs->ip =3D ip; } =20 +/* + * x86control flow change classification + * x86control flow changes include branches, interrupts, traps, faults + */ +enum { + X86_BR_NONE =3D 0, /* unknown */ + + X86_BR_USER =3D 1 << 0, /* branch target is user */ + X86_BR_KERNEL =3D 1 << 1, /* branch target is kernel */ + + X86_BR_CALL =3D 1 << 2, /* call */ + X86_BR_RET =3D 1 << 3, /* return */ + X86_BR_SYSCALL =3D 1 << 4, /* syscall */ + X86_BR_SYSRET =3D 1 << 5, /* syscall return */ + X86_BR_INT =3D 1 << 6, /* sw interrupt */ + X86_BR_IRET =3D 1 << 7, /* return from interrupt */ + X86_BR_JCC =3D 1 << 8, /* conditional */ + X86_BR_JMP =3D 1 << 9, /* jump */ + X86_BR_IRQ =3D 1 << 10,/* hw interrupt or trap or fault */ + X86_BR_IND_CALL =3D 1 << 11,/* indirect calls */ + X86_BR_ABORT =3D 1 << 12,/* transaction abort */ + X86_BR_IN_TX =3D 1 << 13,/* in transaction */ + X86_BR_NO_TX =3D 1 << 14,/* not in transaction */ + X86_BR_ZERO_CALL =3D 1 << 15,/* zero length call */ + X86_BR_CALL_STACK =3D 1 << 16,/* call stack */ + X86_BR_IND_JMP =3D 1 << 17,/* indirect jump */ + + X86_BR_TYPE_SAVE =3D 1 << 18,/* indicate to save branch type */ + +}; + +#define X86_BR_PLM (X86_BR_USER | X86_BR_KERNEL) +#define X86_BR_ANYTX (X86_BR_NO_TX | X86_BR_IN_TX) + +#define X86_BR_ANY \ + (X86_BR_CALL |\ + X86_BR_RET |\ + X86_BR_SYSCALL |\ + X86_BR_SYSRET |\ + X86_BR_INT |\ + X86_BR_IRET |\ + X86_BR_JCC |\ + X86_BR_JMP |\ + X86_BR_IRQ |\ + X86_BR_ABORT |\ + X86_BR_IND_CALL |\ + X86_BR_IND_JMP |\ + X86_BR_ZERO_CALL) + +#define X86_BR_ALL (X86_BR_PLM | X86_BR_ANY) + +#define X86_BR_ANY_CALL \ + (X86_BR_CALL |\ + X86_BR_IND_CALL |\ + X86_BR_ZERO_CALL |\ + X86_BR_SYSCALL |\ + X86_BR_IRQ |\ + X86_BR_INT) + +int common_branch_type(int type); +int branch_type(unsigned long from, unsigned long to, int abort); + ssize_t x86_event_sysfs_show(char *page, u64 config, u64 event); ssize_t intel_event_sysfs_show(char *page, u64 config); =20 diff --git a/arch/x86/events/utils.c b/arch/x86/events/utils.c new file mode 100644 index 000000000000..a32368945462 --- /dev/null +++ b/arch/x86/events/utils.c @@ -0,0 +1,216 @@ +// SPDX-License-Identifier: GPL-2.0 +#include + +#include "perf_event.h" + +/* + * return the type of control flow change at address "from" + * instruction is not necessarily a branch (in case of interrupt). + * + * The branch type returned also includes the priv level of the + * target of the control flow change (X86_BR_USER, X86_BR_KERNEL). + * + * If a branch type is unknown OR the instruction cannot be + * decoded (e.g., text page not present), then X86_BR_NONE is + * returned. + */ +int branch_type(unsigned long from, unsigned long to, int abort) +{ + struct insn insn; + void *addr; + int bytes_read, bytes_left; + int ret =3D X86_BR_NONE; + int ext, to_plm, from_plm; + u8 buf[MAX_INSN_SIZE]; + int is64 =3D 0; + + to_plm =3D kernel_ip(to) ? X86_BR_KERNEL : X86_BR_USER; + from_plm =3D kernel_ip(from) ? X86_BR_KERNEL : X86_BR_USER; + + /* + * maybe zero if lbr did not fill up after a reset by the time + * we get a PMU interrupt + */ + if (from =3D=3D 0 || to =3D=3D 0) + return X86_BR_NONE; + + if (abort) + return X86_BR_ABORT | to_plm; + + if (from_plm =3D=3D X86_BR_USER) { + /* + * can happen if measuring at the user level only + * and we interrupt in a kernel thread, e.g., idle. + */ + if (!current->mm) + return X86_BR_NONE; + + /* may fail if text not present */ + bytes_left =3D copy_from_user_nmi(buf, (void __user *)from, + MAX_INSN_SIZE); + bytes_read =3D MAX_INSN_SIZE - bytes_left; + if (!bytes_read) + return X86_BR_NONE; + + addr =3D buf; + } else { + /* + * The LBR logs any address in the IP, even if the IP just + * faulted. This means userspace can control the from address. + * Ensure we don't blindly read any address by validating it is + * a known text address. + */ + if (kernel_text_address(from)) { + addr =3D (void *)from; + /* + * Assume we can get the maximum possible size + * when grabbing kernel data. This is not + * _strictly_ true since we could possibly be + * executing up next to a memory hole, but + * it is very unlikely to be a problem. + */ + bytes_read =3D MAX_INSN_SIZE; + } else { + return X86_BR_NONE; + } + } + + /* + * decoder needs to know the ABI especially + * on 64-bit systems running 32-bit apps + */ +#ifdef CONFIG_X86_64 + is64 =3D kernel_ip((unsigned long)addr) || any_64bit_mode(current_pt_regs= ()); +#endif + insn_init(&insn, addr, bytes_read, is64); + if (insn_get_opcode(&insn)) + return X86_BR_ABORT; + + switch (insn.opcode.bytes[0]) { + case 0xf: + switch (insn.opcode.bytes[1]) { + case 0x05: /* syscall */ + case 0x34: /* sysenter */ + ret =3D X86_BR_SYSCALL; + break; + case 0x07: /* sysret */ + case 0x35: /* sysexit */ + ret =3D X86_BR_SYSRET; + break; + case 0x80 ... 0x8f: /* conditional */ + ret =3D X86_BR_JCC; + break; + default: + ret =3D X86_BR_NONE; + } + break; + case 0x70 ... 0x7f: /* conditional */ + ret =3D X86_BR_JCC; + break; + case 0xc2: /* near ret */ + case 0xc3: /* near ret */ + case 0xca: /* far ret */ + case 0xcb: /* far ret */ + ret =3D X86_BR_RET; + break; + case 0xcf: /* iret */ + ret =3D X86_BR_IRET; + break; + case 0xcc ... 0xce: /* int */ + ret =3D X86_BR_INT; + break; + case 0xe8: /* call near rel */ + if (insn_get_immediate(&insn) || insn.immediate1.value =3D=3D 0) { + /* zero length call */ + ret =3D X86_BR_ZERO_CALL; + break; + } + fallthrough; + case 0x9a: /* call far absolute */ + ret =3D X86_BR_CALL; + break; + case 0xe0 ... 0xe3: /* loop jmp */ + ret =3D X86_BR_JCC; + break; + case 0xe9 ... 0xeb: /* jmp */ + ret =3D X86_BR_JMP; + break; + case 0xff: /* call near absolute, call far absolute ind */ + if (insn_get_modrm(&insn)) + return X86_BR_ABORT; + + ext =3D (insn.modrm.bytes[0] >> 3) & 0x7; + switch (ext) { + case 2: /* near ind call */ + case 3: /* far ind call */ + ret =3D X86_BR_IND_CALL; + break; + case 4: + case 5: + ret =3D X86_BR_IND_JMP; + break; + } + break; + default: + ret =3D X86_BR_NONE; + } + /* + * interrupts, traps, faults (and thus ring transition) may + * occur on any instructions. Thus, to classify them correctly, + * we need to first look at the from and to priv levels. If they + * are different and to is in the kernel, then it indicates + * a ring transition. If the from instruction is not a ring + * transition instr (syscall, systenter, int), then it means + * it was a irq, trap or fault. + * + * we have no way of detecting kernel to kernel faults. + */ + if (from_plm =3D=3D X86_BR_USER && to_plm =3D=3D X86_BR_KERNEL + && ret !=3D X86_BR_SYSCALL && ret !=3D X86_BR_INT) + ret =3D X86_BR_IRQ; + + /* + * branch priv level determined by target as + * is done by HW when LBR_SELECT is implemented + */ + if (ret !=3D X86_BR_NONE) + ret |=3D to_plm; + + return ret; +} + +#define X86_BR_TYPE_MAP_MAX 16 + +static int branch_map[X86_BR_TYPE_MAP_MAX] =3D { + PERF_BR_CALL, /* X86_BR_CALL */ + PERF_BR_RET, /* X86_BR_RET */ + PERF_BR_SYSCALL, /* X86_BR_SYSCALL */ + PERF_BR_SYSRET, /* X86_BR_SYSRET */ + PERF_BR_UNKNOWN, /* X86_BR_INT */ + PERF_BR_ERET, /* X86_BR_IRET */ + PERF_BR_COND, /* X86_BR_JCC */ + PERF_BR_UNCOND, /* X86_BR_JMP */ + PERF_BR_IRQ, /* X86_BR_IRQ */ + PERF_BR_IND_CALL, /* X86_BR_IND_CALL */ + PERF_BR_UNKNOWN, /* X86_BR_ABORT */ + PERF_BR_UNKNOWN, /* X86_BR_IN_TX */ + PERF_BR_UNKNOWN, /* X86_BR_NO_TX */ + PERF_BR_CALL, /* X86_BR_ZERO_CALL */ + PERF_BR_UNKNOWN, /* X86_BR_CALL_STACK */ + PERF_BR_IND, /* X86_BR_IND_JMP */ +}; + +int common_branch_type(int type) +{ + int i; + + type >>=3D 2; /* skip X86_BR_USER and X86_BR_KERNEL */ + + if (type) { + i =3D __ffs(type); + if (i < X86_BR_TYPE_MAP_MAX) + return branch_map[i]; + } + + return PERF_BR_UNKNOWN; +} --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3995C25B0C for ; Thu, 11 Aug 2022 12:36:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235291AbiHKMgz (ORCPT ); Thu, 11 Aug 2022 08:36:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235283AbiHKMgw (ORCPT ); Thu, 11 Aug 2022 08:36:52 -0400 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2073.outbound.protection.outlook.com [40.107.237.73]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 560111144E; Thu, 11 Aug 2022 05:36:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U+c4TuCNOEQzppBX1dziu6tiSooHHZmMUw8n2axmugdB2TkAR0n4OPiI7Sgx/GwoXIVwGcMi55BwImMOX6xmgG0+nCn4w8CScMgmcuDpOHUsCBUCpvhhOjfUQRYZNKHXBa/xtuSDv4q6iZIMiXYbRvzwTTsHLebeXn5M65trBWt09LCsFG87O9p0OAgkdfWyWGX8C93qEf+1kGiZz5ZZ0zJ9NysoM2TlFYfT12Wl/WqpCBDf3RNAoABK7o0nnpZEBed49YKVDoLie8rg9g6GkwPv6I3f81ubYHuba1klptZWAks69Y1ZiJbU5MNImGTcWZP673YkSJnOMC50a2E4yA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aRhbQZlBRmMsALmcrlo+FaGXFHPTpoQk3uvWQWSEYhg=; b=DWFVX8AmzTcV+YcsQAUdOIo1ZSDhgfRH4v6IJALNhifOuRab8fmxXxbM7tf/IuSEQBqm+3NxM4mWozGSV2gnmJao0SEA2gctuQaZKTZvrgJro/kN87TCR7P6GBRFvxC4JemitVgwl5oF5fF7DPfyu+15V8/D1F96wzDLOZkYKwRC7VCftwpRrWWN64VoKvKq7orqEnelDRXFVKivbp24+Y8tGmMAq1c//KNsJY1kHN8oRERpdG+v3A/1xhduMGBCVRo4cCnzsB2XkL+FQ7dWlfYGJZ6l1Lr2kAXYLG3P2ctttDCfMmnYiXnLcgvVHfMOoEgYBrPMA1ZtUbxL1KiScg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aRhbQZlBRmMsALmcrlo+FaGXFHPTpoQk3uvWQWSEYhg=; b=R4wt/YkKJfEOGh2m+1Z3apkB5T/x8GwPpEF1EiUDewKpN0WDojbnLdrlCX/JLUPZ6KAIx2jT0C1oVXZ1KVqfAU7da+/Rw3LpwCl9KBe1IhpywiKDM5aB0G6/RPAt1aE0GDIX9EU9aVKvdjJWLMHn9QFRy5V3BbvkKABEIURgLns= Received: from MW4PR04CA0302.namprd04.prod.outlook.com (2603:10b6:303:82::7) by BY5PR12MB3970.namprd12.prod.outlook.com (2603:10b6:a03:1ac::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Thu, 11 Aug 2022 12:36:47 +0000 Received: from CO1NAM11FT041.eop-nam11.prod.protection.outlook.com (2603:10b6:303:82:cafe::81) by MW4PR04CA0302.outlook.office365.com (2603:10b6:303:82::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:36:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT041.mail.protection.outlook.com (10.13.174.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:36:46 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:36:41 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 09/13] perf/x86/amd/lbr: Add LbrExtV2 software branch filter support Date: Thu, 11 Aug 2022 17:59:57 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ca1782fb-954e-4608-71c5-08da7b9627d9 X-MS-TrafficTypeDiagnostic: BY5PR12MB3970:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: demYzxeE0FyiJOmptuvCEvVJxqr07x1jcnafo5bAlDd78ar0pXDDjjwuyWBVgAgF/EnTAfcI68DwHui5jkIKo5S35Nx/o8Uwz9UQFSk8qmTzwZ73i81YLOhrmdr1qub5YHDI/p6VqsuygJ3Pop5tpWG3PbYWHuJfQkt89eOz56Y4Xrjqvb9YMk3zcB9X2BYRFGqPCqP++R6KEnPoNeN7Swwrsm9ZY7XyqGkGhmiOuXi4UodrN46JpLrqFvndgNQpt3RduoOVjwS5Y7C7XT50Kr2eNOzUfjdadXXbPeS4Abv5U2jVWjkuxYNRO3VPSzR29NsOtA7oFlPb155AdhyjSqTPj8NMeJJ2nE4uloMmdFELQogb+73zBAae8/gR2TpzcGB3J2aIgl0/lQQDR4mIBpgjI43NgB6I4F+6vE6RAxyv5QseVXJqWXs7OPoLHMjrsoOeJXPNRRwpQ5f1eJAAzn9jHTeZ5L44IwjpqGtucLO1lDJFaZeX0ag4CrtTQSa2IQTG9UxaqGGjA/Qz7wOOYlcoGMUb/2bVeOiWd5uw/DNQ1ZLcUiZpDh8C4o5dTCAmfkG4ctGftK+RRb+lOQHGPB7xGhQe+8PqcsZ1/9wpae501EuMNX0nL1a/Nh40gA+FFzVTNXFqPttKmI3To3ME+vfhjJ4ZLk4BWo2KlLOsfx/rDEoiOnvPsPxH3pbEuxoCOKQTJFgXYfIbFwkq3gshusWwehleZI21rFaUVkZ9U243ahQfZfwvLMc6dnHSVHVtaKXqoBu1t6DYD66xsh0hpdz0igQiSQkswzw5od8pDM5czarRHJ9te0XBUKSn7uLAUWSm2IkvQYEYNdiUZcHiGg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(136003)(396003)(39860400002)(376002)(346002)(40470700004)(36840700001)(46966006)(41300700001)(6666004)(7696005)(478600001)(26005)(83380400001)(426003)(47076005)(40460700003)(336012)(2616005)(2906002)(186003)(44832011)(7416002)(16526019)(5660300002)(40480700001)(82310400005)(316002)(54906003)(110136005)(8936002)(8676002)(4326008)(70206006)(82740400003)(86362001)(81166007)(70586007)(36756003)(356005)(36860700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:36:46.8188 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ca1782fb-954e-4608-71c5-08da7b9627d9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT041.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB3970 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" With AMD Last Branch Record Extension Version 2 (LbrExtV2), it is necessary to process the branch records further as hardware filtering is not granular enough for identifying certain types of branches. E.g. to record system calls, one should record far branches. The filter captures both far calls and far returns but the irrelevant records are filtered out based on the branch type as seen by the branch classifier. Signed-off-by: Sandipan Das --- arch/x86/events/amd/lbr.c | 92 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 87 insertions(+), 5 deletions(-) diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c index 064a6c8e1597..3699ba53a326 100644 --- a/arch/x86/events/amd/lbr.c +++ b/arch/x86/events/amd/lbr.c @@ -94,6 +94,50 @@ static __always_inline u64 sign_ext_branch_ip(u64 ip) return (u64)(((s64)ip << shift) >> shift); } =20 +static void amd_pmu_lbr_filter(void) +{ + struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + int br_sel =3D cpuc->br_sel, type, i, j; + bool compress =3D false; + u64 from, to; + + /* If sampling all branches, there is nothing to filter */ + if (((br_sel & X86_BR_ALL) =3D=3D X86_BR_ALL) && + ((br_sel & X86_BR_TYPE_SAVE) !=3D X86_BR_TYPE_SAVE)) + return; + + for (i =3D 0; i < cpuc->lbr_stack.nr; i++) { + from =3D cpuc->lbr_entries[i].from; + to =3D cpuc->lbr_entries[i].to; + type =3D branch_type(from, to, 0); + + /* If type does not correspond, then discard */ + if (type =3D=3D X86_BR_NONE || (br_sel & type) !=3D type) { + cpuc->lbr_entries[i].from =3D 0; /* mark invalid */ + compress =3D true; + } + + if ((br_sel & X86_BR_TYPE_SAVE) =3D=3D X86_BR_TYPE_SAVE) + cpuc->lbr_entries[i].type =3D common_branch_type(type); + } + + if (!compress) + return; + + /* Remove all invalid entries */ + for (i =3D 0; i < cpuc->lbr_stack.nr; ) { + if (!cpuc->lbr_entries[i].from) { + j =3D i; + while (++j < cpuc->lbr_stack.nr) + cpuc->lbr_entries[j - 1] =3D cpuc->lbr_entries[j]; + cpuc->lbr_stack.nr--; + if (!cpuc->lbr_entries[i].from) + continue; + } + i++; + } +} + void amd_pmu_lbr_read(void) { struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); @@ -128,6 +172,9 @@ void amd_pmu_lbr_read(void) * LBR To[0] always represent the TOS */ cpuc->lbr_stack.hw_idx =3D 0; + + /* Perform further software filtering */ + amd_pmu_lbr_filter(); } =20 static const int lbr_select_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] =3D { @@ -136,8 +183,8 @@ static const int lbr_select_map[PERF_SAMPLE_BRANCH_MAX_= SHIFT] =3D { [PERF_SAMPLE_BRANCH_HV_SHIFT] =3D LBR_IGNORE, =20 [PERF_SAMPLE_BRANCH_ANY_SHIFT] =3D LBR_ANY, - [PERF_SAMPLE_BRANCH_ANY_CALL_SHIFT] =3D LBR_REL_CALL | LBR_IND_CALL, - [PERF_SAMPLE_BRANCH_ANY_RETURN_SHIFT] =3D LBR_RETURN, + [PERF_SAMPLE_BRANCH_ANY_CALL_SHIFT] =3D LBR_REL_CALL | LBR_IND_CALL | LBR= _FAR, + [PERF_SAMPLE_BRANCH_ANY_RETURN_SHIFT] =3D LBR_RETURN | LBR_FAR, [PERF_SAMPLE_BRANCH_IND_CALL_SHIFT] =3D LBR_IND_CALL, [PERF_SAMPLE_BRANCH_ABORT_TX_SHIFT] =3D LBR_NOT_SUPP, [PERF_SAMPLE_BRANCH_IN_TX_SHIFT] =3D LBR_NOT_SUPP, @@ -150,8 +197,6 @@ static const int lbr_select_map[PERF_SAMPLE_BRANCH_MAX_= SHIFT] =3D { =20 [PERF_SAMPLE_BRANCH_NO_FLAGS_SHIFT] =3D LBR_NOT_SUPP, [PERF_SAMPLE_BRANCH_NO_CYCLES_SHIFT] =3D LBR_NOT_SUPP, - - [PERF_SAMPLE_BRANCH_TYPE_SAVE_SHIFT] =3D LBR_NOT_SUPP, }; =20 static int amd_pmu_lbr_setup_filter(struct perf_event *event) @@ -165,6 +210,41 @@ static int amd_pmu_lbr_setup_filter(struct perf_event = *event) if (!x86_pmu.lbr_nr) return -EOPNOTSUPP; =20 + if (br_type & PERF_SAMPLE_BRANCH_USER) + mask |=3D X86_BR_USER; + + if (br_type & PERF_SAMPLE_BRANCH_KERNEL) + mask |=3D X86_BR_KERNEL; + + /* Ignore BRANCH_HV here */ + + if (br_type & PERF_SAMPLE_BRANCH_ANY) + mask |=3D X86_BR_ANY; + + if (br_type & PERF_SAMPLE_BRANCH_ANY_CALL) + mask |=3D X86_BR_ANY_CALL; + + if (br_type & PERF_SAMPLE_BRANCH_ANY_RETURN) + mask |=3D X86_BR_RET | X86_BR_IRET | X86_BR_SYSRET; + + if (br_type & PERF_SAMPLE_BRANCH_IND_CALL) + mask |=3D X86_BR_IND_CALL; + + if (br_type & PERF_SAMPLE_BRANCH_COND) + mask |=3D X86_BR_JCC; + + if (br_type & PERF_SAMPLE_BRANCH_IND_JUMP) + mask |=3D X86_BR_IND_JMP; + + if (br_type & PERF_SAMPLE_BRANCH_CALL) + mask |=3D X86_BR_CALL | X86_BR_ZERO_CALL; + + if (br_type & PERF_SAMPLE_BRANCH_TYPE_SAVE) + mask |=3D X86_BR_TYPE_SAVE; + + reg->reg =3D mask; + mask =3D 0; + for (i =3D 0; i < PERF_SAMPLE_BRANCH_MAX_SHIFT; i++) { if (!(br_type & BIT_ULL(i))) continue; @@ -220,13 +300,15 @@ void amd_pmu_lbr_reset(void) void amd_pmu_lbr_add(struct perf_event *event) { struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); + struct hw_perf_event_extra *reg =3D &event->hw.branch_reg; =20 if (!x86_pmu.lbr_nr) return; =20 if (has_branch_stack(event)) { cpuc->lbr_select =3D 1; - cpuc->lbr_sel->config =3D event->hw.branch_reg.config; + cpuc->lbr_sel->config =3D reg->config; + cpuc->br_sel =3D reg->reg; } =20 perf_sched_cb_inc(event->ctx->pmu); --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41D41C19F2A for ; Thu, 11 Aug 2022 12:38:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235301AbiHKMii (ORCPT ); Thu, 11 Aug 2022 08:38:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235283AbiHKMif (ORCPT ); Thu, 11 Aug 2022 08:38:35 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2049.outbound.protection.outlook.com [40.107.244.49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB0FD5245E; Thu, 11 Aug 2022 05:38:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F4raxouhj3+vy+0d7dgpSedEijZ5vRcdMmwtjfchkHV+c7XEiclBoFwlfUCR1go4F5OYT4x6G9AwEk3CjIKn8pdt23MJZ0+Q4NWZtS4yhXuIGluS1qatHkfp+w2WyL11HGeqzzF4fGXmU5NEgKdDbwG0O7wyvQjio59j/MmUW+xq8HeRloZXnE5lMQgL65BIoybQpkCAB1G/F3feEnqLNlhvNLj5sCIRsdX3KPs42/AMG9XWRA+sUg4qBK8FjT3MrAX0GZ8vCHMWkzlCcrMlOTZ9xKUavdY68vspSamfQ+Uv02HFRUQ8Vo2xtKw7apnO6W/OfGTPEwJxUcn1A91ttQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0F3Qt7+frbXmBj+cK0wsc3uIDhXET7lHywC5bF9h910=; b=AY0XBIqfXTOGcCb9TC3ptzAYwYBcyA4sFI0FTXjQQqv2v3YsNxoepuEHHgshrbe+XX7n/FyTg99hb7tEyhdjdkxsRz1a/i4eCj2deYfEdsONRLIIcbGnIlCzV6YcbroFtC0lvYE6qZSHfeFMxBvE6Sy1vyB6PZSbIhhChjEQPl5yPRlu3e5Jrt22Z3k6bibtNsU4OcUTAFCboAjpF59IEGBEnhj17CkhgLsYoeRI9FuIeHHkaMEtEIXvFMYA6wbPINo1uwQEc05wafAUuK4ZchqXpNO4hLr6UYUB5WtjdqmsASrDBCi1tEjMxgXg48W6pnO2PnmmZuscZAxkGSZSFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0F3Qt7+frbXmBj+cK0wsc3uIDhXET7lHywC5bF9h910=; b=lLrHrAEQrYJ7Vy/czC4XumppGqAP0rR/Z/0llezxc6Tr8ZIBtdZwPq6slJEykOtZWUnD+imGWZYyEzROVYuZYvX6EAby9PppXg7g8DmtT+GGT/LY+Pf2jTKX9Dy0qtrSpbL+YU24qcncayXKuxI34S9H3muJCMBBvHHRtgn24bc= Received: from MW4PR04CA0189.namprd04.prod.outlook.com (2603:10b6:303:86::14) by BL0PR12MB5540.namprd12.prod.outlook.com (2603:10b6:208:1cb::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Thu, 11 Aug 2022 12:38:32 +0000 Received: from CO1NAM11FT013.eop-nam11.prod.protection.outlook.com (2603:10b6:303:86:cafe::28) by MW4PR04CA0189.outlook.office365.com (2603:10b6:303:86::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:38:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT013.mail.protection.outlook.com (10.13.174.227) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:38:32 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:37:34 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 10/13] perf/x86: Make branch classifier fusion-aware Date: Thu, 11 Aug 2022 17:59:58 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1139e959-7de1-42ef-6c56-08da7b9666b4 X-MS-TrafficTypeDiagnostic: BL0PR12MB5540:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QZmisqu5837xjngcH5kUyIzdaxdbHRWW849hRPVcqJIdI4woGznBxGDnZKtxDhX0aMh/bkENPPzVD6PymexkpudeI2fh8XbvTdmLNVzqR6XWB23Z1JQa6B+XzZwqqJtIe4LGQTz+OFI6e2MKxTYPBq+fU7csvRK/kXial892N+jgfc4NMc6buQMbTKI0OTV4CSv3OVk2qyjdUFpgAZ5JEKq5Wwer0p3tsh/C7eKAj4d09Ucy3lFMYMhOVFNPZ4dsvxxzBwWKD0tAmBGAGC9+LW2wpWeM2101YuPJag/UouePEjAglGpNtQz0DS5Kaaj3VKM2QH3EmVZ0f9oq4oklJnL8NGo/t6tIJfXYUEnMYY0vGzmffsvb8e0MRXJo8m6cjBRgNcPUFMhAaj4duLdk60nr08WsyrVKbXz7Aa9e9xR/bm8r9nynNzfrV1wNl49QvV4APENn0t7G886/D75FyaeqbH7hZd287138FOXyLUJHlC/SL6YAIWLgqPPe5uBUhyaWr4HWWT5d3SeQOWFPE91z0NpSG7X5cDpREnaf+ejKPokd7sketu7EhNl9dT5cr5ctYQfWFurMdXjL63/a7v8534xiqxn+prrBr5C0+IT6gg8HYSnbMapkkpqYYFcVWZc5QllorFoWvp7gLr1krz6SryLeMzLxqAEJwBRohFx64z5z4M8JBFcwvG44gLuheHsO+Qw0QsMKteSc7O+NVcF3v3+amT37bIS6JOEXXpFQkIn+4TV/TSQeSi7wf1xgSM/4l0RdCc+n/+YAZaO23OgcDJJ81y+LAKIqwAWz0i7DB3moKuYpLzNEtqiLwJ7ETLjJE++LVUrQyzTK4a6JGyJErYPjHFwS/F8bRptGI4M+T2OD7BJlRxlXAG8MDnLYc4JbPW+w0Cq2GqMEg+efPQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(136003)(346002)(376002)(39860400002)(396003)(40470700004)(46966006)(36840700001)(26005)(41300700001)(478600001)(54906003)(316002)(40480700001)(4326008)(40460700003)(82310400005)(2906002)(70586007)(110136005)(70206006)(8676002)(44832011)(7416002)(8936002)(5660300002)(82740400003)(36756003)(36860700001)(86362001)(16526019)(81166007)(356005)(426003)(2616005)(47076005)(336012)(186003)(6666004)(7696005)(83380400001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:38:32.2101 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1139e959-7de1-42ef-6c56-08da7b9666b4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT013.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB5540 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" With branch fusion and other optimizations, branch sampling hardware in some processors can report a branch from address that points to an instruction preceding the actual branch by several bytes. In such cases, the classifier cannot determine the branch type which leads to failures such as with the recently added test from commit b55878c90ab9 ("perf test: Add test for branch stack sampling"). Branch information is also easier to consume and annotate if branch from addresses always point to branch instructions. Add a new variant of the branch classifier that can account for instruction fusion. If fusion is expected and the current branch from address does not point to a branch instruction, it attempts to find the first branch within the next (MAX_INSN_SIZE - 1) bytes and if found, additionally provides the offset between the reported branch from address and the address of the expected branch instruction. Signed-off-by: Sandipan Das --- arch/x86/events/perf_event.h | 2 + arch/x86/events/utils.c | 169 +++++++++++++++++++++-------------- 2 files changed, 102 insertions(+), 69 deletions(-) diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h index 8ac8f9e6affb..eb6227630244 100644 --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -1269,6 +1269,8 @@ enum { =20 int common_branch_type(int type); int branch_type(unsigned long from, unsigned long to, int abort); +int branch_type_fused(unsigned long from, unsigned long to, int abort, + int *offset); =20 ssize_t x86_event_sysfs_show(char *page, u64 config, u64 event); ssize_t intel_event_sysfs_show(char *page, u64 config); diff --git a/arch/x86/events/utils.c b/arch/x86/events/utils.c index a32368945462..e013243f360c 100644 --- a/arch/x86/events/utils.c +++ b/arch/x86/events/utils.c @@ -3,6 +3,68 @@ =20 #include "perf_event.h" =20 +static int decode_branch_type(struct insn *insn) +{ + int ext; + + if (insn_get_opcode(insn)) + return X86_BR_ABORT; + + switch (insn->opcode.bytes[0]) { + case 0xf: + switch (insn->opcode.bytes[1]) { + case 0x05: /* syscall */ + case 0x34: /* sysenter */ + return X86_BR_SYSCALL; + case 0x07: /* sysret */ + case 0x35: /* sysexit */ + return X86_BR_SYSRET; + case 0x80 ... 0x8f: /* conditional */ + return X86_BR_JCC; + } + return X86_BR_NONE; + case 0x70 ... 0x7f: /* conditional */ + return X86_BR_JCC; + case 0xc2: /* near ret */ + case 0xc3: /* near ret */ + case 0xca: /* far ret */ + case 0xcb: /* far ret */ + return X86_BR_RET; + case 0xcf: /* iret */ + return X86_BR_IRET; + case 0xcc ... 0xce: /* int */ + return X86_BR_INT; + case 0xe8: /* call near rel */ + if (insn_get_immediate(insn) || insn->immediate1.value =3D=3D 0) { + /* zero length call */ + return X86_BR_ZERO_CALL; + } + fallthrough; + case 0x9a: /* call far absolute */ + return X86_BR_CALL; + case 0xe0 ... 0xe3: /* loop jmp */ + return X86_BR_JCC; + case 0xe9 ... 0xeb: /* jmp */ + return X86_BR_JMP; + case 0xff: /* call near absolute, call far absolute ind */ + if (insn_get_modrm(insn)) + return X86_BR_ABORT; + + ext =3D (insn->modrm.bytes[0] >> 3) & 0x7; + switch (ext) { + case 2: /* near ind call */ + case 3: /* far ind call */ + return X86_BR_IND_CALL; + case 4: + case 5: + return X86_BR_IND_JMP; + } + return X86_BR_NONE; + } + + return X86_BR_NONE; +} + /* * return the type of control flow change at address "from" * instruction is not necessarily a branch (in case of interrupt). @@ -13,14 +75,22 @@ * If a branch type is unknown OR the instruction cannot be * decoded (e.g., text page not present), then X86_BR_NONE is * returned. + * + * While recording branches, some processors can report the "from" + * address to be that of an instruction preceding the actual branch + * when instruction fusion occurs. If fusion is expected, attempt to + * find the type of the first branch instruction within the next + * MAX_INSN_SIZE bytes and if found, provide the offset between the + * reported "from" address and the actual branch instruction address. */ -int branch_type(unsigned long from, unsigned long to, int abort) +static int get_branch_type(unsigned long from, unsigned long to, int abort, + bool fused, int *offset) { struct insn insn; void *addr; - int bytes_read, bytes_left; + int bytes_read, bytes_left, insn_offset; int ret =3D X86_BR_NONE; - int ext, to_plm, from_plm; + int to_plm, from_plm; u8 buf[MAX_INSN_SIZE]; int is64 =3D 0; =20 @@ -83,77 +153,27 @@ int branch_type(unsigned long from, unsigned long to, = int abort) is64 =3D kernel_ip((unsigned long)addr) || any_64bit_mode(current_pt_regs= ()); #endif insn_init(&insn, addr, bytes_read, is64); - if (insn_get_opcode(&insn)) - return X86_BR_ABORT; + ret =3D decode_branch_type(&insn); + insn_offset =3D 0; =20 - switch (insn.opcode.bytes[0]) { - case 0xf: - switch (insn.opcode.bytes[1]) { - case 0x05: /* syscall */ - case 0x34: /* sysenter */ - ret =3D X86_BR_SYSCALL; - break; - case 0x07: /* sysret */ - case 0x35: /* sysexit */ - ret =3D X86_BR_SYSRET; - break; - case 0x80 ... 0x8f: /* conditional */ - ret =3D X86_BR_JCC; - break; - default: - ret =3D X86_BR_NONE; - } - break; - case 0x70 ... 0x7f: /* conditional */ - ret =3D X86_BR_JCC; - break; - case 0xc2: /* near ret */ - case 0xc3: /* near ret */ - case 0xca: /* far ret */ - case 0xcb: /* far ret */ - ret =3D X86_BR_RET; - break; - case 0xcf: /* iret */ - ret =3D X86_BR_IRET; - break; - case 0xcc ... 0xce: /* int */ - ret =3D X86_BR_INT; - break; - case 0xe8: /* call near rel */ - if (insn_get_immediate(&insn) || insn.immediate1.value =3D=3D 0) { - /* zero length call */ - ret =3D X86_BR_ZERO_CALL; + /* Check for the possibility of branch fusion */ + while (fused && ret =3D=3D X86_BR_NONE) { + /* Check for decoding errors */ + if (insn_get_length(&insn) || !insn.length) break; - } - fallthrough; - case 0x9a: /* call far absolute */ - ret =3D X86_BR_CALL; - break; - case 0xe0 ... 0xe3: /* loop jmp */ - ret =3D X86_BR_JCC; - break; - case 0xe9 ... 0xeb: /* jmp */ - ret =3D X86_BR_JMP; - break; - case 0xff: /* call near absolute, call far absolute ind */ - if (insn_get_modrm(&insn)) - return X86_BR_ABORT; =20 - ext =3D (insn.modrm.bytes[0] >> 3) & 0x7; - switch (ext) { - case 2: /* near ind call */ - case 3: /* far ind call */ - ret =3D X86_BR_IND_CALL; + insn_offset +=3D insn.length; + bytes_read -=3D insn.length; + if (bytes_read < 0) break; - case 4: - case 5: - ret =3D X86_BR_IND_JMP; - break; - } - break; - default: - ret =3D X86_BR_NONE; + + insn_init(&insn, addr + insn_offset, bytes_read, is64); + ret =3D decode_branch_type(&insn); } + + if (offset) + *offset =3D insn_offset; + /* * interrupts, traps, faults (and thus ring transition) may * occur on any instructions. Thus, to classify them correctly, @@ -179,6 +199,17 @@ int branch_type(unsigned long from, unsigned long to, = int abort) return ret; } =20 +int branch_type(unsigned long from, unsigned long to, int abort) +{ + return get_branch_type(from, to, abort, false, NULL); +} + +int branch_type_fused(unsigned long from, unsigned long to, int abort, + int *offset) +{ + return get_branch_type(from, to, abort, true, offset); +} + #define X86_BR_TYPE_MAP_MAX 16 =20 static int branch_map[X86_BR_TYPE_MAP_MAX] =3D { --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D948C3F6B0 for ; Thu, 11 Aug 2022 12:39:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235309AbiHKMi6 (ORCPT ); Thu, 11 Aug 2022 08:38:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235283AbiHKMi4 (ORCPT ); Thu, 11 Aug 2022 08:38:56 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2059.outbound.protection.outlook.com [40.107.243.59]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A9D310A0; Thu, 11 Aug 2022 05:38:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mBdoNzeD9VdtO8gPMV9b/mbFu8l+7LkzeltR0J2qaXyXPNn2sovBi5thXG69hovwQLrXNDI4P7/Z5NYolhQXNJVTAwMH4FGJUdKP1aAQjQBNlhwRUJNrjbucuvMfvPeYIheNJdHLxXffTosxsaS+eyuVGfQoYLuQBWnp3W7N/TIORLfn73e2w5XHZOP2lrmtLJQShaNuhFX0ZsmGDHxUMQDEYTJo6yWcrfNmuKP10AMcWeOs33ZgUfXD4O3CviIKo0jkHyHh75UeKMQ+EnrKuzUYDMd4V0akdM27wxIiCasTdHp5dTVXj18wlfBisze+TPoixfaIEkpTIWnvFZdyDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=//XiLlN83AusKh8pbLJPsHe5bjQYUkAYzwAsvOBMAiw=; b=bfYlHspik7DLtyzVoQITTPpplq6Hcv1ctHY7C+pFCNRzbhsxlZLlFSGOhldzAJPY62DGd9rigyGmJTdU+10UGK8MAQ7ZWXjqDUiVtHOeHSQ8frhN8DyTmE/rgYPBVLy/vts3eILL7AXNdW6QTaLZnp9l+5MBDqLMGye8ZYJkNgMiotdsx5+BPZHf9tJChppkswDtumZkjuhDzVr4otKSRxEq10xmlbYtP2GbZtbe5siK1/fCCujTKYPT92TN2FhdNoJ80wGrmALXusQaXOlvoKnRFlnG67TFdKXdr+sgtVWaL+SIp9oIcXedYUmriuUbFvZOhdS1pBTSIxPkhgI9OA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=//XiLlN83AusKh8pbLJPsHe5bjQYUkAYzwAsvOBMAiw=; b=mXk1W0+nKCRdZkWinu5WCF6gDlDWlY7YQxPU1ZugLExFA07tCfeW3uT5e+IJfh31q7ix5MtfGaOAsW7eLcvSpIqhBNl4ba6pLPS/x6g2EP7vSmixVaFhYSUuQ1tuVC+q94LQCWaJ8atDwnKLvDY8wwT5xqsRuWiiSxcyVeesIrE= Received: from MW2PR2101CA0019.namprd21.prod.outlook.com (2603:10b6:302:1::32) by PH7PR12MB6786.namprd12.prod.outlook.com (2603:10b6:510:1ac::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.10; Thu, 11 Aug 2022 12:38:51 +0000 Received: from CO1NAM11FT045.eop-nam11.prod.protection.outlook.com (2603:10b6:302:1:cafe::3b) by MW2PR2101CA0019.outlook.office365.com (2603:10b6:302:1::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.11 via Frontend Transport; Thu, 11 Aug 2022 12:38:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT045.mail.protection.outlook.com (10.13.175.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:38:51 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:38:44 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 11/13] perf/x86/amd/lbr: Use fusion-aware branch classifier Date: Thu, 11 Aug 2022 17:59:59 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bbf358ad-d921-402c-83b0-08da7b9671e1 X-MS-TrafficTypeDiagnostic: PH7PR12MB6786:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1HzaFiKTqTRpMWDKCKjXEPIZb2SNSj17fYF5oKH0gGINaD6e19P2kSBTvg5yuNwkJKpKzl/HG2uNW9sbZajKLBeOMEjQwXTbAdg1A84mZjmFVuQr8/r8gUBVx/93dgT7TzakS9J6/pIfkzY6VUO1PA72UwiNiGL/4GJQO8w8e5VoLM/aUL2kHMV80hAC67h6fMtda/QCOqiyg6jMexHSLeSpRbj1z1aMzbF0/A1rkpKedXSRFErGXl5IUhgpSraEITMBNVUjAbBlD7/EBDThoxfWaGdBHmJOe8JtukywFRDd+Hadydy6pD8NMQGjItrcQjSHWFp1ImpondBziIHW/Lx/zKNH+EmdcCCvRbuHU1DFmOxN7BIjW9s1xMnWuHpyqI+PbDXTTlZ1HUJYXValzHJZ/1GsD6YQhO8lMXIz1B1GTSoZm172PGBJERQNmjwwLWnL7BbDoCdxz+fT4j6AaFZoQtrhB6bwT0AiBSZ8138e4a8QyJbJob9DiifOKg1eSLnT18RoFaZ1LZR++tFkCR+n4l5XeaYgo1n53SoD4Og4te4/EwEvPxLA6bGuMwnC2ORJLAfHMJLi236M7n8+4f2Vim8OvWw+2EZNPDQq0Ua2gl32Lc1T/+Z+dxhcIXXmLJA+eUbMVkBOQArCvO8x5G5gix+wFqThPYK31jYRHCkVu+eWACWBZhcVu7ZC5+s5HSmCeMDud1sEtmPf7IY0xVX/eD2uRoNbtAD4OBIR1KUaEvN/4YtwtxQTVnfbcs1WVqhpwSLnjn2HA5rJVZvudcHiJq8JpoovVxs1yBThRLlpsv4bWYqB/D9/9pGscPdaYKbjCf9cltn35EtmdicZaw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(376002)(396003)(136003)(36840700001)(46966006)(40470700004)(6666004)(26005)(70586007)(81166007)(7696005)(70206006)(8676002)(4326008)(478600001)(356005)(316002)(82740400003)(41300700001)(54906003)(110136005)(36860700001)(36756003)(2906002)(2616005)(16526019)(336012)(186003)(47076005)(426003)(40480700001)(82310400005)(5660300002)(40460700003)(8936002)(7416002)(86362001)(44832011)(83380400001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:38:51.0387 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bbf358ad-d921-402c-83b0-08da7b9671e1 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT045.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6786 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" AMD Last Branch Record Extension Version 2 (LbrExtV2) can report a branch from address that points to an instruction preceding the actual branch by several bytes due to branch fusion and further optimizations in Zen4 processors. In such cases, software should move forward sequentially in the instruction stream from the reported address and the address of the first branch encountered should be used instead. Hence, use the fusion-aware branch classifier to determine the correct branch type and get the offset for adjusting the branch from address. Signed-off-by: Sandipan Das --- arch/x86/events/amd/lbr.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c index 3699ba53a326..5fa985cd44cb 100644 --- a/arch/x86/events/amd/lbr.c +++ b/arch/x86/events/amd/lbr.c @@ -97,7 +97,7 @@ static __always_inline u64 sign_ext_branch_ip(u64 ip) static void amd_pmu_lbr_filter(void) { struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); - int br_sel =3D cpuc->br_sel, type, i, j; + int br_sel =3D cpuc->br_sel, offset, type, i, j; bool compress =3D false; u64 from, to; =20 @@ -109,7 +109,15 @@ static void amd_pmu_lbr_filter(void) for (i =3D 0; i < cpuc->lbr_stack.nr; i++) { from =3D cpuc->lbr_entries[i].from; to =3D cpuc->lbr_entries[i].to; - type =3D branch_type(from, to, 0); + type =3D branch_type_fused(from, to, 0, &offset); + + /* + * Adjust the branch from address in case of instruction + * fusion where it points to an instruction preceding the + * actual branch + */ + if (offset) + cpuc->lbr_entries[i].from +=3D offset; =20 /* If type does not correspond, then discard */ if (type =3D=3D X86_BR_NONE || (br_sel & type) !=3D type) { --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68216C25B0C for ; Thu, 11 Aug 2022 12:39:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235326AbiHKMja (ORCPT ); Thu, 11 Aug 2022 08:39:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235311AbiHKMj1 (ORCPT ); Thu, 11 Aug 2022 08:39:27 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2060a.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eab::60a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 017392F668; Thu, 11 Aug 2022 05:39:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cjrpt2Q99gHdLGO6qySJB9FdmdSY+RQfCM6rp+gqvzxLy+TOUyJ8CvDd8WkHOvzff5qNIFo3X5HvHqWfvLwKFCpLxtJuWrG4CxNHxvpDVLnUfHO+zqoKz35+JCy2WGxRG152X/bOJOXqik6msCbmlwzS/ihVWRCo89cBQLIto7yaS3fqy56NSd0d9FTSePhyMD+nlftG1U6JhlFOsoKG5ZtXShpscqZ8MbpNTUMwfQM7SC0b6s7iOBavBhB4KCRkjFctDzzhJEljuDN0Q4VGZdPSWm8JeUcn3TZ1iMSa7fRkV78lKgl2c5SVafkkKT2kxQWDifb3B3LXUeXUv6aDQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C2+iFOgVS7l5nXc91WMm2jx5sfefUwgDb5VFh4qcR9E=; b=oZUMeVb3xtuzMnW8dLhDAdeFs3edQXkwEG8cQlehPmiNaoYRMT6l/53nBw1AFidTGOLa/ccmlv5gwkBzZMXmQAX6ND6FLntfVXiCLOFxSkb/Wb9HUQ0LQDJ5YqjF2vT7aKjcQBj+lp8YGB3B4SMJ/NRDEkkKX17i8mWLLbSsugLHM+4GfE2g8CvGD7n7FJXsBTp7ko2QbuBIXhQ12nkzwJieAlz0o7lCzsbzpfiOsrdyKH+KzbKBsh1XsVzfjgyOIF9I1o7vY71Xnd5+R++Fg46oXh6Tip3AJjSaT5eUfhdb/XRJyySVXEYj8PLi1gxFT5uRRZcEdb/zWgO+XaZ5Zg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C2+iFOgVS7l5nXc91WMm2jx5sfefUwgDb5VFh4qcR9E=; b=AX1P7kRyAngFBy+Yja74l5VU1FWk+ol2Q614SSBuC6YqlblUpN79NU1jp+gXPAFoaObj5zOe+ViigbrmI41AVajdnuhdq0Cie7d9dMN9hVCZDgZbXlULHWUTfuZIRvC5e8KCXRzxneRNCS3Mpu6uhB/4abyZOb0AI8sj+ZplTdM= Received: from MW4PR03CA0256.namprd03.prod.outlook.com (2603:10b6:303:b4::21) by MN2PR12MB4253.namprd12.prod.outlook.com (2603:10b6:208:1de::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.11; Thu, 11 Aug 2022 12:39:23 +0000 Received: from CO1NAM11FT015.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b4:cafe::1a) by MW4PR03CA0256.outlook.office365.com (2603:10b6:303:b4::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.17 via Frontend Transport; Thu, 11 Aug 2022 12:39:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT015.mail.protection.outlook.com (10.13.175.130) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:39:22 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:39:16 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 12/13] perf/core: Add speculation info to branch entries Date: Thu, 11 Aug 2022 18:00:00 +0530 Message-ID: <834088c302faf21c7b665031dd111f424e509a64.1660211399.git.sandipan.das@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9d9348ed-25f7-4f04-5d67-08da7b9684a4 X-MS-TrafficTypeDiagnostic: MN2PR12MB4253:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tIxpKGAMdgC/aCY2RS2HRUfAXiY/5y/UsHCre3eJn97EW7+ZuNDyIR71u4ChFsUgxHR789kljMsaPH8IhmAYfI8CYTsrvCH2zbMTIJd6j/NReQr2qFK79psyAaW2QZ33/pW08zerkWwQf6ijjEcDgAv/ktXryxtCSwJuBbOeidJiYV2z+BWmstH2vkkHkteG4PhA3dp9KIteBg6H3D7P5OJrBezQWRlU1xSfS41WicU6t8y2ffnDIl4O1cEj+iVmiZ3TyBeVlbKlf+CcxaXTs/S6JcbDwBfS9JQpsT+8ENMcf73GH87NOyZ+dLw2LNUXVpNmamlefmIWw3664WaX6rLoeitDhFFnpRsX5lMpOEFDrucAMo/gfW4g9rDHxTOllXdYxPbNZksCZMRRAP34mLwiLT+d9sOy7U6nRQlVWq8jFIGUmkKWcmh1eumScyN5zW34vcocU9iXjcP18JwKg1XE6mLjKja1vR9Ng/K3pHgkmQMaS4ebglKxQJ4Llk7Ep0TtHW/YzaRKeZ/uh41srS7uQoZZRp41XPnHJtDuyE6Pr9/2dN0G7ppCQzJmX6uvwI0XoT8ruO1Pv/7zVfY2o/rGT7IjZpiUtu8H3XGPszJm0fIgHrIJZK1wr9u24Osxz7yu2KvQUHDnhsZzhvw+O38yfhVFesdke7FcW5beglGfotfS1tBjiwudT0SHRctkMXQ/Kg4Pqj/zIFauwAwIPqq7HnpvU8mJq4s9Ldb+QTLuIEZBFka8hmARz5vQj6eIHQBzeOHm/cYg0WzALLkk0+hOw4I9cGdocnY/Ylzb15i/Ph081+Am3o/av5SHn1YlJwY57cbL7siSUHNJG4tdWQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(346002)(376002)(396003)(136003)(46966006)(36840700001)(40470700004)(26005)(356005)(81166007)(82310400005)(82740400003)(7416002)(44832011)(5660300002)(40460700003)(8936002)(86362001)(7696005)(478600001)(83380400001)(8676002)(6666004)(41300700001)(4326008)(70586007)(2906002)(70206006)(40480700001)(36756003)(16526019)(110136005)(2616005)(47076005)(54906003)(186003)(426003)(336012)(36860700001)(316002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:39:22.5302 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9d9348ed-25f7-4f04-5d67-08da7b9684a4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT015.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4253 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a new "spec" bitfield to branch entries for providing speculation information. This will be populated using hints provided by branch sampling features on supported hardware. The following cases are covered: * No branch speculation information is available * Branch is speculative but taken on the wrong path * Branch is non-speculative but taken on the correct path * Branch is speculative and taken on the correct path Suggested-by: Stephane Eranian Signed-off-by: Sandipan Das --- include/linux/perf_event.h | 1 + include/uapi/linux/perf_event.h | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index ee8b9ecdc03b..ae30c61957d2 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1078,6 +1078,7 @@ static inline void perf_clear_branch_entry_bitfields(= struct perf_branch_entry *b br->abort =3D 0; br->cycles =3D 0; br->type =3D 0; + br->spec =3D PERF_BR_SPEC_NA; br->reserved =3D 0; } =20 diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_even= t.h index 0474ee362151..9656733a95a5 100644 --- a/include/uapi/linux/perf_event.h +++ b/include/uapi/linux/perf_event.h @@ -256,6 +256,17 @@ enum { PERF_BR_MAX, }; =20 +/* + * Common branch speculation outcome classification + */ +enum { + PERF_BR_SPEC_NA =3D 0, /* Not available */ + PERF_BR_SPEC_WRONG_PATH =3D 1, /* Speculative but on wrong path */ + PERF_BR_NON_SPEC_CORRECT_PATH =3D 2, /* Non-speculative but on correct pa= th */ + PERF_BR_SPEC_CORRECT_PATH =3D 3, /* Speculative and on correct path */ + PERF_BR_SPEC_MAX, +}; + #define PERF_SAMPLE_BRANCH_PLM_ALL \ (PERF_SAMPLE_BRANCH_USER|\ PERF_SAMPLE_BRANCH_KERNEL|\ @@ -1363,6 +1374,7 @@ union perf_mem_data_src { * abort: aborting a hardware transaction * cycles: cycles from last branch (or 0 if not supported) * type: branch type + * spec: branch speculation info (or 0 if not supported) */ struct perf_branch_entry { __u64 from; @@ -1373,7 +1385,8 @@ struct perf_branch_entry { abort:1, /* transaction abort */ cycles:16, /* cycle count to last branch */ type:4, /* branch type */ - reserved:40; + spec:2, /* branch speculation info */ + reserved:38; }; =20 union perf_sample_weight { --=20 2.34.1 From nobody Mon Apr 13 13:54:31 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B79EC19F2D for ; Thu, 11 Aug 2022 12:39:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235187AbiHKMjv (ORCPT ); Thu, 11 Aug 2022 08:39:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234084AbiHKMjs (ORCPT ); Thu, 11 Aug 2022 08:39:48 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2087.outbound.protection.outlook.com [40.107.243.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5046167DF; Thu, 11 Aug 2022 05:39:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vr4sFIjP3NzJndP1fhCO/7FRD/q5Bip7Irazvk4qAHSaBpvt/hWLGmYH/Y5/EFfZMU2XfQioNLOVd8TK7rfHR9LrH2ns//yWqq6On2Ch/ZXwSbehLmMpP78ffv46v0YGii0lLDMQg54Nsz8MqiDgYlhwkV7C84uXxG/DnbM5VroFNt0/Jny7cBZ01VreYg4hhSMt1RnzhOb0nJ6timdO3fH6djdvvnUykCZewHe9Qjpqjc7DcHzvnxvqGgWIFL6dcM7tV46ppOvEnlT5TnSXt2A9kYDqSwwL4vG7piow4bjwpPWIjBZksosnZya6ZrZtw1y/Olfqi1U8VZu438y03A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qYCpHGH1VgtMWzcYYpOhNUiLopgN+3M9EpvYJtDg8Cg=; b=iPVbmk+i9PZQjDW+0E2WsZoJfFZMMIkRDX8LuarUIsxVT7ROmZdNaqhRBY0IOo3oZmHV4d0DYfC7aV6pv/nPfXF4dqbhTVGtt+VT3LDxJ/xhMOxBCr2ot2C17GWU9KvOyvpVAOxnfuS3dsqJJ+QGqwcq1+Z6U68K29ux9vXL6ICuctnww5GK96H5RhSuKOwnivq5/e79QsWGOS3nFmEXDnoHhnzRabxJalP7ftsNUbPAf4yQ2tBchTlmJhP+6xz/jMgi08wMLKJ0CB7mJQxkZlU0byCKdOZ7FeXNQbAwGFMZoWw4P58By3Fv0hrXN9jEggqEbb8oN/WqttjT6eGpQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qYCpHGH1VgtMWzcYYpOhNUiLopgN+3M9EpvYJtDg8Cg=; b=wQ3uejQYGjfDnTXP0YCuev1tRrfKGkQOOGTfBuXsqZGzxFQIW0RNxwIB3/TdGY+3e2U7xK9TZ7QhSkYIt/LtUwfZiXX6hqbmDAOQ9CmvJPeWbhX32zkYlkn66Mk60OUQ7ImN4ZOZuwwy9zLC9+SIa47srxsXMjFAVUdrdkdDxhQ= Received: from MW4PR03CA0150.namprd03.prod.outlook.com (2603:10b6:303:8c::35) by DM5PR12MB1418.namprd12.prod.outlook.com (2603:10b6:3:7a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.15; Thu, 11 Aug 2022 12:39:42 +0000 Received: from CO1NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8c:cafe::69) by MW4PR03CA0150.outlook.office365.com (2603:10b6:303:8c::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5525.12 via Frontend Transport; Thu, 11 Aug 2022 12:39:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT031.mail.protection.outlook.com (10.13.174.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5525.11 via Frontend Transport; Thu, 11 Aug 2022 12:39:41 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Thu, 11 Aug 2022 07:39:35 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , Subject: [PATCH 13/13] perf/x86/amd/lbr: Add LbrExtV2 branch speculation info support Date: Thu, 11 Aug 2022 18:00:01 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b1af6e6f-cdc9-4c5e-03bf-08da7b96900d X-MS-TrafficTypeDiagnostic: DM5PR12MB1418:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9Kba9rEMJ7OaPKD4vbD9xf6Hx2Czop5JdNQZzhW1/X5oQxrBItKbjDW3gah9BSgZ+pvvOAObtbBEgc+QZUVf0HAcxiGymfTOJxgR+svcku7Q7fAR9czhAPecK0qg3LA77X4ca3XH7INfq1T5Y2rjIGumyjBuSLs05tMrsI75+8mfbcH/KASU5XbNTBS6JyO3VdMRCR+AtdGAiUVDuwX8MGtCxTqkz4txyclonWyq+sKSKBUpB5W9NjyX2s+n5BGqgYxI+1DjElzvEPYGDKK4Am/wKr71YVWloYhtlMb7USgJc3sK43q+RoieC7Ckzy1WilVDxO75By5g7pI2sQKPjAJqmmzfiNCE7z3RgXX2q+g3sDpH3EqC7A/hEnNIRbrGTu5vZt2K/sC9r6cf2x0145YYG13A14jpbemoEsvZWIhJkPoGe5HdBsth5k8WBs1jfOCHijgbYvgywu1M4Duv0Gzbe+eBbsWg51i/mSJqcTuKceg82bFMey/fubHVKFTn40AGVJFbSZpSkPBrs+WaB5SsWGio6ljQM7kMOznbI3Sy6m1Bqc3rPIA4OXPwgAVewMdfnCxzBxQu0cB7Qy/rLX4MGEd09mejnzOT6x6RviCOGMQb3HrmFjvQTCdpphTZtOyOA/Y221phYa+5gFlibUXIMeE/Vk8e9s3C0yZWWJwrRSiX1uHrEJVPMM6CR0cfbvdJ1cF8KnXxrUR2DLZe+IT7EqnUSk4WBt46c01N0SM2acu/zkdeFGNGVHb0Vb5b3eiFnay7RwYSd/ae7hAZq4MTs8OIfWaMmNN+e/YMB+mOJwhycFiX7C9q54njjVGaODKog17ukD1ClyKuOmniPg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(376002)(396003)(136003)(39860400002)(40470700004)(46966006)(36840700001)(26005)(356005)(5660300002)(36756003)(82740400003)(336012)(41300700001)(478600001)(2906002)(7696005)(81166007)(6666004)(40480700001)(82310400005)(70206006)(83380400001)(2616005)(8676002)(110136005)(86362001)(54906003)(7416002)(4326008)(426003)(36860700001)(16526019)(316002)(40460700003)(70586007)(8936002)(44832011)(186003)(47076005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2022 12:39:41.6710 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b1af6e6f-cdc9-4c5e-03bf-08da7b96900d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1418 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Provide branch speculation information captured via AMD Last Branch Record Extension Version 2 (LbrExtV2) by setting the speculation info in branch records. The info is based on the "valid" and "spec" bits in the Branch To registers. Suggested-by: Stephane Eranian Signed-off-by: Sandipan Das --- arch/x86/events/amd/lbr.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c index 5fa985cd44cb..3fd316b6adda 100644 --- a/arch/x86/events/amd/lbr.c +++ b/arch/x86/events/amd/lbr.c @@ -146,12 +146,19 @@ static void amd_pmu_lbr_filter(void) } } =20 +static const int lbr_spec_map[PERF_BR_SPEC_MAX] =3D { + PERF_BR_SPEC_NA, + PERF_BR_SPEC_WRONG_PATH, + PERF_BR_NON_SPEC_CORRECT_PATH, + PERF_BR_SPEC_CORRECT_PATH, +}; + void amd_pmu_lbr_read(void) { struct cpu_hw_events *cpuc =3D this_cpu_ptr(&cpu_hw_events); struct perf_branch_entry *br =3D cpuc->lbr_entries; struct branch_entry entry; - int out =3D 0, i; + int out =3D 0, idx, i; =20 if (!cpuc->lbr_users) return; @@ -160,8 +167,11 @@ void amd_pmu_lbr_read(void) entry.from.full =3D amd_pmu_lbr_get_from(i); entry.to.full =3D amd_pmu_lbr_get_to(i); =20 - /* Check if a branch has been logged */ - if (!entry.to.split.valid) + /* + * Check if a branch has been logged; if valid =3D 0, spec =3D 0 + * then no branch was recorded + */ + if (!entry.to.split.valid && !entry.to.split.spec) continue; =20 perf_clear_branch_entry_bitfields(br + out); @@ -170,6 +180,25 @@ void amd_pmu_lbr_read(void) br[out].to =3D sign_ext_branch_ip(entry.to.split.ip); br[out].mispred =3D entry.from.split.mispredict; br[out].predicted =3D !br[out].mispred; + + /* + * Set branch speculation information using the status of + * the valid and spec bits. + * + * When valid =3D 0, spec =3D 0, no branch was recorded and the + * entry is discarded as seen above. + * + * When valid =3D 0, spec =3D 1, the recorded branch was + * speculative but took the wrong path. + * + * When valid =3D 1, spec =3D 0, the recorded branch was + * non-speculative but took the correct path. + * + * When valid =3D 1, spec =3D 1, the recorded branch was + * speculative and took the correct path + */ + idx =3D (entry.to.split.valid << 1) | entry.to.split.spec; + br[out].spec =3D lbr_spec_map[idx]; out++; } =20 --=20 2.34.1