From nobody Sun Feb 8 20:36:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B076CA0FE5 for ; Fri, 1 Sep 2023 14:35:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350034AbjIAOfQ (ORCPT ); Fri, 1 Sep 2023 10:35:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245262AbjIAOfN (ORCPT ); Fri, 1 Sep 2023 10:35:13 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A405B9 for ; Fri, 1 Sep 2023 07:35:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693578910; x=1725114910; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x8W5RJy52JcHrmK1sJosAK6uMp8pOkYSsoFyAQJuX9g=; b=YikV6Bh4D7pwdgx1ToCxoakU/Y8POsgvTJH9476nnaZs0WAypFRt7PZW oczgHUvFriHDFCEgzRSPi9bM2xkBDJ7MakoacLx/q6s4kDdgIVDldOaNC 8AxTeDn08w2q6AAHFCJ3d37QZct9dH6x3alknge9Y6CdH3D/Samk5z4Dh QvhCbreh4aPPYuA5adS7fEyIDb2raznmWV+R/9VQGU/ve6lS+MPVQcBDA 6F9iBwqVhXL/7uaWaSTVCcyz69Buf89AZdIEXyR3D765Jn7DraVOeFEsB Y8BP7L3ys/YFN/i+ouC6VUvpvAW/xnkOKo3R1GCZA5pk/SsIGRuuhsGR+ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="373627539" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="373627539" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2023 07:35:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10819"; a="805450056" X-IronPort-AV: E=Sophos;i="6.02,219,1688454000"; d="scan'208";a="805450056" Received: from sunyi-station.sh.intel.com (HELO ysun46-mobl.sh.intel.com) ([10.239.159.10]) by fmsmga008.fm.intel.com with ESMTP; 01 Sep 2023 07:35:03 -0700 From: Yi Sun To: dave.hansen@intel.com, tglx@linutronix.de, linux-kernel@vger.kernel.org, x86@kernel.org Cc: sohil.mehta@intel.com, ak@linux.intel.com, ilpo.jarvinen@linux.intel.com, heng.su@intel.com, tony.luck@intel.com, dave.hansen@linux.intel.com, yi.sun@intel.intel.com, Yi Sun Subject: [PATCH v6 2/3] tools/testing/fpu: Add script to consume trace log of xsaves latency Date: Fri, 1 Sep 2023 22:34:13 +0800 Message-Id: <20230901143414.1664368-3-yi.sun@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230901143414.1664368-1-yi.sun@intel.com> References: <20230901143414.1664368-1-yi.sun@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Consume the trace log dumped by tracepoints x86_fpu_latency_xsave and x86_fpu_latency_xrstor, calculate latency ranges for each RFBM and XINUSE combination including min, max, average and 97% tail latency. Add the average of 97% tail latency to remove the unreasonable data which is introduced by interrupts or other noise. By adding the experimental code disabling interrupts before the calculation of latency, it's obvious to get the 3% tail latency has been filtered. Make use of sqlite3 to make the data statistics more efficient and concise. The output looks like follows: EVENTs RFBM XINUSE lat_min lat_max lat_avg lat_avg(97%) Reviewed-by: Tony Luck ---------------------- ------- ------ ------- ------- ------- ------------ x86_fpu_latency_xrstor 0x206e7 0x0 364 364 364 364 x86_fpu_latency_xrstor 0x206e7 0x202 112 1152 300 276 x86_fpu_latency_xsave 0x206e7 0x202 80 278 141 137 x86_fpu_latency_xsave 0x206e7 0x246 108 234 180 177 The XSAVES/XRSTORS latency trace logs can be got by two ways: 1. Generated by Kernel debugfs echo 1 > /sys/kernel/debug/tracing/events/x86_fpu/enable cat /sys/kernel/debug/tracing/trace_pipe > trace-log 2. Generated by helper tool like 'trace-cmd' trace-cmd record -e x86_fpu -F trace-cmd report > trace-log Reviewed-by: Tony Luck Signed-off-by: Yi Sun diff --git a/tools/testing/fpu/xsave-latency-trace.sh b/tools/testing/fpu/x= save-latency-trace.sh new file mode 100755 index 000000000000..d45563984fd6 --- /dev/null +++ b/tools/testing/fpu/xsave-latency-trace.sh @@ -0,0 +1,227 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# (c) 2022 Yi Sun + +trace_log=3D$1 +trace_lat_log=3D".trace_lat_log" +db_name=3D"db_trace" +db_file=3D"${db_name}.db" +table_raw=3D"t_trace" +table_tail=3D"t_trace_tail" +table_results=3D"t_results" +events=3D"x86_fpu_latency_xsave|x86_fpu_latency_xrstor" + +# The regex for the trace log. The rough pattern: +# (proc) (No.cpu) (flags) (timestamp): (tracepoint): latency:(123) RFBM:0x= (123) XINUSE:0x(123)$ +# Fold the regex into 3 parts making it easier to read. +regex1=3D"([^\ ]*)[[:space:]]*\[([0-9]+)\][[:space:]]*(.....\ )?[[:space:]= ]*" +regex2=3D"([0-9.]*):[[:space:]]*([^\ :]*):.*latency:([0-9]*)[[:space:]]*" +regex3=3D"RFBM:(0x[0-9a-f]*)[[:space:]]*XINUSE:(0x[0-9a-f]*)$" + +function usage() { + echo "This script consumes the tracepoint data, and dumps out the" + echo "latency ranges for each RFBM combination." + echo "Usage:" + echo "$0 " + echo " trace-log:" + echo " Either generated by Kernel sysfs:" + echo " echo 1 > /sys/kernel/debug/tracing/events/x86_fpu/enable" + echo " cat /sys/kernel/debug/tracing/trace_pipe > trace-log" + echo "" + echo " Or generate by helper tool like 'trace-cmd':" + echo " trace-cmd record -e x86_fpu" + echo " trace-cmd report > trace-log" +} + +# Check the dependent tools +# {@}: a list of third-part tools +function check_packages() { + for pack in "$@"; do + which $pack >& /dev/null + if [[ $? !=3D 0 ]]; then + echo "Please install $pack before running this script." + exit 1 + fi + done +} + +# Run SQL command with sqlite3 +# ${*}: SQL command fed to sqlite3 +function SQL_CMD() { + sqlite3 $db_file "$*" +} + +# Run SQL command with sqlite3 and format the output with headers and colu= mn. +# ${*}: SQL command fed to sqlite3 +function SQL_CMD_HEADER() { + sqlite3 -column -header $db_file "$*" +} + +# Create a table in the DB +# ${1}=EF=BC=9A name of the table +function create_table() { + if [[ "$1" =3D=3D "" ]]; then + echo "Empty table name!" + exit 1 + fi + SQL_CMD "create table $1 ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + process TEXT, + cpu INT, + timestamp FLOAT, + event_name TEXT, + lat INT, + RFBM INT, + XINUSE INT);" +} + +# Round to the nearest whole number +# ${1}: a float number +# Output: integer +function round() { + echo "scale=3D0; ($1+0.5)/1" | bc +} + +# Insert a record in the trace table +# +# process cpu timestamp event_name lat RFBM XINUSE +# $2 $3 $4 $5 $6 $7 $8 + +function insert_line() { + if [[ "$1" =3D=3D "" ]]; then + echo "Empty table name!" + exit 1 + fi + SQL_CMD "INSERT INTO $1 (process, cpu, timestamp, event_name, lat, RFBM, = XINUSE) + VALUES (\"$2\", $3, $4, \"$5\", $6, $7, $8);" +} + +# Show the results of the trace statistics +function get_latency_stat() { + SQL_CMD "create table $table_results ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + event_name TEXT, + RFBM INT, + XINUSE INT, + lat_min INT, + lat_max INT, + lat_avg INT, + lat_tail_avg INT);" + + for((i=3D0; i<$cnt; i++));do + event_name=3D`get_comb_item $table_raw $i event_name` + RFBM=3D`get_comb_item $table_raw $i RFBM` + XINUSE=3D`get_comb_item $table_raw $i XINUSE` + lat_min=3D`get_comb_item $table_raw $i min\(lat\)` + lat_max=3D`get_comb_item $table_raw $i max\(lat\)` + lat_avg=3D`get_comb_item $table_raw $i avg\(lat\)` + lat_tail_avg=3D`get_comb_item $table_tail $i avg\(lat\)` + + lat_avg=3D`round $lat_avg` + lat_tail_avg=3D`round $lat_tail_avg` + + SQL_CMD "INSERT INTO $table_results + (event_name, RFBM,XINUSE, lat_min, lat_max, lat_avg, lat_tail_avg) + VALUES (\"$event_name\", $RFBM, $XINUSE, $lat_min, $lat_max, + $lat_avg, $lat_tail_avg);" + done + + SQL_CMD_HEADER "select event_name[EVENTs],printf('0x%x',RFBM)[RFBM], + printf('0x%x',XINUSE)[XINUSE],lat_min,lat_max,lat_avg, + lat_tail_avg[lat_avg(97%)] + from $table_results;" +} + +# Get the count of the combination of event_name, RFBM, XINUSE amount all = lat trace records +function get_combs_cnt() { + SQL_CMD "SELECT event_name, RFBM, XINUSE from $table_raw + group by event_name,RFBM,XINUSE;" | wc -l +} + +# Get a specified combination from a table +# ${1}: name of table +# ${2}: the order of the combination of event_name, RFBM, XINUSE +# ${3}: the items which are wanted to be shown +function get_comb_item() { + table=3D$1 + cnt=3D$2 + col=3D$3 + SQL_CMD "SELECT $col from $table group by event_name,RFBM,XINUSE limit $c= nt,1;" +} + +# Get count of the records in a given table +# ${1}: name of the table +function get_rows_cnt() { + table=3D$1 + SQL_CMD "SELECT count(*) from $table;" +} + +# Generate a new table from the raw trace table removing 3% tail traces. +function gen_tail_lat() { + cnt=3D`get_combs_cnt` + create_table $table_tail + + for((i=3D0; i<$cnt; i++));do + create_table t$i + event_name=3D`get_comb_item $table_raw $i event_name` + RFBM=3D`get_comb_item $table_raw $i RFBM` + XINUSE=3D`get_comb_item $table_raw $i XINUSE` + + SQL_CMD "insert into t$i(process,cpu,timestamp,event_name,lat,RFBM,XINUS= E) + select process,cpu,timestamp,event_name,lat,RFBM,XINUSE + from $table_raw where event_name=3D\"$event_name\" and RFBM=3D$RFBM and + XINUSE=3D$XINUSE ORDER BY lat ASC;" + + row=3D`get_rows_cnt t$i` + row=3D`echo "scale=3D0; ($row*0.97 + 0.5)/1" | bc` + + SQL_CMD "insert into $table_tail + (process,cpu,timestamp,event_name,lat,RFBM,XINUSE) + select process,cpu,timestamp,event_name,lat,RFBM,XINUSE + from t$i limit 0,$row;" + done + +} + +if [[ ! -e "$trace_log" || $# !=3D 1 ]];then + usage + exit 1 +fi + +# Check dependency +# Make sure having following packages +check_packages sqlite3 bc wc cut + +# Filter trace log keeping latency related lines only +grep -E "$events" $trace_log > $trace_lat_log +cnt_lines=3D`wc -l $trace_lat_log | cut -d' ' -f1` +# Remove the old db file if it existed before creating +[[ -f $db_file ]] && rm -rf $db_file + +create_table $table_raw + +# Read each line from the temp file and insert into the table +i=3D0 +while IFS=3D read -r line; +do + ((i =3D i + 1)) + echo -ne "(${i}/$cnt_lines) Importing trace log into database!\r" + if [[ "$line" =3D~ ${regex1}${regex2}${regex3} ]]; then + pname=3D${BASH_REMATCH[1]} + cpu=3D${BASH_REMATCH[2]} + ts=3D${BASH_REMATCH[4]} + ename=3D${BASH_REMATCH[5]} + lat=3D${BASH_REMATCH[6]} + ((rfbm=3D${BASH_REMATCH[7]})) + ((xinuse=3D${BASH_REMATCH[8]})) + + insert_line $table_raw $pname $cpu $ts $ename $lat $rfbm $xinuse + fi +done < $trace_lat_log + +gen_tail_lat +get_latency_stat + +# Cleanup +rm -rf $trace_lat_log $db_file --=20 2.34.1