From nobody Sun Feb 8 08:43:35 2026 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C1AF1D221A for ; Tue, 19 Nov 2024 18:01:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732039297; cv=none; b=gSMw5NDowsPygCShYlmLf30l//J7WGIdmVRzxpahrJc2KQkoTkpMBQ0jrwkbHjPWRtMTDhLpA5wnPpl7CaChYk68OPGe4UdKkFdJLwIo/MtNH68okR3W2rhZ9ZaiAsqIPnK0Xa3i/lGh/DukZolX6lp9tPl0Tbc+FFhGaIrrFVA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732039297; c=relaxed/simple; bh=FRCD0mklqsM8YzYKgxccyE8kvTxTAZqJyW2FOK9fwEU=; h=Date:Message-Id:Mime-Version:Subject:From:To:Cc:Content-Type; b=Ia6EObg4Ew5s0pzbx2yUtgsZV8s9LlMbhXKzIUS/1tcGOqMlfd7VPsVRC0JY2xAjqCU88dagpsc8/j3SvZ6YfBfbodabkhmtNQIMzReQ6wxhfii1Hz+EOZKRvZzdJdU+W6Ahnh+GvQomdr6qjFW7E4hCccxr4yOETKgae42IA/w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mDxLJvKl; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mDxLJvKl" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6ee57ae1133so52941927b3.0 for ; Tue, 19 Nov 2024 10:01:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732039294; x=1732644094; darn=vger.kernel.org; h=cc:to:from:subject:mime-version:message-id:date:from:to:cc:subject :date:message-id:reply-to; bh=TQFCW5HyokM6GPCpSGZP+kz1l/ptP37eeUF9xzXvHb0=; b=mDxLJvKlRVvGJrydo4xi2l7efx1WLQdpGZ1g3a9gpZHD6ESGF7Mb6bMDIb9ljZETgH 4zO57Jlu/+w0CO5A/0J0mPA9+Luqm4Tv1C6LMM59x/0/awq6vjwIBWX3LACuaVwBeA+B r7SLcAeLib39Ag4zy+qWsaFaJC/UEEVqZ8dlYd91AUEZ5hifpZwJ9v7sYJ1bCJar78y/ pFzbt8aALg4bRD4omusDd6Zg/fDgi77+fih32Nn5V9+Z/ZltomI9r6uHOFoqVtWJwOmv sP4nbbAiBZ6kkIQKl6DwxLyKMxGMyo8bD6TEi8CafxP0va39wRSyyCjxVbMrV9RppiFX mqFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732039294; x=1732644094; h=cc:to:from:subject:mime-version:message-id:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=TQFCW5HyokM6GPCpSGZP+kz1l/ptP37eeUF9xzXvHb0=; b=mGhXZUchrgMOuGtyFVfSJf9u+HpVI7bnpAm1YCwbfEtV5rUoKECWqu1LXs3gbLTpSD EF8byKga8CCy9HroEvHFlPKYod/sVB8GKJkLK3fkhMwk/hkil+g4KsG+s0h/Wm8xXWY4 LSCBfYEd3t3X1SbPuk4DayA0y/B4j1L7o+T5XcSf6bmo0ZBuUe8tYsuqIMTpT07f4oxA iUtc1R6V754cln3sXY0uLD3AJeivacQLDxSEhbpOFOp+kMaLFugCmB3NY5bw2rj76YjN NLxgUDf+GCNXlgpssuIIaBMZR/jv+sswu7nY8EecjocqL8F8HzM6pUqlNGdW4OvmjMGV zorA== X-Forwarded-Encrypted: i=1; AJvYcCVsKpRguKkp05EWGwu6c03i9ZbT1MM2TAXZkHXAm6IPVINoNU95CGEs2LI+PSXl3aI7VCXO8nM5mlN/6Zk=@vger.kernel.org X-Gm-Message-State: AOJu0YymP2QFqHp29JS4dSaU4hrghZ/31SpmFirlHB6mQRJrvJsiJLZ2 RY3SyFi6lzXFhSVoV/PXMHnwD3aRY7kEeYPEkMVRQQ1eos93nm2bz5NDdI5q7g05ZKKHaEjw4QO aoY/alw== X-Google-Smtp-Source: AGHT+IEaamOMz7Fi+3YLgkkmVXxvJJt7gmHsMBKdyAI1rOlv3UWwoAKvdNTU/6CyNZsrYp71LWCKSZKKZmJq X-Received: from irogers.svl.corp.google.com ([2620:15c:2c5:11:89d:ee73:740b:dfb3]) (user=irogers job=sendgmr) by 2002:a05:690c:8c1b:b0:6e3:1702:b3e6 with SMTP id 00721157ae682-6ee55c8cf88mr588147b3.4.1732039294050; Tue, 19 Nov 2024 10:01:34 -0800 (PST) Date: Tue, 19 Nov 2024 10:01:30 -0800 Message-Id: <20241119180130.19160-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.47.0.371.ga323438b13-goog Subject: [PATCH v1] perf script python: Improve physical mem type resolution From: Ian Rogers To: Kan Liang Cc: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Previously system RAM and persistent memory were hard code matched, change so that the label of the memory region is just read from /proc/iomem. This avoids frequent N/A samples. Change the /proc/iomem reading, event processing and output so that nested entries appear and their counts count toward their parent. As labels may be repeated, include the memory ranges in the output to make it clear why, for example, "System RAM" appears twice. Before: ``` Event: mem_inst_retired.all_loads:P Memory type count percentage Acked-by: Kan Liang ---------------------------------------- ---------- ---------- System RAM 9460 96.5% N/A 998 3.5% ``` After: ``` Event: mem_inst_retired.all_loads:P Memory type count percentage ---------------------------------------- ---------- ---------- 100000000-105f7fffff : System RAM 36741 96.5 841400000-8416599ff : Kernel data 89 0.2 840800000-8412a6fff : Kernel rodata 60 0.2 841ebe000-8423fffff : Kernel bss 34 0.1 0-fff : Reserved 1345 3.5 100000-89dd9fff : System RAM 2 0.0 ``` Before: ``` Event: mem_inst_retired.any:P Memory type count percentage ---------------------------------------- ----------- ----------- System RAM 9460 90.5% N/A 998 9.5% ``` After: ``` Event: mem_inst_retired.any:P Memory type count percentage ---------------------------------------- ---------- ---------- 100000000-105f7fffff : System RAM 9460 90.5 841400000-8416599ff : Kernel data 45 0.4 840800000-8412a6fff : Kernel rodata 19 0.2 841ebe000-8423fffff : Kernel bss 12 0.1 0-fff : Reserved 998 9.5 ``` The code has been updated to python 3 with type hints and resolving issues reported by mypy and pylint. Tabs are swapped to spaces as preferred in PEP8, because most lines of code were modified (of this small file) and this makes pylint significantly less noisy. Signed-off-by: Ian Rogers --- tools/perf/scripts/python/mem-phys-addr.py | 177 ++++++++++++--------- 1 file changed, 102 insertions(+), 75 deletions(-) diff --git a/tools/perf/scripts/python/mem-phys-addr.py b/tools/perf/script= s/python/mem-phys-addr.py index 1f332e72b9b0..5e237a5a5f1b 100644 --- a/tools/perf/scripts/python/mem-phys-addr.py +++ b/tools/perf/scripts/python/mem-phys-addr.py @@ -3,98 +3,125 @@ # # Copyright (c) 2018, Intel Corporation. =20 -from __future__ import division -from __future__ import print_function - import os import sys -import struct import re import bisect import collections +from dataclasses import dataclass +from typing import (Dict, Optional) =20 sys.path.append(os.environ['PERF_EXEC_PATH'] + \ - '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + +@dataclass(frozen=3DTrue) +class IomemEntry: + """Read from a line in /proc/iomem""" + begin: int + end: int + indent: int + label: str =20 -#physical address ranges for System RAM -system_ram =3D [] -#physical address ranges for Persistent Memory -pmem =3D [] -#file object for proc iomem -f =3D None -#Count for each type of memory -load_mem_type_cnt =3D collections.Counter() -#perf event name -event_name =3D None +# Physical memory layout from /proc/iomem. Key is the indent and then +# a list of ranges. +iomem: Dict[int, list[IomemEntry]] =3D collections.defaultdict(list) +# Child nodes from the iomem parent. +children: Dict[IomemEntry, set[IomemEntry]] =3D collections.defaultdict(se= t) +# Maximum indent seen before an entry in the iomem file. +max_indent: int =3D 0 +# Count for each range of memory. +load_mem_type_cnt: Dict[IomemEntry, int] =3D collections.Counter() +# Perf event name set from the first sample in the data. +event_name: Optional[str] =3D None =20 def parse_iomem(): - global f - f =3D open('/proc/iomem', 'r') - for i, j in enumerate(f): - m =3D re.split('-|:',j,2) - if m[2].strip() =3D=3D 'System RAM': - system_ram.append(int(m[0], 16)) - system_ram.append(int(m[1], 16)) - if m[2].strip() =3D=3D 'Persistent Memory': - pmem.append(int(m[0], 16)) - pmem.append(int(m[1], 16)) + """Populate iomem from /proc/iomem file""" + global iomem + global max_indent + global children + with open('/proc/iomem', 'r', encoding=3D'ascii') as f: + for line in f: + indent =3D 0 + while line[indent] =3D=3D ' ': + indent +=3D 1 + if indent > max_indent: + max_indent =3D indent + m =3D re.split('-|:', line, 2) + begin =3D int(m[0], 16) + end =3D int(m[1], 16) + label =3D m[2].strip() + entry =3D IomemEntry(begin, end, indent, label) + # Before adding entry, search for a parent node using its begi= n. + if indent > 0: + parent =3D find_memory_type(begin) + assert parent, f"Given indent expected a parent for {label= }" + children[parent].add(entry) + iomem[indent].append(entry) =20 -def print_memory_type(): - print("Event: %s" % (event_name)) - print("%-40s %10s %10s\n" % ("Memory type", "count", "percentage"), end= =3D'') - print("%-40s %10s %10s\n" % ("----------------------------------------", - "-----------", "-----------"), - end=3D''); - total =3D sum(load_mem_type_cnt.values()) - for mem_type, count in sorted(load_mem_type_cnt.most_common(), \ - key =3D lambda kv: (kv[1], kv[0]), reverse =3D True): - print("%-40s %10d %10.1f%%\n" % - (mem_type, count, 100 * count / total), - end=3D'') +def find_memory_type(phys_addr) -> Optional[IomemEntry]: + """Search iomem for the range containing phys_addr with the maximum in= dent""" + for i in range(max_indent, -1, -1): + if i not in iomem: + continue + position =3D bisect.bisect_right(iomem[i], phys_addr, + key=3Dlambda entry: entry.begin) + if position is None: + continue + iomem_entry =3D iomem[i][position-1] + if iomem_entry.begin <=3D phys_addr <=3D iomem_entry.end: + return iomem_entry + print(f"Didn't find {phys_addr}") + return None =20 -def trace_begin(): - parse_iomem() +def print_memory_type(): + print(f"Event: {event_name}") + print(f"{'Memory type':<40} {'count':>10} {'percentage':>10}") + print(f"{'-' * 40:<40} {'-' * 10:>10} {'-' * 10:>10}") + total =3D sum(load_mem_type_cnt.values()) + # Add count from children into the parent. + for i in range(max_indent, -1, -1): + if i not in iomem: + continue + for entry in iomem[i]: + global children + for child in children[entry]: + if load_mem_type_cnt[child] > 0: + load_mem_type_cnt[entry] +=3D load_mem_type_cnt[child] =20 -def trace_end(): - print_memory_type() - f.close() + def print_entries(entries): + """Print counts from parents down to their children""" + global children + for entry in sorted(entries, + key =3D lambda entry: load_mem_type_cnt[entry], + reverse =3D True): + count =3D load_mem_type_cnt[entry] + if count > 0: + mem_type =3D ' ' * entry.indent + f"{entry.begin:x}-{entry= .end:x} : {entry.label}" + percent =3D 100 * count / total + print(f"{mem_type:<40} {count:>10} {percent:>10.1f}") + print_entries(children[entry]) =20 -def is_system_ram(phys_addr): - #/proc/iomem is sorted - position =3D bisect.bisect(system_ram, phys_addr) - if position % 2 =3D=3D 0: - return False - return True + print_entries(iomem[0]) =20 -def is_persistent_mem(phys_addr): - position =3D bisect.bisect(pmem, phys_addr) - if position % 2 =3D=3D 0: - return False - return True +def trace_begin(): + parse_iomem() =20 -def find_memory_type(phys_addr): - if phys_addr =3D=3D 0: - return "N/A" - if is_system_ram(phys_addr): - return "System RAM" +def trace_end(): + print_memory_type() =20 - if is_persistent_mem(phys_addr): - return "Persistent Memory" +def process_event(param_dict): + if "sample" not in param_dict: + return =20 - #slow path, search all - f.seek(0, 0) - for j in f: - m =3D re.split('-|:',j,2) - if int(m[0], 16) <=3D phys_addr <=3D int(m[1], 16): - return m[2] - return "N/A" + sample =3D param_dict["sample"] + if "phys_addr" not in sample: + return =20 -def process_event(param_dict): - name =3D param_dict["ev_name"] - sample =3D param_dict["sample"] - phys_addr =3D sample["phys_addr"] + phys_addr =3D sample["phys_addr"] + entry =3D find_memory_type(phys_addr) + if entry: + load_mem_type_cnt[entry] +=3D 1 =20 - global event_name - if event_name =3D=3D None: - event_name =3D name - load_mem_type_cnt[find_memory_type(phys_addr)] +=3D 1 + global event_name + if event_name is None: + event_name =3D param_dict["ev_name"] --=20 2.47.0.338.g60cca15819-goog