From nobody Thu Nov 28 13:00:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1692780943; cv=none; d=zohomail.com; s=zohoarc; b=j/cXv3mZcIwbzSFRv02enHJ8BUGxCN0hfSKg3dgBldFSgAvnxTbW030krSD/vtlifjpZQphD51jRkewxF7kWaYt0/xROACVmV1Hze84Q1GZ7IK5g0KpK/6kEXMg1MozPsOU8eYa7U9qh9/UbfFmNsA5ielxyWSennuekgtgRZXA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1692780943; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SvjOE1wSVteO9a7jCIZM0X0ZH3YH3HxN3uWZIwehWM8=; b=mIusu1IbNM41H9aA9HHv62ZIoFernsuo9JxXxp1jjDiduLG2mC52IBIuWEYVwJaxuu0EMnbHOA4L19Ksk77c/TRB9zHelVHN9KJssemiySG9nWvOmpCFQvEgFECz24NWP71rbDYhLQfugAWth8duzo5ci7t3KWZKlc7iHGTMtzI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1692780943582110.1307056073997; Wed, 23 Aug 2023 01:55:43 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdn-00044G-R6; Wed, 23 Aug 2023 04:55:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdl-0003wE-Tp for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:01 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdj-0008GU-SS for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:01 -0400 Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id BEF533A0A86; Wed, 23 Aug 2023 08:54:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780898; bh=SvjOE1wSVteO9a7jCIZM0X0ZH3YH3HxN3uWZIwehWM8=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=hV8j+JieG5en8jZ/lcROw+1/yt7WESpBq5G6+tECGmPYR20eCqsaH9ap9rF17U8Zk 498IaI6mCUCx0JxBkjct6FTn5cXgl2AmQz0zb8ssTcppu0z48swq2vX/2KiwZ42c8s n3ERrKbTnxf3iUVQG4gOXM8pmyv3sTNc7WTYIP80drDXmsxLPz/EDICPJ06qi/VAqX rGPQyuEKc9DRIP92Opmf5jkuMtd98X2laaLWQBnsqUQQi1lSgGqzsglpS8bom8XSij 1O0/BmPSl+kRbZjh9+mwhMiJCP/C88lXXU6pgP6tqs3q5QAsLzqhzFx+8TQzt9muEh XajvMCUxTV9Fg== From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 09/14] simpletrace: refactor to separate responsibilities Date: Wed, 23 Aug 2023 10:54:24 +0200 Message-Id: <20230823085429.20519-10-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-ORIG-GUID: n_1LVRq1CPhgEd0p9c8nQ_RgawW7HRAM X-Proofpoint-GUID: n_1LVRq1CPhgEd0p9c8nQ_RgawW7HRAM X-Proofpoint-Virus-Version: =?UTF-8?Q?vendor=3Dfsecure_engine=3D1.1.170-22c6f66c430a71ce266a39bfe25bc?= =?UTF-8?Q?2903e8d5c8f:6.0.517,18.0.883,17.11.64.514.0000000_definitions?= =?UTF-8?Q?=3D2022-06-21=5F08:2022-06-21=5F01,2022-06-21=5F08,2022-02-23?= =?UTF-8?Q?=5F01_signatures=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=999 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ynddal.dk) X-ZM-MESSAGEID: 1692780945604100007 Content-Type: text/plain; charset="utf-8" From: Mads Ynddal Moved event_mapping and event_id_to_name down one level in the function call-stack to keep variable instantiation and usage closer (`process` and `run` has no use of the variables; `read_trace_records` does). Instead of passing event_mapping and event_id_to_name to the bottom of the call-stack, we move their use to `read_trace_records`. This separates responsibility and ownership of the information. `read_record` now just reads the arguments from the file-object by knowning the total number of bytes. Parsing it to specific arguments is moved up to `read_trace_records`. Special handling of dropped events removed, as they can be handled by the general code. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 115 +++++++++++++++++++---------------------- 1 file changed, 53 insertions(+), 62 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 7f514d1577..0826aef283 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -31,6 +31,7 @@ =20 log_header_fmt =3D '=3DQQQ' rec_header_fmt =3D '=3DQQII' +rec_header_fmt_len =3D struct.calcsize(rec_header_fmt) =20 class SimpleException(Exception): pass @@ -43,35 +44,6 @@ def read_header(fobj, hfmt): raise SimpleException('Error reading header. Wrong filetype provid= ed?') return struct.unpack(hfmt, hdr) =20 -def get_record(event_mapping, event_id_to_name, rechdr, fobj): - """Deserialize a trace record from a file into a tuple - (name, timestamp, pid, arg1, ..., arg6).""" - event_id, timestamp_ns, length, pid =3D rechdr - if event_id !=3D dropped_event_id: - name =3D event_id_to_name[event_id] - try: - event =3D event_mapping[name] - except KeyError as e: - raise SimpleException( - f'{e} event is logged but is not declared in the trace eve= nts' - 'file, try using trace-events-all instead.' - ) - - rec =3D (name, timestamp_ns, pid) - for type, name in event.args: - if is_string(type): - l =3D fobj.read(4) - (len,) =3D struct.unpack('=3DL', l) - s =3D fobj.read(len) - rec =3D rec + (s,) - else: - (value,) =3D struct.unpack('=3DQ', fobj.read(8)) - rec =3D rec + (value,) - else: - (dropped_count,) =3D struct.unpack('=3DQ', fobj.read(8)) - rec =3D ("dropped", timestamp_ns, pid, dropped_count) - return rec - def get_mapping(fobj): (event_id, ) =3D struct.unpack('=3DQ', fobj.read(8)) (len, ) =3D struct.unpack('=3DL', fobj.read(4)) @@ -79,10 +51,11 @@ def get_mapping(fobj): =20 return (event_id, name) =20 -def read_record(event_mapping, event_id_to_name, fobj): - """Deserialize a trace record from a file into a tuple (event_num, tim= estamp, pid, arg1, ..., arg6).""" - rechdr =3D read_header(fobj, rec_header_fmt) - return get_record(event_mapping, event_id_to_name, rechdr, fobj) +def read_record(fobj): + """Deserialize a trace record from a file into a tuple (event_num, tim= estamp, pid, args).""" + event_id, timestamp_ns, record_length, record_pid =3D read_header(fobj= , rec_header_fmt) + args_payload =3D fobj.read(record_length - rec_header_fmt_len) + return (event_id, timestamp_ns, record_pid, args_payload) =20 def read_trace_header(fobj): """Read and verify trace file header""" @@ -97,17 +70,28 @@ def read_trace_header(fobj): if log_version !=3D 4: raise ValueError(f'Log format {log_version} not supported with thi= s QEMU release!') =20 -def read_trace_records(event_mapping, event_id_to_name, fobj): - """Deserialize trace records from a file, yielding record tuples (even= t_num, timestamp, pid, arg1, ..., arg6). - - Note that `event_id_to_name` is modified if the file contains mapping = records. +def read_trace_records(events, fobj, read_header): + """Deserialize trace records from a file, yielding record tuples (even= t, event_num, timestamp, pid, arg1, ..., arg6). =20 Args: event_mapping (str -> Event): events dict, indexed by name - event_id_to_name (int -> str): event names dict, indexed by event = ID fobj (file): input file + read_header (bool): whether headers were read from fobj =20 """ + frameinfo =3D inspect.getframeinfo(inspect.currentframe()) + dropped_event =3D Event.build("Dropped_Event(uint64_t num_events_dropp= ed)", + frameinfo.lineno + 1, frameinfo.filename) + + event_mapping =3D {e.name: e for e in events} + event_mapping["dropped"] =3D dropped_event + event_id_to_name =3D {dropped_event_id: "dropped"} + + # If there is no header assume event ID mapping matches events list + if not read_header: + for event_id, event in enumerate(events): + event_id_to_name[event_id] =3D event.name + while True: t =3D fobj.read(8) if len(t) =3D=3D 0: @@ -115,12 +99,35 @@ def read_trace_records(event_mapping, event_id_to_name= , fobj): =20 (rectype, ) =3D struct.unpack('=3DQ', t) if rectype =3D=3D record_type_mapping: - event_id, name =3D get_mapping(fobj) - event_id_to_name[event_id] =3D name + event_id, event_name =3D get_mapping(fobj) + event_id_to_name[event_id] =3D event_name else: - rec =3D read_record(event_mapping, event_id_to_name, fobj) + event_id, timestamp_ns, pid, args_payload =3D read_record(fobj) + event_name =3D event_id_to_name[event_id] + + try: + event =3D event_mapping[event_name] + except KeyError as e: + raise SimpleException( + f'{e} event is logged but is not declared in the trace= events' + 'file, try using trace-events-all instead.' + ) + + offset =3D 0 + args =3D [] + for type, _ in event.args: + if is_string(type): + (length,) =3D struct.unpack_from('=3DL', args_payload,= offset=3Doffset) + offset +=3D 4 + s =3D args_payload[offset:offset+length] + offset +=3D length + args.append(s) + else: + (value,) =3D struct.unpack_from('=3DQ', args_payload, = offset=3Doffset) + offset +=3D 8 + args.append(value) =20 - yield rec + yield (event_mapping[event_name], event_name, timestamp_ns, pi= d) + tuple(args) =20 class Analyzer: """A trace file analyzer which processes trace records. @@ -202,20 +209,6 @@ def process(events, log, analyzer, read_header=3DTrue): if read_header: read_trace_header(log) =20 - frameinfo =3D inspect.getframeinfo(inspect.currentframe()) - dropped_event =3D Event.build("Dropped_Event(uint64_t num_events_dropp= ed)", - frameinfo.lineno + 1, frameinfo.filename) - event_mapping =3D {"dropped": dropped_event} - event_id_to_name =3D {dropped_event_id: "dropped"} - - for event in events_list: - event_mapping[event.name] =3D event - - # If there is no header assume event ID mapping matches events list - if not read_header: - for event_id, event in enumerate(events_list): - event_id_to_name[event_id] =3D event.name - def build_fn(analyzer, event): if isinstance(event, str): return analyzer.catchall @@ -238,12 +231,10 @@ def build_fn(analyzer, event): =20 with analyzer: fn_cache =3D {} - for rec in read_trace_records(event_mapping, event_id_to_name, log= ): - event_num =3D rec[0] - event =3D event_mapping[event_num] - if event_num not in fn_cache: - fn_cache[event_num] =3D build_fn(analyzer, event) - fn_cache[event_num](event, rec) + for event, event_id, timestamp_ns, record_pid, *rec_args in read_t= race_records(events, log, read_header): + if event_id not in fn_cache: + fn_cache[event_id] =3D build_fn(analyzer, event) + fn_cache[event_id](event, (event_id, timestamp_ns, record_pid,= *rec_args)) =20 if close_log: log.close() --=20 2.38.1