From nobody Wed Feb 11 02:07:19 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1676970169; cv=none; d=zohomail.com; s=zohoarc; b=PrR64JyqCbw091OKUi62hHFTS6quWKlxRvf2ScbHMia5JTCHyOgoS3Z5+vtM8tpkdI4cF/eA/waXPMgp3gAB0swHMq4q6JgBkpQOAGQqmdE8VXNcRMdpRH40TJhM9XpJK1BNwpnVHFYlbk/cbrEdm0mU11oW3v5YNZhFiviSm4o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1676970169; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OLFAeG3fivFW5vWdAB9lVX4ycP8IQRSmjytnCKj6xhE=; b=H7X/dLf1/EU4POhCbkS8QuAJbj0JGY311FNDnhsV+tJrctXOEhcuh2KOvteWmOTUk/pvTiBw9obWbV6J87giyBbeyvxwlzF7u+AUDIdvByyopXZrYJIvxM+LupVbKiik/uKwNLELgi1p29nAjfiqUtW8mT/irBJtdwWF7FElInI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1676970169613816.3924423046847; Tue, 21 Feb 2023 01:02:49 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pUOWi-0007r9-3k; Tue, 21 Feb 2023 04:01:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pUOWf-0007mN-W0 for qemu-devel@nongnu.org; Tue, 21 Feb 2023 04:01:30 -0500 Received: from mr85p00im-ztdg06011201.me.com ([17.58.23.181]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pUOWd-0005Y6-UL for qemu-devel@nongnu.org; Tue, 21 Feb 2023 04:01:29 -0500 Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06011201.me.com (Postfix) with ESMTPSA id B183B962FAB; Tue, 21 Feb 2023 09:01:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1676970087; bh=OLFAeG3fivFW5vWdAB9lVX4ycP8IQRSmjytnCKj6xhE=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=LY/XIO8wXACGZ2TaipOY/t0M+fquattBfWc8wv4XRSytBrSmIyVQu5wMiXPvzQcGj l49IPJVjwLzmTRWIVQInM2Xp4VfVmXZTcBg3LrvhqUw16NoZ9g+vrafIj9qXOvjllk gvzlnDl0kUIPhlCHPivnjvwRkrZfONmUDP+Xelw5S4LAP9RQpK65w+Sj7IbjHUF0uG rEuW0G/b8tOALdJBptc/lI2Lefrxwdeiz/QK8f21RfrRxqOPGiaOlbtx7FZRnHP5U2 t8GNJyKQzgGpEL8yBhjn1Iv91x0x0FM/5eO6y8psTe3iL5HLz0CAAMWSFBWTW5oKCs /s2PmZVw3U9gg== From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH 9/9] simpletrace: Refactor to separate responsibilities Date: Tue, 21 Feb 2023 10:01:04 +0100 Message-Id: <20230221090104.86103-10-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230221090104.86103-1-mads@ynddal.dk> References: <20230221090104.86103-1-mads@ynddal.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Proofpoint-ORIG-GUID: poc2_AyTv1_52Cqnd9C-C_IGex4TFzoO X-Proofpoint-GUID: poc2_AyTv1_52Cqnd9C-C_IGex4TFzoO X-Proofpoint-Virus-Version: =?UTF-8?Q?vendor=3Dfsecure_engine=3D1.1.170-22c6f66c430a71ce266a39bfe25bc?= =?UTF-8?Q?2903e8d5c8f:6.0.425,18.0.816,17.0.605.474.0000000_definitions?= =?UTF-8?Q?=3D2022-01-18=5F01:2022-01-14=5F01,2022-01-18=5F01,2020-01-23?= =?UTF-8?Q?=5F02_signatures=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxlogscore=973 phishscore=0 adultscore=0 spamscore=0 clxscore=1030 bulkscore=0 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2302210078 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=17.58.23.181; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06011201.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ynddal.dk) X-ZM-MESSAGEID: 1676970171421100001 Content-Type: text/plain; charset="utf-8" From: Mads Ynddal Moved event_mapping and event_id_to_name down one level in the function call-stack to keep variable instantiation and usage closer (`process` and `run` has no use of the variables; `read_trace_records` does). Instead of passing event_mapping and event_id_to_name to the bottom of the call-stack, we move their use to `read_trace_records`. This separates responsibility and ownership of the information. `read_record` now just reads the arguments from the file-object by knowning the total number of bytes. Parsing it to specific arguments is moved up to `read_trace_records`. Special handling of dropped events removed, as they can be handled by the general code. Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 121 +++++++++++++++++++---------------------- 1 file changed, 55 insertions(+), 66 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 35d2a757c4..465fcac609 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -24,6 +24,7 @@ =20 log_header_fmt =3D '=3DQQQ' # trace/simple.c::TraceLogHeader rec_header_fmt =3D '=3DQQII' # trace/simple.c::TraceRecord +rec_header_fmt_len =3D struct.calcsize(rec_header_fmt) =20 class SimpleException(Exception): pass @@ -36,35 +37,6 @@ def read_header(fobj, hfmt): raise SimpleException('Error reading header. Wrong filetype provid= ed?') return struct.unpack(hfmt, hdr) =20 -def get_record(event_mapping, event_id_to_name, rechdr, fobj): - """Deserialize a trace record from a file into a tuple - (name, timestamp, pid, arg1, ..., arg6).""" - event_id, timestamp_ns, length, pid =3D rechdr - if event_id !=3D dropped_event_id: - name =3D event_id_to_name[event_id] - try: - event =3D event_mapping[name] - except KeyError as e: - raise SimpleException( - f'{e} event is logged but is not declared in the trace eve= nts' - 'file, try using trace-events-all instead.' - ) - - rec =3D (name, timestamp_ns, pid) - for type, name in event.args: - if is_string(type): - l =3D fobj.read(4) - (len,) =3D struct.unpack('=3DL', l) - s =3D fobj.read(len) - rec =3D rec + (s,) - else: - (value,) =3D struct.unpack('=3DQ', fobj.read(8)) - rec =3D rec + (value,) - else: - (dropped_count,) =3D struct.unpack('=3DQ', fobj.read(8)) - rec =3D ("dropped", timestamp_ns, pid, dropped_count) - return rec - def get_mapping(fobj): (event_id, ) =3D struct.unpack('=3DQ', fobj.read(8)) (len, ) =3D struct.unpack('=3DL', fobj.read(4)) @@ -72,10 +44,11 @@ def get_mapping(fobj): =20 return (event_id, name) =20 -def read_record(event_mapping, event_id_to_name, fobj): - """Deserialize a trace record from a file into a tuple (event_num, tim= estamp, pid, arg1, ..., arg6).""" - rechdr =3D read_header(fobj, rec_header_fmt) - return get_record(event_mapping, event_id_to_name, rechdr, fobj) +def read_record(fobj): + """Deserialize a trace record from a file into a tuple (event_num, tim= estamp, pid, args).""" + event_id, timestamp_ns, record_length, record_pid =3D read_header(fobj= , rec_header_fmt) + args_payload =3D fobj.read(record_length - rec_header_fmt_len) + return (event_id, timestamp_ns, record_pid, args_payload) =20 def read_trace_header(fobj): """Read and verify trace file header""" @@ -90,30 +63,60 @@ def read_trace_header(fobj): if log_version !=3D 4: raise ValueError(f'Log format {log_version} not supported with thi= s QEMU release!') =20 -def read_trace_records(event_mapping, event_id_to_name, fobj): - """Deserialize trace records from a file, yielding record tuples (even= t_num, timestamp, pid, arg1, ..., arg6). - - Note that `event_id_to_name` is modified if the file contains mapping = records. +def read_trace_records(events, fobj, read_header): + """Deserialize trace records from a file, yielding record tuples (even= t, event_num, timestamp, pid, arg1, ..., arg6). =20 Args: event_mapping (str -> Event): events dict, indexed by name - event_id_to_name (int -> str): event names dict, indexed by event = ID fobj (file): input file + read_header (bool): whether to headers were read from fobj =20 """ - while True: - t =3D fobj.read(8) - if len(t) =3D=3D 0: - break + frameinfo =3D inspect.getframeinfo(inspect.currentframe()) + dropped_event =3D Event.build("Dropped_Event(uint64_t num_events_dropp= ed)", + frameinfo.lineno + 1, frameinfo.filename) + + event_mapping =3D {e.name: e for e in events} + event_mapping["dropped"] =3D dropped_event + event_id_to_name =3D {dropped_event_id: "dropped"} + + # If there is no header assume event ID mapping matches events list + if not read_header: + for event_id, event in enumerate(events): + event_id_to_name[event_id] =3D event.name =20 + while t :=3D fobj.read(8): (rectype, ) =3D struct.unpack('=3DQ', t) if rectype =3D=3D record_type_mapping: - event_id, name =3D get_mapping(fobj) - event_id_to_name[event_id] =3D name + event_id, event_name =3D get_mapping(fobj) + event_id_to_name[event_id] =3D event_name else: - rec =3D read_record(event_mapping, event_id_to_name, fobj) - - yield rec + event_id, timestamp_ns, pid, args_payload =3D read_record(fobj) + event_name =3D event_id_to_name[event_id] + + try: + event =3D event_mapping[event_name] + except KeyError as e: + raise SimpleException( + f'{e} event is logged but is not declared in the trace= events' + 'file, try using trace-events-all instead.' + ) + + offset =3D 0 + args =3D [] + for type, _ in event.args: + if is_string(type): + (len,) =3D struct.unpack_from('=3DL', args_payload, of= fset=3Doffset) + offset +=3D 4 + s =3D args_payload[offset:offset+len] + offset +=3D len + args.append(s) + else: + (value,) =3D struct.unpack_from('=3DQ', args_payload, = offset=3Doffset) + offset +=3D 8 + args.append(value) + + yield (event_mapping[event_name], event_name, timestamp_ns, pi= d) + tuple(args) =20 class Analyzer: """A trace file analyzer which processes trace records. @@ -167,28 +170,15 @@ def __exit__(self, _type, value, traceback): """Called at the end of the trace.""" pass =20 -def process(events, log, analyzer_class, read_header=3DTrue): +def process(events_fobj, trace_event_path, log_fobj, analyzer_class, read_= header=3DTrue): """Invoke an analyzer on each event in a log.""" if read_header: - read_trace_header(log) - - frameinfo =3D inspect.getframeinfo(inspect.currentframe()) - dropped_event =3D Event.build("Dropped_Event(uint64_t num_events_dropp= ed)", - frameinfo.lineno + 1, frameinfo.filename) - event_mapping =3D {"dropped": dropped_event} - event_id_to_name =3D {dropped_event_id: "dropped"} + read_trace_header(log_fobj) =20 - for event in events: - event_mapping[event.name] =3D event - - # If there is no header assume event ID mapping matches events list - if not read_header: - for event_id, event in enumerate(events): - event_id_to_name[event_id] =3D event.name + events =3D read_events(events_fobj, trace_event_path) =20 with analyzer_class() as analyzer: - for event_id, timestamp_ns, record_pid, *rec_args in read_trace_re= cords(event_mapping, event_id_to_name, log): - event =3D event_mapping[event_id] + for event, event_id, timestamp_ns, record_pid, *rec_args in read_t= race_records(events, log_fobj, read_header): fn =3D getattr(analyzer, event.name, analyzer.catchall) fn(*rec_args, event=3Devent, event_id=3Devent_id, timestamp_ns= =3Dtimestamp_ns, pid=3Drecord_pid) =20 @@ -207,8 +197,7 @@ def run(analyzer): raise SimpleException(f'usage: {sys.argv[0]} [--no-header] \n') =20 with open(trace_event_path, 'r') as events_fobj, open(trace_file_path,= 'rb') as log_fobj: - events =3D read_events(events_fobj, trace_event_path) - process(events, log_fobj, analyzer, read_header=3Dnot no_header) + process(events_fobj, trace_event_path, log_fobj, analyzer, read_he= ader=3Dnot no_header) =20 if __name__ =3D=3D '__main__': class Formatter(Analyzer): --=20 2.38.1