From patchwork Wed Aug 23 08:54:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824551 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=QSD4RDUf; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Sv3xL5z1yNm for ; Wed, 23 Aug 2023 18:56:47 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdX-0003rr-21; Wed, 23 Aug 2023 04:54:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdV-0003r5-FC for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:45 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdU-00086t-1j for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780878; bh=0fJy1sjStRMrctTrd8eWHr4fZQUPa8Euy65mZZsWwKQ=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=QSD4RDUflQ0CRj05c4kiHavEzY74JPNDEbdpsj7F8MGp4PVYnIC6+Vp+/urJIUrXc oTuH0IATRAo6lOK2uDu430SJslR2gayssA6nmAN4SBjcodYdBAfl6XK/f0hO0+JHhl G14LvRzqNEeOwWclBT44hshNgPpqYIp1Xg6RZMckyKDi6zgubkXn83GAbeCvGXo3fs fMa+qHrZgSDpn7zfr+VkhJUQf6pT/gJz4EM79qxTYsAuDFS513bOJ/+8imNYDDCPps XUll+ecRcBmxa/UmLwW6alu4fV9qrDj3CcnV6f9a6Zp/FPGQJ9WRCJdJe1yaO/QTnn 96bMlLXe8+hcg== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id DDDE13A0B0B; Wed, 23 Aug 2023 08:54:35 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 01/14] simpletrace: add __all__ to define public interface Date: Wed, 23 Aug 2023 10:54:16 +0200 Message-Id: <20230823085429.20519-2-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: KB9SXGAJ9n3Oe4RRIGWykMgv4TZGwKWS X-Proofpoint-GUID: KB9SXGAJ9n3Oe4RRIGWykMgv4TZGwKWS X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=489 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal It was unclear what was the supported public interface. I.e. when refactoring the code, what functions/classes are important to retain. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 1f6d1ae1f3..b221d9a241 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -14,6 +14,8 @@ from tracetool import read_events, Event from tracetool.backend.simple import is_string +__all__ = ['Analyzer', 'process', 'run'] + header_event_id = 0xffffffffffffffff header_magic = 0xf2b177cb0aa429b4 dropped_event_id = 0xfffffffffffffffe From patchwork Wed Aug 23 08:54:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824553 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=G/CYibv+; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Sw01ZGz1yh4 for ; Wed, 23 Aug 2023 18:56:48 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdV-0003r4-9N; Wed, 23 Aug 2023 04:54:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdT-0003qk-LE for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:43 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdS-00087Q-7J for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780880; bh=LDRzB8NZ046ueAIigp31NIDUT523AGKGB/KGni40fIc=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=G/CYibv+LSvWZ96R0XtfVsV9nzVxzkuSkXU2roLKPapif2d41n5iGdj3zx8QSo+MU un4sNYKNUdWFV1m2iU3v38510U1k/qw1Pi9VCQXn+a6ovKiMewd/eJx9zYGZCJTCk5 1MfCl0H8ehRGTs9PGoge/Eoe9CBwlf8clSF9C89eRk5qt08UHPpThtIe1ERy6BPC06 9m0l+rdY9wrbAucn7/JJiB78LN7C+V7G2mJjRb/67eMwEAzMKBev7tkTu0zx4XMz5X B4soxsBusV5YDpqLBNVkMF7/EBlxFsvghnU8mXCZfT/KVqxoetO75WpauO6W36XTpv 0nNVvI3ukUCNw== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 99B433A0B70; Wed, 23 Aug 2023 08:54:38 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 02/14] simpletrace: annotate magic constants from QEMU code Date: Wed, 23 Aug 2023 10:54:17 +0200 Message-Id: <20230823085429.20519-3-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 4AZ2DnAWlEUY5XSzk-01RX3RYhcevmUw X-Proofpoint-GUID: 4AZ2DnAWlEUY5XSzk-01RX3RYhcevmUw X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=298 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal It wasn't clear where the constants and structs came from, so I added comments to help. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index b221d9a241..5c230a1b74 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -16,6 +16,11 @@ __all__ = ['Analyzer', 'process', 'run'] +# This is the binary format that the QEMU "simple" trace backend +# emits. There is no specification documentation because the format is +# not guaranteed to be stable. Trace files must be parsed with the +# same trace-events-all file and the same simpletrace.py file that +# QEMU was built with. header_event_id = 0xffffffffffffffff header_magic = 0xf2b177cb0aa429b4 dropped_event_id = 0xfffffffffffffffe From patchwork Wed Aug 23 08:54:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824542 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=hujTVxBY; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Qw09bpz1ybW for ; Wed, 23 Aug 2023 18:55:04 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdY-0003sb-R9; Wed, 23 Aug 2023 04:54:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdW-0003ri-9r for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:46 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdU-00089n-H9 for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780883; bh=tRX60fqCDLTAh6rk1c2F28puMh0+hQQ2UdFppuN+qzw=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=hujTVxBYx8DOG5HkgEcZxDVloLGfRlEcWef5gehQHRcc5VQCK6mRUTwPbjA/i7zD4 0S3k0N/P8SC0/9PO7AFyKN+r0hZIw74SHeAQi8LtshGQMD60kzhXRiYl1Qx5NaIez2 DcANC+A8+UpTgJxmwnDOz9HPhFqexJ9sYIaW5m28M6J1wxq5dqsSIYV4xG0XnpvFmp ustb3GyMAwxwdBkcnBVg+QJjwJCEbJ5m/R3fw83jZgzV7Lv4GVVP320QpJp1NUyoOU XHp2BwQZXBnOyFmW+LqG57GdKrtf2Q7k7jcBN2DfmkEoyUkkLkW0c5kiS/0Tv+kglY fct23OdvJfDiA== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 1FDE73A09FD; Wed, 23 Aug 2023 08:54:40 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 03/14] simpletrace: improve parsing of sys.argv; fix files never closed. Date: Wed, 23 Aug 2023 10:54:18 +0200 Message-Id: <20230823085429.20519-4-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: iMe5a3eH0NtAnc3b2_trxUtkugqKIUub X-Proofpoint-GUID: iMe5a3eH0NtAnc3b2_trxUtkugqKIUub X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=697 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal The arguments extracted from `sys.argv` named and unpacked to make it clear what the arguments are and what they're used for. The two input files were opened, but never explicitly closed. File usage changed to use `with` statement to take care of this. At the same time, ownership of the file-object is moved up to `run` function. Added option to process to support file-like objects. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 50 ++++++++++++++++++++++++++++-------------- 1 file changed, 34 insertions(+), 16 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 5c230a1b74..283b5918a1 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -9,6 +9,7 @@ # # For help see docs/devel/tracing.rst +import sys import struct import inspect from tracetool import read_events, Event @@ -51,7 +52,6 @@ def get_record(edict, idtoname, rechdr, fobj): try: event = edict[name] except KeyError as e: - import sys sys.stderr.write('%s event is logged but is not declared ' \ 'in the trace events file, try using ' \ 'trace-events-all instead.\n' % str(e)) @@ -172,11 +172,28 @@ def end(self): pass def process(events, log, analyzer, read_header=True): - """Invoke an analyzer on each event in a log.""" + """Invoke an analyzer on each event in a log. + Args: + events (file-object or list or str): events list or file-like object or file path as str to read event data from + log (file-object or str): file-like object or file path as str to read log data from + analyzer (Analyzer): Instance of Analyzer to interpret the event data + read_header (bool, optional): Whether to read header data from the log data. Defaults to True. + """ + if isinstance(events, str): - events = read_events(open(events, 'r'), events) + with open(events, 'r') as f: + events_list = read_events(f, events) + elif isinstance(events, list): + # Treat as a list of events already produced by tracetool.read_events + events_list = events + else: + # Treat as an already opened file-object + events_list = read_events(events, events.name) + + close_log = False if isinstance(log, str): log = open(log, 'rb') + close_log = True if read_header: read_trace_header(log) @@ -187,12 +204,12 @@ def process(events, log, analyzer, read_header=True): edict = {"dropped": dropped_event} idtoname = {dropped_event_id: "dropped"} - for event in events: + for event in events_list: edict[event.name] = event # If there is no header assume event ID mapping matches events list if not read_header: - for event_id, event in enumerate(events): + for event_id, event in enumerate(events_list): idtoname[event_id] = event.name def build_fn(analyzer, event): @@ -225,24 +242,25 @@ def build_fn(analyzer, event): fn_cache[event_num](event, rec) analyzer.end() + if close_log: + log.close() + def run(analyzer): """Execute an analyzer on a trace file given on the command-line. This function is useful as a driver for simple analysis scripts. More advanced scripts will want to call process() instead.""" - import sys - - read_header = True - if len(sys.argv) == 4 and sys.argv[1] == '--no-header': - read_header = False - del sys.argv[1] - elif len(sys.argv) != 3: - sys.stderr.write('usage: %s [--no-header] ' \ - '\n' % sys.argv[0]) + + try: + # NOTE: See built-in `argparse` module for a more robust cli interface + *no_header, trace_event_path, trace_file_path = sys.argv[1:] + assert no_header == [] or no_header == ['--no-header'], 'Invalid no-header argument' + except (AssertionError, ValueError): + sys.stderr.write(f'usage: {sys.argv[0]} [--no-header] \n') sys.exit(1) - events = read_events(open(sys.argv[1], 'r'), sys.argv[1]) - process(events, sys.argv[2], analyzer, read_header=read_header) + with open(trace_event_path, 'r') as events_fobj, open(trace_file_path, 'rb') as log_fobj: + process(events_fobj, log_fobj, analyzer, read_header=not no_header) if __name__ == '__main__': class Formatter(Analyzer): From patchwork Wed Aug 23 08:54:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824543 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=dDOp8E5v; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0R93Nnpz1yNm for ; Wed, 23 Aug 2023 18:55:17 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjda-0003sy-Nb; Wed, 23 Aug 2023 04:54:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdY-0003sO-MI for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:48 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdX-0008AG-1V for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780886; bh=P9WjvnEfg+gRxF9hXm33GvQAhcZGvYrETbTjf9cgSgk=; h=From:To:Subject:Date:Message-Id:MIME-Version:Content-Type; b=dDOp8E5vQTxmCZ8KajmXnUeZznimJ3AdPYiyFhaS6x4eT5hkfDbGZfGVj6bHb+9G3 VEGzS7/9bkP8SHg5bAj3vbII8U3SZTxr7gy87NC1aWweB2863eTttqXlPSD8IJe8KY dKS/NOxOSQcZeg13SxT8tiW9IxIff3HFwdSdA//tRc011iDqNkYdZ9+46guDPuuyDM VdNgqTeDnQbiVVz0EQm1WEpqZBg1F3AueR2usJvxQBAU25L/EL7bF2grvWgihdm5aQ tY7oPd5RTq/psoOPSfCG7cQiUkqxPpBOpNah4nL1P/DWNud9rxdaX50eocumWToPFt ygFyePqwpsxEw== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 002643A0B00; Wed, 23 Aug 2023 08:54:43 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v4 04/14] simpletrace: changed naming of edict and idtoname to improve readability Date: Wed, 23 Aug 2023 10:54:19 +0200 Message-Id: <20230823085429.20519-5-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: _Yt6AhzD1DslkMWYOdlddaImtaLfMRH4 X-Proofpoint-GUID: _Yt6AhzD1DslkMWYOdlddaImtaLfMRH4 X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=815 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Readability is subjective, but I've expanded the naming of the variables and arguments, to help with understanding for new eyes on the code. Signed-off-by: Mads Ynddal Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 283b5918a1..09511f624d 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -40,17 +40,17 @@ def read_header(fobj, hfmt): return None return struct.unpack(hfmt, hdr) -def get_record(edict, idtoname, rechdr, fobj): +def get_record(event_mapping, event_id_to_name, rechdr, fobj): """Deserialize a trace record from a file into a tuple (name, timestamp, pid, arg1, ..., arg6).""" if rechdr is None: return None if rechdr[0] != dropped_event_id: event_id = rechdr[0] - name = idtoname[event_id] + name = event_id_to_name[event_id] rec = (name, rechdr[1], rechdr[3]) try: - event = edict[name] + event = event_mapping[name] except KeyError as e: sys.stderr.write('%s event is logged but is not declared ' \ 'in the trace events file, try using ' \ @@ -79,10 +79,10 @@ def get_mapping(fobj): return (event_id, name) -def read_record(edict, idtoname, fobj): +def read_record(event_mapping, event_id_to_name, fobj): """Deserialize a trace record from a file into a tuple (event_num, timestamp, pid, arg1, ..., arg6).""" rechdr = read_header(fobj, rec_header_fmt) - return get_record(edict, idtoname, rechdr, fobj) + return get_record(event_mapping, event_id_to_name, rechdr, fobj) def read_trace_header(fobj): """Read and verify trace file header""" @@ -103,14 +103,14 @@ def read_trace_header(fobj): raise ValueError('Log format %d not supported with this QEMU release!' % log_version) -def read_trace_records(edict, idtoname, fobj): +def read_trace_records(event_mapping, event_id_to_name, fobj): """Deserialize trace records from a file, yielding record tuples (event_num, timestamp, pid, arg1, ..., arg6). - Note that `idtoname` is modified if the file contains mapping records. + Note that `event_id_to_name` is modified if the file contains mapping records. Args: - edict (str -> Event): events dict, indexed by name - idtoname (int -> str): event names dict, indexed by event ID + event_mapping (str -> Event): events dict, indexed by name + event_id_to_name (int -> str): event names dict, indexed by event ID fobj (file): input file """ @@ -122,9 +122,9 @@ def read_trace_records(edict, idtoname, fobj): (rectype, ) = struct.unpack('=Q', t) if rectype == record_type_mapping: event_id, name = get_mapping(fobj) - idtoname[event_id] = name + event_id_to_name[event_id] = name else: - rec = read_record(edict, idtoname, fobj) + rec = read_record(event_mapping, event_id_to_name, fobj) yield rec @@ -201,16 +201,16 @@ def process(events, log, analyzer, read_header=True): frameinfo = inspect.getframeinfo(inspect.currentframe()) dropped_event = Event.build("Dropped_Event(uint64_t num_events_dropped)", frameinfo.lineno + 1, frameinfo.filename) - edict = {"dropped": dropped_event} - idtoname = {dropped_event_id: "dropped"} + event_mapping = {"dropped": dropped_event} + event_id_to_name = {dropped_event_id: "dropped"} for event in events_list: - edict[event.name] = event + event_mapping[event.name] = event # If there is no header assume event ID mapping matches events list if not read_header: for event_id, event in enumerate(events_list): - idtoname[event_id] = event.name + event_id_to_name[event_id] = event.name def build_fn(analyzer, event): if isinstance(event, str): @@ -234,9 +234,9 @@ def build_fn(analyzer, event): analyzer.begin() fn_cache = {} - for rec in read_trace_records(edict, idtoname, log): + for rec in read_trace_records(event_mapping, event_id_to_name, log): event_num = rec[0] - event = edict[event_num] + event = event_mapping[event_num] if event_num not in fn_cache: fn_cache[event_num] = build_fn(analyzer, event) fn_cache[event_num](event, rec) From patchwork Wed Aug 23 08:54:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824544 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=CS57kwG5; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0RH6gcZz1yNm for ; Wed, 23 Aug 2023 18:55:23 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdc-0003tD-9k; Wed, 23 Aug 2023 04:54:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdb-0003t0-27 for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:51 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdZ-0008CD-Ja for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780888; bh=2eo6M7Po98p7txvfneyaWrztwzb3AvIPCmOUr/Bqc1k=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=CS57kwG5jna64i8qoPa6jwntKUD+E4JlxBOvkRvYWJtZk9hbvrE8L/MutmaR4bsSJ ROUBBsU7A0z7z/rdOLaOWuxyxv8deMVdxLiKWCZNiIcQ415idXWXX4IDUciH1JFp4V gwp4SwrmiI0bGwP9qGyHsocOFJwvBOcia0S0H1KLekQuqchC5F7AI52xR8t2e4iCge boZSr0A2MTVe5fL+Q7zNxkuBH3hxOmT/PJAXOFwoZ4f8stLDoRI1QZu4rIDoji3CcU BphIIw+Hwl7vf3QP9KpySsCUXPmt1NazSwbg+3kq7EDWJbTR+IBVZfGEEd1xDoKmbN 1NvmdjcWVhvYg== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 9C3F43A0AF2; Wed, 23 Aug 2023 08:54:46 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 05/14] simpletrace: update code for Python 3.11 Date: Wed, 23 Aug 2023 10:54:20 +0200 Message-Id: <20230823085429.20519-6-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: j9D0rmyqpV-B1m3S0Q05juaIEVHGsuBZ X-Proofpoint-GUID: j9D0rmyqpV-B1m3S0Q05juaIEVHGsuBZ X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=904 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal The call to `getargspec` was deprecated and in Python 3.11 it has been removed in favor of `getfullargspec`. `getfullargspec` is compatible with QEMU's requirement of at least Python version 3.6. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé --- scripts/simpletrace.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 09511f624d..971b2a0f6a 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -221,7 +221,7 @@ def build_fn(analyzer, event): return analyzer.catchall event_argcount = len(event.args) - fn_argcount = len(inspect.getargspec(fn)[0]) - 1 + fn_argcount = len(inspect.getfullargspec(fn)[0]) - 1 if fn_argcount == event_argcount + 1: # Include timestamp as first argument return lambda _, rec: fn(*(rec[1:2] + rec[3:3 + event_argcount])) From patchwork Wed Aug 23 08:54:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824549 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=dBbeqQoX; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0SN4vxrz1yNm for ; Wed, 23 Aug 2023 18:56:20 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdf-0003uP-MM; Wed, 23 Aug 2023 04:54:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdd-0003uE-P5 for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:53 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdc-0008Dx-4T for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780891; bh=tv8iQdRgnNjq6DBzxFJS9huknQBhYuHeuL8iwQFn9rw=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=dBbeqQoXHUh1hqUds9kCBcbuNqKInd1DUkd4H7HC1Lbs7hWC6lg/zp54ZTL+/eoTA skPY/KdxCMiixVSKAACd0Kyf3sc5NrkM+Z4ZWthFx9tcIM4KnvHGdIvC1pR3Y7FV41 aoMRNNWBIOqKHIVxNduyuwpL2ZHuuCSjJWtEdfGl8tjHKHWP5A0V2LP4R2rnlsQ/ky ANt55dOQrCkK4dy+Hh10yPBb3QP24SRpTPOaOdZSyiERJQvPPeAunqdetbbrBq0/AJ 5b7pTnoYkE5MwPh/OIWGCxGDV7auzLVhgrxkcOiefOKzLWlg9JoxU4ohquAujxPth9 p5CV/WkxBJdmg== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 31D9D3A0BAE; Wed, 23 Aug 2023 08:54:48 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 06/14] simpletrace: improved error handling on struct unpack Date: Wed, 23 Aug 2023 10:54:21 +0200 Message-Id: <20230823085429.20519-7-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: -4RPrpLXnOBi2F8ztseGgY9LxKMcEA_T X-Proofpoint-GUID: -4RPrpLXnOBi2F8ztseGgY9LxKMcEA_T X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=366 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal A failed call to `read_header` wouldn't be handled the same for the two different code paths (one path would try to use `None` as a list). Changed to raise exception to be handled centrally. This also allows for easier unpacking, as errors has been filtered out. Signed-off-by: Mads Ynddal Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 41 ++++++++++++++++------------------------- 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 971b2a0f6a..8aea0d169b 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -37,26 +37,24 @@ def read_header(fobj, hfmt): hlen = struct.calcsize(hfmt) hdr = fobj.read(hlen) if len(hdr) != hlen: - return None + raise ValueError('Error reading header. Wrong filetype provided?') return struct.unpack(hfmt, hdr) def get_record(event_mapping, event_id_to_name, rechdr, fobj): """Deserialize a trace record from a file into a tuple (name, timestamp, pid, arg1, ..., arg6).""" - if rechdr is None: - return None - if rechdr[0] != dropped_event_id: - event_id = rechdr[0] + event_id, timestamp_ns, length, pid = rechdr + if event_id != dropped_event_id: name = event_id_to_name[event_id] - rec = (name, rechdr[1], rechdr[3]) try: event = event_mapping[name] except KeyError as e: - sys.stderr.write('%s event is logged but is not declared ' \ + sys.stderr.write(f'{e} event is logged but is not declared ' \ 'in the trace events file, try using ' \ - 'trace-events-all instead.\n' % str(e)) + 'trace-events-all instead.\n') sys.exit(1) + rec = (name, timestamp_ns, pid) for type, name in event.args: if is_string(type): l = fobj.read(4) @@ -67,9 +65,8 @@ def get_record(event_mapping, event_id_to_name, rechdr, fobj): (value,) = struct.unpack('=Q', fobj.read(8)) rec = rec + (value,) else: - rec = ("dropped", rechdr[1], rechdr[3]) - (value,) = struct.unpack('=Q', fobj.read(8)) - rec = rec + (value,) + (dropped_count,) = struct.unpack('=Q', fobj.read(8)) + rec = ("dropped", timestamp_ns, pid, dropped_count) return rec def get_mapping(fobj): @@ -86,22 +83,16 @@ def read_record(event_mapping, event_id_to_name, fobj): def read_trace_header(fobj): """Read and verify trace file header""" - header = read_header(fobj, log_header_fmt) - if header is None: - raise ValueError('Not a valid trace file!') - if header[0] != header_event_id: - raise ValueError('Not a valid trace file, header id %d != %d' % - (header[0], header_event_id)) - if header[1] != header_magic: - raise ValueError('Not a valid trace file, header magic %d != %d' % - (header[1], header_magic)) - - log_version = header[2] + _header_event_id, _header_magic, log_version = read_header(fobj, log_header_fmt) + if _header_event_id != header_event_id: + raise ValueError(f'Not a valid trace file, header id {_header_event_id} != {header_event_id}') + if _header_magic != header_magic: + raise ValueError(f'Not a valid trace file, header magic {_header_magic} != {header_magic}') + if log_version not in [0, 2, 3, 4]: - raise ValueError('Unknown version of tracelog format!') + raise ValueError(f'Unknown version {log_version} of tracelog format!') if log_version != 4: - raise ValueError('Log format %d not supported with this QEMU release!' - % log_version) + raise ValueError(f'Log format {log_version} not supported with this QEMU release!') def read_trace_records(event_mapping, event_id_to_name, fobj): """Deserialize trace records from a file, yielding record tuples (event_num, timestamp, pid, arg1, ..., arg6). From patchwork Wed Aug 23 08:54:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824548 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=em+v1+5U; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0SB6WPfz1yNm for ; Wed, 23 Aug 2023 18:56:10 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdh-0003uq-8k; Wed, 23 Aug 2023 04:54:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdg-0003uh-7J for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:56 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjde-0008Ey-Nj for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780893; bh=W83Ca9yBFXSN52i0UPeLf92nJZuoNUL362rz88h/WN4=; h=From:To:Subject:Date:Message-Id:MIME-Version:Content-Type; b=em+v1+5U/ZU12COjuos8Yhd9JdUZ9UeiRscZ3+cTrpc40a4FRkG5DMERzJWhPNK+a tixEPp70RQ3ie9knPMDkmLjKwnAaiBcFbIBKsywVBgnPFLSysIOP45EVMlCmWtwv1o 3GuVI0JIkwUe1m5uQiZgDTdiDIkNMqG+mnbOC+z4hiOWmee4JNLssuvAEhCaSnuqE1 ieXWez8o2sj2EeRUTwkV5e/bzK3j24UuFniRsTNcwycL1POCDmWpB/MstSNE3YIuO4 NLjGlpGg2RnFAC/XzkPfXEdjAvOw9bdyqpQA8jdmJwTxaV6Ndp0RX4xqAy5+XJAG/W joboAlJSP6y5Q== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 9B8613A0B7A; Wed, 23 Aug 2023 08:54:51 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v4 07/14] simpletrace: define exception and add handling Date: Wed, 23 Aug 2023 10:54:22 +0200 Message-Id: <20230823085429.20519-8-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: _o6R2X_eXKb8Xn-HxNpjaBJaEb-ZS-q- X-Proofpoint-GUID: _o6R2X_eXKb8Xn-HxNpjaBJaEb-ZS-q- X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=690 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Define `SimpleException` to differentiate our exceptions from generic exceptions (IOError, etc.). Adapted simpletrace to support this and output to stderr. Signed-off-by: Mads Ynddal Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 8aea0d169b..229b10aa99 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -32,12 +32,15 @@ log_header_fmt = '=QQQ' rec_header_fmt = '=QQII' +class SimpleException(Exception): + pass + def read_header(fobj, hfmt): '''Read a trace record header''' hlen = struct.calcsize(hfmt) hdr = fobj.read(hlen) if len(hdr) != hlen: - raise ValueError('Error reading header. Wrong filetype provided?') + raise SimpleException('Error reading header. Wrong filetype provided?') return struct.unpack(hfmt, hdr) def get_record(event_mapping, event_id_to_name, rechdr, fobj): @@ -49,10 +52,10 @@ def get_record(event_mapping, event_id_to_name, rechdr, fobj): try: event = event_mapping[name] except KeyError as e: - sys.stderr.write(f'{e} event is logged but is not declared ' \ - 'in the trace events file, try using ' \ - 'trace-events-all instead.\n') - sys.exit(1) + raise SimpleException( + f'{e} event is logged but is not declared in the trace events' + 'file, try using trace-events-all instead.' + ) rec = (name, timestamp_ns, pid) for type, name in event.args: @@ -247,8 +250,7 @@ def run(analyzer): *no_header, trace_event_path, trace_file_path = sys.argv[1:] assert no_header == [] or no_header == ['--no-header'], 'Invalid no-header argument' except (AssertionError, ValueError): - sys.stderr.write(f'usage: {sys.argv[0]} [--no-header] \n') - sys.exit(1) + raise SimpleException(f'usage: {sys.argv[0]} [--no-header] \n') with open(trace_event_path, 'r') as events_fobj, open(trace_file_path, 'rb') as log_fobj: process(events_fobj, log_fobj, analyzer, read_header=not no_header) @@ -276,4 +278,8 @@ def catchall(self, event, rec): i += 1 print(' '.join(fields)) - run(Formatter()) + try: + run(Formatter()) + except SimpleException as e: + sys.stderr.write(str(e) + "\n") + sys.exit(1) From patchwork Wed Aug 23 08:54:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824552 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=fW06zrHo; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Sv5MjYz1ybW for ; Wed, 23 Aug 2023 18:56:47 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdj-0003vZ-O2; Wed, 23 Aug 2023 04:54:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdi-0003vF-Ia for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:58 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdh-0008Fi-5S for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:54:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780896; bh=zbUuQMImgFXI+shnIyEaHz7dPAc3pcIDgFyYbkPOa8A=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=fW06zrHo7yluhE1CxpV4pu6CpB4OFieztqjAZXN6ticAg4xVPhHqvpVPWJCFPLmkb +NuPm53nOhAYylkbU15M7pEQarVkeRzewCyyGc5dXcccGTWBB38uDbgDLNeevWXnbM pIAd4T1KzeYwaZAX0w9KmVic0k6TVKQtCN303reD6jfi+0P9s84E3CpwSfMWWR0LLu uMsdv9TziI0bT4kzP4M6tntHxF/tGydRpaIHyvRsneufw5o/Oxxnt1mq1EKcNvtu5s 0hh0iNrydVIHHcpBWxSZsDnBf/yjktkf7gFUxXEGnDrp5qhlxXpFXSmXtSSR0TPl63 BL1NE/UBtZx/A== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 50C493A0A87; Wed, 23 Aug 2023 08:54:54 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 08/14] simpletrace: made Analyzer into context-manager Date: Wed, 23 Aug 2023 10:54:23 +0200 Message-Id: <20230823085429.20519-9-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: LodQfIG7zHL0lRsb6zQjP6kLAthaWsK9 X-Proofpoint-GUID: LodQfIG7zHL0lRsb6zQjP6kLAthaWsK9 X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=800 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Instead of explicitly calling `begin` and `end`, we can change the class to use the context-manager paradigm. This is mostly a styling choice, used in modern Python code. But it also allows for more advanced analyzers to handle exceptions gracefully in the `__exit__` method (not demonstrated here). Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 229b10aa99..7f514d1577 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -122,12 +122,13 @@ def read_trace_records(event_mapping, event_id_to_name, fobj): yield rec -class Analyzer(object): +class Analyzer: """A trace file analyzer which processes trace records. An analyzer can be passed to run() or process(). The begin() method is invoked, then each trace record is processed, and finally the end() method - is invoked. + is invoked. When Analyzer is used as a context-manager (using the `with` + statement), begin() and end() are called automatically. If a method matching a trace event name exists, it is invoked to process that trace record. Otherwise the catchall() method is invoked. @@ -165,6 +166,15 @@ def end(self): """Called at the end of the trace.""" pass + def __enter__(self): + self.begin() + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + if exc_type is None: + self.end() + return False + def process(events, log, analyzer, read_header=True): """Invoke an analyzer on each event in a log. Args: @@ -226,15 +236,14 @@ def build_fn(analyzer, event): # Just arguments, no timestamp or pid return lambda _, rec: fn(*rec[3:3 + event_argcount]) - analyzer.begin() - fn_cache = {} - for rec in read_trace_records(event_mapping, event_id_to_name, log): - event_num = rec[0] - event = event_mapping[event_num] - if event_num not in fn_cache: - fn_cache[event_num] = build_fn(analyzer, event) - fn_cache[event_num](event, rec) - analyzer.end() + with analyzer: + fn_cache = {} + for rec in read_trace_records(event_mapping, event_id_to_name, log): + event_num = rec[0] + event = event_mapping[event_num] + if event_num not in fn_cache: + fn_cache[event_num] = build_fn(analyzer, event) + fn_cache[event_num](event, rec) if close_log: log.close() From patchwork Wed Aug 23 08:54:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824545 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=hV8j+Jie; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Rh3G65z1yNm for ; Wed, 23 Aug 2023 18:55:44 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdn-00044G-R6; Wed, 23 Aug 2023 04:55:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdl-0003wE-Tp for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:01 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdj-0008GU-SS for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780898; bh=SvjOE1wSVteO9a7jCIZM0X0ZH3YH3HxN3uWZIwehWM8=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=hV8j+JieG5en8jZ/lcROw+1/yt7WESpBq5G6+tECGmPYR20eCqsaH9ap9rF17U8Zk 498IaI6mCUCx0JxBkjct6FTn5cXgl2AmQz0zb8ssTcppu0z48swq2vX/2KiwZ42c8s n3ERrKbTnxf3iUVQG4gOXM8pmyv3sTNc7WTYIP80drDXmsxLPz/EDICPJ06qi/VAqX rGPQyuEKc9DRIP92Opmf5jkuMtd98X2laaLWQBnsqUQQi1lSgGqzsglpS8bom8XSij 1O0/BmPSl+kRbZjh9+mwhMiJCP/C88lXXU6pgP6tqs3q5QAsLzqhzFx+8TQzt9muEh XajvMCUxTV9Fg== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id BEF533A0A86; Wed, 23 Aug 2023 08:54:56 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 09/14] simpletrace: refactor to separate responsibilities Date: Wed, 23 Aug 2023 10:54:24 +0200 Message-Id: <20230823085429.20519-10-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: n_1LVRq1CPhgEd0p9c8nQ_RgawW7HRAM X-Proofpoint-GUID: n_1LVRq1CPhgEd0p9c8nQ_RgawW7HRAM X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=999 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Moved event_mapping and event_id_to_name down one level in the function call-stack to keep variable instantiation and usage closer (`process` and `run` has no use of the variables; `read_trace_records` does). Instead of passing event_mapping and event_id_to_name to the bottom of the call-stack, we move their use to `read_trace_records`. This separates responsibility and ownership of the information. `read_record` now just reads the arguments from the file-object by knowning the total number of bytes. Parsing it to specific arguments is moved up to `read_trace_records`. Special handling of dropped events removed, as they can be handled by the general code. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 115 +++++++++++++++++++---------------------- 1 file changed, 53 insertions(+), 62 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 7f514d1577..0826aef283 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -31,6 +31,7 @@ log_header_fmt = '=QQQ' rec_header_fmt = '=QQII' +rec_header_fmt_len = struct.calcsize(rec_header_fmt) class SimpleException(Exception): pass @@ -43,35 +44,6 @@ def read_header(fobj, hfmt): raise SimpleException('Error reading header. Wrong filetype provided?') return struct.unpack(hfmt, hdr) -def get_record(event_mapping, event_id_to_name, rechdr, fobj): - """Deserialize a trace record from a file into a tuple - (name, timestamp, pid, arg1, ..., arg6).""" - event_id, timestamp_ns, length, pid = rechdr - if event_id != dropped_event_id: - name = event_id_to_name[event_id] - try: - event = event_mapping[name] - except KeyError as e: - raise SimpleException( - f'{e} event is logged but is not declared in the trace events' - 'file, try using trace-events-all instead.' - ) - - rec = (name, timestamp_ns, pid) - for type, name in event.args: - if is_string(type): - l = fobj.read(4) - (len,) = struct.unpack('=L', l) - s = fobj.read(len) - rec = rec + (s,) - else: - (value,) = struct.unpack('=Q', fobj.read(8)) - rec = rec + (value,) - else: - (dropped_count,) = struct.unpack('=Q', fobj.read(8)) - rec = ("dropped", timestamp_ns, pid, dropped_count) - return rec - def get_mapping(fobj): (event_id, ) = struct.unpack('=Q', fobj.read(8)) (len, ) = struct.unpack('=L', fobj.read(4)) @@ -79,10 +51,11 @@ def get_mapping(fobj): return (event_id, name) -def read_record(event_mapping, event_id_to_name, fobj): - """Deserialize a trace record from a file into a tuple (event_num, timestamp, pid, arg1, ..., arg6).""" - rechdr = read_header(fobj, rec_header_fmt) - return get_record(event_mapping, event_id_to_name, rechdr, fobj) +def read_record(fobj): + """Deserialize a trace record from a file into a tuple (event_num, timestamp, pid, args).""" + event_id, timestamp_ns, record_length, record_pid = read_header(fobj, rec_header_fmt) + args_payload = fobj.read(record_length - rec_header_fmt_len) + return (event_id, timestamp_ns, record_pid, args_payload) def read_trace_header(fobj): """Read and verify trace file header""" @@ -97,17 +70,28 @@ def read_trace_header(fobj): if log_version != 4: raise ValueError(f'Log format {log_version} not supported with this QEMU release!') -def read_trace_records(event_mapping, event_id_to_name, fobj): - """Deserialize trace records from a file, yielding record tuples (event_num, timestamp, pid, arg1, ..., arg6). - - Note that `event_id_to_name` is modified if the file contains mapping records. +def read_trace_records(events, fobj, read_header): + """Deserialize trace records from a file, yielding record tuples (event, event_num, timestamp, pid, arg1, ..., arg6). Args: event_mapping (str -> Event): events dict, indexed by name - event_id_to_name (int -> str): event names dict, indexed by event ID fobj (file): input file + read_header (bool): whether headers were read from fobj """ + frameinfo = inspect.getframeinfo(inspect.currentframe()) + dropped_event = Event.build("Dropped_Event(uint64_t num_events_dropped)", + frameinfo.lineno + 1, frameinfo.filename) + + event_mapping = {e.name: e for e in events} + event_mapping["dropped"] = dropped_event + event_id_to_name = {dropped_event_id: "dropped"} + + # If there is no header assume event ID mapping matches events list + if not read_header: + for event_id, event in enumerate(events): + event_id_to_name[event_id] = event.name + while True: t = fobj.read(8) if len(t) == 0: @@ -115,12 +99,35 @@ def read_trace_records(event_mapping, event_id_to_name, fobj): (rectype, ) = struct.unpack('=Q', t) if rectype == record_type_mapping: - event_id, name = get_mapping(fobj) - event_id_to_name[event_id] = name + event_id, event_name = get_mapping(fobj) + event_id_to_name[event_id] = event_name else: - rec = read_record(event_mapping, event_id_to_name, fobj) + event_id, timestamp_ns, pid, args_payload = read_record(fobj) + event_name = event_id_to_name[event_id] + + try: + event = event_mapping[event_name] + except KeyError as e: + raise SimpleException( + f'{e} event is logged but is not declared in the trace events' + 'file, try using trace-events-all instead.' + ) + + offset = 0 + args = [] + for type, _ in event.args: + if is_string(type): + (length,) = struct.unpack_from('=L', args_payload, offset=offset) + offset += 4 + s = args_payload[offset:offset+length] + offset += length + args.append(s) + else: + (value,) = struct.unpack_from('=Q', args_payload, offset=offset) + offset += 8 + args.append(value) - yield rec + yield (event_mapping[event_name], event_name, timestamp_ns, pid) + tuple(args) class Analyzer: """A trace file analyzer which processes trace records. @@ -202,20 +209,6 @@ def process(events, log, analyzer, read_header=True): if read_header: read_trace_header(log) - frameinfo = inspect.getframeinfo(inspect.currentframe()) - dropped_event = Event.build("Dropped_Event(uint64_t num_events_dropped)", - frameinfo.lineno + 1, frameinfo.filename) - event_mapping = {"dropped": dropped_event} - event_id_to_name = {dropped_event_id: "dropped"} - - for event in events_list: - event_mapping[event.name] = event - - # If there is no header assume event ID mapping matches events list - if not read_header: - for event_id, event in enumerate(events_list): - event_id_to_name[event_id] = event.name - def build_fn(analyzer, event): if isinstance(event, str): return analyzer.catchall @@ -238,12 +231,10 @@ def build_fn(analyzer, event): with analyzer: fn_cache = {} - for rec in read_trace_records(event_mapping, event_id_to_name, log): - event_num = rec[0] - event = event_mapping[event_num] - if event_num not in fn_cache: - fn_cache[event_num] = build_fn(analyzer, event) - fn_cache[event_num](event, rec) + for event, event_id, timestamp_ns, record_pid, *rec_args in read_trace_records(events, log, read_header): + if event_id not in fn_cache: + fn_cache[event_id] = build_fn(analyzer, event) + fn_cache[event_id](event, (event_id, timestamp_ns, record_pid, *rec_args)) if close_log: log.close() From patchwork Wed Aug 23 08:54:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824547 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=b1fljrCL; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Rv1wylz1yNm for ; Wed, 23 Aug 2023 18:55:55 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjdr-0004Mg-62; Wed, 23 Aug 2023 04:55:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdp-0004DD-5N for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:05 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdn-0008Jl-1q for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780901; bh=WJybbYg8Al1st0uItsQ36eA7qhiUWArDe/Aix+sMO1c=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=b1fljrCLJHkoFTFmY/fVYnIblyXoIz7pjgw4XBg14OlBxcAOWrrGZtg9Jgf7/uYOe vksk0RcrLrfDUY4+yc8POsctOkaTkJNPmgz4q7oIPpmO+e8lf8G/CLu8aIvqKYz3vC D+7F7YwBWcwYhAVp5qOLvumn2muEOs+01sRCYfbj7wWIw317p8FvhqO/wm6idq6lYf lzyqFg1NC683OmzBZfXkk9L4Gko62+dk9n6FWvUxYxqZQejoqLPb/ZrkUqNuYxXl8w 4U8CgqwFgErMDF0F3HFiJbip+ymTvnmpF2oKHCCzAKloL9eXZjRUQwM/FFD6JZDuD5 PhIJEbMcg/mbw== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 82E053A0B73; Wed, 23 Aug 2023 08:54:59 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 10/14] simpletrace: move logic of process into internal function Date: Wed, 23 Aug 2023 10:54:25 +0200 Message-Id: <20230823085429.20519-11-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: YAo8A56RG4VI0VC7Qbue8xSWRzeTGqFo X-Proofpoint-GUID: YAo8A56RG4VI0VC7Qbue8xSWRzeTGqFo X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=791 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal To avoid duplicate code depending on input types and to better handle open/close of log with a context-manager, we move the logic of process into _process. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 0826aef283..6969fdd59a 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -201,13 +201,26 @@ def process(events, log, analyzer, read_header=True): # Treat as an already opened file-object events_list = read_events(events, events.name) - close_log = False if isinstance(log, str): - log = open(log, 'rb') - close_log = True + with open(log, 'rb') as log_fobj: + _process(events_list, log_fobj, analyzer, read_header) + else: + # Treat `log` as an already opened file-object. We will not close it, + # as we do not own it. + _process(events_list, log, analyzer, read_header) + +def _process(events, log_fobj, analyzer, read_header=True): + """Internal function for processing + + Args: + events (list): list of events already produced by tracetool.read_events + log_fobj (file): file-object to read log data from + analyzer (Analyzer): the Analyzer to interpret the event data + read_header (bool, optional): Whether to read header data from the log data. Defaults to True. + """ if read_header: - read_trace_header(log) + read_trace_header(log_fobj) def build_fn(analyzer, event): if isinstance(event, str): @@ -231,14 +244,11 @@ def build_fn(analyzer, event): with analyzer: fn_cache = {} - for event, event_id, timestamp_ns, record_pid, *rec_args in read_trace_records(events, log, read_header): + for event, event_id, timestamp_ns, record_pid, *rec_args in read_trace_records(events, log_fobj, read_header): if event_id not in fn_cache: fn_cache[event_id] = build_fn(analyzer, event) fn_cache[event_id](event, (event_id, timestamp_ns, record_pid, *rec_args)) - if close_log: - log.close() - def run(analyzer): """Execute an analyzer on a trace file given on the command-line. From patchwork Wed Aug 23 08:54:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824546 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=eBFZPj/+; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Rq0rzCz1yNm for ; Wed, 23 Aug 2023 18:55:51 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjds-0004U8-Kl; Wed, 23 Aug 2023 04:55:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdr-0004O4-Au for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:07 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYjdp-000051-AM for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780903; bh=IU4eVSgFHWDP+jifjfGdkBe+orZ14DeYHyLKEqxQd3o=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=eBFZPj/+QRjOlp0Jdbscy36waqUe5sjFsWU+MmSJ4SzJ9Cv7qPiAhB8x5Oa3I5IcE 2BqeRv5upE57h2tgiF7O1pQF6HUhxaj7ZPE9il1K7pDJoo4zfgOPlUobDQrbnIzww1 ueKxG36/7VPnxmClfCoRVpG7sz7wauQBYoC626fJFF/p1Fu515l6HNmCsa0BJtMC6J OKCjBx1Bop6UlesuLH8BC8uXxowVENPIjkV6atLkrxhJJegTmPoUNrMBiNeX/Tb5gr zgjef10nGQCHzItYSj0Qz194tTorN+WHL3AeE1Nt2Gou7CdPmmF4TRYil4mJBtfj2M B29UZYE3omtdw== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 044763A0B91; Wed, 23 Aug 2023 08:55:01 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 11/14] simpletrace: move event processing to Analyzer class Date: Wed, 23 Aug 2023 10:54:26 +0200 Message-Id: <20230823085429.20519-12-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: guMPptZvj-eM39Vgg2ucavEE4OC-PvId X-Proofpoint-GUID: guMPptZvj-eM39Vgg2ucavEE4OC-PvId X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=999 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Moved event processing to the Analyzer class to separate specific analyzer logic (like caching and function signatures) from the _process function. This allows for new types of Analyzer-based subclasses without changing the core code. Note, that the fn_cache is important for performance in cases where the analyzer is branching away from the catch-all a lot. The cache has no measurable performance penalty. Signed-off-by: Mads Ynddal Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 60 +++++++++++++++++++++++++----------------- 1 file changed, 36 insertions(+), 24 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 6969fdd59a..4136d00600 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -169,6 +169,35 @@ def catchall(self, event, rec): """Called if no specific method for processing a trace event has been found.""" pass + def _build_fn(self, event): + fn = getattr(self, event.name, None) + if fn is None: + # Return early to avoid costly call to inspect.getfullargspec + return self.catchall + + event_argcount = len(event.args) + fn_argcount = len(inspect.getfullargspec(fn)[0]) - 1 + if fn_argcount == event_argcount + 1: + # Include timestamp as first argument + return lambda _, rec: fn(*(rec[1:2] + rec[3:3 + event_argcount])) + elif fn_argcount == event_argcount + 2: + # Include timestamp and pid + return lambda _, rec: fn(*rec[1:3 + event_argcount]) + else: + # Just arguments, no timestamp or pid + return lambda _, rec: fn(*rec[3:3 + event_argcount]) + + def _process_event(self, rec_args, *, event, event_id, timestamp_ns, pid, **kwargs): + if not hasattr(self, '_fn_cache'): + # NOTE: Cannot depend on downstream subclasses to have + # super().__init__() because of legacy. + self._fn_cache = {} + + rec = (event_id, timestamp_ns, pid, *rec_args) + if event_id not in self._fn_cache: + self._fn_cache[event_id] = self._build_fn(event) + self._fn_cache[event_id](event, rec) + def end(self): """Called at the end of the trace.""" pass @@ -222,32 +251,15 @@ def _process(events, log_fobj, analyzer, read_header=True): if read_header: read_trace_header(log_fobj) - def build_fn(analyzer, event): - if isinstance(event, str): - return analyzer.catchall - - fn = getattr(analyzer, event.name, None) - if fn is None: - return analyzer.catchall - - event_argcount = len(event.args) - fn_argcount = len(inspect.getfullargspec(fn)[0]) - 1 - if fn_argcount == event_argcount + 1: - # Include timestamp as first argument - return lambda _, rec: fn(*(rec[1:2] + rec[3:3 + event_argcount])) - elif fn_argcount == event_argcount + 2: - # Include timestamp and pid - return lambda _, rec: fn(*rec[1:3 + event_argcount]) - else: - # Just arguments, no timestamp or pid - return lambda _, rec: fn(*rec[3:3 + event_argcount]) - with analyzer: - fn_cache = {} for event, event_id, timestamp_ns, record_pid, *rec_args in read_trace_records(events, log_fobj, read_header): - if event_id not in fn_cache: - fn_cache[event_id] = build_fn(analyzer, event) - fn_cache[event_id](event, (event_id, timestamp_ns, record_pid, *rec_args)) + analyzer._process_event( + rec_args, + event=event, + event_id=event_id, + timestamp_ns=timestamp_ns, + pid=record_pid, + ) def run(analyzer): """Execute an analyzer on a trace file given on the command-line. From patchwork Wed Aug 23 08:54:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824555 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=OI2sb9NR; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0T32LWQz1yNm for ; Wed, 23 Aug 2023 18:56:55 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjeM-0005Bp-3a; Wed, 23 Aug 2023 04:55:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYje4-0004oG-5A for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:23 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYje1-00006s-NQ for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780906; bh=A1z+bpLi9INQ8yEVKt1RtbSbm6CFFHNePYE+j7k4KDU=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=OI2sb9NRUDJtSBZrvcnrSUv5FNqkAzeeRi1/XDBk67JaH6MRQIyB/SblsebZOCJgf SLT8Wh74mwKidwXEXZXb/4Bx/T7uP1/vWDcUN4UtS404sqVzcxWja2mH/h2aH/z7Gf Dd8ZhgzBb4Xe6hvta3GtycJ9DzFbrx+YDCn+I5czoVyDBhaHtHV5CeVKL/OVsyOHP/ CYnMMoYApR3At8LO1vroiO0TeXPtcZl+SvPq7Nuc1dJhGwip2ryL5KHoMzCWW78TCn 8iL1/v7IfMBuPyXPnHY7n6b90ZZ6ZFTahXgKGiATex0GEtBsowqslTwhRq8xi4c4cp rgM6DZKuISPmg== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 6ED973A0B5F; Wed, 23 Aug 2023 08:55:04 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 12/14] simpletrace: added simplified Analyzer2 class Date: Wed, 23 Aug 2023 10:54:27 +0200 Message-Id: <20230823085429.20519-13-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: vqzlbXczv9uul3SwpoloWbDsp9FVGnml X-Proofpoint-GUID: vqzlbXczv9uul3SwpoloWbDsp9FVGnml X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=885 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal By moving the dynamic argument construction to keyword-arguments, we can remove all of the specialized handling, and streamline it. If a tracing method wants to access these, they can define the kwargs, or ignore it be placing `**kwargs` at the end of the function's arguments list. Added deprecation warning to Analyzer class to make users aware of the Analyzer2 class. No removal date is planned. Signed-off-by: Mads Ynddal --- scripts/simpletrace-benchmark.zip | Bin 0 -> 4809 bytes scripts/simpletrace.py | 78 +++++++++++++++++++++++++++++- 2 files changed, 76 insertions(+), 2 deletions(-) create mode 100644 scripts/simpletrace-benchmark.zip diff --git a/scripts/simpletrace-benchmark.zip b/scripts/simpletrace-benchmark.zip new file mode 100644 index 0000000000000000000000000000000000000000..1d696a8d5c49d4725c938af8bf25a59090192986 GIT binary patch literal 4809 zcmb7|^-~m#x5jtLrD4ehq*FRXq`O;??nb%c@2}tInKREhbLN~s;G?6CjzIOwzjqB z^$ax7Cjek4BSY+t{;7Y{F8~h!#JB(g0RQ{|01nLF`z$IYKjr4yFu~&OG`dpv7e;He znoe1lQxC!H%PeWl6BVnB{0xqmXh!%lpLBOyEU#Mm1ClW8JT~{VxYJb8CM6{$AFlmu zdnfrUU`%24s6j5ct15zP#`!eJS4@mmdb5(l-_iJ2O5`Ssnh(=8TReo*o0`A2sEbEk z4y{gjWjCG#nvsNBDVpcq{^lq9{qZ3N5B?LF6geLN%GIA^dNZzc1R;YGX<6BV;Hb*v#H;roy4XLg&L1yqSv8O|2OHKdoeU(#f) z+fNkiI(^ADca?ZrRVLGST3!|A6KSN#jUdH?J@C)#_9Uvbdkn7m2|xnWH&ThVE9_~g zpnH-0lf!p#sH0uhrgP_qPH138f`^h4?3gI6>I&`BMINt_>B$z=(?TE0%I<g%Ra$uv8 z6=PbEGNv(Hf|td|qb~ZHb<+ZAZcN_HO~_P0hAd68D8s1g1j$H5?MEjltRq|>+y*U4k^&^o_I{2Euf9>F} zq2VY^=m@y6+;s}ii5B!I{&fmlQ@N0#L@p_}_;mRXHh0fM)|~caA80naEAKal-4{W- zK5KCCaK{*u8d7e(adQ2`GhW9{+r?RdAm*$0c(bYCpb`Pdrx7|LndKIVu}2661?O~z zbKb1$dFzpGaFx<>S2rb}v`AoIDJhkCvc@Tg4!^#RR*DQ_UzIndt36)ncVPng*ym6| zhFQ=DlEA-on#=Y2;>aDCi=Wql+4>0`>`ba&pM{U2DOb&0PU~xmMil!(pV~zp6X|mE z23yqRo&_@Qh})4SjTwJnCeqW)4$;5czzy^EKII|h-WHD5bapSoH=e>UICf~Xw5gl$ zVcB$0!fm%+UH|H;B$$=D#U*eNUXzsv?*Dv78vAJh&kFL*#;wC);_HIQtgtOA;p`gn zr;n51ZAEzvn`6-d$E4)#9@%f+A&o5;{fc2&{lvFZe2{TcO#q5b*$t+o7|JX>x=#*? zalnv|Qhjf}P##10S4^{g;^SUg=JL6vFmJ|ro~m#tQCC(nkCD<2NP1ENWESG2brS0% zA_7+7RMjACZ8ClN#$o>X^x2=U#PEP3ZVYJE#xi0{KFA7Ix1@&s_C9` z&Rb)I>axD%Ai+}uQTln`7fghU^Tb|ltxBwz5+-u$Sm`=|AQk&K&ZZ9SS$zq((J3MX z1{rnfRCYG4!zTLgj694Xn`?2%0U6cq1KAk~MB(y{CC#PPUo>53t z>BSYi-1R~0O3n(Y0!;hnAL53;18VdEMVF&Q;Np-gZag#QjJ!ITu9Qmdq8-@sx9OBY za2^D|#Z(AD-GtzkDF?-S=9x&;(n1smuQ*~j1Hr|sdDM`gB)KuAv+%W6HCEJA3D`%M%20vbTO@4FNaGO2(+4)h6?1&WlkT6p zTTpVe74&@+)2#iT8&(SUYUr0ZR=22)Q`H@|vD7>uAc(>Z#{)3@{mE;6e1Np*M#X>} zr71XFf-gMnmLd#WD4p6&+)VH&2&UpJ&)!hrAv_j>g4{RlQ)tXnC%aVR+bpYf6U&hA z;^CGM?JAn!zNZqY9}1-?>3OY)*JT<;UmaiO9Qf`!-FMO@TH}sza^VNt=LZ)s0Pt-y zdtSW89R_?K@n0JBuIzqQ2~S0PR%#F_i@_z+X7M%DWY&mUF_OG?#*1&`jkSR3EWvH2 zpwWsFDRjm=JTRhZ;WQvpc=D1eSW~#nH8337Wh(DF>*-*EQoj&6KvfuCfW7^0e^d{nO0#FVHE4BBk$>UDFfR2bFzR8J%^H1n3bR6#F&Fan?3{tCl_agJ-v$%1dI^nw=Sp zo!?EDKA6YnGi#*wdj-WvnEX@&?!r&0#6KnOS>+!KM)(q0VHM9%EzT>68`-F3C7Sv) zTx&i5#acCs<4&w6u}@y4<&=y&m&ii)Lb)OfsZrGcR@7M>(5I3~CdYOb&Myjnr%gi4 zEjCs3&Yxf_mX$I&YD^{S()uY61t_$6*WN1V=jK+!|J!ILhW86D0~|jxb9v09_G3kr zv*HAc6SrRV8uu2EZ(Mw*%JsRi;)@2v-n|;R_@ULyu%zmG!L!&MhdcneJ8Wg{R4vlx z+^4Sf_E3y#z#n$z6}Y?hc#cLP@>lQ3NkSVBhM4a&v5Rjm?G=`#wnE0c+} zJ%XN{ki8)FOC9S@yYt_bO3PAMDE58PwSE}*Wg_F6jK#rfhM#nLh8VJuCvduw|18En zFUV_}RNL`Tx54+TJtX6zYmyx-ZAFoaql5+3<>xB+v1RFtxhAZXrkdan4%#iu)bDZf0Z8TREp)?D2p zo35a9ch+z;1#io=*F2hh8Z4ALk$iJqqE=$Ghv%o_!DjXhqIxFp0i~GhfBe8+-j$8W z7#0(`^&3#KWl4dW2hUaZXI=@@yr1j86Sn?iE?loQt^fd$ks7$3HkIC*m@8BZrju2G zP>VdN6Ze+n#}2{+YTO^u#9$Y=MOcu}6{Y(2v>U<4A$;PA>!zm52&rP&Rz-8^SB1gz zpim)JM^x~W%`=NM&FnUo?HFIr;vvBX4D=7ZKDy-T18~~OQNg0#G+AfHKUBH3X1RR4 z?p8tf*Fh)LQq!z08He*{jRTpySl@Am@Qx$cD;0Q()NdubkNygB{eCR;CBu`#Wi@7H zbvS3oo3C0&Tyz;#H(EG~RrtHj7DhjwkXmN{_VM7@MG{mol-n{PMMdi*c|Q8ZluwS8 z{VD z=bU@Tyf|5y8Gp^7zdI>vPfk>Uw)004Z>V6pEB~;Ne40kfuTbods|=B2z~>%+maaBe zpg}#vzL;)H#$3FMXPbmy1LM_Lb-i||iqCXx@ft;xy3=8KW$I3~{(*u<=<9B269;mP$kt>4j6HK+vob6KV@3|20-E_aIB>y1K<4+J)eDaP0=FLd7w$Jy3r_Zg^BwuN{)^mM?Oib7lIh>@s$r;uTD!e zPSb77e?^h|DfWaa$p)i8j$04OyAv$bJ;{?zQhfxO*YRqBT2pw^*QTi5zp0j=JhS`z z6s10z(cZLT8{Er)O~}Z>0ezo1Hb2xc++@(_b;w8>5vFV{)c1#>z#?i^zf!hLv)TU9 zoBdsIx*e=IHuS?QEoGrF=4&go>vO1u!i4vcQ}Ev?`@40xV`v5-#iG|5_uGQ&5d6;} zyZo>Qf@Z(^yuNOzXNbEOTaQus3~2*mLl>>)A&M&<6%Bt5$`TzsjrOWY*u0p%MSfBx`fTyn^Li{HX)pQL?8<~zvTbC#1(7M-W+ zJGrj_@MU4j?O{{-h3#0+C41!UZeu$&G%q`7@gT-l)}^DFL*8HOxe5-s7A5sAcR-7E z{DbR2G&e5?nZThY2S6MNWhijTdPcPdlnA7oJOCzGjKQ;z47A?6ja^QL(R-50p zij>|w#NXMrispn=s+}~PM?x@XLp$5$7Q-5q0qYY0FZebo_g zj~|{3JhlkT0Ukdo1MU;TH~&I_N89JhOsjbD;iJY`5yVFG!=-)^Cq;E(OKCB3Ydx+l zz!NXsW-Bj`3Ng_`%{NiVf${~JrWK8B&nD?zK(hRJV0fiDpcy-mGJ#I%4f^DODpYQd zj;@;Ch`!-Wr1h|7$%3JUFIQ|Tw0l3wnWp+ZV15;cNbb06YYDK9zO(3g&)FMdz6%gO zqAZA}wqw^g(lo^W`@rHVe+lerAY6($P(c4cgvjqYf1h{n)h!v`j=csnS|KII7yRfR zD{UwJSLTU-)A2Uto09B!;w?$!1?D3`0Ffn8ci8=cF|hH&L&+`&rc1Zd26k@;G>zzH zqY*ttUDP}=b43~ze>ZS@#LTdT?fI(QQ<^TFvE$5jVlKQNr=}t{2}vDx+6Cu?qVjdK zxT{?vhK=GRk_7@)Kg*wtb$Wm3*kyTAp${uNPEH7+7y5VvokqOWLR*_>FP%I<7KJ?$ zH3L$C-dV_{nHu7{VFm+~0j=gEG>Vgt^NCUwEqJG^8o_Ty4z!gYeqxGq z%+j;Z^vO2yiedKT-zw;Dq`ux@*Eudq;gExG=R5~f87&U3B&RsLfgZ`XP{Cri^g8O; zphe27{W}r>;9q#8qYgwPL;ruU^q;$b!P5WfAmZQdzr##A>X=ynUIG8H**|Fr{&)Hx DD^M;a literal 0 HcmV?d00001 diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 4136d00600..fdeeed7cd6 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -12,10 +12,11 @@ import sys import struct import inspect +import warnings from tracetool import read_events, Event from tracetool.backend.simple import is_string -__all__ = ['Analyzer', 'process', 'run'] +__all__ = ['Analyzer', 'Analyzer2', 'process', 'run'] # This is the binary format that the QEMU "simple" trace backend # emits. There is no specification documentation because the format is @@ -130,7 +131,9 @@ def read_trace_records(events, fobj, read_header): yield (event_mapping[event_name], event_name, timestamp_ns, pid) + tuple(args) class Analyzer: - """A trace file analyzer which processes trace records. + """[Deprecated. Refer to Analyzer2 instead.] + + A trace file analyzer which processes trace records. An analyzer can be passed to run() or process(). The begin() method is invoked, then each trace record is processed, and finally the end() method @@ -188,6 +191,11 @@ def _build_fn(self, event): return lambda _, rec: fn(*rec[3:3 + event_argcount]) def _process_event(self, rec_args, *, event, event_id, timestamp_ns, pid, **kwargs): + warnings.warn( + "Use of deprecated Analyzer class. Refer to Analyzer2 instead.", + DeprecationWarning, + ) + if not hasattr(self, '_fn_cache'): # NOTE: Cannot depend on downstream subclasses to have # super().__init__() because of legacy. @@ -211,6 +219,56 @@ def __exit__(self, exc_type, exc_val, exc_tb): self.end() return False +class Analyzer2(Analyzer): + """A trace file analyzer which processes trace records. + + An analyzer can be passed to run() or process(). The begin() method is + invoked, then each trace record is processed, and finally the end() method + is invoked. When Analyzer is used as a context-manager (using the `with` + statement), begin() and end() are called automatically. + + If a method matching a trace event name exists, it is invoked to process + that trace record. Otherwise the catchall() method is invoked. + + The methods are called with a set of keyword-arguments. These can be ignored + using `**kwargs` or defined like any keyword-argument. + + The following keyword-arguments are available, but make sure to have an + **kwargs to allow for unmatched arguments in the future: + event: Event object of current trace + event_id: The id of the event in the current trace file + timestamp_ns: The timestamp in nanoseconds of the trace + pid: The process id recorded for the given trace + + Example: + The following method handles the runstate_set(int new_state) trace event:: + + def runstate_set(self, new_state, **kwargs): + ... + + The method can also explicitly take a timestamp keyword-argument with the + trace event arguments:: + + def runstate_set(self, new_state, *, timestamp_ns, **kwargs): + ... + + Timestamps have the uint64_t type and are in nanoseconds. + + The pid can be included in addition to the timestamp and is useful when + dealing with traces from multiple processes: + + def runstate_set(self, new_state, *, timestamp_ns, pid, **kwargs): + ... + """ + + def catchall(self, *rec_args, event, timestamp_ns, pid, event_id, **kwargs): + """Called if no specific method for processing a trace event has been found.""" + pass + + def _process_event(self, rec_args, *, event, **kwargs): + fn = getattr(self, event.name, self.catchall) + fn(*rec_args, event=event, **kwargs) + def process(events, log, analyzer, read_header=True): """Invoke an analyzer on each event in a log. Args: @@ -300,6 +358,22 @@ def catchall(self, event, rec): i += 1 print(' '.join(fields)) + class Formatter2(Analyzer2): + def __init__(self): + self.last_timestamp_ns = None + + def catchall(self, *rec_args, event, timestamp_ns, pid, event_id): + if self.last_timestamp_ns is None: + self.last_timestamp_ns = timestamp_ns + delta_ns = timestamp_ns - self.last_timestamp_ns + self.last_timestamp_ns = timestamp_ns + + fields = [ + f'{name}={r}' if is_string(type) else f'{name}=0x{r:x}' + for r, (type, name) in zip(rec_args, event.args) + ] + print(f'{event.name} {delta_ns / 1000:0.3f} {pid=} ' + ' '.join(fields)) + try: run(Formatter()) except SimpleException as e: From patchwork Wed Aug 23 08:54:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824554 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=KDTEWFLg; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0Sx2xXSz1yNm for ; Wed, 23 Aug 2023 18:56:49 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjeM-0005Eo-De; Wed, 23 Aug 2023 04:55:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYje6-0004t4-Sd for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:24 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYje3-0000CH-Te for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780908; bh=GO5SwThOEmujCtZHTQiwWv3gNgM24dsJJCUk62NRIOE=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=KDTEWFLgTc4L9unBjPIJw6qRjzHUdz0/SULCZIzwUzJBkyaNryy8V5nXDzWZX4blZ uKjACb13iMJahRzlzklMbUNZkPJJrJ/J/HBs9uX9LUawaIbQaH+fZRzt5KiAcAN9sK RLxo7ViNbD/dfTwWbflgZYLMUu/P2pb51XSzyeRF1RI1ctMJLltIpYOzwZsnPWgEVA f8FCIO1POpYqSjQ3xXrdMjRwsxHgLa9lXavwM0wXzkSKt0KshGI9pi/Pkn77w8YPqU udx8HhexdqwNXZn38IkADk3OuRGDIIpb/MdlLN44++VP21rScsIyAtqxXBm92ZbuHp yUaSSD/7fan2g== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 01D923A0A27; Wed, 23 Aug 2023 08:55:06 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 13/14] MAINTAINERS: add maintainer of simpletrace.py Date: Wed, 23 Aug 2023 10:54:28 +0200 Message-Id: <20230823085429.20519-14-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: Kjx1VVzRcwp7ClD0nIZKR-b_OnO2CUra X-Proofpoint-GUID: Kjx1VVzRcwp7ClD0nIZKR-b_OnO2CUra X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=875 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal In my work to refactor simpletrace.py, I noticed that there's no maintainer of it, and has the status of "odd fixes". I'm using it from time to time, so I'd like to maintain the script. I've added myself as reviewer under "Tracing" to be informed of changes that might affect simpletrace.py. Signed-off-by: Mads Ynddal Reviewed-by: Philippe Mathieu-Daudé --- MAINTAINERS | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 6111b6b4d9..2ffb608dec 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3162,6 +3162,7 @@ F: stubs/ Tracing M: Stefan Hajnoczi +R: Mads Ynddal S: Maintained F: trace/ F: trace-events @@ -3174,6 +3175,11 @@ F: docs/tools/qemu-trace-stap.rst F: docs/devel/tracing.rst T: git https://github.com/stefanha/qemu.git tracing +Simpletrace +M: Mads Ynddal +S: Maintained +F: scripts/simpletrace.py + TPM M: Stefan Berger S: Maintained From patchwork Wed Aug 23 08:54:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1824556 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=RWtipdC1; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RW0T54B2Dz1yNm for ; Wed, 23 Aug 2023 18:56:57 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYjeM-0005LC-W1; Wed, 23 Aug 2023 04:55:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYje8-0004t7-0A for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:26 -0400 Received: from pv50p00im-tydg10021701.me.com ([17.58.6.54]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYje6-0000DF-IR for qemu-devel@nongnu.org; Wed, 23 Aug 2023 04:55:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1692780911; bh=0TM9Xz67OYngDEuTQ62kPsInUQpx3kaf2lzF824GJ1Y=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=RWtipdC1lk21Q/Zh+3FDVEe3H1V+TXayzAccMFEJcodylzygVFi5Re02veD9zBGOt V935lQv2rjn4/hGqKAi/E/CeZTbx94m5duZx8kOeM5ZyQ8qMdnWdAkxLYgSrIKARRq AwTrm5d/R62+cAMqDWql0BrefnbJmW86/mbH7GVCH78XcW2IibZeTGPsJ3VXwJU0jb N8uWeZY6NB6dv/tR6xsqcfskbNF7eEcHqJggDDwQaHgLzHb9tVXkc24VlYvGpluG+i bI2aNCcb+Sbjd1EXoJ8Xscqs8mX7669Cx4LWDoeGTdsk56bX3ITzhZebnazWeY3lxF 3hRaIg43KPnRA== Received: from localhost.localdomain (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-tydg10021701.me.com (Postfix) with ESMTPSA id 790D33A0BB3; Wed, 23 Aug 2023 08:55:09 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: John Snow , Mads Ynddal , Stefan Hajnoczi , Cleber Rosa , Mads Ynddal Subject: [PATCH v4 14/14] scripts/analyse-locks-simpletrace.py: changed iteritems() to items() Date: Wed, 23 Aug 2023 10:54:29 +0200 Message-Id: <20230823085429.20519-15-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230823085429.20519-1-mads@ynddal.dk> References: <20230823085429.20519-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: r3A3Wdx7tdfGdOFPyKmoQsz50lmSAhq_ X-Proofpoint-GUID: r3A3Wdx7tdfGdOFPyKmoQsz50lmSAhq_ X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E517=2C18=2E0?= =?utf-8?q?=2E883=2C17=2E11=2E64=2E514=2E0000000_definitions=3D2022-06-21=5F?= =?utf-8?q?08=3A2022-06-21=5F01=2C2022-06-21=5F08=2C2022-02-23=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxlogscore=562 clxscore=1030 bulkscore=0 adultscore=0 phishscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2308230080 Received-SPF: pass client-ip=17.58.6.54; envelope-from=mads@ynddal.dk; helo=pv50p00im-tydg10021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Python 3 removed `dict.iteritems()` in favor of `dict.items()`. This means the script curerntly doesn't work on Python 3. Signed-off-by: Mads Ynddal Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Hajnoczi --- scripts/analyse-locks-simpletrace.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/analyse-locks-simpletrace.py b/scripts/analyse-locks-simpletrace.py index 63c11f4fce..d650dd7140 100755 --- a/scripts/analyse-locks-simpletrace.py +++ b/scripts/analyse-locks-simpletrace.py @@ -75,7 +75,7 @@ def get_args(): (analyser.locks, analyser.locked, analyser.unlocks)) # Now dump the individual lock stats - for key, val in sorted(analyser.mutex_records.iteritems(), + for key, val in sorted(analyser.mutex_records.items(), key=lambda k_v: k_v[1]["locks"]): print ("Lock: %#x locks: %d, locked: %d, unlocked: %d" % (key, val["locks"], val["locked"], val["unlocked"]))