From patchwork Thu Jun 22 18:17:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siddhesh Poyarekar X-Patchwork-Id: 779672 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wtqZC3dM6z9t0j for ; Fri, 23 Jun 2017 04:17:55 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="lDZK1BoM"; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:in-reply-to :references; q=dns; s=default; b=iLaGwxiwyOraHL99sDW0K85HiJkaDQo 4+VP3PqQa6EsrtJfvgaaWc3jtxJXwRkDBAJmDRyzdUSoA/MpfhZsHHAFwSQemZvC wqbcMoV8Bpmh7AL9UMumNH5UTX2gJQl9+9TIS3VhOeQQX2RFZjTz7gZ2DmRYVIwQ T5IzAcv65qsk= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id:in-reply-to :references; s=default; bh=yTIVIB1ikXpDThsJBFhzn9bEeHQ=; b=lDZK1 BoMHWMyfldRHtRypZV4XYkRoqbNKnJGwq3MU3xlAZ/VAF6Enk6XgGXgN0R2P90IM r93JysRKSkfLFXsneGE2UYZqIdoWZznp7vauq4JIYj0at+pbiQGCLDuaP2tiWlI4 u3XQInr/jbjb+HA/8gq0gKjPalyemw83TOSejU= Received: (qmail 13856 invoked by alias); 22 Jun 2017 18:17:34 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 13760 invoked by uid 89); 22 Jun 2017 18:17:33 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_NEUTRAL autolearn=ham version=3.3.2 spammy= X-HELO: homiemail-a50.g.dreamhost.com From: Siddhesh Poyarekar To: libc-alpha@sourceware.org Subject: [COMMITTED] benchtests: New script to parse memcpy results Date: Thu, 22 Jun 2017 23:47:14 +0530 Message-Id: <1498155434-22182-3-git-send-email-siddhesh@sourceware.org> In-Reply-To: <1498155434-22182-1-git-send-email-siddhesh@sourceware.org> References: <1498155434-22182-1-git-send-email-siddhesh@sourceware.org> Read the memcpy results in json and print out the results in tabular form, in addition to generating a graph of the results to compare all of the implementations. The format of the output is extensible enough to allow this kind of analysis to be done on other string functions as well. * benchtests/scripts/benchout_strings.schema.json: New file. * benchtests/scripts/compare_strings.py: New file. --- benchtests/scripts/benchout_strings.schema.json | 44 ++++++++ benchtests/scripts/compare_strings.py | 129 ++++++++++++++++++++++++ 2 files changed, 173 insertions(+) create mode 100644 benchtests/scripts/benchout_strings.schema.json create mode 100755 benchtests/scripts/compare_strings.py diff --git a/benchtests/scripts/benchout_strings.schema.json b/benchtests/scripts/benchout_strings.schema.json new file mode 100644 index 0000000..7c9a671 --- /dev/null +++ b/benchtests/scripts/benchout_strings.schema.json @@ -0,0 +1,44 @@ +{ + "title": "string benchmark", + "type": "object", + "properties": { + "timing_type": { + "type": "string" + }, + "functions": { + "title": "Associative array of functions", + "type": "object", + "patternProperties": { + "^[_a-zA-Z][_a-zA-Z0-9]+$": { + "title": "Function names", + "type": "object", + "properties": { + "bench-variant": {"type": "string"}, + "ifuncs": { + "type": "array", + "items": {"type": "string"} + }, + "results": { + "type": "array", + "items": { + "type": "object", + "properties": { + "timings": { + "type": "array", + "items": {"type": "number"} + } + }, + "additionalProperties": {"type": "number"}, + "minProperties": 2 + } + } + }, + "additionalProperties": false + } + }, + "minProperties": 1 + } + }, + "required": ["timing_type", "functions"], + "additionalProperties": false +} diff --git a/benchtests/scripts/compare_strings.py b/benchtests/scripts/compare_strings.py new file mode 100755 index 0000000..9d73ec4 --- /dev/null +++ b/benchtests/scripts/compare_strings.py @@ -0,0 +1,129 @@ +#!/usr/bin/python +# Copyright (C) 2017 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . +"""Compare results of string functions + +Given a string benchmark result file, print a table with comparisons with a +baseline. The baseline is the first function, which typically is the builtin +function. +""" +import sys +import os +import json +import pylab + +try: + import jsonschema as validator +except ImportError: + print('Could not find jsonschema module.') + raise + + +def parse_file(filename, schema_filename): + with open(schema_filename, 'r') as schemafile: + schema = json.load(schemafile) + with open(filename, 'r') as benchfile: + bench = json.load(benchfile) + validator.validate(bench, schema) + return bench + + +def draw_graph(f, v, ifuncs, results): + """Plot graphs for functions + + Plot line graphs for each of the ifuncs + + Args: + f: Function name + v: Benchmark variant for the function. + ifuncs: List of ifunc names + results: Dictionary of results for each test criterion + """ + xkeys = results.keys() + + pylab.clf() + fig = pylab.figure(frameon=False) + fig.set_size_inches(32, 18) + pylab.ylabel('Performance improvement from base') + X = range(len(xkeys)) + pylab.xticks(X, xkeys) + + i = 0 + + while i < len(ifuncs): + Y = [results[k][i] for k in xkeys] + lines = pylab.plot(X, Y, label=':'+ifuncs[i]) + i = i + 1 + + pylab.legend() + pylab.grid() + pylab.savefig('%s-%s.png' % (f, v), bbox_inches='tight') + + +def process_results(results, attrs): + """ Process results and print them + + Args: + results: JSON dictionary of results + attrs: Attributes that form the test criteria + """ + + for f in results['functions'].keys(): + print('Function: %s' % f) + print('\t'.join(results['functions'][f]['ifuncs'])) + v = results['functions'][f]['bench-variant'] + print('Variant: %s' % v) + print("=" * 80) + graph_res = {} + for res in results['functions'][f]['results']: + attr_list = ['%s=%s' % (a, res[a]) for a in attrs] + first = True + key = ','.join(attr_list) + sys.stdout.write('%s: \t' % key) + graph_res[key] = res['timings'] + for t in res['timings']: + sys.stdout.write ('%.2f' % t) + if first: + first = False + else: + diff = (res['timings'][0] - t) * 100 / res['timings'][0] + + sys.stdout.write (' (%.2f%%)' % diff) + sys.stdout.write('\t') + print('') + draw_graph(f, v, results['functions'][f]['ifuncs'], graph_res) + + +def main(args): + """Program Entry Point + + Take a string benchmark output file and compare timings. + """ + if len(args) < 3: + print('Usage: %s attr1 [attr2 ...]' % sys.argv[0]) + sys.exit(os.EX_USAGE) + + filename = args[0] + schema_filename = args[1] + attrs = args[2:] + + results = parse_file(filename, schema_filename) + process_results(results, attrs) + + +if __name__ == '__main__': + main(sys.argv[1:])