From patchwork Thu Jul 1 23:40:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugene Rozenfeld X-Patchwork-Id: 1499809 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=TgF1oLAL; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GGF8J4xQ7z9sjD for ; Fri, 2 Jul 2021 09:40:47 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3999239724BA for ; Thu, 1 Jul 2021 23:40:44 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3999239724BA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1625182844; bh=uT/3OKAmx9YzGzNGi/EbSogGJ6Axw1SLhbkKBA+KiTs=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=TgF1oLALWAA5Iy2R55w452dJw69Q5PN9y/0uFDibO1kXtDDc/7Kek7BJgev+C/i1M ap5bIQJ8BKbl+PzYstSk0P4fCsybVEb9KqQHcnAPgdF0Cn28gCt1/3RYhh5Zzp45/d 2Ve7rAhOHmt4BURdY3cCpYHrxau6g3Nfq3zbeOd4= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2118.outbound.protection.outlook.com [40.107.244.118]) by sourceware.org (Postfix) with ESMTPS id 1253C3857C6D for ; Thu, 1 Jul 2021 23:40:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 1253C3857C6D ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WRmoAWvNy+6w/g4P8gjirmyu1r/lJWU8IeT6FHOjTTa+eSHNxMRX8lcXOv+AOUPccS+0e1apopQ9n14IYuzpKCfISjLtDYN50OFBpBuOGbgsiXM24tphCm/VOjXvRA0tOerUoxhBo09RwzRqyxoT/N984LVsvHP7CoLHGO/2I5v9wQxlcJTbw38VojGRtxuco3T6k3yX4lmxDzkZ4RXO4cdPRhgPm7Zx9ISlRBW1ymOUGOZaOo9iSjKnSrtmt6tAVcmOh1d0mFb8tg07wpwEVwzhxqOr97LnxLWcB3nBYvYLJHOG0XpgFyah26hHKpPNDNtmf96RYQ2+kDKw9arFTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uT/3OKAmx9YzGzNGi/EbSogGJ6Axw1SLhbkKBA+KiTs=; b=N8OJn+CrjSW/G4cfdsLDJCf4zPQ6sVEi2HLQ3cRe+VNwgoMphLzgqc0q48KciOt/blwv56VjSJ3vocE7w1W2ZH2vbkh1JWFl0gopyQZcyw+OdZwLDvyQOVWyHlnpzeDiGSTQ8fRXdAcojr/3lS9aZ9hYZkKUqEkbnjZ6DWkwo0cq47eayor7j1vdqE5wE4lfZnnhAq1rLFHRgOnzWavU9IsYYVIS4ur4kP5bcEqEbDrp9Jw5MMLv5AZ8MsKQ/8mFbb0Xl4B6xoyoUKStNchhoRHFxllnucEl77yN/pXzv23y8ZDjVzAFelKJccbePBl0ulBzcNKpsPQKwwqjl/iUMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none Received: from MWHPR21MB0798.namprd21.prod.outlook.com (2603:10b6:300:128::18) by MWHPR21MB0512.namprd21.prod.outlook.com (2603:10b6:300:df::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.1; Thu, 1 Jul 2021 23:40:20 +0000 Received: from MWHPR21MB0798.namprd21.prod.outlook.com ([fe80::1d02:9a92:e9:c946]) by MWHPR21MB0798.namprd21.prod.outlook.com ([fe80::1d02:9a92:e9:c946%14]) with mapi id 15.20.4308.012; Thu, 1 Jul 2021 23:40:20 +0000 To: "gcc-patches@gcc.gnu.org" , Andi Kleen Subject: [PATCH] Update gen_autofdo_event.py and gcc-auto-profile Thread-Topic: [PATCH] Update gen_autofdo_event.py and gcc-auto-profile Thread-Index: Addu0nNvveGbpSs/RmKDFI4/j+1QOg== Date: Thu, 1 Jul 2021 23:40:20 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=ac75ac69-b8dd-45d2-89ba-7e5ddbb3c6c3; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-07-01T23:37:09Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 18d03819-4c42-4d84-4be0-08d93ce996b8 x-ms-traffictypediagnostic: MWHPR21MB0512: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1060; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: u92ohORkuTICQhn5gbv4b7VDV/3Gf2RU+bV8FkjdxwB3u2qk7Tvozyk3U5g5JfhGEQVFJ4A95gT5D0DSIjL4QPo4hPopOnkbf73avLVmvS+HI/qEbi5trgwZ4VYSMe9icaQ8hoIKUdgYgt0UdGc5ViKXulz1eL5d1Kw40nZw1sgKcFIOWkjAk93EEwkKRYMbuVpLWWMkCrxGXIgHh2F/BzR0NLv4WdSklN5nLdD+Nv/dl1ikXwgAIFOcNCsCjL2KaTKHGOm5m/bsa9j2akeGXq1zxMe+yyg/uyt/oail7ZZfiz8SgRTDtFlPzyOs4DwJ5ByDykcq2726ZlLts4lMIIFc6noJ6aCHYyqSNqGJuTPlyp0NSmEpINgg7uOmLq1UHsZSJsbck9PwS+QyyvcsCrPLJlVTffkJKU+wcbd4H+BHs7q/vzHUzZ+loWHSm7EuFR2ns1PsXMXIFKK7DKRdzrjB8nAumnQ/DYpT9oD+F4k3nvmB5D+ilouoPfJpfERQgBnUkpFcBKK5cK/aqJ9WkuQLNpHGSJAPJWkPtqS/8RaSv/8DMD0FWzb4HpoEgv4iuCFuHJGL/EzomJgTBYk+KG4457rWZbUaS8u40Xz8uuySkDUuarTILS9WWOo86PpmeSoZS1Do+G+IZW2dibShqBxOizRV2KTRAcdUKWbnk/SexuqiqnlemQQ4Q2nu808YhUePC4M5M8W+J9fVlyZN6bByxJGu225zaqzBr7oYr8TMVODUJSLgUYz88yfCmN0555NxK7XT7W5QkN3SWVEqcA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR21MB0798.namprd21.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(478600001)(66946007)(76116006)(66476007)(66556008)(66446008)(10290500003)(316002)(5660300002)(52536014)(71200400001)(110136005)(2906002)(33656002)(64756008)(83380400001)(8990500004)(82960400001)(38100700002)(122000001)(82950400001)(86362001)(8936002)(8676002)(186003)(15650500001)(966005)(9686003)(55016002)(6506007)(7696005); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: fxfpT3HjsM7QWzqMumVb1HHK6kKtzZ2InyNaXEuZnqe4TEMwjYJy28/FIQw0/xIcte7YjJRtWenp6csbIsG7YltHxy0Nv+9ocHcKri2mjywgSq1h3J78Awk+Rax3vO64qLULqn/tsQycc0AbQYixCLMImiSNIQUUAyAYTr+ZdGdyLMGjbUYjhH8WC/YKh85kYBDiKM/4Kg5UwmgPYsR3fRQQWaSuTnTzuQAdWlV7lUHpCWEbwlWydZrbZBRG92qFCmscm5D00Ix7vqPmWRKeqc9AA+IiKY8gBB/8MQRlzceG1vQ4rJ3ev0hTx2bmwcZxFjKGhPzReE+XHeVoMUzCpNN1z1BKji/sNS7hU0Uxd9YUijbofbEe+npDnfja1KI/NEyPQax72KS1htBXSJ+zNr8ikfSGglpUVr8dtW2dnlGkd4D/mXNPajT0lX6iX8fEf3TgbrJVzHj0m8/bPWiCnnpKOfbYCyXncMrWU+nssywuuF7ii/Pzbl+Y5QS9/Bw5gxt/uN7WfQdTCVQ4p4n51L4CIC48y3sjtvjlTrM+TvKvvPCQdcFWMfBs/YL7fZ7bTDAiSKJTPp0bbdtJcB2WRlpeB5EeWyTNgipkHyYLj3v50nkbBBHv9HBbp1R0YIggkMMlMT3FjKUjKhLS+JDbInCiFNQZJlrY7naHi8a7GzHpBsGhzwadjIYZPPp99bsCVxBOY6wA9iXq0wTSDs+ov/NT8xGk9Ff2A54a15eWOpYWwkghp9qOECnaCmLKh/QhML0iRSZcrCVqem5thfKy/ptQSuh2KIi3/je3i7oJ1Qmnee5rp4JMU5CZsHs8muYud9bEPxewgo0aTnECqh/G9hhtZwfEBzd143vo+4arlNWUBjIWUZB3Gx4zoRD1RGkZU2yIMU6MeHGdSnwUgnvRkQSPESORBiLarYAb80REvPycuxb40YuNwmrzpjgoIyK24zzBAOFC8F80obr6ub8tEo14yqM3xbK7wDonbbZ/f8UuM2Mh39DydsVJB5b993Hc2cDuLVEPo+SOEkyEPZvXb3Z2/yDMify1EgCRNXuPOdKQgUKgPw4GYIxkTpaCR61ax5Nm0V96s56OvrkWNDGObaG36ua+tMeZHofgcO2vwpD2/Rm5/3qBOm1Bm2HTxpzFDPF9p71Ogtz8luk3edsTRc7avu4q7n1LXiT/erx1rmmJXjgTNLDmjyzjiRKrQcp5Ni49UB5l3FiVpFhslpn6dZ1vcCic3iR/nao3dnsnm6FfVJm8qkPCNsfaRRpQLOwjvpda06lSfSr8+UBCOOMru+N3DzL0j3D6R99tJGuVgi9u3ayaI3FtOxmX6e26hewDCmZDitz5hFqrv3XMkpI0apymOEG3YyT5mrJ8HZ6JjxI= x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR21MB0798.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 18d03819-4c42-4d84-4be0-08d93ce996b8 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2021 23:40:20.2343 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Qhoz3WDVEFllN8lrs9ws9qGkZyzZ7XVC9dTdANOZz2WJzm0S5o749n88WYccVHCOhhh0aa2E1A/5WXJtIylGsQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR21MB0512 X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_SPF_HELO, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Eugene Rozenfeld via Gcc-patches From: Eugene Rozenfeld Reply-To: Eugene Rozenfeld Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" gen_autofdo_event.py was stumbling on models with stepping so I updated the script to handle this case similar to the code in https://github.com/andikleen/pmu-tools/blob/c6a5f63aede19def8886d6a8b74d7a55c38ca947/event_download.py The second change was to tolerate cases when the CPU supports PEBS but the perf command with /p fails. This can happen in, e.g., a virtual machine. I regenerated gcc-auto-profile using the updated script. contrib/ChangeLog: * gen_autofdo_event.py: handle stepping, non-working PEBS gcc/ChangeLog: * config/i386/gcc-auto-profile: regenerate --- contrib/gen_autofdo_event.py | 54 ++++++++++++++++++++++---------- gcc/config/i386/gcc-auto-profile | 41 +++++++++++++++++++----- 2 files changed, 71 insertions(+), 24 deletions(-) diff --git a/contrib/gen_autofdo_event.py b/contrib/gen_autofdo_event.py index c97460c61c6..1eb6f1d6d85 100755 --- a/contrib/gen_autofdo_event.py +++ b/contrib/gen_autofdo_event.py @@ -46,20 +46,29 @@ args = ap.parse_args() eventmap = collections.defaultdict(list) -def get_cpu_str(): - with open('/proc/cpuinfo', 'r') as c: - vendor, fam, model = None, None, None - for j in c: - n = j.split() - if n[0] == 'vendor_id': - vendor = n[2] - elif n[0] == 'model' and n[1] == ':': - model = int(n[2]) - elif n[0] == 'cpu' and n[1] == 'family': - fam = int(n[3]) - if vendor and fam and model: - return "%s-%d-%X" % (vendor, fam, model), model - return None, None +def get_cpustr(): + cpuinfo = os.getenv("CPUINFO") + if cpuinfo is None: + cpuinfo = '/proc/cpuinfo' + f = open(cpuinfo, 'r') + cpu = [None, None, None, None] + for j in f: + n = j.split() + if n[0] == 'vendor_id': + cpu[0] = n[2] + elif n[0] == 'model' and n[1] == ':': + cpu[2] = int(n[2]) + elif n[0] == 'cpu' and n[1] == 'family': + cpu[1] = int(n[3]) + elif n[0] == 'stepping' and n[1] == ':': + cpu[3] = int(n[2]) + if all(v is not None for v in cpu): + break + # stepping for SKX only + stepping = cpu[0] == "GenuineIntel" and cpu[1] == 6 and cpu[2] == 0x55 + if stepping: + return "%s-%d-%X-%X" % tuple(cpu) + return "%s-%d-%X" % tuple(cpu)[:3] def find_event(eventurl, model): print >>sys.stderr, "Downloading", eventurl @@ -81,7 +90,7 @@ def find_event(eventurl, model): return found if not args.all: - cpu, model = get_cpu_str() + cpu = get_cpu_str() if not cpu: sys.exit("Unknown CPU type") @@ -94,7 +103,8 @@ for j in u: n = j.rstrip().split(',') if len(n) >= 4 and (args.all or n[0] == cpu) and n[3] == "core": if args.all: - vendor, fam, model = n[0].split("-") + components = n[0].split("-") + model = components[2] model = int(model, 16) cpufound += 1 found += find_event(baseurl + n[2], model) @@ -146,7 +156,17 @@ case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo && echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to update script." exit 1 ;;''' print "esac" - print 'exec perf record -e $E -b "$@"' + print "set -x" + print 'if ! perf record -e $E -b "$@" ; then' + print ' # PEBS may not actually be working even if the processor supports it' + print ' # (e.g., in a virtual machine). Trying to run without /p.' + print ' set +x' + print ' echo >&2 "Retrying without /p."' + print ' E="$(echo "${E}" | sed -e \'s/\/p/\//\')"' + print ' set -x' + print ' exec perf record -e $E -b "$@"' + print ' set +x' + print 'fi' if cpufound == 0 and not args.all: sys.exit('CPU %s not found' % cpu) diff --git a/gcc/config/i386/gcc-auto-profile b/gcc/config/i386/gcc-auto-profile index 5da5c63cd84..56f64cbff1f 100755 --- a/gcc/config/i386/gcc-auto-profile +++ b/gcc/config/i386/gcc-auto-profile @@ -1,7 +1,7 @@ #!/bin/sh -# profile workload for gcc profile feedback (autofdo) using Linux perf -# auto generated. to regenerate for new CPUs run -# contrib/gen_autofdo_event.py --shell --all in gcc source +# Profile workload for gcc profile feedback (autofdo) using Linux perf. +# Auto generated. To regenerate for new CPUs run +# contrib/gen_autofdo_event.py --script --all in gcc source # usages: # gcc-auto-profile program (profile program and children) @@ -10,7 +10,7 @@ # gcc-auto-profile --kernel -a sleep X (profile kernel) # gcc-auto-profile --all -a sleep X (profile kernel and user space) -# identify branches taken event for CPU +# Identify branches taken event for CPU. # FLAGS=u @@ -37,7 +37,12 @@ case `egrep -q "^cpu family\s*: 6" /proc/cpuinfo && egrep "^model\s*:" /proc/cpuinfo | head -n1` in model*:\ 55|\ model*:\ 77|\ -model*:\ 76) E="cpu/event=0xC4,umask=0xFE/p$FLAGS" ;; +model*:\ 76|\ +model*:\ 92|\ +model*:\ 95|\ +model*:\ 87|\ +model*:\ 133|\ +model*:\ 122) E="cpu/event=0xC4,umask=0xFE/p$FLAGS" ;; model*:\ 42|\ model*:\ 45|\ model*:\ 58|\ @@ -48,9 +53,16 @@ model*:\ 70|\ model*:\ 63|\ model*:\ 61|\ model*:\ 71|\ +model*:\ 79|\ model*:\ 86|\ model*:\ 78|\ -model*:\ 94) E="cpu/event=0xC4,umask=0x20/p$FLAGS" ;; +model*:\ 94|\ +model*:\ 142|\ +model*:\ 158|\ +model*:\ 165|\ +model*:\ 166|\ +model*:\ 85|\ +model*:\ 85) E="cpu/event=0xC4,umask=0x20/p$FLAGS" ;; model*:\ 46|\ model*:\ 30|\ model*:\ 31|\ @@ -63,8 +75,23 @@ model*:\ 38|\ model*:\ 39|\ model*:\ 54|\ model*:\ 53) E="cpu/event=0x88,umask=0x41/p$FLAGS" ;; +model*:\ 126|\ +model*:\ 140|\ +model*:\ 141|\ +model*:\ 106|\ +model*:\ 108) E="cpu/event=0xc4,umask=0x20/p$FLAGS" ;; *) echo >&2 "Unknown CPU. Run contrib/gen_autofdo_event.py --all --script to update script." exit 1 ;; esac -exec perf record -e $E -b "$@" +set -x +if ! perf record -e $E -b "$@" ; then + # PEBS may not actually be working even if the processor supports it + # (e.g., in a virtual machine). Trying to run without /p. + set +x + echo >&2 "Retrying without /p." + E="$(echo "${E}" | sed -e 's/\/p/\//')" + set -x + exec perf record -e $E -b "$@" + set +x +fi