From patchwork Tue Aug 31 13:30:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1522624 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=AKh69+f5; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GzSmq2TtHz9sR4 for ; Tue, 31 Aug 2021 23:32:59 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BBBF5385800F for ; Tue, 31 Aug 2021 13:32:56 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org BBBF5385800F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1630416776; bh=SPvIHd9jZpf7fQeK1MMP5lskGBqxjK0Zs+0g6nvkw9g=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=AKh69+f5p3ILsI4iLQW0S9xCq1bUXPS8c4mIzquYuBdd8RXDjWB14QUohshAvU7xL mDWrmmn5oC6qbJWYgABl3AONmpl13dr5KQmzyLB4Tr/XGwCFzKnsx+Iv2AZ7nrLWr7 /v+JtJqgrALwAgdzbY8abyndmp1XYs1BdXCqiBbs= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00086.outbound.protection.outlook.com [40.107.0.86]) by sourceware.org (Postfix) with ESMTPS id 768393858438 for ; Tue, 31 Aug 2021 13:30:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 768393858438 Received: from AM5PR0601CA0048.eurprd06.prod.outlook.com (2603:10a6:203:68::34) by AM9PR08MB7273.eurprd08.prod.outlook.com (2603:10a6:20b:434::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Tue, 31 Aug 2021 13:30:55 +0000 Received: from AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:68:cafe::cf) by AM5PR0601CA0048.outlook.office365.com (2603:10a6:203:68::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend Transport; Tue, 31 Aug 2021 13:30:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;gcc.gnu.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT038.mail.protection.outlook.com (10.152.17.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend Transport; Tue, 31 Aug 2021 13:30:55 +0000 Received: ("Tessian outbound f11f34576ce3:v103"); Tue, 31 Aug 2021 13:30:55 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: ba5f0b8e07f70c80 X-CR-MTA-TID: 64aa7808 Received: from d1063e7e5cda.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 568AC231-FE34-4217-BC4E-A48C31158210.1; Tue, 31 Aug 2021 13:30:49 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d1063e7e5cda.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 31 Aug 2021 13:30:49 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Nqrxu5Ebg5JGGtYL6OK4N7iu6+oYILuyYiJHXQP8IyJ8o1hYKOkIpao75hJb4uLQo0z4VM3SQ4JTgk4g6fQhHZ/jCiHsZUC2Xp3y+iFExBtFMCreRFxz7Wy/Y8Rw5BcV1k2Uvi8F8AeR/QnMm0rxAz1P69EgCdJQeJkHuuzBOcwMUlLPQglFzOGtIwxDx+NYLArQGtOh2W5KQCQ1m3EbT0uQVKOyWQMznsGqgCIFZsBDjs9NJlVRHSScUkDQOPhuGrHuww/YipeLdLz41xP+j7adUJE95q2NHjkFIrTRqvecPruE3S8uKa4GZeNXd+Dy09EOguCBK39KCNmr9zbQdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SPvIHd9jZpf7fQeK1MMP5lskGBqxjK0Zs+0g6nvkw9g=; b=lwfzxZ89rYTha++nt+Es4ueb0ZOBzNqFdLz+EvYNYh4jrDfVXVIAuEISglIFBj9Q/c4A8008TjXwUy3PGlibIeNzrVlFJglAWQ0B5Ry6Ya8+JkomSXTONAOnuI0K3WoPCGEZt3L7At/nOIsA3+EEfIaF2/bu/JdzHaRBBnuZDJDuZ0ybJmTj/VNnUXSLwro4MpyQDQr4fv1epzv4bmxCLuo/Lw2wl18zZRBaOvWQ4j43DPEjQcxNKRjOvrmy0OeV9XYawd9ZfezY1MAsNhQPetFdXgSZ8vd+55kBerVxSsqoELFii9BRXXwdy9MlvncfQywzbMQgYndiEtx24otC+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by VI1PR08MB3166.eurprd08.prod.outlook.com (2603:10a6:803:42::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Tue, 31 Aug 2021 13:30:36 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a%4]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 13:30:36 +0000 Date: Tue, 31 Aug 2021 14:30:32 +0100 To: gcc-patches@gcc.gnu.org Subject: [PATCH 1/5]AArch64 sve: combine inverted masks into NOTs Message-ID: Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: LO4P123CA0259.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:194::12) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from arm.com (217.140.106.53) by LO4P123CA0259.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:194::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.22 via Frontend Transport; Tue, 31 Aug 2021 13:30:35 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4653d089-ea6b-4a9a-fd9d-08d96c838f99 X-MS-TrafficTypeDiagnostic: VI1PR08MB3166:|AM9PR08MB7273: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:4303;OLM:4303; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: K61GaOK2EuEILJXTehTzFaa0iWoLekFOWOzRnRtkc2bBup2II6AKcIAWCF3zD1JIDWoawwAMtzIoTdoPar8xa1A/x/mluDWW94aNSGjsfRuAa35dvGC8gMxYGwmgB1PKBLfyR7h5k07qnJbznz0I7ffVuQkVY76t95JEWWQGSMYgB8Ky73aNC+jC9JixlmCdqgWiPu8Ko5xaq31LOBRnudvyuKi6jHG3M64+x21hrBdoyZjBjyUyUZOz4tgT5lQgbLCN19vyUGKuBFCCkN13nUEIIzWVsm4v0KPmtjTBvREVJRQJM8joQ1LByfjU7WtQvA6pgpwjV5rxWhNaF4rAf4ckiu8EOwB6RDoHMcR1WBrnYHtYcDiFtRINAtmNjiDa4XayTTRJqeclSFN7eEe1ZL+mfUGCs9VjJbVQZJ1VK3znVFO+3WmlJm8Ct3NfXTGaBGjeGVZQDP/TAQRML5zwtxf1JjICpwxepUiUGd8bSuUWdn0+1maQa49eA0DIclN2l6g4VKKdEHnj8dJo44yidI6S2gFSg/QKUCi4uoiRiICyKDp692IwLKkPNapRUEtazZeWZNqM0zj/TpWPfqTYqeGPsrEVgdNOiqI99a/h2r1TIrPWz92gg+AY5JDTYmJ49IJbb0+8pQ3x6PIGZ2Y64jRDY8SKtn0e+AxFl6OnG1Gyabjl+fAvVFlD7Bz+ZlvGXmNk1cZZOwjvTl0pxxd+Khi27EIcT0kFLv/Q7Iyv517EZnP7pgTXnS0ONu++2rex+eGKih2vIOLr60TQGelsdUUPkbwRvpNbTcR2BdmCWsZmtl4bmRg0fJVx6Ytz2JYIBJSCEGVwaXDo5vajSYwFoWmidJSt8ZEZSnIuMoT+mtA= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(376002)(396003)(136003)(39860400002)(52116002)(36756003)(4743002)(26005)(8886007)(316002)(44832011)(8676002)(7696005)(4326008)(956004)(2616005)(44144004)(8936002)(2906002)(33964004)(6666004)(66616009)(66476007)(66556008)(55016002)(86362001)(38350700002)(235185007)(66946007)(38100700002)(478600001)(6916009)(186003)(83380400001)(5660300002)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?y6S0Chgal+/e3U1FLtsBlYJR9OAj?= =?utf-8?q?+HlXRUu+CXaCtb9DRQi4cg8IvrBf2LuETq1OwNLbSwcFvzaVXRz0g1zAHdiByqJa3?= =?utf-8?q?piL29jhc7cuwP1N9z65wuQzED1OYnXrMIjNxl2OS2W8aq9L/JV2doQftjv1bK6BPZ?= =?utf-8?q?zLgjfJlq6QksU63lL6lj5EfFTV4pY1OXUjsVAnUNzBG6QR16SvtjWNtBZbGEF3azg?= =?utf-8?q?pPLgp+AYKfVvrOGDPSUFQ9o5VIN/3tUU+/6gAfVbX/XmWAQR2Ib/8htpvvCbPkJif?= =?utf-8?q?8xPat0MOwhHybv265X8cLnAZlQsXBmYZBFXCu3DH7lVEulNc1drAVSP9tmHxzbxdG?= =?utf-8?q?X8BjppDS2qL9wQbnACiW8XHRWyYyIbdzRjF74RCNvi8L7h846iCUn5GJpi/jfROmK?= =?utf-8?q?qiJz2G1zrN/XYE+k0LGFUOhQcucIVPuTbwWTKGszAMFTnUfra0Bw244c5SKuS3aEu?= =?utf-8?q?lpXjKmEDiFtRSoN5Qga7t5dE1/VFTykWTlPduQXgFRwiSI5hyOPrVCjYInzxjRzgX?= =?utf-8?q?bJTkvZxVixPKEQlZ97nMDQ0OzOKuqGoqVoxqF77SKuY+3zybWT80DEiznUGSf/n7U?= =?utf-8?q?ZPkBiop6/qsx1R/0ovjuXn1K4nU8lkjbxixADmNDwaggxMu9s9BSipTnfHgEvKW9V?= =?utf-8?q?Zm0t4pQW33Y/UqaL7hyER9I5syDEVqZMfRyEI8ZDg5cyjN9WUgUJvGVp1tVcVz5w3?= =?utf-8?q?rlrQasHGbFxyN2E8AhQZdfSYl/D2W4JD65nKy3vUWQrtqLprMtXcnpX55vHkNYhdO?= =?utf-8?q?ItfsJb+qp9n+3nAOogB7uF0h8/FQReXWWHYSvWb+2u3kCpKAchoxTMnIOkZuYUoBm?= =?utf-8?q?A0fYMZaXbVbJsytCcHzYolEqNU1w1eVx+0wh8iN7fU3AjxRDzW0s8iI6/ZftDMwp4?= =?utf-8?q?/dGEIsBzTk4Xz4ASempKlXx0OsRw6gYqmm5/XhpUJpENRH6kA6Hm48M7xE3u1RPSw?= =?utf-8?q?zmV5oOutMuiXs07A3K8PzyrzbCSOiENCFXT+T5rCyeWjXmcLtnc0SvppRo/Up/VAX?= =?utf-8?q?/u+JSGswHJwICOz3aX/dHZA9Pm02u1HU3OfssS+RlaUS99GLPQaw8XDPzSZDBsLbo?= =?utf-8?q?rYeVU1XBcYWJqIIueCh+dxUoS8AD34nq3+d0uIsxs0EBZbCHmtM6U4aG+Ox1cLqic?= =?utf-8?q?0PDB0HbJ+OLk/xUd4gPVk18BEJV9+UeE8Ei741mHzYk+u0XMBo2Tzu+DGEPv7FKu9?= =?utf-8?q?/voNIO8+Q7wpMet7K6cwtkKzaIbBZtpVaiRqg7h6kbmFPj/QMaUrlJCYip5FK28fn?= =?utf-8?q?vpDzm8imftwmF9r2?= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3166 Original-Authentication-Results: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 204c5690-789f-48cb-12de-08d96c8383f4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: o8F4GVLunjpOv4EYVYXayJvPaX2W71EUfzWqFxUauGNnm+ShE/II4PZKeAQWtW/OgF6RLeldBdz3xIrjrd7UncIkJBn6eDwr5P3FSs8W2BXtjOuFCxSdk/TzwHlGxixF0ItHFVK0noFzlpg+1bbut9UQ38JY483wBAA+cLwSMXziWVV9i77QAWZ5O42vo4Edj1m6qAXDweMMJrTtERTCHdKXxfbiG/uMioFcgrMhvXFLKA0UtVV+Zu+A6pfX3MiUwHcQGeV7e6UkQziMqiPESk3q3JxocLGTngvflO0V8UCTZhmi3/3b1f0iFwJYhb99iNaA4nE0dkj+DUbX7bA2lKWnTzcZCtTvMamzFzYKugQckrtIeHyMW2FpXisA2P1vOXMzeI9yMQ9vVPOaasDB8+uQzIp1xF3iHTiTbLWTwmL9MQGVy+iJ7y6fRATIFk1+eMZ3Mg8L6mnqNW5EwYnbuiM05PMlFbyp2W0v4khq0c4cytgCbEpyMLSX+7JhP6jGJMFxUkj+Gd/YxavrRGf25ydgq4aroR9g3PlXglaYNVhnGFW4ziB/C4Azc2ZkWKsXHYPeUdJ4fCr5IhSq92ZtDHUHiGKM+W5aDSQcTVE5jPeZY/fO/p1qDGLn8aapQpIAPhTMMz0MQqxUHTlO2xYmuI8Sk6xAJROR/Y+MTU3gKTHZN53cSapUN1iIeJhD8nGsFCmX9a9M4NyjZHQ5/t59LSI5SoPnfZsiwVl9wLjah4O3N8XWHtwqmz/E5NE0nBrAFkLRhAzqKz9Q+Z8/UH365Pxg+Rj6RnH/h6Lw8preUoznGdWLvs7fZWJnZ/CoxQ7g+suzu3EKybJ0CmT7eOSa7MEsPPG0Jdu6TwSC1GQg1eI= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(376002)(346002)(136003)(396003)(39850400004)(46966006)(36840700001)(44832011)(235185007)(336012)(83380400001)(70586007)(8936002)(36860700001)(47076005)(44144004)(5660300002)(316002)(8676002)(33964004)(2906002)(2616005)(4743002)(36756003)(6916009)(8886007)(478600001)(26005)(6666004)(82740400003)(86362001)(70206006)(956004)(82310400003)(186003)(55016002)(7696005)(81166007)(4326008)(66616009)(356005)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 13:30:55.4625 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4653d089-ea6b-4a9a-fd9d-08d96c838f99 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB7273 X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_LOTSOFHASH, KAM_SHORT, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Cc: Richard.Earnshaw@arm.com, nd@arm.com, richard.sandiford@arm.com, Marcus.Shawcroft@arm.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi All, The following example void f10(double * restrict z, double * restrict w, double * restrict x, double * restrict y, int n) { for (int i = 0; i < n; i++) { z[i] = (w[i] > 0) ? x[i] + w[i] : y[i] - w[i]; } } generates currently: ld1d z1.d, p1/z, [x1, x5, lsl 3] fcmgt p2.d, p1/z, z1.d, #0.0 fcmgt p0.d, p3/z, z1.d, #0.0 ld1d z2.d, p2/z, [x2, x5, lsl 3] bic p0.b, p3/z, p1.b, p0.b ld1d z0.d, p0/z, [x3, x5, lsl 3] where a BIC is generated between p1 and p0 where a NOT would be better here since we won't require the use of p3 and opens the pattern up to being CSEd. After this patch using a 2 -> 2 split we generate: ld1d z1.d, p0/z, [x1, x5, lsl 3] fcmgt p2.d, p0/z, z1.d, #0.0 not p1.b, p0/z, p2.b The additional scratch is needed such that we can CSE the two operations. If both statements wrote to the same register then CSE won't be able to CSE the values if there are other statements in between that use the register. Note: This patch series is working incrementally towards generating the most efficient code for this and other loops in small steps. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * config/aarch64/aarch64-sve.md (*mask_inv_combine): New. gcc/testsuite/ChangeLog: * gcc.target/aarch64/sve/pred-not-gen.c: New test. --- inline copy of patch -- diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index 359fe0e457096cf4042a774789a5c241420703d3..2c23c6b12bafb038d82920e7141a418e078a2c65 100644 diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index 359fe0e457096cf4042a774789a5c241420703d3..2c23c6b12bafb038d82920e7141a418e078a2c65 100644 --- a/gcc/config/aarch64/aarch64-sve.md +++ b/gcc/config/aarch64/aarch64-sve.md @@ -8126,6 +8126,42 @@ (define_insn_and_split "*fcmuo_and_combine" UNSPEC_COND_FCMUO))] ) +;; Make sure that inversions of masked comparisons are always on the mask +;; instead of on the operation. +(define_insn_and_split "*mask_inv_combine" + [(set (match_operand: 0 "register_operand" "=Upa") + (and: + (and: + (not: + (unspec: + [(match_operand: 1) + (const_int SVE_KNOWN_PTRUE) + (match_operand:SVE_FULL_F 2 "register_operand" "w") + (match_operand:SVE_FULL_F 3 "aarch64_simd_reg_or_zero" "wDz")] + SVE_COND_FP_CMP_I0)) + (match_operand: 4 "register_operand" "Upa")) + (match_dup: 1))) + (clobber (match_scratch: 5 "=&Upa"))] + "TARGET_SVE" + "#" + "&& 1" + [(set (match_dup 5) + (unspec: + [(match_dup 4) + (const_int SVE_MAYBE_NOT_PTRUE) + (match_dup 2) + (match_dup 3)] + SVE_COND_FP_CMP_I0)) + (set (match_dup 0) + (and: + (not: + (match_dup 5)) + (match_dup 4)))] +{ + operands[5] = gen_reg_rtx (mode); +} +) + ;; ------------------------------------------------------------------------- ;; ---- [FP] Absolute comparisons ;; ------------------------------------------------------------------------- diff --git a/gcc/testsuite/gcc.target/aarch64/sve/pred-not-gen.c b/gcc/testsuite/gcc.target/aarch64/sve/pred-not-gen.c new file mode 100644 index 0000000000000000000000000000000000000000..a5bb616ef505a63075cf33203de8cf8e8c38b95d --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/pred-not-gen.c @@ -0,0 +1,56 @@ +/* { dg-do assemble { target aarch64_asm_sve_ok } } */ +/* { dg-options "-O3 --save-temps -fno-schedule-insns -fno-schedule-insns2" } */ +/* { dg-final { check-function-bodies "**" "" "" } } */ + +/* +** f10: +** ... +** ld1d z1.d, p0/z, \[x1, x5, lsl 3\] +** fcmgt p2.d, p0/z, z1.d, #0.0 +** ld1d z2.d, p2/z, \[x2, x5, lsl 3\] +** not p1.b, p0/z, p2.b +** ld1d z0.d, p1/z, \[x3, x5, lsl 3\] +** ... +*/ + +void f10(double * restrict z, double * restrict w, double * restrict x, double * restrict y, int n) +{ + for (int i = 0; i < n; i++) { + z[i] = (w[i] > 0) ? x[i] + w[i] : y[i] - w[i]; + } +} + +/* +** f11: +** ... +** ld1d z0.d, p0/z, \[x1, x2, lsl 3\] +** fcmgt p2.d, p3/z, z0.d, #0.0 +** fcmgt p1.d, p0/z, z0.d, #0.0 +** not p1.b, p0/z, p1.b +** ld1d z1.d, p1/z, \[x3, x2, lsl 3\] +** ... +*/ + +void f11(double * restrict z, double * restrict w, double * restrict x, double * restrict y, int n) +{ + for (int i = 0; i < n; i++) { + z[i] = (w[i] > 0) ? w[i] : y[i]; + } +} + + +/* +** f12: +** ... +** ld1w z1.s, p0/z, \[x1, x2, lsl 2\] +** cmple p1.s, p0/z, z1.s, #0 +** ld1w z0.s, p1/z, \[x3, x2, lsl 2\] +** ... +*/ + +void f12(int * restrict z, int * restrict w, int * restrict x, int * restrict y, int n) +{ + for (int i = 0; i < n; i++) { + z[i] = (w[i] > 0) ? w[i] : y[i]; + } +} From patchwork Tue Aug 31 13:31:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1522627 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=En/LecBf; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GzSnz3jm2z9sPf for ; Tue, 31 Aug 2021 23:33:59 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 842DC385842A for ; Tue, 31 Aug 2021 13:33:56 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 842DC385842A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1630416836; bh=BWAPhAJt/X8SKho3gPLmFxOA4r8c5Wt8a1/oYjAyWVE=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=En/LecBfjm4IRZpNgkv4DuyZ7C9Ll8aFDcyp+vp37KKd5v10lhi7s06B2PFkRhQOi wLqPa5KZjPwHJyBiPuRWsN1JVviSQFJhSrqRr+hwOD+1qYuLxn/cYerg2DqgwBjEcN 5QJbFOPvkmFN/s6Ed3kArJrVkUOiLOjmxrC6LZ/c= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40071.outbound.protection.outlook.com [40.107.4.71]) by sourceware.org (Postfix) with ESMTPS id E55E3385843F for ; Tue, 31 Aug 2021 13:32:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E55E3385843F Received: from AM7PR04CA0029.eurprd04.prod.outlook.com (2603:10a6:20b:110::39) by DBAPR08MB5621.eurprd08.prod.outlook.com (2603:10a6:10:1a3::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Tue, 31 Aug 2021 13:32:13 +0000 Received: from AM5EUR03FT023.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:110:cafe::cc) by AM7PR04CA0029.outlook.office365.com (2603:10a6:20b:110::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.20 via Frontend Transport; Tue, 31 Aug 2021 13:32:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;gcc.gnu.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT023.mail.protection.outlook.com (10.152.16.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend Transport; Tue, 31 Aug 2021 13:32:13 +0000 Received: ("Tessian outbound 5918cb94a4b5:v103"); Tue, 31 Aug 2021 13:32:12 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 173f2207168ac47a X-CR-MTA-TID: 64aa7808 Received: from 47ebf33aec40.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 0F90FDFD-92A9-4F49-98EC-801EE577F364.1; Tue, 31 Aug 2021 13:32:03 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 47ebf33aec40.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 31 Aug 2021 13:32:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SSiD2leyQj+4VQm9F8caGIwmfnQ4ZVlA0hukcpUG9Xi4x6FeSNOcJWZiN+A4fxxQASMpvFwMDuSOAboyi0IjYgJoS7hGZLE+Gts9V4lWHrHWCg36WrJIx7e56rfLPmawqGfY3uQqBHb6ECJXXSiO2e7eyrrTDMhgI+mKduGCYQaaK1+V5a2ktR+Fe0I+LsZUmmBRj7Ps3ahwM3peiIKTgSbRO3RA7dG0MB7Xem9KWPEHPYjn3X5iHgxHDpb426iJ8hm6Yc2v5z/DEyTZFYKi+GqMLIe+uWHOuKbsJWLMMinloiIFEipuHrWFrBIYXmKXf49QMwJmywWX5ZPvtvflCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BWAPhAJt/X8SKho3gPLmFxOA4r8c5Wt8a1/oYjAyWVE=; b=ewM6GXRFAy81b1obnAzYVS9jge9Z1kLju+uHKN+FwaRx4Gi8+CAa+aS+Z9FEfHt1N/BsQnXz9STXqs7We4dOQjgcrnXzT76PXGfc5WN2hFXvKkyHwBgsjROPr9wTmu9jw0gefmiV20OyQMb5ve+/Eb9fvT+kYYpU4p5iUjBd6v6FKdG/KIe0snauVJ/j3KyAJdEVoyCLQitcpUzvFJvqiR8OxbWej33+BD1u9HWpMZ6eVaiyLib/oAaI3dSx/BGqXZ33tB3XNxoSN6FgQPTdEIIKpCxouhU6ryI0t9lEaOVlYO2EIRj5/QCfbtt9ftJXU0MCjSsd2kp9HTYnMc+R8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by VE1PR08MB5136.eurprd08.prod.outlook.com (2603:10a6:803:107::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.18; Tue, 31 Aug 2021 13:31:58 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a%4]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 13:31:58 +0000 Date: Tue, 31 Aug 2021 14:31:56 +0100 To: gcc-patches@gcc.gnu.org Subject: [PATCH 2/5]AArch64 sve: combine nested if predicates Message-ID: Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: LO3P123CA0008.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:ba::13) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from arm.com (217.140.106.53) by LO3P123CA0008.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:ba::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.19 via Frontend Transport; Tue, 31 Aug 2021 13:31:57 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ab27e2f2-6967-41b5-a4b5-08d96c83bdf5 X-MS-TrafficTypeDiagnostic: VE1PR08MB5136:|DBAPR08MB5621: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:1148;OLM:1148; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: XxEnIHnPqkhByFpebSKJuO28vq7+aKxiJF2XRPktqKcmDAWZIlQWRGb7VIEvSqBuB5yUEvedLPX/nAkBTiKYTAHMonSMt2MFimJzw1U6aL198x72fiKLmo38WYaUZofEYYSqoJC5H2P108T1MFoLnSwY7O5dcYqSra1dc5f/7hnQ3ACrEcJ5y23pytzjfchAnJDVipXOjYRCwgz7H7kUMUfXBbX6SToj6AsktqAe0jN5jaHRovWKkj5Uze58Og0Xg83sV7opDrmNEtriImabW13DEnJzIsQir//V3KAjM/ksKiz/siSSxPxO+1uvbrwbdbwlCFLydAzjxOo5tm1l2Q2wLY/PvBJB6abCI6vNMqnUL2kvzUADwAeU3vz4n3KvAtCd84IffhXcqPF2DftjyZuuCyJyL7IMOuIoJGrdufjPE+Bdh0rFDs2oYWEytrKXqbipU33h/+zPqSWQvWl9Mqf1QGCty17ahdpINZ+xMzoi8ddQES3X82pudcFa1N3UUs1R6BRG+PMgvAXGOBvL/lh9ZkRTSoayqhjsGTJp0htsY/QvmV5anro7xd+Zyt+0Y0+km54hAvXe2HZ1S5aFIzJPrJr+e5QeK3koSBurgBVKYewiEtrTRFENg3uMZay0ZrJRLHXZM5LlF+0Q9k3PfOtohXemYlIMIbkdki1HAOE1/1DjTrSWlNxeo4LtfI7eNe1B2dbrJg2vRfaNar5yNi4kAmos/lJZTlGw/NPaL/o1tRTdJ3UiZpnRD+wynyXys6ZKUz0OJvK/0d6mZHs+A/AZ+6fSpXBwHkDFfHc7OaPSoavkarHMDl3aRPLHxYPBVygwgih2ziIh1VbUeGqwPXz4QsBYp9C0czIVosq2WEV4m75MjH29sJlXtC8FC5zW X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(136003)(346002)(366004)(396003)(55016002)(66616009)(66556008)(8936002)(44144004)(52116002)(7696005)(316002)(956004)(66946007)(66476007)(2616005)(44832011)(33964004)(8676002)(36756003)(83380400001)(2906002)(8886007)(235185007)(5660300002)(38350700002)(6916009)(478600001)(86362001)(4326008)(38100700002)(26005)(186003)(4743002)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?0OkCLu5brbdaU1Rj+sy60Y2lumAH?= =?utf-8?q?dhgA0rRQWivTh0BJKxNYb7WByoIIikdDb7Ps5xbaqvgQ6xq6bsRHfL9aKIHDpbY0V?= =?utf-8?q?6KhsrOm9DbkKUaWsI25tF/LUw0qqIlF0azxJ5fks9tWSplpI0q+ufJ1ZtL7wUbhSq?= =?utf-8?q?SbYk5OEKqD4zk/kZp+qmTSnqazSsKJg4WA8dxHechgV40fprD7VWqPtrVp+SRK4u7?= =?utf-8?q?h6/v7r3vDC/dIfLdaKO5rg8ZLLSCrPqqTMRkt4elwSLO1LcTvpi8Lh6KFBVNKy3xK?= =?utf-8?q?lXhfW0ZqnVZgY81Z6a6vROnzsRo/3bV9tdjhP7MMmL2A07VmDN6hPmxuRXfSsK8ri?= =?utf-8?q?UunnaS3IItyRt10OiTNfUE3SS5WlmhoQPufodAwTFl8G5cOCz+nQBxI39/loeWkus?= =?utf-8?q?MKZ7DlJnNIoQPWFJaJLkCOOKNYvxGiCBqhNAIZ5VARj7kEhPdcWqGuYgYNujUX/XT?= =?utf-8?q?fd+VDRvC92m+cbACgk4ebGkmy1wmCrDB0Qw/y0RL4BQojQqCe0Y3lAm7E4dOVcDOr?= =?utf-8?q?Gl2JFA8pImh1DhsRobdQuzy25pWlb4O52D32Zx7qAtqgxTfvjFZGTrwx2g74wRAHN?= =?utf-8?q?EfPV/nRo3VfMjI2MGSC4vUXAgj5KCqlty0y9IVGg/QxIuI+TTIxzVvwjiqCageyg3?= =?utf-8?q?XrQV9RSizsXGH9raZ9Ig/TZw4m04upyCkK0j8bDHKyBdUuibr7d1BVhNmxmKaQwnO?= =?utf-8?q?9iDNV6tYUiliLxLMNUQT4hw2QcOo8TrxSQ3WOqXw0zH8lvMjNZbLOf7fxBEqgxH15?= =?utf-8?q?XQqrQMK1W4Nhtj5KdwNS/rvM0yaSoT9dvU8mZrY7rFfAb6gtWokk+/asg2HmDE/M2?= =?utf-8?q?md6PgJK9jpo32yNNIHEZOYsHUA6sImZjwqclYTIANPlUWVzFCyerwuCr1YaTZWTSo?= =?utf-8?q?jNUBRvWzboHea6Uxhe9mg+XCePbrfio5jbpQ6cxYDNjZV6NkqywgSaY5Kwd9lX4hb?= =?utf-8?q?1yJwhJaJt+HdloTn7pOPIpwgfi7UbXVBAED82JIk41GZBGKLsRzdltO6z75hMVh/t?= =?utf-8?q?g86/VrEEebhSQH9TUNyNfI7OE0cSXGR25af+OSVxOQHlrFiOloPRqY4kauhfV3DWR?= =?utf-8?q?Be5ZoLzxXyqIo+1Ca7lWwpDCre77Y7CGXvlp+6t0Y1b9yubK0YDUDfUBBcyAeFwTO?= =?utf-8?q?kdJmgWxP6427YPm+C5qGej14Zh1gsuLdG5B1UZQGv1XG+n294XU9fhvXoIbqhuUcM?= =?utf-8?q?DItpAD3QHErtEXYxHO9Dhz5Fhy5VVkJ4dDICRtnZYmzsfwDJaD6wo6NdlT2noDM0J?= =?utf-8?q?4HqyZ4ChiTO6M1lX?= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5136 Original-Authentication-Results: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT023.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: b3f1c4fe-9897-4a6a-e9d6-08d96c83b4e8 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gF1Kx0IJpnjxzI+oQu9KisSTWjZyUGZBEjrT7cOYbUVYseGM9AnUIGyPPCfJZZc7KXECr0Y+kAF6lBDM7I2oZtE4mrgncBAnllylmXRSWHyznP9nj/vMrqJYesOR6OY3oQI8kO1Wtk6NTVSuSV0bXyxtOCy7CPjNekJF38vDGQjUR7LspISTWhUWGvEpnp1Lb+zmTr1bbSz3HGvA+1n5sx0iAbS3f7ikyFCntOqzzecfFDsWsPLEwzdYDUlzU7eGnoVFYu8t42hkNoOdfW1tGRQI5F3vs9uq2X9hy9bT1mdHgGCI6fqgpn4TDPsszBgQacT1SHPFwWv24Xsi5Qpek1qrWI6uyetP9S5JYHqcS55plSbZolexc3/ZQMIUTeVJHxj1TTGgWVX6c7/ZOOLmZ3yQPblivE7NSsV+bpF5Vk+h3C1urI73f7azdUrVQgzKV0qLkuPvCJsB/XZQAn6hjkxPwfSlM7ZVtm5187izPFepVrOdwLkjy79FZBdSfDkYJuS2x8u6MXa3S4/NnJqLSKhlCxrOCCKltIPCL1rfz2DbbByjlMPRj1yMBb+BUry0jNd3FIj/Dr/plAHKiibLLPijdh/ivfBQyNLTuT2EUsvQMVygm31jgCz/bWIYvet4I27m7lukTNVYcNrncVTQ+DL/DzVU85P8t8on5u/h0x6tKR0Vb/8vBsMlf3+DrEa9NwL84jhGNVuYEtrkXCk7Yt4xvCSNs0MwHOA2JAr1u54gXIR+1Qaoo2m4xC0j18UOYc8+KEgd/Ulnev1CSOVyIsHE0wkys2rgAefYtPqYBrntNl/IjDfOA+i5zbbsNtX2lpnBztVm+NOpfNSyLLUImflGhfraLbbTBH9q6/NAR3AkHhUY+dRl+fklp+NfQTcW X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(376002)(346002)(136003)(39860400002)(396003)(46966006)(36840700001)(235185007)(55016002)(4743002)(5660300002)(956004)(86362001)(47076005)(2616005)(6916009)(70586007)(8676002)(36756003)(8936002)(4326008)(336012)(478600001)(36860700001)(44832011)(8886007)(316002)(44144004)(33964004)(186003)(81166007)(70206006)(66616009)(356005)(83380400001)(7696005)(26005)(82740400003)(82310400003)(2906002)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 13:32:13.2355 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ab27e2f2-6967-41b5-a4b5-08d96c83bdf5 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT023.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5621 X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_LOTSOFHASH, KAM_SHORT, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Cc: Richard.Earnshaw@arm.com, nd@arm.com, richard.sandiford@arm.com, Marcus.Shawcroft@arm.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi All, The following example void f5(float * restrict z0, float * restrict z1, float *restrict x, float * restrict y, float c, int n) { for (int i = 0; i < n; i++) { float a = x[i]; float b = y[i]; if (a > b) { z0[i] = a + b; if (a > c) { z1[i] = a - b; } } } } generates currently: ptrue p3.b, all ld1w z1.s, p1/z, [x2, x5, lsl 2] ld1w z2.s, p1/z, [x3, x5, lsl 2] fcmgt p0.s, p3/z, z1.s, z0.s fcmgt p2.s, p1/z, z1.s, z2.s fcmgt p0.s, p0/z, z1.s, z2.s The conditions for a > b and a > c become separate comparisons. After this patch using a 2 -> 2 split we generate: ld1w z1.s, p0/z, [x2, x5, lsl 2] ld1w z2.s, p0/z, [x3, x5, lsl 2] fcmgt p1.s, p0/z, z1.s, z2.s fcmgt p1.s, p1/z, z1.s, z0.s Where the condition a > b && a > c are folded by using the predicate result of the previous compare and thus allows the removal of one of the compares. Note: This patch series is working incrementally towards generating the most efficient code for this and other loops in small steps. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * config/aarch64/aarch64-sve.md (*mask_cmp_and_combine): New. gcc/testsuite/ChangeLog: * gcc.target/aarch64/sve/pred-combine-and.c: New test. --- inline copy of patch -- diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index 2c23c6b12bafb038d82920e7141a418e078a2c65..ee9d32c0a5534209689d9d3abaa560ee5b66347d 100644 diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index 2c23c6b12bafb038d82920e7141a418e078a2c65..ee9d32c0a5534209689d9d3abaa560ee5b66347d 100644 --- a/gcc/config/aarch64/aarch64-sve.md +++ b/gcc/config/aarch64/aarch64-sve.md @@ -8162,6 +8162,48 @@ (define_insn_and_split "*mask_inv_combine" } ) +;; Combine multiple masks where the comparisons operators are the same and +;; each comparison has one parameter shared. e.g. combine a > b && a > c +(define_insn_and_split "*mask_cmp_and_combine" + [(set (match_operand: 0 "register_operand" "=Upa") + (and: + (and: + (unspec: + [(match_operand: 1) + (const_int SVE_KNOWN_PTRUE) + (match_operand:SVE_FULL_F 2 "register_operand" "w") + (match_operand:SVE_FULL_F 3 "aarch64_simd_reg_or_zero" "wDz")] + SVE_COND_FP_CMP_I0) + (unspec: + [(match_dup 1) + (const_int SVE_KNOWN_PTRUE) + (match_dup 2) + (match_operand:SVE_FULL_F 4 "aarch64_simd_reg_or_zero" "wDz")] + SVE_COND_FP_CMP_I0)) + (match_operand: 5 "register_operand" "Upa"))) + (clobber (match_scratch: 6 "=&Upa"))] + "TARGET_SVE" + "#" + "&& 1" + [(set (match_dup 6) + (unspec: + [(match_dup 5) + (const_int SVE_MAYBE_NOT_PTRUE) + (match_dup 2) + (match_dup 3)] + SVE_COND_FP_CMP_I0)) + (set (match_dup 0) + (unspec: + [(match_dup 6) + (const_int SVE_MAYBE_NOT_PTRUE) + (match_dup 2) + (match_dup 4)] + SVE_COND_FP_CMP_I0))] +{ + operands[6] = gen_reg_rtx (mode); +} +) + ;; ------------------------------------------------------------------------- ;; ---- [FP] Absolute comparisons ;; ------------------------------------------------------------------------- diff --git a/gcc/testsuite/gcc.target/aarch64/sve/pred-combine-and.c b/gcc/testsuite/gcc.target/aarch64/sve/pred-combine-and.c new file mode 100644 index 0000000000000000000000000000000000000000..d395b7f84bb15b588493611df5a47549726ac24a --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/pred-combine-and.c @@ -0,0 +1,18 @@ +/* { dg-do assemble { target aarch64_asm_sve_ok } } */ +/* { dg-options "-O3 --save-temps" } */ + +void f5(float * restrict z0, float * restrict z1, float *restrict x, float * restrict y, float c, int n) +{ + for (int i = 0; i < n; i++) { + float a = x[i]; + float b = y[i]; + if (a > b) { + z0[i] = a + b; + if (a > c) { + z1[i] = a - b; + } + } + } +} + +/* { dg-final { scan-assembler-times {\tfcmgt\tp[0-9]+\.s, p[0-9]+/z, z[0-9]+\.s, z[0-9]+\.s} 2 } } */ From patchwork Tue Aug 31 13:35:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1522629 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=iYL/b5SQ; dkim-atps=neutral Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GzSrM4gdQz9sXM for ; Tue, 31 Aug 2021 23:36:02 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 28620385802E for ; Tue, 31 Aug 2021 13:35:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 28620385802E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1630416959; bh=Sg378PwNcOQM2nI8na0WxeHvvRQ1GHYBwV23WBEcbWY=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=iYL/b5SQmKcXAalJ2uF6XH2ZBVu1BXJgmZbD05ACy2PAsuV8NcmNBYCtxld8Q/Q9h OOLx9ZMpRUyZVvEXaUluQOo4/FmCuu6X5Fgmbbr5MgivPwfDV6R5KtbbmHGxSgooND fcRnPTrDEnlu30Q1DzsSta5PRJi8LFux7eoyu47s= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2067.outbound.protection.outlook.com [40.107.21.67]) by sourceware.org (Postfix) with ESMTPS id 416433858C27 for ; Tue, 31 Aug 2021 13:35:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 416433858C27 Received: from DB6PR0202CA0041.eurprd02.prod.outlook.com (2603:10a6:4:a5::27) by DU2PR08MB7254.eurprd08.prod.outlook.com (2603:10a6:10:2d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Tue, 31 Aug 2021 13:35:26 +0000 Received: from DB5EUR03FT054.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:a5:cafe::e3) by DB6PR0202CA0041.outlook.office365.com (2603:10a6:4:a5::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend Transport; Tue, 31 Aug 2021 13:35:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;gcc.gnu.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT054.mail.protection.outlook.com (10.152.20.248) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend Transport; Tue, 31 Aug 2021 13:35:26 +0000 Received: ("Tessian outbound 8b41f5fb4e9e:v103"); Tue, 31 Aug 2021 13:35:26 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 2aa3c9b4de3d818a X-CR-MTA-TID: 64aa7808 Received: from ae3cfd90cae9.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id FE009B6A-8C5F-4CAB-8F3C-331F27ABA2C6.1; Tue, 31 Aug 2021 13:35:14 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ae3cfd90cae9.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 31 Aug 2021 13:35:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iLNb39AhwZ9OJUgp7GrjvWZVW/bzbyAhpJU+Prh4WyYC7DDIxxh+HObuMJyoMk4IwBgIHsILEa+rIB17X02edp5Z9+fc9uTwq6BcxbhuqRlmEQe0HmvdEO2fiKirDpPv0IplJmtc4yLn1+8qOSC1vFxvNNXpthUjwW7SBJhqPCyZqpOAVsjz3NgiaO3XbhwgPP347ILmgdMic7W91ow1KLLGEOXgo2drFGxFVIiwB/YG5ueRn7rXokj0F1dVbJDYhUiEdiJdrOfbvg5SsJKqs2FiY0b4yM7RmuheuFGnDvtTDzwXFx5GfXtxqBJ55EZscrsN62AztrtFIWdZG9SZvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Sg378PwNcOQM2nI8na0WxeHvvRQ1GHYBwV23WBEcbWY=; b=d28AQF012r8B+CiPfuc38nvMH89TQjZsgtU36UZ0VgeU1zjGyN75OIkj0ZPjvHi7rDE6CfE8S1Zt8iS/8VJyJVgcBVDpVD38GJD6u6aUJmgAHbI73zLSVLEav2NxsNXfRNP2AeBRhmlA0gG5iMoHBKtMGu9/IpOjPfrWAgmVtS/aqInQlGGA8RJrxmadnEC0d2sd+udKvb1JBT/VNWdrH5aYCSwTcGkoirw0fU212gG8mgN/ntW2yztdL9CveIuvLWHB91Rfx/gwJSPOTtnUt/YFTCKTQ5xdDmko4xMKFWiUueyc2AjPTw6XOQi3KZR9uqFmNLpdV9Bt4bP8AcAj/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by VI1PR0801MB2128.eurprd08.prod.outlook.com (2603:10a6:800:5c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23; Tue, 31 Aug 2021 13:35:12 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a%4]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 13:35:12 +0000 Date: Tue, 31 Aug 2021 14:35:10 +0100 To: gcc-patches@gcc.gnu.org Subject: [PATCH 3/5]AArch64 sve: do not keep negated mask and inverse mask live at the same time Message-ID: Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: LO4P123CA0388.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18f::15) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from arm.com (217.140.106.53) by LO4P123CA0388.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.23 via Frontend Transport; Tue, 31 Aug 2021 13:35:12 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6368340d-c063-44ec-825e-08d96c8430dc X-MS-TrafficTypeDiagnostic: VI1PR0801MB2128:|DU2PR08MB7254: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: t0rak29Qst4e09CuaWsorEBi12aC+6Xm0BWPNxyGQBWdR8v64VKhUctuhgM5wUkas0M9OyB2DOXqDAiC1rWqNu6NL/8kE4FOwP05nvxidVzl3qmYTSeH8ZtD0x/ijA9XWTXzNV1s4UtU7pqMr4OSQW490zzhd0XutBBNP7AgbP0M9CwVG2PbuvVe2GCkT1PJdgPQkx/cHx86UmpHWcL95GwRvUvf8TH0VRRmgFhgOuBCCOBGa3QtwjbZIU6luMYxjf9kzLJgCcK/sivTWbVYs5U9SCjtn+MK/tNZ3YF5LRuC86QF66850SFHSEQL+eW9MX1nd7anO8ANVfgi1zYxO9dLB1P6kvf1Zxnr0z+hcjdvQXahM4YKDNUsis5uKzvcM8Q8dx8rnhNxmYVducrXfWKELP/BaByJiQ4bhm9wfeMJgnJCWVrYAGL9BEgJKDQkmFyZF47BpsGydX4CYuphil+N8k4IupCF3oHxeojqRjaTjBVAVc0S8qlUo+/WV8OQjtxvJKqaEjj8SiPZysTS0RdyqAPPovVb8nQbEnmLPdt3oAJSMvKntWNomTY3Jrq77mQnyTR6DmCUVQFLlMBS+nsHijMTdGWAJYFWyyx2lguUfovzAyXk6G7Tb8Kwwg3Lil+v0takIfdJFOxWOVAo5ltwC7g3GjbqIUOdU2Ey+RImyriZIrwwa+XnqeClXKm+z8yIcIoa3mXwL205qdLjH1HdpvAGQXhK2Vs2HhcCGcxGaZYHKFxiypKVhSqdO8bPluJ6s0/itOPdTmg5NEZdyOgvYdBxqyIPVPaTZRu0YvMmnuFtMf9X759taxPvxj1FOJntP70Cox+XU7betIxrUH/5dP8arjUp9LDK/x3qpZ+V0gZNtRZZNOeZL/Dpoz+M X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39850400004)(366004)(396003)(346002)(376002)(136003)(86362001)(6916009)(316002)(44144004)(8886007)(186003)(55016002)(956004)(33964004)(52116002)(26005)(38350700002)(4326008)(36756003)(2906002)(5660300002)(4743002)(44832011)(2616005)(38100700002)(7696005)(235185007)(8676002)(66616009)(66476007)(8936002)(83380400001)(478600001)(66556008)(66946007)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?JBIfT7YrQwXcCFh8qhlj4Xr8+2L9?= =?utf-8?q?/kDAfyAD1dYEYDNdTMWGinGN+IyMlAh/guWgz8qiDhWc1QLx3d69wPG+RptmOSgIw?= =?utf-8?q?HXgOqgtdCevLXiQZ28mH2MZGTlzY4ZpjkDYQwvK+nNcY9Akgc/6o1nzHhaRnAUx0z?= =?utf-8?q?9YO1y2iybKMG11fHHxRqsUXyil/D0lHNEocumU3PrwiM/D3d87eyDzVcrEDR9mWhz?= =?utf-8?q?eQ8MtDWVXUzqDbxqkYFgnLH7h1CVoLu5OBvomzJXp2gV2rykuf+sdMXOnJRev2BkI?= =?utf-8?q?+8DznoxZfvAo6uWkH3sj7u6uTAOT9lgxe9bXVzIN5ply3Dmkbhv692jJSsGjQld4a?= =?utf-8?q?PuHzwjTLObOQxV5bQqkvgz0v5sf8XfNDsqKuuAbZsMnjFNoItdcmqvxxQUL+PW7ni?= =?utf-8?q?sSDzEbZyoCo77R4fjhpWsoX6RjqhfmbuQk1L1OUfBB7TwaSU7/dnfDA8ohbagsK4i?= =?utf-8?q?dx18o1dgbQHlB1tttHRbyVhBS8m6JXKu1B4MqfhWgcugvzNRyd6Sc26og4Z70hZnH?= =?utf-8?q?qz3eh/Fm3AqkTp5UDTtF1Sr5SL4mkQrhxuGm8RFp0QoPCXAjZIk4Q/rT1ARbuXDmW?= =?utf-8?q?y0WYnPKo28FhwRJklUkcK5REFi9vqcg3PawsNMtqFZWLB63KrkWc/LSMkrs1moTtA?= =?utf-8?q?Ji9uFB67o9uQEY8Ie9AbRvyr2FR4Opp3MxVLtmVbOBcApjCNg9dmeE8dioU/ug8ap?= =?utf-8?q?piE7ClyIZ4lSR783eSQEsuUGzdqP4WJW4IIgYnjxS2TVMBog0jGM72RhuxScFaE8N?= =?utf-8?q?XGmSOQoEt5RWXDKDJjA16gN2uks12Ek7ckXnoSqYr1Y0INsobC2fTco+6az0fP3EP?= =?utf-8?q?YoHwC/re2S4hne36TZfmeFBAmaqGWd1kQlEX3PjHKfVAYoJDjqLj5hMtZtzEFIEXk?= =?utf-8?q?xhWBq2u7o2mEZqbIMuJn5mU8W466HS0nsq/tTQUKpoGO4vTZYAqyiq8zELo4oTK3W?= =?utf-8?q?g1OSiJVo5mJUhuI2Bay9Xw+GxqL+Mezkwu70uNInwyxTsDmZQZWKaqCz23JcogrDt?= =?utf-8?q?rfTMa0d7U5SjrO12ivoJRu6AWtSgR80tgz5s0Mg7hdcZRToqHndXb90FLGkCBFXXT?= =?utf-8?q?jp8GIxj/TiqvvMR/EJuOArssthhNgGep5PdjA3fpDho0sV2jloofOIGRIklVEjvHX?= =?utf-8?q?HX2wpXbepNG6HkI8VC6ArFV9ZcqhUUU4+vZZi28s0uH4/7Saq8XvKIUiBJqn8f0i0?= =?utf-8?q?EYOUKX80WfWjrwpOKKpjqUkbFX2geer75CO+BgsP0BRqR57Lm+Kw/5Ggd3mta239D?= =?utf-8?q?w3919JHP3+mixPBr?= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB2128 Original-Authentication-Results: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT054.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 0fc58628-3f7a-49b2-c311-08d96c8428b4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NVq7z40MVQbRn9/cHN7eLjswzVpl+5RSjICGc02WEEjkvJA5frX8LuYkcGMBQ/ov+zZ4t4tjgSy+XVGaZmtnWP8zGclA4JOml0fUzv0FxrRSoElL00bqgTp+CtifjlW5lN/l3IgegOSRkg5jZo4E39ZaWxoKo/G6AdmpiCRtZ7ee2mJ8IqcmNhSGqNqWseKyzafni1DcXLWxjZYROyPwSP0txLpWM0rBpyWSVkacElXL6QCHXSE4L5VZE6HNXyCY7pcZsOThKiRZ874FHcDfLIkfnTIIGB7T5Y1CVysvpRaW08urUTh2IdNfOBLIK+xYCgf6ZW5CE06OqskVcmrpRUfsynOyJSdO6wRNoQHU8NyKfGOcz0Ok5MnMkF3/zHMTez3jfUCRmPpEQ4cA0zM4klxZAYaDKu8J8knr1JLwH5YCyeCQ75Z2Kg4p/72VOPiarn2Vxu3wrdBYeB6uE6NcBzB92sVcGz6h4AXJpBJ43NHet28i2XkNhk2/RUWoDekD/oWy/im+N7Df/cxHKX7kfF+pJuS4Kq/6/CT/n+/XF7CyiukQ5kOJ/qyPh2RnTU+jCjiu5u9gG5DN5CUthZI6tPIw1UpHk6t1daxrFuU5sUpFA8eZ2nGUd6sz03F5IAq1J2LvTndmapncUMU1L9eJA9rcCWX2H2QKMfJMvVhi2jMNBGAMyvSNY5F4H/q1WQfBMo1AdlPaZIUBYImXDNLVwTSQiosAXwcmXJJpEJixmuv2qxP8pAuvZtcXmM7d83QCLx4XCxBAGnli3rYDNXthEs43i9TuOlz3T4guaLb+qO02vqvvWT9tBPHuru1x/F1AD9iDdRcraVVHpVHwk98u95V1DZIF4mW6b2Or8eheMllJUORhk9ap2fb69os/RBBR X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(39860400002)(136003)(346002)(396003)(376002)(36840700001)(46966006)(8886007)(336012)(55016002)(36756003)(26005)(86362001)(2616005)(8936002)(186003)(316002)(4743002)(2906002)(8676002)(6916009)(83380400001)(5660300002)(82310400003)(478600001)(44144004)(36860700001)(44832011)(4326008)(66616009)(235185007)(47076005)(33964004)(70586007)(81166007)(70206006)(356005)(7696005)(956004)(82740400003)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 13:35:26.0791 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6368340d-c063-44ec-825e-08d96c8430dc X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT054.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR08MB7254 X-Spam-Status: No, score=-13.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_LOTSOFHASH, KAM_SHORT, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Cc: Richard.Earnshaw@arm.com, nd@arm.com, richard.sandiford@arm.com, Marcus.Shawcroft@arm.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi All, The following example: void f11(double * restrict z, double * restrict w, double * restrict x, double * restrict y, int n) { for (int i = 0; i < n; i++) { z[i] = (w[i] > 0) ? w[i] : y[i]; } } Generates currently: ptrue p2.b, all ld1d z0.d, p0/z, [x1, x2, lsl 3] fcmgt p1.d, p2/z, z0.d, #0.0 bic p3.b, p2/z, p0.b, p1.b ld1d z1.d, p3/z, [x3, x2, lsl 3] and after the previous patches generates: ptrue p3.b, all ld1d z0.d, p0/z, [x1, x2, lsl 3] fcmgt p1.d, p0/z, z0.d, #0.0 fcmgt p2.d, p3/z, z0.d, #0.0 not p1.b, p0/z, p1.b ld1d z1.d, p1/z, [x3, x2, lsl 3] where a duplicate comparison is performed for w[i] > 0. This is because in the vectorizer we're emitting a comparison for both a and ~a where we just need to emit one of them and invert the other. After this patch we generate: ld1d z0.d, p0/z, [x1, x2, lsl 3] fcmgt p1.d, p0/z, z0.d, #0.0 mov p2.b, p1.b not p1.b, p0/z, p1.b ld1d z1.d, p1/z, [x3, x2, lsl 3] In order to perform the check I have to fully expand the NOT stmts when recording them as the SSA names for the top level expressions differ but their arguments don't. e.g. in _31 = ~_34 the value of _34 differs but not the operands in _34. But we only do this when the operation is an ordered one because mixing ordered and unordered expressions can lead to de-optimized code. Note: This patch series is working incrementally towards generating the most efficient code for this and other loops in small steps. The mov is created by postreload when it does a late CSE. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * fold-const.c (tree_comparison_ordered_p): New. * fold-const.h (tree_comparison_ordered_p): New. * tree-vect-stmts.c (vectorizable_condition): Check if inverse of mask is live. * tree-vectorizer.c (scalar_cond_masked_key::get_cond_ops_from_tree): Register mask inverses. gcc/testsuite/ChangeLog: * gcc.target/aarch64/sve/pred-not-gen.c: Update testcase. --- inline copy of patch -- diff --git a/gcc/fold-const.h b/gcc/fold-const.h index 7bac84ba33145c17d1dac9afe70bbd1c89a4b3fa..852fc37b25023a108410fcf375604d082357efa2 100644 diff --git a/gcc/fold-const.h b/gcc/fold-const.h index 7bac84ba33145c17d1dac9afe70bbd1c89a4b3fa..852fc37b25023a108410fcf375604d082357efa2 100644 --- a/gcc/fold-const.h +++ b/gcc/fold-const.h @@ -144,6 +144,7 @@ extern enum tree_code swap_tree_comparison (enum tree_code); extern bool ptr_difference_const (tree, tree, poly_int64_pod *); extern enum tree_code invert_tree_comparison (enum tree_code, bool); +extern bool tree_comparison_ordered_p (enum tree_code); extern bool inverse_conditions_p (const_tree, const_tree); extern bool tree_unary_nonzero_warnv_p (enum tree_code, tree, tree, bool *); diff --git a/gcc/fold-const.c b/gcc/fold-const.c index 7dcecc9a5c08d56703075229f762f750ed6c5d93..04991457db7e5166e8ce17d4bfa3b107f619dbc1 100644 --- a/gcc/fold-const.c +++ b/gcc/fold-const.c @@ -2669,6 +2669,37 @@ invert_tree_comparison (enum tree_code code, bool honor_nans) } } +/* Given a tree comparison code return whether the comparison is for an + ordered expression or not. */ + +bool +tree_comparison_ordered_p (enum tree_code code) +{ + switch (code) + { + case EQ_EXPR: + case NE_EXPR: + case GT_EXPR: + case GE_EXPR: + case LT_EXPR: + case LE_EXPR: + case LTGT_EXPR: + return true; + case UNEQ_EXPR: + case UNGT_EXPR: + case UNGE_EXPR: + case UNLT_EXPR: + case UNLE_EXPR: + case ORDERED_EXPR: + case UNORDERED_EXPR: + return false; + default: + gcc_unreachable (); + } +} + + + /* Similar, but return the comparison that results if the operands are swapped. This is safe for floating-point. */ diff --git a/gcc/testsuite/gcc.target/aarch64/sve/pred-not-gen.c b/gcc/testsuite/gcc.target/aarch64/sve/pred-not-gen.c index 18d5cf8dcb46e227aecfcbacb833670427ed0586..e4251de32fe347d6193d6f964a74d30e28f5d128 100644 --- a/gcc/testsuite/gcc.target/aarch64/sve/pred-not-gen.c +++ b/gcc/testsuite/gcc.target/aarch64/sve/pred-not-gen.c @@ -24,7 +24,6 @@ void f10(double * restrict z, double * restrict w, double * restrict x, double * ** f11: ** ... ** ld1d z0.d, p0/z, \[x1, x2, lsl 3\] -** fcmgt p2.d, p3/z, z0.d, #0.0 ** fcmgt p1.d, p0/z, z0.d, #0.0 ** not p1.b, p0/z, p1.b ** ld1d z1.d, p1/z, \[x3, x2, lsl 3\] @@ -55,5 +54,3 @@ void f12(int * restrict z, int * restrict w, int * restrict x, int * restrict y, } } -/* { dg-final { scan-assembler-not {\tbic\t} } } */ -/* { dg-final { scan-assembler-times {\tnot\tp[0-9]+\.b, p[0-9]+/z, p[0-9]+\.b\n} 2 } } */ diff --git a/gcc/tree-vect-stmts.c b/gcc/tree-vect-stmts.c index 074dfdcf385f31f2ba753012131985544dfd69f8..54cce92066c058d85ad010091c0c0eb6716f8979 100644 --- a/gcc/tree-vect-stmts.c +++ b/gcc/tree-vect-stmts.c @@ -10216,6 +10216,7 @@ vectorizable_condition (vec_info *vinfo, else { bool honor_nans = HONOR_NANS (TREE_TYPE (cond.op0)); + tree_code orig_code = cond.code; cond.code = invert_tree_comparison (cond.code, honor_nans); if (loop_vinfo->scalar_cond_masked_set.contains (cond)) { @@ -10223,6 +10224,21 @@ vectorizable_condition (vec_info *vinfo, cond_code = cond.code; swap_cond_operands = true; } + else if (tree_comparison_ordered_p (orig_code)) + { + /* Try the inverse of the current mask. We check if the + inverse mask is live and if so we generate a negate of + the current mask such that we still honor NaNs. */ + cond.code = invert_tree_comparison (orig_code, false); + if (loop_vinfo->scalar_cond_masked_set.contains (cond)) + { + bitop1 = GT_EXPR; + bitop2 = BIT_NOT_EXPR; + masks = &LOOP_VINFO_MASKS (loop_vinfo); + cond_code = cond.code; + swap_cond_operands = true; + } + } } } } diff --git a/gcc/tree-vectorizer.c b/gcc/tree-vectorizer.c index b9709a613d557445c060669f5b4517a15058f89d..c2d9970d79f6a9afaf0ad1fbb80a2d5a97bab89e 100644 --- a/gcc/tree-vectorizer.c +++ b/gcc/tree-vectorizer.c @@ -1682,6 +1682,22 @@ scalar_cond_masked_key::get_cond_ops_from_tree (tree t) this->op1 = gimple_assign_rhs2 (stmt); return; } + else if (code == BIT_NOT_EXPR) + { + tree n_op = gimple_assign_rhs1 (stmt); + if ((stmt = dyn_cast (SSA_NAME_DEF_STMT (n_op)))) + { + code = gimple_assign_rhs_code (stmt); + if (TREE_CODE_CLASS (code) == tcc_comparison + && tree_comparison_ordered_p (code)) + { + this->code = invert_tree_comparison (code, false); + this->op0 = gimple_assign_rhs1 (stmt); + this->op1 = gimple_assign_rhs2 (stmt); + return; + } + } + } } this->code = NE_EXPR; From patchwork Tue Aug 31 13:35:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1522632 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=RJBfs946; dkim-atps=neutral Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GzSsb1ttJz9sW4 for ; Tue, 31 Aug 2021 23:37:07 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C1E613858013 for ; Tue, 31 Aug 2021 13:37:04 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C1E613858013 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1630417024; bh=4O1J08TvCmUngehT6ua+k4aK4dHEc2l6w/xJmYr30oY=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=RJBfs9460T2UUEc8WqVUabHuj7suYLzvTm725FvnuxbcAkqfQ446EHtLhM5S1amAS EXx17QS6TUwH5QEWk3v09T9GTakskmZRfgKwLECrmtplHAPIIWQFq50BiDSXq79BZw epX1XX/Iq+0QNdqhu1CCprBSkCp9WQ35XS5jjx5Q= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40054.outbound.protection.outlook.com [40.107.4.54]) by sourceware.org (Postfix) with ESMTPS id 884793858409 for ; Tue, 31 Aug 2021 13:36:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 884793858409 Received: from PR3P191CA0009.EURP191.PROD.OUTLOOK.COM (2603:10a6:102:54::14) by VI1PR08MB4384.eurprd08.prod.outlook.com (2603:10a6:803:fd::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17; Tue, 31 Aug 2021 13:36:06 +0000 Received: from VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:102:54:cafe::a5) by PR3P191CA0009.outlook.office365.com (2603:10a6:102:54::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend Transport; Tue, 31 Aug 2021 13:36:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;gcc.gnu.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT004.mail.protection.outlook.com (10.152.18.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.17 via Frontend Transport; Tue, 31 Aug 2021 13:36:05 +0000 Received: ("Tessian outbound 8b41f5fb4e9e:v103"); Tue, 31 Aug 2021 13:36:04 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: ca690377300fecc3 X-CR-MTA-TID: 64aa7808 Received: from 9bba67777500.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 6CBF1615-4100-4E7E-B966-9277ACDC9A75.1; Tue, 31 Aug 2021 13:35:58 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9bba67777500.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 31 Aug 2021 13:35:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FvMiDomXVtogZduAe1lUETHDD2ML46ujpr20AaYWs3WWoZfPbEzrrJ0lyrr0+y9PbWCX2s5aYoFyfFzKpoemZbvJCLb6NMKRNI43O6JC8JgHzczZXzHqa3eVs4dFIzPGhNJXwSr3JXWPDO3GzQesnQYgusnXg1JrgAn1d8YWJQIOPKcoDE6kvF3o3QgpytVOgyOnop4o0jKXhWAbKXK+oHtQ18QouDwxtZQiMTY40veLLtnJbgaBuSfkKqtsLJfsNoMGTZGM0svuRuOdPe3yifS8P/gbRu1MnO6mCpr8w4D5q4VEEqnPL5amtkXUJEBxTP5y930NmzTF6EYhgYVvLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4O1J08TvCmUngehT6ua+k4aK4dHEc2l6w/xJmYr30oY=; b=VDCz9jsLCWe+322VcnQm+UeEkYkQV81jpD9sR4eFizH/LtTsqxf8X9zT3R3e/MDhEBq6LIJKP9xNpitZjTnbj9UgDGu8M4Y8h4wwt8QL1pG2i0pqclNdZM7adzRmRWGDtZRoSihzDToNURQ30KL0o0nxnEI7Bhg2d0crRCcBuNaWTzpOtMxZ5IKhtyO0UEkUqgxpXVslA1idtzJJKgONqcwIHZBLNf7Yr6VJJUQY1q1kXOSFEjaupQ8rxkIUDM+yzEyUCU8NDYktdD16AeqXnNh27+tibxflEqcUq07psWsxzpz4fccaU1RZ7BVaa0jvU5LmEJ5y0oQI6g2GqCmgwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by VI1PR08MB3936.eurprd08.prod.outlook.com (2603:10a6:803:e4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.18; Tue, 31 Aug 2021 13:35:55 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::bd45:5ad5:f666:272a%4]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 13:35:55 +0000 Date: Tue, 31 Aug 2021 14:35:47 +0100 To: gcc-patches@gcc.gnu.org Subject: [PATCH 4/5]AArch64 sve: optimize add reduction patterns Message-ID: Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: SA9P223CA0025.NAMP223.PROD.OUTLOOK.COM (2603:10b6:806:26::30) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from arm.com (217.140.106.53) by SA9P223CA0025.NAMP223.PROD.OUTLOOK.COM (2603:10b6:806:26::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.21 via Frontend Transport; Tue, 31 Aug 2021 13:35:53 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c8a12eaa-4c40-4f58-d3af-08d96c844857 X-MS-TrafficTypeDiagnostic: VI1PR08MB3936:|VI1PR08MB4384: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:2887;OLM:2887; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: RtwazozbA7ceBchQcNbY79MKDYn9GW6l9Dw+R17Y7v1O5qUegX3sndqYVwNn+KaXDD+hN2zpaT+3nIUOqBhwxIiDovsmF0DUIltT7QhorQFSopGiLdp+Xw7eCD/TS4DKVSmhzJnxK3STJ21ny14oyTmNePxIqkyIH5rHs/elEeAP1yrFNlvOtEu5FurJSvs3MbVdWAAdSxJVBWK4sJdEVwtaRqFUaBJdXnEF/OJUMlJtMP6fVwYS/4gWSBzX8QzJm3mbhfGNPPVOJUHDsOhzgkkc3pEHx11cY8W3WpCco61cf3t8oVYUyztwZeoKfF6gaarz+LQE1VvvVPecUCxNG9c4KiFcdtcSWgJc39aLYxFdLHi3iH95EXcC2uTeVKXwH9kbrnyyZ1st6J+gBUW9Ba+uEGenISfHjlitJJgZvpc+iNHiVN9OmGpZ+6aRoQBRrEapcT9u/Iy+ZctbW/r/3h27MX3bVcyJLTBzcTbnxF4M5/gdBuHD/CdjdfZYSD0Abe0lim8Yo4WRmXZdajLLZW0dd0QoNtK0lzvg1sAvCb/pK3QyP+TaNvLt+M1f04HGfoty6KhGuzTdYpsE7+ky/6pMe30mPMgCdh/+tH1xQddZkqYOSh0LlI+Ktp+1q6Ooqc4obVPMq4UAE1+UyXPoxOXPzPq6a7p4p5tbFJ6J6ZV3joN/7ALyKAJE9ayofPn42RS4mJnMuRcrUG66gI/LYhXf5oXry9DKGdtmYYiUV0Atx52lh3cH1qAgtTkjDSgy2val+i4it1nMNtulTZBqP8OqeIb9UoTD+l2Bf5IWJy4K5ki/Bsrg2/7+97267VGzP11f3Rsr9ARlDqnrVa1JVygUsLJhh2rYgHAfHoPQ7YY4w7mHDOqBtGwVPXFmGzkr X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(39850400004)(346002)(136003)(396003)(376002)(38350700002)(33964004)(83380400001)(478600001)(86362001)(4743002)(6916009)(7696005)(44144004)(52116002)(36756003)(55016002)(316002)(38100700002)(5660300002)(235185007)(26005)(44832011)(8886007)(8936002)(66616009)(2616005)(66556008)(66476007)(956004)(186003)(8676002)(2906002)(4326008)(66946007)(6666004)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?eSLQiIxiqj+UeDzjwR5J4x55W5RV?= =?utf-8?q?xvxxR1iBJ8+eanpFmN25uy/lT/yxR+cTeY7/J5+e1IpS4mnwQsozU1JYufK9im7Du?= =?utf-8?q?5YJSglX855/hmETVLm7sOPtOMwuBLj9Tp0QyVNdiSYr2uPaEMmlZaBidKf6g9OPjQ?= =?utf-8?q?2xJSMj/7RzlZXPrDDo6v6mnh0frZB9a7UWCCQb+jeRupScFHWa9VSg7h+dAOn7Cem?= =?utf-8?q?Q55jkEdovNynzPlAZiTmvydFgg5GXPcufVUKIz+GwjpGDU7EfRKYkXe7OvydGJAO0?= =?utf-8?q?xECmOcwZypFdQbYvxjBA6xrCMnSd9vYIjybGb53yX9GNZ5eXVkdVv+2Yhyf+oRIr8?= =?utf-8?q?HDHKPy/AChNqlGLz69x5zJzs4x6ZodSTk3NzcTsUsaZSKmMAJXnCk0BgEf+3KxQaR?= =?utf-8?q?39un68XU+cSf59lsmr84MxIf0EDCQ64zFIJJoSGHGMNqv6PjSZL2dYupjujsA3wKp?= =?utf-8?q?3egNqL3NjK0FIdsnEurs/J5zynddl4topHQyQxwMR8gePs3jl6cl/IjWFXVPJYyqo?= =?utf-8?q?tBrTKV+U6YS9+CjfsuAr+hQqJd3bq4AbgizcWVsUWrLDdXVIq0tKF+1E2C03MC8Y4?= =?utf-8?q?ZCz1Hg31ZE7xehzDdxF9A0Zjt0Tdz3/t4kk60SIuxINqn504ZD7LXrmodwRJeWwFC?= =?utf-8?q?EfyVW+2v6bnXicDVQQk26xAX3OTE04Y4Eu204xidK6Vzvinv+AusJYWPN37D7jPFC?= =?utf-8?q?4qsS4nAnqpIxJuFbkoz3/q4XjhCxgqJj/5YCW4vNKQ/R/J1rROAydrNgtR+Mqv0Ii?= =?utf-8?q?0Zjj4r7RVGkudl5+9islTNG8wtGY5S8cw/eiY62zPa4U1vkXyim7s373U1LwNCF8p?= =?utf-8?q?+me1BD6YJGFPQQSVS/47e7E2XCDIpDBYp/9t9YlX9jCbiQbczq8ntF+eqKdmfd88R?= =?utf-8?q?A4vjOnuv4KCJ8XskTtZA7BduBw9Oj+bTq5s/zECktjY2Q6iHa0k7mEicPlz02w/uS?= =?utf-8?q?rYZC2EwZiJlJwp4XAAOj8Fep794pLcnv2uaDr0/3m0e+Fzzi29OacMFxA+ehrSpZ8?= =?utf-8?q?RugQU2KQ8OtDVGXDuYEsx1WurEZM6t7oGGXUJZ50GeT/r31jlOTOg26KaasowYnpX?= =?utf-8?q?aMwBeLOisf52rGA8kdkiJ6T0NNkc0GeF5hmqqFa0WldAmMKC6S/jeJqvshfpSzCuv?= =?utf-8?q?ggqYRw8SsUvZEHshOGio38fz+xsXi09Gk9ujs/zeADG2wTQhDzFNpDKfk4m/PxLAA?= =?utf-8?q?O1zicnZLoLGVLSMKlwu21i5Wjl1GGRY6cEPhETHJOy68Tti5m2mt6r23PeU76Ubx/?= =?utf-8?q?CvlAXMfP5DUJsS5B?= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3936 Original-Authentication-Results: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: c152079b-137b-46f0-b878-08d96c844259 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: c8idwFsAqH4KLk84TXGG+UWG0l7zr97CxfTeyofGAJDregfuRK5Wrr18Ou8IWzyiYAysSavGsyCkC4ZaFAed7lwpQwRLKar0cPUkEH1tIEkH2f0w3k9yGXIEE5zU3n4lnnKgIZl9KIGDPdKPvDjiAdZnZi2b8M91iA3AChNzwtjmTBiTzE+VL8D/DysT/3n8BTUR0dwNvetOYF7rAvVbqI4djAatyov3rfMVtsvIOo2dmP4Hwzd6eK0isSEoD7G8F+3JaXeWmQybctFhObiMlUp1OgTexsX7NU8fWSKMpMKrYM+9Dth1LBIPS7VFGL5QsHyzU/RCY/mxBwO6P+Dsej7PMPU0Rf3hPAeltKxgebPRkYN4k6g0Ynz81l+gjByvmHwL+OHWZ3FXNcE8uw0LJ0h68QE17To0preNWNlQUqNZsRgeSSfNkcdmj63KvEeyAj7u9GiKtFhB86grz/TmswZVVyF4/jNm38vB5SQ0dDUIjPv7g1x+QQuTeFxGRa6JD20u6iVlH/rezHiInP7Z6w+VDWXl+Se7iRq5u+wp2GuLI+ptTV/5+Zcy6B3y+Fz3VwfIUQhAErjb5ZD10WCFmKkZ1IIAz4xKmBuCg4uecsjOlapSo4OSBhFGloPEwg2CBh7LFOv4kzek2RhdMFGb2oVq8jeGL2PgsxuM5FQMhlYyK2MDW+ZnlZL17Y+aEW03UR+kBXZvLi7nW8AFbQUrZMtqaUFNCMiQyLIutp9+HdLUsBpCsh1t+Ftlli8p3SRLK2Dr3tqY/EO5eabOP2BrvrWn8fnp5e4WwNZJk1HOm4PwKhx0uMswrAh4AqOxWZemnlcP/FBKBARW5xyTe+lP+pRYrdH7OKEuygqkfV1IqW6FogI6TwGB4KfBmdXLfg0x X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(376002)(346002)(39850400004)(136003)(396003)(36840700001)(46966006)(83380400001)(6916009)(36756003)(956004)(8676002)(82310400003)(316002)(356005)(2616005)(8886007)(36860700001)(66616009)(82740400003)(4326008)(47076005)(44144004)(44832011)(55016002)(81166007)(4743002)(7696005)(235185007)(70586007)(26005)(5660300002)(86362001)(6666004)(70206006)(8936002)(186003)(336012)(478600001)(2906002)(33964004)(4216001)(2700100001)(357404004); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 13:36:05.3774 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c8a12eaa-4c40-4f58-d3af-08d96c844857 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4384 X-Spam-Status: No, score=-13.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_LOTSOFHASH, KAM_SHORT, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Cc: Richard.Earnshaw@arm.com, nd@arm.com, richard.sandiford@arm.com, Marcus.Shawcroft@arm.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi All, The following loop does a conditional reduction using an add: #include int32_t f (int32_t *restrict array, int len, int min) { int32_t iSum = 0; for (int i=0; i= min) iSum += array[i]; } return iSum; } for this we currently generate: mov z1.b, #0 mov z2.s, w2 mov z3.d, z1.d ptrue p2.b, all ld1w z0.s, p0/z, [x0, x3, lsl 2] cmpge p1.s, p2/z, z0.s, z2.s add x3, x3, x4 sel z0.s, p1, z0.s, z3.s add z1.s, p0/m, z1.s, z0.s whilelo p0.s, w3, w1 where the SEL is unneeded as it's selecting between 0 or a value. This can be optimized to just doing the conditional add on p1 instead of p0. After this patch we generate: mov z2.s, w2 mov z0.b, #0 ptrue p1.b, all ld1w z1.s, p0/z, [x0, x3, lsl 2] cmpge p0.s, p0/z, z1.s, z2.s add x3, x3, x4 add z0.s, p0/m, z0.s, z1.s whilelo p0.s, w3, w1 and so we drop the SEL and the 0 move. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * match.pd: New rule. gcc/testsuite/ChangeLog: * gcc.target/aarch64/sve/pred-cond-reduc.c: New test. --- inline copy of patch -- diff --git a/gcc/match.pd b/gcc/match.pd index 19cbad7592787a568d4a7cfd62746d5844c0be5f..ec98a302ac773647413f776fba15930ad247c747 100644 diff --git a/gcc/match.pd b/gcc/match.pd index 19cbad7592787a568d4a7cfd62746d5844c0be5f..ec98a302ac773647413f776fba15930ad247c747 100644 --- a/gcc/match.pd +++ b/gcc/match.pd @@ -6978,6 +6978,18 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT) && element_precision (type) == element_precision (op_type)) (view_convert (cond_op @2 @3 @4 @5 (view_convert:op_type @1))))))) +/* Detect simplication for a conditional reduction where + + a = mask1 ? b : 0 + c = mask2 ? d + a : d + + is turned into + + c = mask1 && mask2 ? d + b : d. */ +(simplify + (IFN_COND_ADD @0 @1 (vec_cond @2 @3 integer_zerop) @1) + (IFN_COND_ADD (bit_and @0 @2) @1 @3 @1)) + /* For pointers @0 and @2 and nonnegative constant offset @1, look for expressions like: diff --git a/gcc/testsuite/gcc.target/aarch64/sve/pred-cond-reduc.c b/gcc/testsuite/gcc.target/aarch64/sve/pred-cond-reduc.c new file mode 100644 index 0000000000000000000000000000000000000000..bd53025d3f17224004244dadc88e0c68ded23f12 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sve/pred-cond-reduc.c @@ -0,0 +1,18 @@ +/* { dg-do assemble { target aarch64_asm_sve_ok } } */ +/* { dg-options "-O3 --save-temps" } */ + +#include + +int32_t f (int32_t *restrict array, int len, int min) +{ + int32_t iSum = 0; + + for (int i=0; i= min) + iSum += array[i]; + } + return iSum; +} + + +/* { dg-final { scan-assembler-not {\tsel\tz[0-9]+\.s, p1, z[0-9]+\.s, z[0-9]+\.s} } } */