From patchwork Tue Jun 25 12:18:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1952074 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=alszccn2; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=selector1 header.b=alszccn2; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W7kQt6sJnz20X6 for ; Tue, 25 Jun 2024 22:19:18 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5915638425AC for ; Tue, 25 Jun 2024 12:19:16 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on20601.outbound.protection.outlook.com [IPv6:2a01:111:f403:260d::601]) by sourceware.org (Postfix) with ESMTPS id 7C742386182A for ; Tue, 25 Jun 2024 12:18:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 7C742386182A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 7C742386182A Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=2a01:111:f403:260d::601 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1719317933; cv=pass; b=un8WTz2FQcityb0QWxTlUM8q/GkjsL8UZLvI9CwaLxmY5U+qyH2MNnGsGQvgi4GAIfaiuuAI9Y59XGoPO9SxogS0UILIWPY4yrZfkplj0AuPw05v9zuTvIPaIsM0F1HHJUnY48n+xxR7XrlmmbCd7ogPCiiSL+2ZaRHfRs6nfsI= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1719317933; c=relaxed/simple; bh=sX65J1N8w6ErZ01WxFYkLTvOqc5Je8u68U13On2On8A=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=GmrttJEHQ+W3WVVr8LC/GLNuh9malu4up5ZFhUkFABTEV6wS6FW6wPi+tEm4ZVMm7qxwZe1VljwPyGpCvh2qKhBGHFUifiZdKSDH1Cc1V4w+FspMefCOcboow0KGjncHXlkbGJQksO/zrSzh+B6rwCHUhFxOmnDv4aSqRMjlw4U= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=jsedIB27+z+8O4GMTsJWUfNJFPQUeFHP24KsEVcO+3i3CLA7m2A9rzb+gaD068uVLCweGrBhv/S8OR7NN3Z2kswBFO+DlQ2U6k/wJ2BIT/QMbW+HUsenlBFfkmqyaS9tfnYyYgA7gjg3nFfPF8lc0ayB1BAvag4E0oiEJ7B1n7YAv1ogl6dIFAPt5M2zRSe1Rn/4vaadAC0Vzw2jnVYf8wyHRiwyRm2u3XHy5Zi3zSVznsiDAPF8RD7ol2ak82nUExBZTccxFy6d+RHBl3YtCpS/oSlwmYiaFqevtrr9kbG0j2Jh0nMDxBFbJGdmO8OQsmGQAa9PvxOsIwSYnU1J3g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UntpWsGV/uOBjr54MfcGsCY1mZ2OitTopuI7DXnew8Q=; b=fP+aDTKRLTTYob4K6oYrGn2bQIJxEZMM8YbtLKbyuSLSHzHjEHs66RWJrwPwvtLWlfCUaaG0T2MPG5kkJV/n6BDt44om3vZVPEuzYdiSZTIf0yNjy1lqqQKPyKLoFJxIdF50i4Ser0pSQYmPIiZl7XTCb+S1m9zp/3RskLsFqtAyUPcp5MJf2a4cM73k8oM8p4RXd9tF82Qs0icvAIBEnMvVUVreZgFOcJ6dDBe1n0K+D6q3DDv5etEF6G9fswCmuyEoNdKbgzxflcYfDznINEdwwcRiC1KtKQGRUS9jS825ntvxnwZqzDVPVl5zROtKlyfzWYKBVLIyV318EAO/Qw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UntpWsGV/uOBjr54MfcGsCY1mZ2OitTopuI7DXnew8Q=; b=alszccn2u1Ww3eHCjjUsoXTi38vW4O0s8byGq0cy01L7qjK5dY2IBYoQIMMumKXCV2oTvNH7ci5vKe69zD96RIYveAtGCu8L1i3DPNsWt8m95Iw/JZ4Fj1NSaDK+yS0v552AqOrs2sQ57Adbba2P7SIDZKEnfJd54lryxw5HXk8= Received: from DB7PR02CA0033.eurprd02.prod.outlook.com (2603:10a6:10:52::46) by PAXPR08MB7333.eurprd08.prod.outlook.com (2603:10a6:102:230::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7698.30; Tue, 25 Jun 2024 12:18:44 +0000 Received: from DB5PEPF00014B88.eurprd02.prod.outlook.com (2603:10a6:10:52:cafe::cc) by DB7PR02CA0033.outlook.office365.com (2603:10a6:10:52::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.37 via Frontend Transport; Tue, 25 Jun 2024 12:18:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5PEPF00014B88.mail.protection.outlook.com (10.167.8.196) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.7677.15 via Frontend Transport; Tue, 25 Jun 2024 12:18:42 +0000 Received: ("Tessian outbound a2b2216f02e2:v347"); Tue, 25 Jun 2024 12:18:42 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 093bfcce631f247e X-CR-MTA-TID: 64aa7808 Received: from f0deb20213c8.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 80661480-21CA-49E0-BB37-94EB6FAAF1C5.1; Tue, 25 Jun 2024 12:18:35 +0000 Received: from EUR02-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f0deb20213c8.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 25 Jun 2024 12:18:35 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ll7lwd/u3NBWP/B5clsnqPvpRLz2oxcP6op2s6RPUPrHSo6OdAC4+CEWGknl6rujW1S13drgIVztpks/FXfUpyJD1w+DIPyHHLZZXLcl+4uCbykhUHjOME9XhfyEQvWWjltu8ac3ZGvPZlOWWyakamur7MrGLduHzVszoLN9p5AeJ14JGKumjKPgyD6NybyX/b9wkB6WBW0e0a3mmcGD23XzMRwfWEpk2QEMbOfHSJYZShVFs/BVDnhaWsgt7xwy435MhKMUF8RUpeauXB7LYEBYWJLoliWTEcPpas7ZvVuJ0g5FxApi2JIq4rSoPFcm0eUMgNt7Z+ioiCxMHUQ46A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UntpWsGV/uOBjr54MfcGsCY1mZ2OitTopuI7DXnew8Q=; b=mKtuTtPjgrkIlJQVhZf/kOkvPC1ls36mfJ7dbR0ke2j10JkvhTHhEYyKkSDxThHudOPfDgZVHcyWdyPVlVxORWiik3AOL/iC01K1VlY1BDZ3ehXB8FarfnWHHESv2Ksnm3O1jzH+0BhR8yi4SJhSTzJcIu4J6uvfrc04T0fwfHSt6GM8uCEO6A0/i8qpwkzS6q7I9Q0NXExvnREnM9CmF/NxEmpfFvWtAICxZ7oSQHsutBbrqXSpkQi/sxKhHMSrXGvK/rg8edZOqlpdwfhVfeOsro63EP5bon2EOh1yrd2hEl4d/8vOmeddF5jDCxN7A234Bx0lC98GqyI/+yy9OQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UntpWsGV/uOBjr54MfcGsCY1mZ2OitTopuI7DXnew8Q=; b=alszccn2u1Ww3eHCjjUsoXTi38vW4O0s8byGq0cy01L7qjK5dY2IBYoQIMMumKXCV2oTvNH7ci5vKe69zD96RIYveAtGCu8L1i3DPNsWt8m95Iw/JZ4Fj1NSaDK+yS0v552AqOrs2sQ57Adbba2P7SIDZKEnfJd54lryxw5HXk8= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DB4PR08MB9238.eurprd08.prod.outlook.com (2603:10a6:10:3fa::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7698.30; Tue, 25 Jun 2024 12:18:32 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::89dc:c731:362b:7c69]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::89dc:c731:362b:7c69%7]) with mapi id 15.20.7698.025; Tue, 25 Jun 2024 12:18:31 +0000 Date: Tue, 25 Jun 2024 13:18:29 +0100 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH]middle-end: Implement conditonal store vectorizer pattern [PR115531] Message-ID: Content-Disposition: inline X-ClientProxiedBy: LO4P302CA0025.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c1::13) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|DB4PR08MB9238:EE_|DB5PEPF00014B88:EE_|PAXPR08MB7333:EE_ X-MS-Office365-Filtering-Correlation-Id: 450f46f2-76c6-40a9-4d47-08dc9510f3e6 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0;ARA:13230037|366013|1800799021|376011; X-Microsoft-Antispam-Message-Info-Original: PtSNbZqbHIGKIYZCwpfQ3+9ybwuhWIZvIzDo1BZi/VPXroWTzrc9En5u8N9x/gVeWRM5y3G1AxKxdyMWFr1Kxx25JOlnvY3s64vJ1QwZGnYMaqyRGCDIQzuHDwbIPxoxOpJMGumZp5KZPkrF5AmlQd9HBVEgAn+jDH1sl18t4pRWyztTZlAwU7QgOszgvgSl6zwaAh8g/CgpdVKod1XtdrqE1LMj+xep4sqdOYQhjGFdW8Ku0tgX197kUZhdReFbjOvbj318favssAQZhZc6aI+CqNrVanCompzfvfwOPHONqRGg8GdkWGHkSv76AzaljP/mdlp/9K+bmEQNHvv1EkTxyvIQTrUBAeiXkmQDYF3TS36JSn/jScVESAfsk+etfYemAppl5M1yAL/Ze5lD68msZTQzy7cMXthkHUlH1oyX9/wxGt7ggkZ+vKmF1fOKbIU5Nrm0afSt/O+uEA+pQFdE7i7JZXKm5MEvHGXL+MqfTDlQ8Nl5tqfuDJG4ZfhaNWrnutYJeYN66YSPNofhte1wltTwP0XZL6MgrCbvKFB8MzEV9SLW8N0R02yjH1lsd1hvM+XO0I83nlVZmuaJ87kiUNv5ZntkNp2vxvY7eJYy2F8o6NL1opKpKqQwOveiHv/EQn/1wtbiDhP1VTvxYuiIb4GXmFsD5oo58Od42JVPn3nVEsJEG/D2+xI4re+ZCexeBCp+2nGutqadG61JNbTS16ZknJ1xojYDOat7i/Hl2ETZSbAlIm6zY01O+ygqE49HRgIUlvw0+GJrSAFuILwl5PMKzv/spjScUOqG3UcHE9m8vRUpafKwmO2YFq17gutbVE7U9ViowOQSfOcDUWzPZ7E599oujHadQV8mGsDCdVfS0LqY/WmsPBt5re+ubLynF3mvq4DiC9F9i0ZuwRMCtcp1P6aKikA7mU4wFgYV10X5ioGbLDdwhElWk5g5E0wtk3l6h4Y0HLlbPF+81ZNZq8NmphWFojmSmOxaVIDvfD4YoRLqJ4QCa3ZeXS+8uYgbOykuYxGSg34EwosMy/pr+8HUAkLwbnKbXhW98OAN2vPWfTRHlM3sXrPHmNfJJ/lDRRfD+qeichwD8DYM1+mFMJHLFpSruI83tU57tV2Ir+trFnVlpAHCN0s9nTNI5c7IgKCvmFGf+pCLVb524GNDWyNn/Uo6UtdeQbOAZvXZjN2izha8UPq21PM2sLI92lV39kUzTnH0Z9yGIDVQeMwLlpx5DnmejnLEk1AXMKJ09QNfsy+U4FD42Q2rxD3m965ojFZeihlNHrH5DCFrDA== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230037)(366013)(1800799021)(376011); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB9238 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5PEPF00014B88.eurprd02.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: eeeaf266-b2e1-421a-a6d3-08dc9510ed81 X-Microsoft-Antispam: BCL:0; ARA:13230037|36860700010|82310400023|376011|35042699019|1800799021; X-Microsoft-Antispam-Message-Info: =?utf-8?q?gm22uHtdKIxtATAmWOQSyHia17h2HSl?= =?utf-8?q?U0xEA/8WBWittKhHRHjBVxLSUbKcHUGl5e9rUr7awZD4YLasEWXuRgvF/V3zjHvHt?= =?utf-8?q?HCbb6k08QfRzilE6DRifQ5sazadPF9H54HgqwWyujgB0/muN+l24CVp1j68knYZYN?= =?utf-8?q?tyaH49NaDpLNamFQDZiEOMJcN4Gr8qMLzfkHTXrrLDvoawENPQvwsTGL/PK4rWoPi?= =?utf-8?q?usDDe63A/NNZ3eu5jbe03OjCknyr0CBZhapBm0lFKAJtUwVP655xXdBNw8nSeLSfs?= =?utf-8?q?hiRupdsufYRRcKz3slcrhZ7JB3vBWY+JRCoSrka/EKOEf8plV4HxXytgNVOnaKjvP?= =?utf-8?q?+QmZGAP8TFewOlrIOAoMF4HDw0pT6JkMyHdt8JcMQkT0SnTIuU6ipTvx+0W1tpUdM?= =?utf-8?q?BafYXws2SmGMMbnrJYcOV3sCrbTEdYkRM3b6p+yPy4EYUN3XBa+XLc57kzb22M6Bp?= =?utf-8?q?eSoENGUpgtZWX9NjlVBuZ3duuImVVvk7pfjWXqEAzhSkccBgbdJ1B8U6XDPXhCvpw?= =?utf-8?q?TOc3+6S4/2dnrtaHkT1T6bSefDp1+LpWWjCziFbVeWvS+9LZRpKNWLFKJfFtdPDLW?= =?utf-8?q?XI0YYkLK1mXKY6y2ZhCtOfZGw9fVRUV8xrJF9Z+eh4BFWn/GH/MIHl15OAHR7DpU/?= =?utf-8?q?70VHvvNqm40sgN1vlvxfis72hqemulNeidWevjL/JPHJpZ6wmZl8u2MnUS9s1H2I9?= =?utf-8?q?+AcdIF6OeUi0YOTKicsyOyIbEuc4E5ISsXkxLg6HeWEjTz5HXla6FaUaB4/W13JoG?= =?utf-8?q?Z2eIEumUWuL/KQnmufjbv/zWEZ8x9tAPAgVCn01/rw5KjrVruUhA4rcEHjKL10jld?= =?utf-8?q?f7Hsmd/pJgcS+HVIF6scNO02F/816orcDLy7nTSaHSHHdrOB16EYDWbC1tdOkZQoI?= =?utf-8?q?FARNSfQTt7T+KpoUvbePzu8dFc6g/IIHTYPFfd7tmUkEg6jzlZSq26524e5LZrpt+?= =?utf-8?q?FB99rx8Yl00HY7c/qV8YsvF6qrdOd01Ypv4Bwnkb+cSZnQnrUZh1ppwDZrlRMcrIF?= =?utf-8?q?lYqHijQON3rt8MJOB/uwMBbMocWsM4REg9pA94HNXsd24pXqHS4/WcB01iIHGG9eg?= =?utf-8?q?4ry5PMPY1/nCqszuIUP+doRSkR4Owc0ZcZ5m+DG0c6F7b6HrfHuQf/AVbxUdSlKtn?= =?utf-8?q?Xh/SK8mSJXP46pLuQDWUFnwdJcQ+Qj6fkrnSOldamFqHgqA+lG0wl8bBoE+scV77p?= =?utf-8?q?I4lptPASzDOS6akfoRQacQUHxLRKijIFJIPBCGgg/c8+m2T+OqTegYg54W5tJtVfj?= =?utf-8?q?1baJ9+EeHsAWSLe75jk2EkeviI52R3a/Kcji2Fx/eXcXyqsX3TQNTFyg=3D?= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230037)(36860700010)(82310400023)(376011)(35042699019)(1800799021); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jun 2024 12:18:42.3583 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 450f46f2-76c6-40a9-4d47-08dc9510f3e6 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5PEPF00014B88.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7333 X-Spam-Status: No, score=-12.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_LOTSOFHASH, SPF_HELO_NONE, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Hi All, This adds a conditional store optimization for the vectorizer as a pattern. The vectorizer already supports modifying memory accesses because of the pattern based gather/scatter recognition. Doing it in the vectorizer allows us to still keep the ability to vectorize such loops for architectures that don't have MASK_STORE support, whereas doing this in ifcvt makes us commit to MASK_STORE. Concretely for this loop: void foo1 (char *restrict a, int *restrict b, int *restrict c, int n, int stride) { if (stride <= 1) return; for (int i = 0; i < n; i++) { int res = c[i]; int t = b[i+stride]; if (a[i] != 0) res = t; c[i] = res; } } today we generate: .L3: ld1b z29.s, p7/z, [x0, x5] ld1w z31.s, p7/z, [x2, x5, lsl 2] ld1w z30.s, p7/z, [x1, x5, lsl 2] cmpne p15.b, p6/z, z29.b, #0 sel z30.s, p15, z30.s, z31.s st1w z30.s, p7, [x2, x5, lsl 2] add x5, x5, x4 whilelo p7.s, w5, w3 b.any .L3 which in gimple is: vect_res_18.9_68 = .MASK_LOAD (vectp_c.7_65, 32B, loop_mask_67); vect_t_20.12_74 = .MASK_LOAD (vectp.10_72, 32B, loop_mask_67); vect__9.15_77 = .MASK_LOAD (vectp_a.13_75, 8B, loop_mask_67); mask__34.16_79 = vect__9.15_77 != { 0, ... }; vect_res_11.17_80 = VEC_COND_EXPR ; .MASK_STORE (vectp_c.18_81, 32B, loop_mask_67, vect_res_11.17_80); A MASK_STORE is already conditional, so there's no need to perform the load of the old values and the VEC_COND_EXPR. This patch makes it so we generate: vect_res_18.9_68 = .MASK_LOAD (vectp_c.7_65, 32B, loop_mask_67); vect__9.15_77 = .MASK_LOAD (vectp_a.13_75, 8B, loop_mask_67); mask__34.16_79 = vect__9.15_77 != { 0, ... }; .MASK_STORE (vectp_c.18_81, 32B, mask__34.16_79, vect_res_18.9_68); which generates: .L3: ld1b z30.s, p7/z, [x0, x5] ld1w z31.s, p7/z, [x1, x5, lsl 2] cmpne p7.b, p7/z, z30.b, #0 st1w z31.s, p7, [x2, x5, lsl 2] add x5, x5, x4 whilelo p7.s, w5, w3 b.any .L3 Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: PR tree-optimization/115531 * tree-vect-patterns.cc (vect_cond_store_pattern_same_ref): New. (vect_recog_cond_store_pattern): New. (vect_vect_recog_func_ptrs): Use it. gcc/testsuite/ChangeLog: PR tree-optimization/115531 * gcc.dg/vect/vect-conditional_store_1.c: New test. * gcc.dg/vect/vect-conditional_store_2.c: New test. * gcc.dg/vect/vect-conditional_store_3.c: New test. * gcc.dg/vect/vect-conditional_store_4.c: New test. --- -- diff --git a/gcc/testsuite/gcc.dg/vect/vect-conditional_store_1.c b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_1.c new file mode 100644 index 0000000000000000000000000000000000000000..3884a3c3d0a2dc2258097348c75bb7c0b3b37c72 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_1.c @@ -0,0 +1,24 @@ +/* { dg-do assemble } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-require-effective-target vect_masked_store } */ + +/* { dg-additional-options "-mavx2" { target avx2 } } */ +/* { dg-additional-options "-march=armv9-a" { target aarch64-*-* } } */ + +void foo1 (char *restrict a, int *restrict b, int *restrict c, int n, int stride) +{ + if (stride <= 1) + return; + + for (int i = 0; i < n; i++) + { + int res = c[i]; + int t = b[i+stride]; + if (a[i] != 0) + res = t; + c[i] = res; + } +} + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ +/* { dg-final { scan-tree-dump-not "VEC_COND_EXPR " "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-conditional_store_2.c b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_2.c new file mode 100644 index 0000000000000000000000000000000000000000..bc965a244f147c199b1726e5f6b44229539cd225 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_2.c @@ -0,0 +1,24 @@ +/* { dg-do assemble } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-require-effective-target vect_masked_store } */ + +/* { dg-additional-options "-mavx2" { target avx2 } } */ +/* { dg-additional-options "-march=armv9-a" { target aarch64-*-* } } */ + +void foo2 (char *restrict a, int *restrict b, int *restrict c, int n, int stride) +{ + if (stride <= 1) + return; + + for (int i = 0; i < n; i++) + { + int res = c[i]; + int t = b[i+stride]; + if (a[i] != 0) + t = res; + c[i] = t; + } +} + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ +/* { dg-final { scan-tree-dump-not "VEC_COND_EXPR " "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-conditional_store_3.c b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_3.c new file mode 100644 index 0000000000000000000000000000000000000000..ab6889f967b330a652917925c2748b16af59b9fd --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_3.c @@ -0,0 +1,24 @@ +/* { dg-do assemble } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-require-effective-target vect_masked_store } */ + +/* { dg-additional-options "-mavx2" { target avx2 } } */ +/* { dg-additional-options "-march=armv9-a" { target aarch64-*-* } } */ + +void foo3 (float *restrict a, int *restrict b, int *restrict c, int n, int stride) +{ + if (stride <= 1) + return; + + for (int i = 0; i < n; i++) + { + int res = c[i]; + int t = b[i+stride]; + if (a[i] >= 0) + t = res; + c[i] = t; + } +} + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ +/* { dg-final { scan-tree-dump-not "VEC_COND_EXPR " "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/vect-conditional_store_4.c b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_4.c new file mode 100644 index 0000000000000000000000000000000000000000..3bfe2f81dc2d47096aa23529d43263be52cd422c --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-conditional_store_4.c @@ -0,0 +1,28 @@ +/* { dg-do assemble } */ +/* { dg-require-effective-target vect_int } */ +/* { dg-require-effective-target vect_masked_store } */ + +/* { dg-additional-options "-mavx2" { target avx2 } } */ +/* { dg-additional-options "-march=armv9-a" { target aarch64-*-* } } */ + +void foo4 (signed char *restrict a, int *restrict b, int *restrict c, int *restrict d, int n, int stride) +{ + if (stride <= 1) + return; + + for (int i = 0; i < n; i++) + { + int res1 = c[i]; + int res2 = d[i]; + int t = b[i+stride]; + if (a[i] > 0) + t = res1; + else if (a[i] < 0) + t = res2 * 2; + + c[i] = t; + } +} + +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" } } */ +/* { dg-final { scan-tree-dump-not "VEC_COND_EXPR " "vect" } } */ diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index cef901808eb97c0c92b51da20535ea7f397b4742..f0da6c932f48d2992a501d0ced3efc8924912c77 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -6397,6 +6397,177 @@ vect_recog_gather_scatter_pattern (vec_info *vinfo, return pattern_stmt; } +/* Helper method of vect_recog_cond_store_pattern, checks to see if COND_ARG + is points to a load statement that reads the same data as that of + STORE_VINFO. */ + +static bool +vect_cond_store_pattern_same_ref (loop_vec_info loop_vinfo, + stmt_vec_info store_vinfo, tree cond_arg) +{ + stmt_vec_info load_stmt_vinfo = loop_vinfo->lookup_def (cond_arg); + if (!load_stmt_vinfo + || !STMT_VINFO_DATA_REF (load_stmt_vinfo) + || DR_IS_WRITE (STMT_VINFO_DATA_REF (load_stmt_vinfo)) + || !same_data_refs (STMT_VINFO_DATA_REF (store_vinfo), + STMT_VINFO_DATA_REF (load_stmt_vinfo))) + return false; + + return true; +} + +/* Function vect_recog_cond_store_pattern + + Try to find the following pattern: + + x = *_3; + c = a CMP b; + y = c ? t_20 : x; + *_3 = y; + + where the store of _3 happens on a conditional select on a value loaded + from the same location. In such case we can elide the initial load if + MASK_STORE is supported and instead only conditionally write out the result. + + The pattern produces for the above: + + c = a CMP b; + .MASK_STORE (_3, c, t_20) + + Input: + + * STMT_VINFO: The stmt from which the pattern search begins. In the + example, when this function is called with _3 then the search begins. + + Output: + + * TYPE_OUT: The type of the output of this pattern. + + * Return value: A new stmt that will be used to replace the sequence. */ + +static gimple * +vect_recog_cond_store_pattern (vec_info *vinfo, + stmt_vec_info stmt_vinfo, tree *type_out) +{ + loop_vec_info loop_vinfo = dyn_cast (vinfo); + if (!loop_vinfo) + return NULL; + + gimple *store_stmt = STMT_VINFO_STMT (stmt_vinfo); + + /* Needs to be a gimple store where we have DR info for. */ + if (!STMT_VINFO_DATA_REF (stmt_vinfo) + || !DR_IS_WRITE (STMT_VINFO_DATA_REF (stmt_vinfo)) + || !gimple_store_p (store_stmt)) + return NULL; + + tree st_rhs = gimple_assign_rhs1 (store_stmt); + tree st_lhs = gimple_assign_lhs (store_stmt); + + if (TREE_CODE (st_rhs) != SSA_NAME) + return NULL; + + gassign *cond_stmt = dyn_cast (SSA_NAME_DEF_STMT (st_rhs)); + if (!cond_stmt || gimple_assign_rhs_code (cond_stmt) != COND_EXPR) + return NULL; + + /* Check if the else value matches the original loaded one. */ + bool invert = false; + tree cmp_ls = gimple_arg (cond_stmt, 0); + tree cond_arg1 = gimple_arg (cond_stmt, 1); + tree cond_arg2 = gimple_arg (cond_stmt, 2); + + if (!vect_cond_store_pattern_same_ref (loop_vinfo, stmt_vinfo, cond_arg2) + && !(invert = vect_cond_store_pattern_same_ref (loop_vinfo, stmt_vinfo, + cond_arg1))) + return NULL; + + vect_pattern_detected ("vect_recog_cond_store_pattern", store_stmt); + + tree scalar_type = TREE_TYPE (st_rhs); + if (VECTOR_TYPE_P (scalar_type)) + return NULL; + + tree vectype = get_vectype_for_scalar_type (vinfo, scalar_type); + if (vectype == NULL_TREE) + return NULL; + + machine_mode mask_mode; + machine_mode vecmode = TYPE_MODE (vectype); + if (!targetm.vectorize.get_mask_mode (vecmode).exists (&mask_mode) + || !can_vec_mask_load_store_p (vecmode, mask_mode, false)) + return NULL; + + /* Convert the mask to the right form. */ + tree gs_vectype = get_vectype_for_scalar_type (loop_vinfo, scalar_type); + tree cookie = build_int_cst (build_pointer_type (scalar_type), + TYPE_ALIGN (scalar_type)); + tree base = TREE_OPERAND (st_lhs, 0); + tree cond_store_arg = cond_arg1; + + /* If we have to invert the condition, i.e. use the true argument rather than + the false argument, we should check whether we can just invert the + comparison or if we have to negate the result. */ + if (invert) + { + gimple *cond = SSA_NAME_DEF_STMT (cmp_ls); + /* We need to use the false parameter of the conditional select. */ + cond_store_arg = cond_arg2; + tree_code new_code = ERROR_MARK; + tree mask_vec_type, itype; + gassign *conv; + tree var = vect_recog_temp_ssa_var (boolean_type_node, NULL); + + if (is_gimple_assign (cond) + && TREE_CODE_CLASS (gimple_assign_rhs_code (cond)) == tcc_comparison) + { + tree_code cond_code = gimple_assign_rhs_code (cond); + tree cond_expr0 = gimple_assign_rhs1 (cond); + tree cond_expr1 = gimple_assign_rhs2 (cond); + + /* We have to invert the comparison, see if we can flip it. */ + bool honor_nans = HONOR_NANS (TREE_TYPE (cond_expr0)); + new_code = invert_tree_comparison (cond_code, honor_nans); + if (new_code != ERROR_MARK) + { + itype = TREE_TYPE(cond_expr0); + conv = gimple_build_assign (var, new_code, cond_expr0, + cond_expr1); + } + } + + if (new_code == ERROR_MARK) + { + /* We couldn't flip the condition, so invert the mask instead. */ + itype = TREE_TYPE (cmp_ls); + conv = gimple_build_assign (var, BIT_XOR_EXPR, cmp_ls, + build_int_cst (itype, 1)); + } + + mask_vec_type = get_mask_type_for_scalar_type (loop_vinfo, itype); + append_pattern_def_seq (vinfo, stmt_vinfo, conv, mask_vec_type, itype); + /* Then prepare the boolean mask as the mask conversion pattern + won't hit on the pattern statement. */ + cmp_ls = build_mask_conversion (vinfo, var, gs_vectype, stmt_vinfo); + } + + tree mask = vect_convert_mask_for_vectype (cmp_ls, gs_vectype, stmt_vinfo, + loop_vinfo); + gcall *call + = gimple_build_call_internal (IFN_MASK_STORE, 4, base, cookie, mask, + cond_store_arg); + gimple_set_location (call, gimple_location (store_stmt)); + gimple_set_lhs (call, make_ssa_name (integer_type_node)); + + /* Copy across relevant vectorization info and associate DR with the + new pattern statement instead of the original statement. */ + stmt_vec_info pattern_stmt_info = loop_vinfo->add_stmt (call); + loop_vinfo->move_dr (pattern_stmt_info, stmt_vinfo); + + *type_out = vectype; + return call; +} + /* Return true if TYPE is a non-boolean integer type. These are the types that we want to consider for narrowing. */ @@ -7061,6 +7232,7 @@ static vect_recog_func vect_vect_recog_func_ptrs[] = { of mask conversion that are needed for gather and scatter internal functions. */ { vect_recog_gather_scatter_pattern, "gather_scatter" }, + { vect_recog_cond_store_pattern, "cond_store" }, { vect_recog_mask_conversion_pattern, "mask_conversion" }, { vect_recog_widen_plus_pattern, "widen_plus" }, { vect_recog_widen_minus_pattern, "widen_minus" },