From patchwork Mon Nov 27 22:40:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1869035 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=armh.onmicrosoft.com header.i=@armh.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-armh-onmicrosoft-com header.b=0giRHmPA; dkim=pass (1024-bit key) header.d=armh.onmicrosoft.com header.i=@armh.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-armh-onmicrosoft-com header.b=0giRHmPA; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SfLDW0Vmqz1yRW for ; Tue, 28 Nov 2023 09:41:47 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8D684385700A for ; Mon, 27 Nov 2023 22:41:44 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2069.outbound.protection.outlook.com [40.107.21.69]) by sourceware.org (Postfix) with ESMTPS id 2FF853857723 for ; Mon, 27 Nov 2023 22:41:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2FF853857723 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2FF853857723 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.21.69 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1701124877; cv=pass; b=Ip3EElfloEJEUxolJ72z95VUZsKFO96JsNxsCG1A0Vh8NjC+avaVGWD9gdOsQBUGN7QnIwK+NfhVpIx8VAgOCQ2X7WTTPVe8xmbwXSrsaGpqwG5vQKR3Ucr8UuXTPHxCPRd+K/HT3M38Gu2l0peIwIkmvXAju1MkY3s6ExQOd0g= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1701124877; c=relaxed/simple; bh=ocYqZFaq9L653w5gkR6ROp+1ncSvKRRmV6sBZqo3fqU=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=kni+6Jx34z90tj/jJ0z2iwOxhD0knczeefMNB6ON/+BGQnTmBk4nS7nWRIVe1Wgbg1QOmgvMe24TiWSEdEH06XM8A3DmLdaCZCDUB3mebc94NXo4hPwytQK0HtBkA5k3s9h2gwTn0mJfQzVk1PpeT3Jw7xQLj8t6pIOQ4Jx6H3g= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=YiC2ucZZpfv4ZKz7sLW9kDEV8tm2WG9AcuDQVolM2leanfJzYCGvx9ESszC69jgtmOKnqzeHxk9n5x7oNjIyaXYUf7ryZ3d3/NYjoWeKd7AphepHLvc8KE7G+qniUVNiIv1WRptH0yBPrS51nOcomFgwYmZAcYCOxLqmQz+x0i4057PSWpYABNUDwGDzkqvrZVziMiW68V2/lEBLHDBv0sGwxN+hLV6r/v19fSSfvLKiYzXtwvvDr7agCnoNuh2wDKkYvvDgSf3VBrX7YSlYqhGufzz1qM8Gp4Ve4QqjYwdV9lmey+SMcm53fx6FzNcYXuzSsiiX3eaibbExrl8xsA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2pBtlf7rKdN3EuLeRDOzqMzWZsRiNZYvN4jARh3Swl0=; b=RmuoeN4xHWeIkPkbS0sZfFEu6x/0CIHLTywRYCpjGeqL4w3wCt1Y+LoldWekgmsKLiEJ/2Y218p/fJKranftI/8iOdPfVgh/Ngi4KtoYMREmAYcc55RsmFNi/UK6++AmJdnisJ1YlLRFhsohzh/GSLLPUChcybczACtez1h11I44ZQ2wzmktdeaBsE9DgAPdT7dwe4+aZsOTx9LR++15U8dgCkAIhEJjHYOtz7fJq8SszZlvHvQN906ROyOJFSuGI48Or8jx0VENMtZNgeACCICwOT3OBp0zx5n+iJFG0PnHugVB8izFEn1BsBh9SdhhXZRDxvMgVrAt25sE4LiyEg== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2pBtlf7rKdN3EuLeRDOzqMzWZsRiNZYvN4jARh3Swl0=; b=0giRHmPAF9RGvHFhtA6cHk49gOEEAC3UnIWjL4p3LE9pyhGPGEV2YeR5/rdTzf31JlbSzsKKl695iIOcS8lK0XY6rSNwqdeHRA0E/EoiQlMVZ9c1lAUYFrwrUO3k37w03KZnMz+BVxBR5gJL9q459Hjg71tEfxeWNltgBQAA6ds= Received: from AS9PR05CA0032.eurprd05.prod.outlook.com (2603:10a6:20b:489::30) by AS8PR08MB8037.eurprd08.prod.outlook.com (2603:10a6:20b:573::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.27; Mon, 27 Nov 2023 22:41:12 +0000 Received: from AMS1EPF00000045.eurprd04.prod.outlook.com (2603:10a6:20b:489:cafe::55) by AS9PR05CA0032.outlook.office365.com (2603:10a6:20b:489::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.27 via Frontend Transport; Mon, 27 Nov 2023 22:41:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AMS1EPF00000045.mail.protection.outlook.com (10.167.16.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.17 via Frontend Transport; Mon, 27 Nov 2023 22:41:11 +0000 Received: ("Tessian outbound 8289ea11ec17:v228"); Mon, 27 Nov 2023 22:41:11 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 82863b9b31605ac7 X-CR-MTA-TID: 64aa7808 Received: from 06749449d60e.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 0AEF5CB9-778D-4ECF-A363-F42D4DE5C3C5.1; Mon, 27 Nov 2023 22:41:04 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 06749449d60e.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 27 Nov 2023 22:41:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G1hDxQ5grNjKZXnUwWZ6ysCCFq4iIj72i5D92FNqeg7fNCScPrcfdCtLIKESNkcTuxjV6szrvqgUecj10o8lYbRTQYR6MjVdEDVkhlScsB2XBX3O5AmT0Zzn5W56XtARuEkspjKK4UXtCe2lmIRdssPfrPkgVq4sEsWVYPD6zl2g08AzKc944cUjeqlWeAig/JoLcUB/F9wfopd3A8hcMNZ9TKFuY7BhSukrVy1TawLVfn2OrNxJyFG0H34XM7H647A4yiEXtOxkwFg9fkXR3awJECoNr1KfjfmFLzNVuRg68QwsGT4rVPYsy3U5og/MH5mtinK/BIY2ChL1rFKnQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2pBtlf7rKdN3EuLeRDOzqMzWZsRiNZYvN4jARh3Swl0=; b=BabehVK9E9MG8w2OoIs0mX/PWCWpijoJcOIFENe22Tz75GsIlwTaHet0YVhvjwS306H+L1tH9sIz0jqRonXslFYsz3rqo2wN5sYMr1AjQfbvj/WQDLeVsUf8TQeM5FghL18CzTDxIp9CpfQ6bpQx+Fcoz4bWO+Ge2Z0ZCaJqywKX/I5wLHZk4mmhdYFOBIJOj1YY748WkYwYHgGTcBtJi96jpScpNnFUciCdUofT1uJLEBc0nN+pUifFjl1Ku5UphDw+6YlCgCQP69HIRB+j3XSalkqEAvx6rojjn7Yvrg5Ji7YA9LBy/RxwIAcOG/kPGAg7MDa6zi6tYl/orX99aA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2pBtlf7rKdN3EuLeRDOzqMzWZsRiNZYvN4jARh3Swl0=; b=0giRHmPAF9RGvHFhtA6cHk49gOEEAC3UnIWjL4p3LE9pyhGPGEV2YeR5/rdTzf31JlbSzsKKl695iIOcS8lK0XY6rSNwqdeHRA0E/EoiQlMVZ9c1lAUYFrwrUO3k37w03KZnMz+BVxBR5gJL9q459Hjg71tEfxeWNltgBQAA6ds= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by AS8PR08MB8250.eurprd08.prod.outlook.com (2603:10a6:20b:53e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.27; Mon, 27 Nov 2023 22:41:02 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::9679:2ab0:99c6:54a3%6]) with mapi id 15.20.7025.022; Mon, 27 Nov 2023 22:41:02 +0000 Date: Mon, 27 Nov 2023 22:40:58 +0000 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH]middle-end: refactor vectorizable_live_operation into helper method for codegen Message-ID: Content-Disposition: inline X-ClientProxiedBy: LO4P123CA0269.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:194::22) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|AS8PR08MB8250:EE_|AMS1EPF00000045:EE_|AS8PR08MB8037:EE_ X-MS-Office365-Filtering-Correlation-Id: 69a5ed36-6d5d-44f4-5abc-08dbef99f4dc x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: ujJXsHKV9e5NiWgbq4Yv5b0sAHFx0jmbAguV3SBomF9X2C//+eosNsv0n9DGu0bzrhr5/I2GfEvwRkgtuJAq3xnWThxpA9UDPdzW9NIiH5MnRLIMNDk3DNcrXkRxbE3fKplUnOn9ST257fPerBJ6AAfR2JP/DBrcrwS1LY6cZyA0zLv4FdYEUQhHXsdusL95ukb+yJrqitCm+eggpEMNX58RzEu+SnQ0hIrEYqjTvUrMGEERY87YL22i0elfbLFXuK+g9Donirc+KosNk7nGgPFR2nvhoXKwQh0hX+syvaWo4NrGfJFn2U+VwvLXJ0zN6CBquXc6NFduWlmNgjF7TshLd44NgScZS5wMxBMwYI5d2fSu8SapQkCFUHJa+p4AMaeauy6KLbHGIoQ/oX3q44XDOOhria/L4efoLdiHRTGnEASU1MHhJebrvbRTG1Qov6tC9hSe20wfG+qNu4TqeTM6X/CQo3r/T8cc+doJj3uummWCcSqBRXmKhV+hpIeNvDcNlJb2nxxADK9ezQ7pMQsZf1b7GTRYVbRXrKf+o01WUjnQW9SnSrdsvuSY3nTXVBtyGnTDCImyD2FmNuiRTB/wH83wMPwdGKWg0418O2Y5DKfDZ8MdPj0nTKnVrbuJvU+hUepb/p2vDowBjLiuxg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376002)(366004)(39860400002)(136003)(346002)(396003)(230922051799003)(64100799003)(186009)(451199024)(1800799012)(44144004)(26005)(2616005)(478600001)(6512007)(6666004)(4743002)(6506007)(33964004)(83380400001)(2906002)(5660300002)(235185007)(41300700001)(316002)(66476007)(66556008)(66946007)(6486002)(6916009)(44832011)(8936002)(8676002)(4326008)(38100700002)(86362001)(36756003)(4216001)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8250 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AMS1EPF00000045.eurprd04.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: c14e5fda-035c-4c36-62ce-08dbef99eee3 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: W7Rc07fTd7mQvZX8DxpVMAWcsF+fzwn+1Nces5Gg9aR9o/kXLd+TZk1LDYPHk+arvQBXb4CTjgzLWiBGNTGdbNZy2yrvRrJgh4W/fHvBACw2T7qLtuKz3yqDbOHtNPjWCOi3fXMheqDBffMJ77KS60TKBsLI34RzpjsTa+eg/TGN3xTsOc1AjkIogy5Stn/jTSX5navKkPTxdU18/qYUBIfJ7xJ8iWkF35CgLeLJd4XHAOEsxbFimRaogAPKmXQOHaF7Rye7HVcX7L0t7b9qJrykQayevww8sh9iYG7V6bacbcnuzlmlP9ofx7EzQblNttB1yiJT6/UbvLk/SqZ39oLC6ux0wk+qaD6GgEdP6GGpBg6mSBlhBFH5cu2Qj0C5DLCAxnRr7++SynzCITL35yP6Kaqxa8On9/or1reWD6RIoBxdlNXipwfI2L369X1uPqb/5PHVrDI0nd4O/b3zrtHvvRfwwk+ag1yNEY1QQSWOAYGwG9uQDtZY4AITI5HPqN8t0bNXycZ9I6ANp1KHwiwSdxVzRoS4sVDmxMu1s+sUJ5d4QvoSajihWPNq6C3z3Foxp4OOBjMlUZESPPwGEWoa3nWtr2Uj41YYIKAIcopBbUyi74UOKPLlTWIGwY1y7DlaKDPRWmrJiF/jHyojFMLXmyIZpm4SszXC2Qvg3YAwKd/mmT4QhdspqjFvpaNrVB/HRLSxWRNMM06CB0z2dgwIg5V58gdMZa1oXHOeMfVq4eI7OkhzBbOialwbd5YGt0nRQcPrqUIemrzZL6Kn77wuUb0acGZh1ukZc7DSgh0= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(346002)(396003)(136003)(376002)(230922051799003)(186009)(451199024)(64100799003)(1800799012)(82310400011)(40470700004)(46966006)(36840700001)(6666004)(4326008)(8936002)(8676002)(6512007)(6506007)(44144004)(33964004)(6916009)(316002)(6486002)(40460700003)(478600001)(36860700001)(81166007)(356005)(47076005)(36756003)(40480700001)(41300700001)(86362001)(107886003)(26005)(44832011)(70586007)(336012)(2906002)(2616005)(83380400001)(4743002)(70206006)(82740400003)(235185007)(5660300002)(4216001)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Nov 2023 22:41:11.9392 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 69a5ed36-6d5d-44f4-5abc-08dbef99f4dc X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF00000045.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8037 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Hi All, To make code review of the updates to add multiple exit supports to vectorizable_live_operation easier I've extracted the refactoring part to its own patch. This patch is a straight extract of the function with no functional changes. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-loop.cc (vectorizable_live_operation_1): New. (vectorizable_live_operation): Extract code to vectorizable_live_operation_1. --- inline copy of patch -- diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 8a50380de49bc12105be47ea1d8ee3cf1f2bdab4..df5e1d28fac2ce35e71decdec0d8e31fb75557f5 100644 --- diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 8a50380de49bc12105be47ea1d8ee3cf1f2bdab4..df5e1d28fac2ce35e71decdec0d8e31fb75557f5 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -10481,6 +10481,95 @@ vectorizable_induction (loop_vec_info loop_vinfo, return true; } + +/* Function vectorizable_live_operation_1. + helper function for vectorizable_live_operation. */ +tree +vectorizable_live_operation_1 (loop_vec_info loop_vinfo, + stmt_vec_info stmt_info, edge exit_e, + tree vectype, int ncopies, slp_tree slp_node, + tree bitsize, tree bitstart, tree vec_lhs, + tree lhs_type, gimple_stmt_iterator *exit_gsi) +{ + basic_block exit_bb = exit_e->dest; + gcc_assert (single_pred_p (exit_bb) || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)); + + tree vec_lhs_phi = copy_ssa_name (vec_lhs); + gimple *phi = create_phi_node (vec_lhs_phi, exit_bb); + for (unsigned i = 0; i < gimple_phi_num_args (phi); i++) + SET_PHI_ARG_DEF (phi, i, vec_lhs); + + gimple_seq stmts = NULL; + tree new_tree; + if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) + { + /* Emit: + SCALAR_RES = VEC_EXTRACT + where VEC_LHS is the vectorized live-out result and MASK is + the loop mask for the final iteration. */ + gcc_assert (ncopies == 1 && !slp_node); + gimple_seq tem = NULL; + gimple_stmt_iterator gsi = gsi_last (tem); + tree len = vect_get_loop_len (loop_vinfo, &gsi, + &LOOP_VINFO_LENS (loop_vinfo), + 1, vectype, 0, 0); + /* BIAS - 1. */ + signed char biasval = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); + tree bias_minus_one + = int_const_binop (MINUS_EXPR, + build_int_cst (TREE_TYPE (len), biasval), + build_one_cst (TREE_TYPE (len))); + /* LAST_INDEX = LEN + (BIAS - 1). */ + tree last_index = gimple_build (&stmts, PLUS_EXPR, TREE_TYPE (len), + len, bias_minus_one); + /* This needs to implement extraction of the first index, but not sure + how the LEN stuff works. At the moment we shouldn't get here since + there's no LEN support for early breaks. But guard this so there's + no incorrect codegen. */ + gcc_assert (!LOOP_VINFO_EARLY_BREAKS (loop_vinfo)); + + /* SCALAR_RES = VEC_EXTRACT . */ + tree scalar_res + = gimple_build (&stmts, CFN_VEC_EXTRACT, TREE_TYPE (vectype), + vec_lhs_phi, last_index); + /* Convert the extracted vector element to the scalar type. */ + new_tree = gimple_convert (&stmts, lhs_type, scalar_res); + } + else if (LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) + { + /* Emit: + SCALAR_RES = EXTRACT_LAST + where VEC_LHS is the vectorized live-out result and MASK is + the loop mask for the final iteration. */ + gcc_assert (!slp_node); + tree scalar_type = TREE_TYPE (STMT_VINFO_VECTYPE (stmt_info)); + gimple_seq tem = NULL; + gimple_stmt_iterator gsi = gsi_last (tem); + tree mask = vect_get_loop_mask (loop_vinfo, &gsi, + &LOOP_VINFO_MASKS (loop_vinfo), + 1, vectype, 0); + + gimple_seq_add_seq (&stmts, tem); + tree scalar_res = gimple_build (&stmts, CFN_EXTRACT_LAST, scalar_type, + mask, vec_lhs_phi); + /* Convert the extracted vector element to the scalar type. */ + new_tree = gimple_convert (&stmts, lhs_type, scalar_res); + } + else + { + tree bftype = TREE_TYPE (vectype); + if (VECTOR_BOOLEAN_TYPE_P (vectype)) + bftype = build_nonstandard_integer_type (tree_to_uhwi (bitsize), 1); + new_tree = build3 (BIT_FIELD_REF, bftype, vec_lhs_phi, bitsize, bitstart); + new_tree = force_gimple_operand (fold_convert (lhs_type, new_tree), + &stmts, true, NULL_TREE); + } + *exit_gsi = gsi_after_labels (exit_bb); + if (stmts) + gsi_insert_seq_before (exit_gsi, stmts, GSI_SAME_STMT); + return new_tree; +} + /* Function vectorizable_live_operation. STMT_INFO computes a value that is used outside the loop. Check if @@ -10690,79 +10779,13 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, gimple *phi = create_phi_node (vec_lhs_phi, exit_bb); SET_PHI_ARG_DEF (phi, LOOP_VINFO_IV_EXIT (loop_vinfo)->dest_idx, vec_lhs); - gimple_seq stmts = NULL; - tree new_tree; - if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) - { - /* Emit: - - SCALAR_RES = VEC_EXTRACT - - where VEC_LHS is the vectorized live-out result and MASK is - the loop mask for the final iteration. */ - gcc_assert (ncopies == 1 && !slp_node); - gimple_seq tem = NULL; - gimple_stmt_iterator gsi = gsi_last (tem); - tree len - = vect_get_loop_len (loop_vinfo, &gsi, - &LOOP_VINFO_LENS (loop_vinfo), - 1, vectype, 0, 0); - - /* BIAS - 1. */ - signed char biasval = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); - tree bias_minus_one - = int_const_binop (MINUS_EXPR, - build_int_cst (TREE_TYPE (len), biasval), - build_one_cst (TREE_TYPE (len))); - - /* LAST_INDEX = LEN + (BIAS - 1). */ - tree last_index = gimple_build (&stmts, PLUS_EXPR, TREE_TYPE (len), - len, bias_minus_one); - - /* SCALAR_RES = VEC_EXTRACT . */ - tree scalar_res - = gimple_build (&stmts, CFN_VEC_EXTRACT, TREE_TYPE (vectype), - vec_lhs_phi, last_index); - - /* Convert the extracted vector element to the scalar type. */ - new_tree = gimple_convert (&stmts, lhs_type, scalar_res); - } - else if (LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) - { - /* Emit: - - SCALAR_RES = EXTRACT_LAST - - where VEC_LHS is the vectorized live-out result and MASK is - the loop mask for the final iteration. */ - gcc_assert (ncopies == 1 && !slp_node); - tree scalar_type = TREE_TYPE (STMT_VINFO_VECTYPE (stmt_info)); - gimple_seq tem = NULL; - gimple_stmt_iterator gsi = gsi_last (tem); - tree mask = vect_get_loop_mask (loop_vinfo, &gsi, - &LOOP_VINFO_MASKS (loop_vinfo), - 1, vectype, 0); - gimple_seq_add_seq (&stmts, tem); - tree scalar_res = gimple_build (&stmts, CFN_EXTRACT_LAST, scalar_type, - mask, vec_lhs_phi); - - /* Convert the extracted vector element to the scalar type. */ - new_tree = gimple_convert (&stmts, lhs_type, scalar_res); - } - else - { - tree bftype = TREE_TYPE (vectype); - if (VECTOR_BOOLEAN_TYPE_P (vectype)) - bftype = build_nonstandard_integer_type (tree_to_uhwi (bitsize), 1); - new_tree = build3 (BIT_FIELD_REF, bftype, - vec_lhs_phi, bitsize, bitstart); - new_tree = force_gimple_operand (fold_convert (lhs_type, new_tree), - &stmts, true, NULL_TREE); - } - - gimple_stmt_iterator exit_gsi = gsi_after_labels (exit_bb); - if (stmts) - gsi_insert_seq_before (&exit_gsi, stmts, GSI_SAME_STMT); + gimple_stmt_iterator exit_gsi; + tree new_tree + = vectorizable_live_operation_1 (loop_vinfo, stmt_info, + LOOP_VINFO_IV_EXIT (loop_vinfo), + vectype, ncopies, slp_node, bitsize, + bitstart, vec_lhs, lhs_type, + &exit_gsi); /* Remove existing phis that copy from lhs and create copies from new_tree. */ --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -10481,6 +10481,95 @@ vectorizable_induction (loop_vec_info loop_vinfo, return true; } + +/* Function vectorizable_live_operation_1. + helper function for vectorizable_live_operation. */ +tree +vectorizable_live_operation_1 (loop_vec_info loop_vinfo, + stmt_vec_info stmt_info, edge exit_e, + tree vectype, int ncopies, slp_tree slp_node, + tree bitsize, tree bitstart, tree vec_lhs, + tree lhs_type, gimple_stmt_iterator *exit_gsi) +{ + basic_block exit_bb = exit_e->dest; + gcc_assert (single_pred_p (exit_bb) || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)); + + tree vec_lhs_phi = copy_ssa_name (vec_lhs); + gimple *phi = create_phi_node (vec_lhs_phi, exit_bb); + for (unsigned i = 0; i < gimple_phi_num_args (phi); i++) + SET_PHI_ARG_DEF (phi, i, vec_lhs); + + gimple_seq stmts = NULL; + tree new_tree; + if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) + { + /* Emit: + SCALAR_RES = VEC_EXTRACT + where VEC_LHS is the vectorized live-out result and MASK is + the loop mask for the final iteration. */ + gcc_assert (ncopies == 1 && !slp_node); + gimple_seq tem = NULL; + gimple_stmt_iterator gsi = gsi_last (tem); + tree len = vect_get_loop_len (loop_vinfo, &gsi, + &LOOP_VINFO_LENS (loop_vinfo), + 1, vectype, 0, 0); + /* BIAS - 1. */ + signed char biasval = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); + tree bias_minus_one + = int_const_binop (MINUS_EXPR, + build_int_cst (TREE_TYPE (len), biasval), + build_one_cst (TREE_TYPE (len))); + /* LAST_INDEX = LEN + (BIAS - 1). */ + tree last_index = gimple_build (&stmts, PLUS_EXPR, TREE_TYPE (len), + len, bias_minus_one); + /* This needs to implement extraction of the first index, but not sure + how the LEN stuff works. At the moment we shouldn't get here since + there's no LEN support for early breaks. But guard this so there's + no incorrect codegen. */ + gcc_assert (!LOOP_VINFO_EARLY_BREAKS (loop_vinfo)); + + /* SCALAR_RES = VEC_EXTRACT . */ + tree scalar_res + = gimple_build (&stmts, CFN_VEC_EXTRACT, TREE_TYPE (vectype), + vec_lhs_phi, last_index); + /* Convert the extracted vector element to the scalar type. */ + new_tree = gimple_convert (&stmts, lhs_type, scalar_res); + } + else if (LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) + { + /* Emit: + SCALAR_RES = EXTRACT_LAST + where VEC_LHS is the vectorized live-out result and MASK is + the loop mask for the final iteration. */ + gcc_assert (!slp_node); + tree scalar_type = TREE_TYPE (STMT_VINFO_VECTYPE (stmt_info)); + gimple_seq tem = NULL; + gimple_stmt_iterator gsi = gsi_last (tem); + tree mask = vect_get_loop_mask (loop_vinfo, &gsi, + &LOOP_VINFO_MASKS (loop_vinfo), + 1, vectype, 0); + + gimple_seq_add_seq (&stmts, tem); + tree scalar_res = gimple_build (&stmts, CFN_EXTRACT_LAST, scalar_type, + mask, vec_lhs_phi); + /* Convert the extracted vector element to the scalar type. */ + new_tree = gimple_convert (&stmts, lhs_type, scalar_res); + } + else + { + tree bftype = TREE_TYPE (vectype); + if (VECTOR_BOOLEAN_TYPE_P (vectype)) + bftype = build_nonstandard_integer_type (tree_to_uhwi (bitsize), 1); + new_tree = build3 (BIT_FIELD_REF, bftype, vec_lhs_phi, bitsize, bitstart); + new_tree = force_gimple_operand (fold_convert (lhs_type, new_tree), + &stmts, true, NULL_TREE); + } + *exit_gsi = gsi_after_labels (exit_bb); + if (stmts) + gsi_insert_seq_before (exit_gsi, stmts, GSI_SAME_STMT); + return new_tree; +} + /* Function vectorizable_live_operation. STMT_INFO computes a value that is used outside the loop. Check if @@ -10690,79 +10779,13 @@ vectorizable_live_operation (vec_info *vinfo, stmt_vec_info stmt_info, gimple *phi = create_phi_node (vec_lhs_phi, exit_bb); SET_PHI_ARG_DEF (phi, LOOP_VINFO_IV_EXIT (loop_vinfo)->dest_idx, vec_lhs); - gimple_seq stmts = NULL; - tree new_tree; - if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) - { - /* Emit: - - SCALAR_RES = VEC_EXTRACT - - where VEC_LHS is the vectorized live-out result and MASK is - the loop mask for the final iteration. */ - gcc_assert (ncopies == 1 && !slp_node); - gimple_seq tem = NULL; - gimple_stmt_iterator gsi = gsi_last (tem); - tree len - = vect_get_loop_len (loop_vinfo, &gsi, - &LOOP_VINFO_LENS (loop_vinfo), - 1, vectype, 0, 0); - - /* BIAS - 1. */ - signed char biasval = LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); - tree bias_minus_one - = int_const_binop (MINUS_EXPR, - build_int_cst (TREE_TYPE (len), biasval), - build_one_cst (TREE_TYPE (len))); - - /* LAST_INDEX = LEN + (BIAS - 1). */ - tree last_index = gimple_build (&stmts, PLUS_EXPR, TREE_TYPE (len), - len, bias_minus_one); - - /* SCALAR_RES = VEC_EXTRACT . */ - tree scalar_res - = gimple_build (&stmts, CFN_VEC_EXTRACT, TREE_TYPE (vectype), - vec_lhs_phi, last_index); - - /* Convert the extracted vector element to the scalar type. */ - new_tree = gimple_convert (&stmts, lhs_type, scalar_res); - } - else if (LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) - { - /* Emit: - - SCALAR_RES = EXTRACT_LAST - - where VEC_LHS is the vectorized live-out result and MASK is - the loop mask for the final iteration. */ - gcc_assert (ncopies == 1 && !slp_node); - tree scalar_type = TREE_TYPE (STMT_VINFO_VECTYPE (stmt_info)); - gimple_seq tem = NULL; - gimple_stmt_iterator gsi = gsi_last (tem); - tree mask = vect_get_loop_mask (loop_vinfo, &gsi, - &LOOP_VINFO_MASKS (loop_vinfo), - 1, vectype, 0); - gimple_seq_add_seq (&stmts, tem); - tree scalar_res = gimple_build (&stmts, CFN_EXTRACT_LAST, scalar_type, - mask, vec_lhs_phi); - - /* Convert the extracted vector element to the scalar type. */ - new_tree = gimple_convert (&stmts, lhs_type, scalar_res); - } - else - { - tree bftype = TREE_TYPE (vectype); - if (VECTOR_BOOLEAN_TYPE_P (vectype)) - bftype = build_nonstandard_integer_type (tree_to_uhwi (bitsize), 1); - new_tree = build3 (BIT_FIELD_REF, bftype, - vec_lhs_phi, bitsize, bitstart); - new_tree = force_gimple_operand (fold_convert (lhs_type, new_tree), - &stmts, true, NULL_TREE); - } - - gimple_stmt_iterator exit_gsi = gsi_after_labels (exit_bb); - if (stmts) - gsi_insert_seq_before (&exit_gsi, stmts, GSI_SAME_STMT); + gimple_stmt_iterator exit_gsi; + tree new_tree + = vectorizable_live_operation_1 (loop_vinfo, stmt_info, + LOOP_VINFO_IV_EXIT (loop_vinfo), + vectype, ncopies, slp_node, bitsize, + bitstart, vec_lhs, lhs_type, + &exit_gsi); /* Remove existing phis that copy from lhs and create copies from new_tree. */