From patchwork Thu Jul 11 08:55:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Feng Xue OS X-Patchwork-Id: 1959161 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.a=rsa-sha256 header.s=selector2 header.b=qlMJD7wK; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WKT9F3lqVz1xqj for ; Thu, 11 Jul 2024 18:56:17 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4631E386180B for ; Thu, 11 Jul 2024 08:56:12 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazlp170110009.outbound.protection.outlook.com [IPv6:2a01:111:f403:c111::9]) by sourceware.org (Postfix) with ESMTPS id 32A3E384A434 for ; Thu, 11 Jul 2024 08:55:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 32A3E384A434 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=os.amperecomputing.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=os.amperecomputing.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 32A3E384A434 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=2a01:111:f403:c111::9 ARC-Seal: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1720688152; cv=pass; b=x1zpstEyzDj9NG9tyiLtw8I/gC8WqMcNB71+Y5SAbaXwqbtW0j2Sxft7mc7cjbhWby+R3wn0Iyzmvdiwzwxy4ojZgpOmXSp9o3YWzd0v8GUtLUADyHPhxw+7mZyB9xlBW9hJzKx3oxtsDGWff/aSSlN/ULjS72qgaWFyl6sNHbM= ARC-Message-Signature: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1720688152; c=relaxed/simple; bh=BUEsrh2d7NH+5U/1cDwCUZQypla79/qCa/A7n3wV9AU=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=Hu9mWv9/B4NrDePxSWCzdUmEP8R5JgHEVfEr7y/eGoccZ988a6Cp19G8kZfAbntORG0MQOFxnvuQ3lLDlFyPsooAdmO7AeH5O2c+12r54LkKlShqY8oVq5iNd+uZj0FJX3owQpzcN4bRfYZ3cfp2lTbMlN7o3NmfYLeFo/HPS+g= ARC-Authentication-Results: i=2; server2.sourceware.org ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=O35oQlrabF4wOhrvt/ZFZC1Z+LQYX30R7lly1Gqj+/4OmBM3/6v+sWyL7PE1ziz0kTj2ARjpzVZaDA57QRoZEB4zemB+1hOQEmMgb5s4DuJLl4eFnNtvHylSoR8xlIJd0nWnlGuFZWsDew6Gf43BO0N2PBrpCVhHDUQEqQfm/mMcEaqeQWuZmEc1yEqCo0LOvSojTbwKYXqmZIxf/vZE6nXWKRvOZWjx/k+VaT5+JNzqtQX9+PG9UQdjaXg2E+jmNMtsLpM3TbvPjhoE3ufbC6i8GOeMOmuDL/NB+04BF+2bhSuRHpxo9OAJnctlZy+hHGayqp3yNuimtcV9u2cy8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=M2pnewTrvMB3/Q7tXoSO9kCmGfZiGvxg5Npt/vDgzvA=; b=il6gOQ8NNMLipHEu/hVBBy0VmX7F5n1BnGiuWnJi3OOeqhmcstVLr++MP76QT+ml4fRBTI4gcI6PVmjZJOaukSJwaBhNTlfQMT8brDu+4ALB7WyCMzObPXydocYwaFqmBL7hGcq9Xbmw+FWm4X53o0MXDMqGiZ9HGduLp3GIuC8ZZwHOK9JEE96H26FzTTQvxskIWGfpcr5j8jQTZGwpkcf3iIQKWuqaXr8KtBLQfvEkSt1xFW7m6+XyaprvPJpimcCCk2erdRvkrEEH5iNvh4xsMEfYdZ0LmofuXWXyw/RLYN0+wG8WCJN/SXJa9/zBa+pxbX0kLnPZXoJAGo2i5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=M2pnewTrvMB3/Q7tXoSO9kCmGfZiGvxg5Npt/vDgzvA=; b=qlMJD7wKB1MaNgOF3yBZlm8HBSCZDc9sIRCwaXsH6RJVGp2XnfqkyauDpAdAbU9XpS2H1ybxiewuFi8MMJL/g9Ztbwh5kURKIMQ/JhIRD4f8ptJ/38aPihxtM+2N5Q2oCiM4tjso1laAFJcJ0qYHegkjXEcXQXZk5KO64C+pwf4= Received: from LV2PR01MB7839.prod.exchangelabs.com (2603:10b6:408:14f::13) by SJ0PR01MB7413.prod.exchangelabs.com (2603:10b6:a03:3df::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.32; Thu, 11 Jul 2024 08:55:46 +0000 Received: from LV2PR01MB7839.prod.exchangelabs.com ([fe80::2ac3:5a77:36fd:9c63]) by LV2PR01MB7839.prod.exchangelabs.com ([fe80::2ac3:5a77:36fd:9c63%4]) with mapi id 15.20.7741.033; Thu, 11 Jul 2024 08:55:46 +0000 From: Feng Xue OS To: Richard Biener , "gcc-patches@gcc.gnu.org" Subject: [PATCH 4/4] vect: Optimize order of lane-reducing statements in loop def-use cycles Thread-Topic: [PATCH 4/4] vect: Optimize order of lane-reducing statements in loop def-use cycles Thread-Index: AQHa03AIi5q3PDrI20mtWyctQryNMA== Date: Thu, 11 Jul 2024 08:55:46 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: msip_labels: MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Enabled=True; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_SiteId=3bc2b170-fd94-476d-b0ce-4229bdc904a7; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_SetDate=2024-07-11T08:55:46.295Z; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Name=Confidential; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_ContentBits=0; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Method=Standard; authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: LV2PR01MB7839:EE_|SJ0PR01MB7413:EE_ x-ms-office365-filtering-correlation-id: f922f9f8-7788-4aaf-4142-08dca187413a x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|366016|1800799024|376014|38070700018; x-microsoft-antispam-message-info: =?iso-2022-jp?b?dzZKaDlmVGhONXJPQUVqL1dy?= =?iso-2022-jp?b?VVRkL2hPQytXbTVvMUIxODRpVi9jb21VNDd1c1RpZWUxZDVqL3NRaFQ1?= =?iso-2022-jp?b?ekdCVFdOR2tiMDNKU1lJSkxrbzNzTmNvcFpDMVY4d0tTa2VQT1RXTHhj?= =?iso-2022-jp?b?MlViMUtOMVh4NnV0NnB1STFaZlIwZzY5c3g1NW1xY3EyY3UydXFja2pI?= =?iso-2022-jp?b?VS9ZcUliMkJ6U0RDSDY2VlRtZURJZWFkdTY2dTNVSHVrbXY4SHFrSExF?= =?iso-2022-jp?b?b0gvNnd2cXBBYU9Rc2Z4NXBsZDBydFlNQ2pQMUVnV2M2RjhNNGd1azlu?= =?iso-2022-jp?b?ZVRFaVdIS0Q3bENuZ2VhZUJqRzNJTkZmM3RST3dST1lTWVo5bUN0Wk82?= =?iso-2022-jp?b?NktHdjhsWVYrbXltdTFVdzNrYlk2UjBHNEVEbURhK0xpbGtEcVcwYXV6?= =?iso-2022-jp?b?bDdqbmpzR2VRVnNrWDdIME9IOGVEbkZnUm5VV1hNdzQyR3hpWkxNZ0tV?= =?iso-2022-jp?b?cGtXK2VtY0xMZ3VSdTJOOEI2Wm9XNHpVOHU1eG9uZ3hPK1RwQStQMlZh?= =?iso-2022-jp?b?V1NneVRLY1NPazR1ZEJyRnZhQlhXUVRrdlpUMmFHSXE1MlczaHpTTE80?= =?iso-2022-jp?b?OExxK2Y2MEFOclhYRjgxR29VQmVqK3J2SHJxUlB2OGdRZDVwL3ZpRlQw?= =?iso-2022-jp?b?S0Q4Mm82YlJGRld3b1E5UDBXZzNwa2pxSHJQWW9EbTF4WGl6ZWlsSTV3?= =?iso-2022-jp?b?czlaU2JIU1JkNCtaVVR4VGR4WkZDK2lGaFB5d1p4UWJNSHVaRUEwMUsz?= =?iso-2022-jp?b?UGp1eG1LYzhwamxTM1BveG5GR1dTV0NHbmNTL2V0NEtCL3FIMHR2VGlm?= =?iso-2022-jp?b?NXRyWTdRSU8raGVteUV4RVppVlFpcHJkc1NUbmJZd2g1K0JTN1poVGN4?= =?iso-2022-jp?b?MU1KRWtnZzFvMWJEUlFaQXVURnV1bkRGV3BqWEFkZmg1VmhTMWxQVVFx?= =?iso-2022-jp?b?cHRCQUNVZmg1VWZhemoxUWVEdzRhUW11TlNJbnBMUWllazVZRnVwTUVa?= =?iso-2022-jp?b?bW1NV0VjcUlmekkya3JRdW5hd0xpMW9rb3FxYW1LUFU3VytKNjJLbjN4?= =?iso-2022-jp?b?L2FpY0l5eS8zYndWRjE0dVd3Q04vclRKWVlzM0dER3VYa3I5d2M3TDVW?= =?iso-2022-jp?b?Q0RubFpGcmRtSkRUcThkU21VK0VjeWpoeXVkVjNRay92VURpZE5SSEwz?= =?iso-2022-jp?b?SnpaNWZvLzdXMEFEOFZsYklxdk4vL0ZtMDZKZloxeGt0LzYwb2hSWnVn?= =?iso-2022-jp?b?Wk1MTDIrWjl1V3pmWU5ETXlRdzEyMGtZWE9IQnFRSkpSQ241b3J3Ny9s?= =?iso-2022-jp?b?TmJpNEZWRDQxcHZndUYzR1JWclZLL0MrQ1BSS25XMytlWGNUZWNnaHl5?= =?iso-2022-jp?b?WnNVL0hUWDJLSEtGa1hlenhGb3phNi91N3B0Y09BYWVLSnVhM3YvRm85?= =?iso-2022-jp?b?SzJHbGZZRVlacG9vZjR3SzdjNmlDNStzR0pVSk1PcGVBaEV6RU1KRHN6?= =?iso-2022-jp?b?eDBkMHVnRklBZjR3bnZLSzd4ekdwd2pOWWlMQVUrOFRkSU1iOE9JaGcz?= =?iso-2022-jp?b?aEUxZFg4enpsQk9aUCtTWG8ydG5BT0JoeVUwRUZtaVU0Y0NIR1IveHdY?= =?iso-2022-jp?b?Q2tWVVN6VVRId3FIWEFwTzlqWUpkanUzYSt3NHhDL2d1bkVjSkZLNTgv?= =?iso-2022-jp?b?VXR0azRyWlpnT21WblJweXpZVkxXWFFkSk56dFh0Ykh6WmVxMkFLUGdz?= =?iso-2022-jp?b?ZUV6T1pjeTd3ZGkvTUc4VmpkNFlLTUZsNjMyMUZQZGJwWGV1alVoWE1Y?= =?iso-2022-jp?b?dWxNS0hqNFZlb0VBVHVwRmgvTFRYdkZZQTJlbUF6NGFGTWo0ZUdFZklu?= =?iso-2022-jp?b?Z1Bhc2REdTlYd2tCUGJmaTdUaTJBUmNRN2N6bWdtSVpkZ0JQVGg2Y1Zj?= =?iso-2022-jp?b?VGFoOXBiR1NLTWVqR3ZLYWk3M1JvVHJOUWxlRDFRR0E5L3NvbFJVNDZG?= =?iso-2022-jp?b?dGhNMkxYZUNJUk9aanhyei9WMDlLdE9wOVE9PQ==?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:ja; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:LV2PR01MB7839.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(376014)(38070700018); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-2022-jp?b?RWkyZ3MzaVBjeklUcXRX?= =?iso-2022-jp?b?d1RWcThVS3MvOG40NFk0OXlQV2Z0c0wrMTEyT20xeUZEcktHUEszUjlQ?= =?iso-2022-jp?b?b3AwTDM0eW5SQnp2MkErT0xQMmduM083WDN1dkFoMTZLQkFWY2wzYzYr?= =?iso-2022-jp?b?UWRodC9TbUR3S3pLRVgvWlA1ZkVIdXhFOHEvaEZFODhydnNacG5hZi9S?= =?iso-2022-jp?b?OC83SUZWbWtmNFR1bkNySzcrWkY2eUVwQUt5b05ZcS82dGJJMzF4a3FE?= =?iso-2022-jp?b?WExscFZ4NGxLRXN4MW1LU0JHZlM5amNDWFduQXBLaTlRSzFTQ0tlMHdq?= =?iso-2022-jp?b?UmhSK3FLYjRlRkNJOUMrQk44dFhIVjNCcWNET25rNFZDcUM0d2xTTFgz?= =?iso-2022-jp?b?WmpwTDhERkUzQlF2c3RyYk8zaHVuYzJxOEJPaXdMZkZ0ZXh0M1AvWHph?= =?iso-2022-jp?b?Q3FOdHA5MEF5WEVHN1czdVF2S2lxQXRNVk5WWEhhdGFBbUVGQ0gwRWJp?= =?iso-2022-jp?b?MnRxT3cxUFNiTVpZMTI4WmlvRmV2WUJHQ1hFRE5yOWt6SXBlU1dnUTZa?= =?iso-2022-jp?b?NjlsQTR1OHFVa2ZZTW5GVzZtNll6ekR6bVgvZVQ2U2xmVndGa3dDUnFI?= =?iso-2022-jp?b?WXhiOHZoaE5uWjFYVU52UWRBUXR4NHFHVm5yZExLZFloMzI4MWdxcEVL?= =?iso-2022-jp?b?Z1lwbkdUZVhrcGlyaHhVUUZBN3FDUExYQTJRakdlTS9OVGF6SWdzZlVE?= =?iso-2022-jp?b?Zk9iTkdWZGpVSlNKdmJ5aStMUXIvOVBxY2M0K3NJU2JBTk5pNW15a3Fy?= =?iso-2022-jp?b?eGJSbmJNditlUUMvTHg4TTBnSisrN0hwTHFtQm9qd2RPTnpGTkhZc2xO?= =?iso-2022-jp?b?RHhKbm0xaFJ3ZjNuZ1lQOWxXSGFhc3FRckgrelU1Y1VHUG1KUGRmVGkz?= =?iso-2022-jp?b?NHNiU3F6T0duTklraTFLSGpDTDZJL3daOXRHZUtJYi8vN1RxMEhUKzNY?= =?iso-2022-jp?b?VHN4YUhoMmF6N2o4YkZ4dk5KVCt3QU03RFFBR2FjK3BaMENxY3NFWUdw?= =?iso-2022-jp?b?NHYvN3ErSldnVkVKR3FlbmlsTjFLdVVTcVdXcTNlMUpvaGRDK0JFeXR6?= =?iso-2022-jp?b?aUs0clpkandEYXBuSytQdGFBQmFFdkl0M2daUDZ3TEdkZExHWlFMK1d4?= =?iso-2022-jp?b?aVRQQm85VGs4VXU1dnhobnF1eUxWaENhU0NnWDF0dEErTkZENTVPUkdq?= =?iso-2022-jp?b?djQvOWZPSzZTZVdoTHpXa013ZGowcGVOZSt5SEVlMGMrb3BTQjE5Sm51?= =?iso-2022-jp?b?VzJOYU00cXkxbXFEd2hUU3dCelpLNTRKSXFPaE1GOEFyOE5BTmJzMVlO?= =?iso-2022-jp?b?bUQ2bEoxckloOW1URXlTVll1a3E2aSsrcmdNVXBrTXJEY1JCc3lVOERG?= =?iso-2022-jp?b?UzE1U25mQ3FHanp5K0Ntd3p5bU9DR1Jla2NsMDZwVEJKODVISEkzUFJC?= =?iso-2022-jp?b?VWdlcldTb2ljdW9YZE5vbHBZUTV1cWVGRlRnL2NLS3RJd2lHazM5VmFm?= =?iso-2022-jp?b?S2RUay9oMUFNWDJkTGYzQWlrVEsySTVrbmFWSTVzd1ROREV4dW9QNVpI?= =?iso-2022-jp?b?aWZFVDZsbmhYWHhKWkxVVUh0RWIxeDdwWHNDVDYzUlh0NDRHMjJQMDQy?= =?iso-2022-jp?b?OWJnV3A4QUJzeEI1V3RRWVErWmRSTzBHNlQ5ckFnYjI0MzVuRCtEMkRz?= =?iso-2022-jp?b?RUxUa282YUdGRTJvMjBmRzFheDdUN0xBenNpWlI0bTdKK3pYM003eTYw?= =?iso-2022-jp?b?eVdiN1lSVlhOSTNRZFVXT3RNRDFMU0RQTmdWNTc2VHRIRVJBa1RyU1Z0?= =?iso-2022-jp?b?VVhOK21pYytiU2NyRlZJL282cWVvcTBOWFZOWDNpMktOMXhVYmY2QzUx?= =?iso-2022-jp?b?Y1RHQkRWR2YxY1NKa0VobUM0TXc4bHREQTh3QXFJQzFldGdzWmJBcDRw?= =?iso-2022-jp?b?S0lsdXlPMVNMcFNjdWpXdTNUZml3ZG44alFLRXRHMUZCZE9ISkFVWFBF?= =?iso-2022-jp?b?RHBHWXlLdFdvN2lTTUE0YzlRcExuSnhsaTM2aVhjR1NqejdhRW5ZSDJi?= =?iso-2022-jp?b?ZWRVendhcjdGc0tGTGJzekdHR04yNEdsbytocURRV3BxR2VmOHoxc3ZZ?= =?iso-2022-jp?b?QjYxUFhFRWl0a1l3ZUxiTHZuT1Y2RlBRejNsemNISDV6SytSMnk3cFVB?= =?iso-2022-jp?b?c25ML21QakZxT3B1Q2ZPM2V4cVZWbUY4OU1rNnJXa0ZsUnhmSWR0Zmg5?= =?iso-2022-jp?b?dkNBc3JlRGQwaFBWR0JkN08wd1IzamJUeCsweDNDNw==?= MIME-Version: 1.0 X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: LV2PR01MB7839.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: f922f9f8-7788-4aaf-4142-08dca187413a X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Jul 2024 08:55:46.6725 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: hLz+GBhKmuW0+qiVgYnXhPNmhhWuploOx4beCEvil6SeA/C1S6jP6CZj7gHeA4nRTfvWcOmPBstd8H0TtidEdu4W8l8NhZ5kQeimhl3/YkM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR01MB7413 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org When transforming multiple lane-reducing operations in a loop reduction chain, originally, corresponding vectorized statements are generated into def-use cycles starting from 0. The def-use cycle with smaller index, would contain more statements, which means more instruction dependency. For example: int sum = 1; for (i) { sum += d0[i] * d1[i]; // dot-prod sum += w[i]; // widen-sum sum += abs(s0[i] - s1[i]); // sad sum += n[i]; // normal } Original transformation result: for (i / 16) { sum_v0 = DOT_PROD (d0_v0[i: 0 ~ 15], d1_v0[i: 0 ~ 15], sum_v0); sum_v1 = sum_v1; // copy sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = WIDEN_SUM (w_v0[i: 0 ~ 15], sum_v0); sum_v1 = sum_v1; // copy sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = SAD (s0_v0[i: 0 ~ 7 ], s1_v0[i: 0 ~ 7 ], sum_v0); sum_v1 = SAD (s0_v1[i: 8 ~ 15], s1_v1[i: 8 ~ 15], sum_v1); sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy ... } For a higher instruction parallelism in final vectorized loop, an optimal means is to make those effective vector lane-reducing ops be distributed evenly among all def-use cycles. Transformed as the below, DOT_PROD, WIDEN_SUM and SADs are generated into disparate cycles, instruction dependency among them could be eliminated. for (i / 16) { sum_v0 = DOT_PROD (d0_v0[i: 0 ~ 15], d1_v0[i: 0 ~ 15], sum_v0); sum_v1 = sum_v1; // copy sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = sum_v0; // copy sum_v1 = WIDEN_SUM (w_v1[i: 0 ~ 15], sum_v1); sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = sum_v0; // copy sum_v1 = sum_v1; // copy sum_v2 = SAD (s0_v2[i: 0 ~ 7 ], s1_v2[i: 0 ~ 7 ], sum_v2); sum_v3 = SAD (s0_v3[i: 8 ~ 15], s1_v3[i: 8 ~ 15], sum_v3); ... } Thanks, Feng --- gcc/ PR tree-optimization/114440 * tree-vectorizer.h (struct _stmt_vec_info): Add a new field reduc_result_pos. (vect_transform_reduction): Add a new parameter of slp_instance type. * tree-vect-stmts.cc (vect_transform_stmt): Add a new argument slp_node_instance to vect_transform_reduction. * tree-vect-loop.cc (vect_transform_reduction): Add a new parameter slp_node_instance. Generate lane-reducing statements in an optimized order. --- gcc/tree-vect-loop.cc | 73 +++++++++++++++++++++++++++++++++++------- gcc/tree-vect-stmts.cc | 3 +- gcc/tree-vectorizer.h | 8 ++++- 3 files changed, 71 insertions(+), 13 deletions(-) From 05e92cb5d93e213ec1bb0d8a02a9ae398bbe4442 Mon Sep 17 00:00:00 2001 From: Feng Xue Date: Wed, 29 May 2024 17:28:14 +0800 Subject: [PATCH 4/4] vect: Optimize order of lane-reducing statements in loop def-use cycles When transforming multiple lane-reducing operations in a loop reduction chain, originally, corresponding vectorized statements are generated into def-use cycles starting from 0. The def-use cycle with smaller index, would contain more statements, which means more instruction dependency. For example: int sum = 1; for (i) { sum += d0[i] * d1[i]; // dot-prod sum += w[i]; // widen-sum sum += abs(s0[i] - s1[i]); // sad sum += n[i]; // normal } Original transformation result: for (i / 16) { sum_v0 = DOT_PROD (d0_v0[i: 0 ~ 15], d1_v0[i: 0 ~ 15], sum_v0); sum_v1 = sum_v1; // copy sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = WIDEN_SUM (w_v0[i: 0 ~ 15], sum_v0); sum_v1 = sum_v1; // copy sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = SAD (s0_v0[i: 0 ~ 7 ], s1_v0[i: 0 ~ 7 ], sum_v0); sum_v1 = SAD (s0_v1[i: 8 ~ 15], s1_v1[i: 8 ~ 15], sum_v1); sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy ... } For a higher instruction parallelism in final vectorized loop, an optimal means is to make those effective vector lane-reducing ops be distributed evenly among all def-use cycles. Transformed as the below, DOT_PROD, WIDEN_SUM and SADs are generated into disparate cycles, instruction dependency among them could be eliminated. for (i / 16) { sum_v0 = DOT_PROD (d0_v0[i: 0 ~ 15], d1_v0[i: 0 ~ 15], sum_v0); sum_v1 = sum_v1; // copy sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = sum_v0; // copy sum_v1 = WIDEN_SUM (w_v1[i: 0 ~ 15], sum_v1); sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy sum_v0 = sum_v0; // copy sum_v1 = sum_v1; // copy sum_v2 = SAD (s0_v2[i: 0 ~ 7 ], s1_v2[i: 0 ~ 7 ], sum_v2); sum_v3 = SAD (s0_v3[i: 8 ~ 15], s1_v3[i: 8 ~ 15], sum_v3); ... } 2024-03-22 Feng Xue gcc/ PR tree-optimization/114440 * tree-vectorizer.h (struct _stmt_vec_info): Add a new field reduc_result_pos. (vect_transform_reduction): Add a new parameter of slp_instance type. * tree-vect-stmts.cc (vect_transform_stmt): Add a new argument slp_node_instance to vect_transform_reduction. * tree-vect-loop.cc (vect_transform_reduction): Add a new parameter slp_node_instance. Generate lane-reducing statements in an optimized order. --- gcc/tree-vect-loop.cc | 73 +++++++++++++++++++++++++++++++++++------- gcc/tree-vect-stmts.cc | 3 +- gcc/tree-vectorizer.h | 8 ++++- 3 files changed, 71 insertions(+), 13 deletions(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index a3374fb2d1a..841ef4c9120 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -8673,7 +8673,8 @@ vect_emulate_mixed_dot_prod (loop_vec_info loop_vinfo, stmt_vec_info stmt_info, bool vect_transform_reduction (loop_vec_info loop_vinfo, stmt_vec_info stmt_info, gimple_stmt_iterator *gsi, - gimple **vec_stmt, slp_tree slp_node) + gimple **vec_stmt, slp_tree slp_node, + slp_instance slp_node_instance) { tree vectype_out = STMT_VINFO_VECTYPE (stmt_info); class loop *loop = LOOP_VINFO_LOOP (loop_vinfo); @@ -8863,6 +8864,7 @@ vect_transform_reduction (loop_vec_info loop_vinfo, sum += d0[i] * d1[i]; // dot-prod sum += w[i]; // widen-sum sum += abs(s0[i] - s1[i]); // sad + sum += n[i]; // normal } The vector size is 128-bit，vectorization factor is 16. Reduction @@ -8880,25 +8882,30 @@ vect_transform_reduction (loop_vec_info loop_vinfo, sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy - sum_v0 = WIDEN_SUM (w_v0[i: 0 ~ 15], sum_v0); - sum_v1 = sum_v1; // copy + sum_v0 = sum_v0; // copy + sum_v1 = WIDEN_SUM (w_v1[i: 0 ~ 15], sum_v1); sum_v2 = sum_v2; // copy sum_v3 = sum_v3; // copy - sum_v0 = SAD (s0_v0[i: 0 ~ 7 ], s1_v0[i: 0 ~ 7 ], sum_v0); - sum_v1 = SAD (s0_v1[i: 8 ~ 15], s1_v1[i: 8 ~ 15], sum_v1); - sum_v2 = sum_v2; // copy + sum_v0 = sum_v0; // copy + sum_v1 = SAD (s0_v1[i: 0 ~ 7 ], s1_v1[i: 0 ~ 7 ], sum_v1); + sum_v2 = SAD (s0_v2[i: 8 ~ 15], s1_v2[i: 8 ~ 15], sum_v2); sum_v3 = sum_v3; // copy + + sum_v0 += n_v0[i: 0 ~ 3 ]; + sum_v1 += n_v1[i: 4 ~ 7 ]; + sum_v2 += n_v2[i: 8 ~ 11]; + sum_v3 += n_v3[i: 12 ~ 15]; } - sum_v = sum_v0 + sum_v1 + sum_v2 + sum_v3; // = sum_v0 + sum_v1 - */ + Moreover, for a higher instruction parallelism in final vectorized + loop, it is considered to make those effective vector lane-reducing + ops be distributed evenly among all def-use cycles. In the above + example, DOT_PROD, WIDEN_SUM and SADs are generated into disparate + cycles, instruction dependency among them could be eliminated. */ unsigned effec_ncopies = vec_oprnds[0].length (); unsigned total_ncopies = vec_oprnds[reduc_index].length (); - if (slp_node) - gcc_assert (effec_ncopies == SLP_TREE_VEC_STMTS_EFFEC_NUM (slp_node)); - gcc_assert (effec_ncopies <= total_ncopies); if (effec_ncopies < total_ncopies) @@ -8909,6 +8916,50 @@ vect_transform_reduction (loop_vec_info loop_vinfo, vec_oprnds[i].safe_grow_cleared (total_ncopies); } } + + unsigned effec_phis_ncopies = total_ncopies; + + if (slp_node) + { + slp_tree reduc_phis = slp_node_instance->reduc_phis; + + effec_phis_ncopies = SLP_TREE_VEC_STMTS_EFFEC_NUM (reduc_phis); + gcc_assert (effec_ncopies <= effec_phis_ncopies); + gcc_assert (effec_ncopies == SLP_TREE_VEC_STMTS_EFFEC_NUM (slp_node)); + } + + if (effec_ncopies < effec_phis_ncopies) + { + /* Find suitable def-use cycles to generate vectorized statements + into, and reorder operands based on the selection. */ + unsigned curr_pos = reduc_info->reduc_result_pos; + unsigned next_pos = (curr_pos + effec_ncopies) % effec_phis_ncopies; + + gcc_assert (curr_pos < effec_phis_ncopies); + reduc_info->reduc_result_pos = next_pos; + + if (curr_pos) + { + unsigned count = effec_phis_ncopies - effec_ncopies; + unsigned start = curr_pos - count; + + if ((int) start < 0) + { + count = curr_pos; + start = 0; + } + + for (unsigned i = 0; i < op.num_ops - 1; i++) + { + for (unsigned j = effec_ncopies; j > start; j--) + { + unsigned k = j - 1; + std::swap (vec_oprnds[i][k], vec_oprnds[i][k + count]); + gcc_assert (!vec_oprnds[i][k]); + } + } + } + } } bool emulated_mixed_dot_prod = vect_is_emulated_mixed_dot_prod (stmt_info); diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 8b9659d221c..3a5c1f7e6cc 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -13458,7 +13458,8 @@ vect_transform_stmt (vec_info *vinfo, case reduc_vec_info_type: done = vect_transform_reduction (as_a (vinfo), stmt_info, - gsi, &vec_stmt, slp_node); + gsi, &vec_stmt, slp_node, + slp_node_instance); gcc_assert (done); break; diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 0914f8064c6..8e6b494be90 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -1412,6 +1412,12 @@ public: /* The vector type for performing the actual reduction. */ tree reduc_vectype; + /* For loop reduction with multiple vectorized results (ncopies > 1), a + lane-reducing operation participating in it may not use all of those + results, this field specifies result index starting from which any + following land-reducing operation would be assigned to. */ + unsigned int reduc_result_pos; + /* If IS_REDUC_INFO is true and if the vector code is performing N scalar reductions in parallel, this variable gives the initial scalar values of those N reductions. */ @@ -2514,7 +2520,7 @@ extern bool vectorizable_induction (loop_vec_info, stmt_vec_info, stmt_vector_for_cost *); extern bool vect_transform_reduction (loop_vec_info, stmt_vec_info, gimple_stmt_iterator *, - gimple **, slp_tree); + gimple **, slp_tree, slp_instance); extern bool vect_transform_cycle_phi (loop_vec_info, stmt_vec_info, gimple **, slp_tree, slp_instance); -- 2.17.1