From patchwork Wed Jun 10 06:34:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kewen.Lin" X-Patchwork-Id: 1306587 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=gcc.gnu.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=rME+4/ea; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49hcfJ11t4z9sRh for ; Wed, 10 Jun 2020 16:34:32 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 80EAE388E802; Wed, 10 Jun 2020 06:34:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 80EAE388E802 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1591770869; bh=wEK0kP6+0XpIOILxn4NoP7WZeWst+1/ZhWVTgHTjjvM=; h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=rME+4/ea5rTgSDwWQmKFIVByFLdjl3bnrttJqnjKZkJLqIexA8FEeJyBXyz8IuHpy 0KMAmGCgy5JDGsXtd+ldP1KCzXv2+Qo2rQ91kJZmZf6ezbB02YZhyfNBjD1tWwMmHn AoLQVjqwZHGMxdOhbr/LDlyG24Y1wsYy/hVGFuWA= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id B34E1388A828 for ; Wed, 10 Jun 2020 06:34:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org B34E1388A828 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05A6Vmnh054940; Wed, 10 Jun 2020 02:34:23 -0400 Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 31gg82jrvb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 Jun 2020 02:34:23 -0400 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 05A6LVVH010442; Wed, 10 Jun 2020 06:34:20 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma03ams.nl.ibm.com with ESMTP id 31g2s7y673-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 Jun 2020 06:34:20 +0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 05A6YISI53149838 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 10 Jun 2020 06:34:18 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0F78CAE05F; Wed, 10 Jun 2020 06:34:18 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 50568AE04D; Wed, 10 Jun 2020 06:34:16 +0000 (GMT) Received: from KewenLins-MacBook-Pro.local (unknown [9.200.53.22]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 10 Jun 2020 06:34:16 +0000 (GMT) Subject: [PATCH 1/4] vect: Rename can_fully_mask_p to can_use_partial_vectors_p To: GCC Patches References: <8107a42b-92e8-56f1-0721-8e594c18b8ed@linux.ibm.com> Message-ID: Date: Wed, 10 Jun 2020 14:34:15 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: <8107a42b-92e8-56f1-0721-8e594c18b8ed@linux.ibm.com> Content-Language: en-US X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-10_02:2020-06-10, 2020-06-10 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 suspectscore=0 adultscore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 phishscore=0 impostorscore=0 clxscore=1015 bulkscore=0 mlxlogscore=999 cotscore=-2147483648 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006100046 X-Spam-Status: No, score=-11.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Kewen.Lin via Gcc-patches" From: "Kewen.Lin" Reply-To: "Kewen.Lin" Cc: Bill Schmidt , Segher Boessenkool Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" gcc/ChangeLog: * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Rename can_fully_mask_p to can_use_partial_vectors_p. (vect_analyze_loop_2): Rename LOOP_VINFO_CAN_FULLY_MASK_P to LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. Rename saved_can_fully_mask_p to saved_can_use_partial_vectors_p. (vectorizable_reduction): Rename LOOP_VINFO_CAN_FULLY_MASK_P to LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P. (vectorizable_live_operation): Likewise. * tree-vect-stmts.c (permute_vec_elements): Likewise. (check_load_store_masking): Likewise. (vectorizable_operation): Likewise. (vectorizable_store): Likewise. (vectorizable_load): Likewise. (vectorizable_condition): Likewise. * tree-vectorizer.h (LOOP_VINFO_CAN_FULLY_MASK_P): Renamed to ... (LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P): ... this. (_loop_vec_info): Rename can_fully_mask_p to can_use_partial_vectors_p. --- --- gcc/tree-vect-loop.c | 24 +++++++++++++----------- gcc/tree-vect-stmts.c | 20 ++++++++++---------- gcc/tree-vectorizer.h | 7 ++++--- 3 files changed, 27 insertions(+), 24 deletions(-) -- diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c index 4f94b4baad9..bb082a9b72f 100644 --- a/gcc/tree-vect-loop.c +++ b/gcc/tree-vect-loop.c @@ -812,7 +812,7 @@ _loop_vec_info::_loop_vec_info (class loop *loop_in, vec_info_shared *shared) vec_outside_cost (0), vec_inside_cost (0), vectorizable (false), - can_fully_mask_p (true), + can_use_partial_vectors_p (true), fully_masked_p (false), peeling_for_gaps (false), peeling_for_niter (false), @@ -2054,7 +2054,8 @@ vect_analyze_loop_2 (loop_vec_info loop_vinfo, bool &fatal, unsigned *n_stmts) vect_optimize_slp (loop_vinfo); } - bool saved_can_fully_mask_p = LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo); + bool saved_can_use_partial_vectors_p + = LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo); /* We don't expect to have to roll back to anything other than an empty set of rgroups. */ @@ -2139,7 +2140,7 @@ start_over: /* Decide whether to use a fully-masked loop for this vectorization factor. */ LOOP_VINFO_FULLY_MASKED_P (loop_vinfo) - = (LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) + = (LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) && vect_verify_full_masking (loop_vinfo)); if (dump_enabled_p ()) { @@ -2374,7 +2375,8 @@ again: LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) = false; LOOP_VINFO_COST_MODEL_THRESHOLD (loop_vinfo) = 0; LOOP_VINFO_VERSIONING_THRESHOLD (loop_vinfo) = 0; - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = saved_can_fully_mask_p; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) + = saved_can_use_partial_vectors_p; goto start_over; } @@ -6786,7 +6788,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo, STMT_VINFO_DEF_TYPE (vect_orig_stmt (tem)) = vect_internal_def; STMT_VINFO_DEF_TYPE (tem) = vect_internal_def; } - else if (loop_vinfo && LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo)) + else if (loop_vinfo && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) { vec_loop_masks *masks = &LOOP_VINFO_MASKS (loop_vinfo); internal_fn cond_fn = get_conditional_internal_fn (code); @@ -6801,7 +6803,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because no" " conditional operation is available.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; } else if (reduction_type == FOLD_LEFT_REDUCTION && reduc_fn == IFN_LAST @@ -6813,7 +6815,7 @@ vectorizable_reduction (loop_vec_info loop_vinfo, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because no" " conditional operation is available.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; } else vect_record_loop_mask (loop_vinfo, masks, ncopies * vec_num, @@ -8012,7 +8014,7 @@ vectorizable_live_operation (loop_vec_info loop_vinfo, if (!vec_stmt_p) { /* No transformation required. */ - if (LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo)) + if (LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) { if (!direct_internal_fn_supported_p (IFN_EXTRACT_LAST, vectype, OPTIMIZE_FOR_SPEED)) @@ -8022,7 +8024,7 @@ vectorizable_live_operation (loop_vec_info loop_vinfo, "can't use a fully-masked loop because " "the target doesn't support extract last " "reduction.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; } else if (slp_node) { @@ -8030,7 +8032,7 @@ vectorizable_live_operation (loop_vec_info loop_vinfo, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because an " "SLP statement is live after the loop.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; } else if (ncopies > 1) { @@ -8038,7 +8040,7 @@ vectorizable_live_operation (loop_vec_info loop_vinfo, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because" " ncopies is greater than 1.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; } else { diff --git a/gcc/tree-vect-stmts.c b/gcc/tree-vect-stmts.c index e7822c44951..fb82c8d940f 100644 --- a/gcc/tree-vect-stmts.c +++ b/gcc/tree-vect-stmts.c @@ -1783,7 +1783,7 @@ static tree permute_vec_elements (vec_info *, tree, tree, tree, stmt_vec_info, its arguments. If the load or store is conditional, SCALAR_MASK is the condition under which it occurs. - Clear LOOP_VINFO_CAN_FULLY_MASK_P if a fully-masked loop is not + Clear LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P if a fully-masked loop is not supported, otherwise record the required mask types. */ static void @@ -1810,7 +1810,7 @@ check_load_store_masking (loop_vec_info loop_vinfo, tree vectype, "can't use a fully-masked loop because the" " target doesn't have an appropriate masked" " load/store-lanes instruction.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; return; } unsigned int ncopies = vect_get_num_copies (loop_vinfo, vectype); @@ -1833,7 +1833,7 @@ check_load_store_masking (loop_vec_info loop_vinfo, tree vectype, "can't use a fully-masked loop because the" " target doesn't have an appropriate masked" " gather load or scatter store instruction.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; return; } unsigned int ncopies = vect_get_num_copies (loop_vinfo, vectype); @@ -1850,7 +1850,7 @@ check_load_store_masking (loop_vec_info loop_vinfo, tree vectype, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because an access" " isn't contiguous.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; return; } @@ -1864,7 +1864,7 @@ check_load_store_masking (loop_vec_info loop_vinfo, tree vectype, "can't use a fully-masked loop because the target" " doesn't have the appropriate masked load or" " store.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; return; } /* We might load more scalars than we need for permuting SLP loads. @@ -6187,7 +6187,7 @@ vectorizable_operation (vec_info *vinfo, should only change the active lanes of the reduction chain, keeping the inactive lanes as-is. */ if (loop_vinfo - && LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) + && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) && reduc_idx >= 0) { if (cond_fn == IFN_LAST @@ -6198,7 +6198,7 @@ vectorizable_operation (vec_info *vinfo, dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, "can't use a fully-masked loop because no" " conditional operation is available.\n"); - LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) = false; + LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) = false; } else vect_record_loop_mask (loop_vinfo, masks, ncopies * vec_num, @@ -7528,7 +7528,7 @@ vectorizable_store (vec_info *vinfo, STMT_VINFO_MEMORY_ACCESS_TYPE (stmt_info) = memory_access_type; if (loop_vinfo - && LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo)) + && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) check_load_store_masking (loop_vinfo, vectype, vls_type, group_size, memory_access_type, &gs_info, mask); @@ -8835,7 +8835,7 @@ vectorizable_load (vec_info *vinfo, STMT_VINFO_MEMORY_ACCESS_TYPE (stmt_info) = memory_access_type; if (loop_vinfo - && LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo)) + && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo)) check_load_store_masking (loop_vinfo, vectype, VLS_LOAD, group_size, memory_access_type, &gs_info, mask); @@ -10280,7 +10280,7 @@ vectorizable_condition (vec_info *vinfo, } if (loop_vinfo - && LOOP_VINFO_CAN_FULLY_MASK_P (loop_vinfo) + && LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P (loop_vinfo) && reduction_type == EXTRACT_LAST_REDUCTION) vect_record_loop_mask (loop_vinfo, &LOOP_VINFO_MASKS (loop_vinfo), ncopies * vec_num, vectype, NULL); diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 2eb3ab5d280..13e6bf461ab 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -620,8 +620,9 @@ public: /* Is the loop vectorizable? */ bool vectorizable; - /* Records whether we still have the option of using a fully-masked loop. */ - bool can_fully_mask_p; + /* Records whether we still have the option of using a partial vectorization + approach for this loop. */ + bool can_use_partial_vectors_p; /* True if have decided to use a fully-masked loop. */ bool fully_masked_p; @@ -687,7 +688,7 @@ public: #define LOOP_VINFO_COST_MODEL_THRESHOLD(L) (L)->th #define LOOP_VINFO_VERSIONING_THRESHOLD(L) (L)->versioning_threshold #define LOOP_VINFO_VECTORIZABLE_P(L) (L)->vectorizable -#define LOOP_VINFO_CAN_FULLY_MASK_P(L) (L)->can_fully_mask_p +#define LOOP_VINFO_CAN_USE_PARTIAL_VECTORS_P(L) (L)->can_use_partial_vectors_p #define LOOP_VINFO_FULLY_MASKED_P(L) (L)->fully_masked_p #define LOOP_VINFO_VECT_FACTOR(L) (L)->vectorization_factor #define LOOP_VINFO_MAX_VECT_FACTOR(L) (L)->max_vectorization_factor