From patchwork Fri Apr 28 12:36:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Andre Vieira (lists)" X-Patchwork-Id: 1774956 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=hLAk8xxn; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q7C0d6hHKz23td for ; Fri, 28 Apr 2023 22:41:53 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DE85F38708CF for ; Fri, 28 Apr 2023 12:41:51 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org DE85F38708CF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1682685711; bh=jztMJ+sAiF9uC3du2aIGBs5+PDIP+7NQ3eEcGPaJ5zY=; h=Date:To:Cc:References:Subject:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=hLAk8xxn7IRVqM2714qb1uWLAYf7t2W8e5HGlXxwsrVSYPIZBkctSQ4hfZF9baEmu QA3bTxPIPP2OnF7AOaQZ1SgsqxouYe58P+eVWzIsYt1d+Ak2pQ2VYr1Cu2BeJFBaPK qFlz6fT/Fufw+8Q+xI2DaDgukl+t27iUSlyAWJ2A= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id A2D38385354F for ; Fri, 28 Apr 2023 12:37:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A2D38385354F Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C61812FC; Fri, 28 Apr 2023 05:37:50 -0700 (PDT) Received: from [10.57.70.10] (unknown [10.57.70.10]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 79B483F5A1; Fri, 28 Apr 2023 05:37:05 -0700 (PDT) Message-ID: Date: Fri, 28 Apr 2023 13:36:59 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Content-Language: en-US To: Richard Biener Cc: Richard Biener , Richard Sandiford , "gcc-patches@gcc.gnu.org" References: <51ce8969-3130-452e-092e-f9d91eff2dad@arm.com> Subject: [PATCH 1/3] Refactor to allow internal_fn's In-Reply-To: X-Spam-Status: No, score=-14.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_LOTSOFHASH, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Andre Vieira \(lists\) via Gcc-patches" From: "Andre Vieira (lists)" Reply-To: "Andre Vieira \(lists\)" Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi, I'm posting the patches separately now with ChangeLogs. I made the suggested changes and tried to simplify the code a bit further. Where internal to tree-vect-stmts I changed most functions to use code_helper to avoid having to check at places we didn't need to. I was trying to simplify things further by also modifying supportable_half_widening_operation and supportable_convert_operation but the result of that was that I ended up moving the code to cast to tree code inside them rather than at the call site and it didn't look simpler, so I left those. Though if we did make those changes we'd no longer need to keep around the tc1 variable in vectorizable_conversion... Let me know what you think. gcc/ChangeLog: 2023-04-28 Andre Vieira Joel Hutton * tree-vect-patterns.cc (vect_gimple_build): New Function. (vect_recog_widen_op_pattern): Refactor to use code_helper. * tree-vect-stmts.cc (vect_gen_widened_results_half): Likewise. (vect_create_vectorized_demotion_stmts): Likewise. (vect_create_vectorized_promotion_stmts): Likewise. (vect_create_half_widening_stmts): Likewise. (vectorizable_conversion): Likewise. (vectorizable_call): Likewise. (supportable_widening_operation): Likewise. (supportable_narrowing_operation): Likewise. (simple_integer_narrowing): Likewise. * tree-vectorizer.h (supportable_widening_operation): Likewise. (supportable_narrowing_operation): Likewise. (vect_gimple_build): New function prototype. * tree.h (code_helper::safe_as_tree_code): New function. (code_helper::safe_as_fn_code): New function. diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 8802141cd6edb298866025b8a55843eae1f0eb17..b35023adade94c1996cd076c4b7419560e819c6b 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -25,6 +25,8 @@ along with GCC; see the file COPYING3. If not see #include "rtl.h" #include "tree.h" #include "gimple.h" +#include "gimple-iterator.h" +#include "gimple-fold.h" #include "ssa.h" #include "expmed.h" #include "optabs-tree.h" @@ -1391,7 +1393,7 @@ vect_recog_sad_pattern (vec_info *vinfo, static gimple * vect_recog_widen_op_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info, tree *type_out, - tree_code orig_code, tree_code wide_code, + tree_code orig_code, code_helper wide_code, bool shift_p, const char *name) { gimple *last_stmt = last_stmt_info->stmt; @@ -1434,7 +1436,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo, vecctype = get_vectype_for_scalar_type (vinfo, ctype); } - enum tree_code dummy_code; + code_helper dummy_code; int dummy_int; auto_vec dummy_vec; if (!vectype @@ -1455,8 +1457,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo, 2, oprnd, half_type, unprom, vectype); tree var = vect_recog_temp_ssa_var (itype, NULL); - gimple *pattern_stmt = gimple_build_assign (var, wide_code, - oprnd[0], oprnd[1]); + gimple *pattern_stmt = vect_gimple_build (var, wide_code, oprnd[0], oprnd[1]); if (vecctype != vecitype) pattern_stmt = vect_convert_output (vinfo, last_stmt_info, ctype, @@ -6406,3 +6407,20 @@ vect_pattern_recog (vec_info *vinfo) /* After this no more add_stmt calls are allowed. */ vinfo->stmt_vec_info_ro = true; } + +/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code, + or internal_fn contained in ch, respectively. */ +gimple * +vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1) +{ + gcc_assert (op0 != NULL_TREE); + if (ch.is_tree_code ()) + return gimple_build_assign (lhs, (tree_code) ch, op0, op1); + + gcc_assert (ch.is_internal_fn ()); + gimple* stmt = gimple_build_call_internal (as_internal_fn ((combined_fn) ch), + op1 == NULL_TREE ? 1 : 2, + op0, op1); + gimple_call_set_lhs (stmt, lhs); + return stmt; +} diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 6b7dbfd4a231baec24e740ffe0ce0b0bf7a1de6b..ce47f4940fa9a1baca4ba1162065cfc3b4072eba 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -3258,13 +3258,13 @@ vectorizable_bswap (vec_info *vinfo, static bool simple_integer_narrowing (tree vectype_out, tree vectype_in, - tree_code *convert_code) + code_helper *convert_code) { if (!INTEGRAL_TYPE_P (TREE_TYPE (vectype_out)) || !INTEGRAL_TYPE_P (TREE_TYPE (vectype_in))) return false; - tree_code code; + code_helper code; int multi_step_cvt = 0; auto_vec interm_types; if (!supportable_narrowing_operation (NOP_EXPR, vectype_out, vectype_in, @@ -3478,7 +3478,7 @@ vectorizable_call (vec_info *vinfo, tree callee = gimple_call_fndecl (stmt); /* First try using an internal function. */ - tree_code convert_code = ERROR_MARK; + code_helper convert_code = MAX_TREE_CODES; if (cfn != CFN_LAST && (modifier == NONE || (modifier == NARROW @@ -3664,8 +3664,8 @@ vectorizable_call (vec_info *vinfo, continue; } new_temp = make_ssa_name (vec_dest); - new_stmt = gimple_build_assign (new_temp, convert_code, - prev_res, half_res); + new_stmt = vect_gimple_build (new_temp, convert_code, + prev_res, half_res); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); } @@ -3755,8 +3755,8 @@ vectorizable_call (vec_info *vinfo, continue; } new_temp = make_ssa_name (vec_dest); - new_stmt = gimple_build_assign (new_temp, convert_code, - prev_res, half_res); + new_stmt = vect_gimple_build (new_temp, convert_code, prev_res, + half_res); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); } else @@ -4768,7 +4768,7 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info, STMT_INFO is the original scalar stmt that we are vectorizing. */ static gimple * -vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code, +vect_gen_widened_results_half (vec_info *vinfo, code_helper ch, tree vec_oprnd0, tree vec_oprnd1, int op_type, tree vec_dest, gimple_stmt_iterator *gsi, stmt_vec_info stmt_info) @@ -4777,12 +4777,11 @@ vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code, tree new_temp; /* Generate half of the widened result: */ - gcc_assert (op_type == TREE_CODE_LENGTH (code)); if (op_type != binary_op) vec_oprnd1 = NULL; - new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0, vec_oprnd1); + new_stmt = vect_gimple_build (vec_dest, ch, vec_oprnd0, vec_oprnd1); new_temp = make_ssa_name (vec_dest, new_stmt); - gimple_assign_set_lhs (new_stmt, new_temp); + gimple_set_lhs (new_stmt, new_temp); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); return new_stmt; @@ -4799,7 +4798,7 @@ vect_create_vectorized_demotion_stmts (vec_info *vinfo, vec *vec_oprnds, stmt_vec_info stmt_info, vec &vec_dsts, gimple_stmt_iterator *gsi, - slp_tree slp_node, enum tree_code code) + slp_tree slp_node, code_helper code) { unsigned int i; tree vop0, vop1, new_tmp, vec_dest; @@ -4811,9 +4810,9 @@ vect_create_vectorized_demotion_stmts (vec_info *vinfo, vec *vec_oprnds, /* Create demotion operation. */ vop0 = (*vec_oprnds)[i]; vop1 = (*vec_oprnds)[i + 1]; - gassign *new_stmt = gimple_build_assign (vec_dest, code, vop0, vop1); + gimple *new_stmt = vect_gimple_build (vec_dest, code, vop0, vop1); new_tmp = make_ssa_name (vec_dest, new_stmt); - gimple_assign_set_lhs (new_stmt, new_tmp); + gimple_set_lhs (new_stmt, new_tmp); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); if (multi_step_cvt) @@ -4861,8 +4860,8 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo, vec *vec_oprnds1, stmt_vec_info stmt_info, tree vec_dest, gimple_stmt_iterator *gsi, - enum tree_code code1, - enum tree_code code2, int op_type) + code_helper ch1, + code_helper ch2, int op_type) { int i; tree vop0, vop1, new_tmp1, new_tmp2; @@ -4878,10 +4877,10 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo, vop1 = NULL_TREE; /* Generate the two halves of promotion operation. */ - new_stmt1 = vect_gen_widened_results_half (vinfo, code1, vop0, vop1, + new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1, op_type, vec_dest, gsi, stmt_info); - new_stmt2 = vect_gen_widened_results_half (vinfo, code2, vop0, vop1, + new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1, op_type, vec_dest, gsi, stmt_info); if (is_gimple_call (new_stmt1)) @@ -4912,7 +4911,7 @@ vect_create_half_widening_stmts (vec_info *vinfo, vec *vec_oprnds1, stmt_vec_info stmt_info, tree vec_dest, gimple_stmt_iterator *gsi, - enum tree_code code1, + code_helper code1, int op_type) { int i; @@ -4942,13 +4941,13 @@ vect_create_half_widening_stmts (vec_info *vinfo, new_stmt2 = gimple_build_assign (new_tmp2, NOP_EXPR, vop1); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt2, gsi); /* Perform the operation. With both vector inputs widened. */ - new_stmt3 = gimple_build_assign (vec_dest, code1, new_tmp1, new_tmp2); + new_stmt3 = vect_gimple_build (vec_dest, code1, new_tmp1, new_tmp2); } else { /* Perform the operation. With the single vector input widened. */ - new_stmt3 = gimple_build_assign (vec_dest, code1, new_tmp1, vop1); - } + new_stmt3 = vect_gimple_build (vec_dest, code1, new_tmp1, vop1); + } new_tmp3 = make_ssa_name (vec_dest, new_stmt3); gimple_assign_set_lhs (new_stmt3, new_tmp3); @@ -4978,8 +4977,9 @@ vectorizable_conversion (vec_info *vinfo, tree scalar_dest; tree op0, op1 = NULL_TREE; loop_vec_info loop_vinfo = dyn_cast (vinfo); - enum tree_code code, code1 = ERROR_MARK, code2 = ERROR_MARK; - enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK; + tree_code tc1; + code_helper code, code1, code2; + code_helper codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK; tree new_temp; enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; int ndts = 2; @@ -5008,31 +5008,43 @@ vectorizable_conversion (vec_info *vinfo, && ! vec_stmt) return false; - gassign *stmt = dyn_cast (stmt_info->stmt); - if (!stmt) + gimple* stmt = stmt_info->stmt; + if (!(is_gimple_assign (stmt) || is_gimple_call (stmt))) return false; - if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME) + if (gimple_get_lhs (stmt) == NULL_TREE + || TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME) return false; - code = gimple_assign_rhs_code (stmt); - if (!CONVERT_EXPR_CODE_P (code) - && code != FIX_TRUNC_EXPR - && code != FLOAT_EXPR - && code != WIDEN_PLUS_EXPR - && code != WIDEN_MINUS_EXPR - && code != WIDEN_MULT_EXPR - && code != WIDEN_LSHIFT_EXPR) + if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME) + return false; + + if (is_gimple_assign (stmt)) + { + code = gimple_assign_rhs_code (stmt); + op_type = TREE_CODE_LENGTH ((tree_code) code); + } + else if (gimple_call_internal_p (stmt)) + { + code = gimple_call_internal_fn (stmt); + op_type = gimple_call_num_args (stmt); + } + else return false; bool widen_arith = (code == WIDEN_PLUS_EXPR - || code == WIDEN_MINUS_EXPR - || code == WIDEN_MULT_EXPR - || code == WIDEN_LSHIFT_EXPR); - op_type = TREE_CODE_LENGTH (code); + || code == WIDEN_MINUS_EXPR + || code == WIDEN_MULT_EXPR + || code == WIDEN_LSHIFT_EXPR); + + if (!widen_arith + && !CONVERT_EXPR_CODE_P (code) + && code != FIX_TRUNC_EXPR + && code != FLOAT_EXPR) + return false; /* Check types of lhs and rhs. */ - scalar_dest = gimple_assign_lhs (stmt); + scalar_dest = gimple_get_lhs (stmt); lhs_type = TREE_TYPE (scalar_dest); vectype_out = STMT_VINFO_VECTYPE (stmt_info); @@ -5070,10 +5082,14 @@ vectorizable_conversion (vec_info *vinfo, if (op_type == binary_op) { - gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR - || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR); + gcc_assert (code == WIDEN_MULT_EXPR + || code == WIDEN_LSHIFT_EXPR + || code == WIDEN_PLUS_EXPR + || code == WIDEN_MINUS_EXPR); + - op1 = gimple_assign_rhs2 (stmt); + op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) : + gimple_call_arg (stmt, 0); tree vectype1_in; if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1, &op1, &slp_op1, &dt[1], &vectype1_in)) @@ -5157,8 +5173,13 @@ vectorizable_conversion (vec_info *vinfo, && code != FLOAT_EXPR && !CONVERT_EXPR_CODE_P (code)) return false; - if (supportable_convert_operation (code, vectype_out, vectype_in, &code1)) + gcc_assert (code.is_tree_code ()); + if (supportable_convert_operation ((tree_code) code, vectype_out, + vectype_in, &tc1)) + { + code1 = tc1; break; + } /* FALLTHRU */ unsupported: if (dump_enabled_p ()) @@ -5169,9 +5190,12 @@ vectorizable_conversion (vec_info *vinfo, case WIDEN: if (known_eq (nunits_in, nunits_out)) { - if (!supportable_half_widening_operation (code, vectype_out, - vectype_in, &code1)) + if (!(code.is_tree_code () + && supportable_half_widening_operation ((tree_code) code, + vectype_out, vectype_in, + &tc1))) goto unsupported; + code1 = tc1; gcc_assert (!(multi_step_cvt && op_type == binary_op)); break; } @@ -5205,14 +5229,17 @@ vectorizable_conversion (vec_info *vinfo, if (GET_MODE_SIZE (rhs_mode) == fltsz) { - if (!supportable_convert_operation (code, vectype_out, - cvt_type, &codecvt1)) + tc1 = ERROR_MARK; + gcc_assert (code.is_tree_code ()); + if (!supportable_convert_operation ((tree_code) code, vectype_out, + cvt_type, &tc1)) goto unsupported; + codecvt1 = tc1; } - else if (!supportable_widening_operation (vinfo, code, stmt_info, - vectype_out, cvt_type, - &codecvt1, &codecvt2, - &multi_step_cvt, + else if (!supportable_widening_operation (vinfo, code, + stmt_info, vectype_out, + cvt_type, &codecvt1, + &codecvt2, &multi_step_cvt, &interm_types)) continue; else @@ -5220,8 +5247,9 @@ vectorizable_conversion (vec_info *vinfo, if (supportable_widening_operation (vinfo, NOP_EXPR, stmt_info, cvt_type, - vectype_in, &code1, &code2, - &multi_step_cvt, &interm_types)) + vectype_in, &code1, + &code2, &multi_step_cvt, + &interm_types)) { found_mode = true; break; @@ -5257,9 +5285,11 @@ vectorizable_conversion (vec_info *vinfo, cvt_type = get_same_sized_vectype (cvt_type, vectype_in); if (cvt_type == NULL_TREE) goto unsupported; - if (!supportable_convert_operation (code, cvt_type, vectype_in, - &codecvt1)) + if (!code.is_tree_code () + || !supportable_convert_operation ((tree_code) code, cvt_type, + vectype_in, &tc1)) goto unsupported; + codecvt1 = tc1; if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_type, &code1, &multi_step_cvt, &interm_types)) @@ -5377,10 +5407,9 @@ vectorizable_conversion (vec_info *vinfo, FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) { /* Arguments are ready, create the new vector stmt. */ - gcc_assert (TREE_CODE_LENGTH (code1) == unary_op); - gassign *new_stmt = gimple_build_assign (vec_dest, code1, vop0); + gimple *new_stmt = vect_gimple_build (vec_dest, code1, vop0); new_temp = make_ssa_name (vec_dest, new_stmt); - gimple_assign_set_lhs (new_stmt, new_temp); + gimple_set_lhs (new_stmt, new_temp); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); if (slp_node) @@ -5410,17 +5439,16 @@ vectorizable_conversion (vec_info *vinfo, for (i = multi_step_cvt; i >= 0; i--) { tree this_dest = vec_dsts[i]; - enum tree_code c1 = code1, c2 = code2; + code_helper c1 = code1, c2 = code2; if (i == 0 && codecvt2 != ERROR_MARK) { c1 = codecvt1; c2 = codecvt2; } if (known_eq (nunits_out, nunits_in)) - vect_create_half_widening_stmts (vinfo, &vec_oprnds0, - &vec_oprnds1, stmt_info, - this_dest, gsi, - c1, op_type); + vect_create_half_widening_stmts (vinfo, &vec_oprnds0, &vec_oprnds1, + stmt_info, this_dest, gsi, c1, + op_type); else vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0, &vec_oprnds1, stmt_info, @@ -5433,9 +5461,8 @@ vectorizable_conversion (vec_info *vinfo, gimple *new_stmt; if (cvt_type) { - gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op); new_temp = make_ssa_name (vec_dest); - new_stmt = gimple_build_assign (new_temp, codecvt1, vop0); + new_stmt = vect_gimple_build (new_temp, codecvt1, vop0); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); } else @@ -5459,10 +5486,8 @@ vectorizable_conversion (vec_info *vinfo, if (cvt_type) FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) { - gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op); new_temp = make_ssa_name (vec_dest); - gassign *new_stmt - = gimple_build_assign (new_temp, codecvt1, vop0); + gimple *new_stmt = vect_gimple_build (new_temp, codecvt1, vop0); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); vec_oprnds0[i] = new_temp; } @@ -12151,9 +12176,11 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype) bool supportable_widening_operation (vec_info *vinfo, - enum tree_code code, stmt_vec_info stmt_info, + code_helper code, + stmt_vec_info stmt_info, tree vectype_out, tree vectype_in, - enum tree_code *code1, enum tree_code *code2, + code_helper *code1, + code_helper *code2, int *multi_step_cvt, vec *interm_types) { @@ -12164,7 +12191,7 @@ supportable_widening_operation (vec_info *vinfo, optab optab1, optab2; tree vectype = vectype_in; tree wide_vectype = vectype_out; - enum tree_code c1, c2; + tree_code c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES; int i; tree prev_type, intermediate_type; machine_mode intermediate_mode, prev_mode; @@ -12174,8 +12201,12 @@ supportable_widening_operation (vec_info *vinfo, if (loop_info) vect_loop = LOOP_VINFO_LOOP (loop_info); - switch (code) + switch (code.safe_as_tree_code ()) { + case MAX_TREE_CODES: + /* Don't set c1 and c2 if code is not a tree_code. */ + break; + case WIDEN_MULT_EXPR: /* The result of a vectorized widening operation usually requires two vectors (because the widened results do not fit into one vector). @@ -12215,8 +12246,9 @@ supportable_widening_operation (vec_info *vinfo, && !nested_in_vect_loop_p (vect_loop, stmt_info) && supportable_widening_operation (vinfo, VEC_WIDEN_MULT_EVEN_EXPR, stmt_info, vectype_out, - vectype_in, code1, code2, - multi_step_cvt, interm_types)) + vectype_in, code1, + code2, multi_step_cvt, + interm_types)) { /* Elements in a vector with vect_used_by_reduction property cannot be reordered if the use chain with this property does not have the @@ -12292,7 +12324,7 @@ supportable_widening_operation (vec_info *vinfo, optab1 = optab_for_tree_code (c1, vectype_out, optab_default); optab2 = optab_for_tree_code (c2, vectype_out, optab_default); } - else if (CONVERT_EXPR_CODE_P (code) + else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ()) && VECTOR_BOOLEAN_TYPE_P (wide_vectype) && VECTOR_BOOLEAN_TYPE_P (vectype) && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype) @@ -12317,8 +12349,12 @@ supportable_widening_operation (vec_info *vinfo, || (icode2 = optab_handler (optab2, vec_mode)) == CODE_FOR_nothing) return false; - *code1 = c1; - *code2 = c2; + if (code.is_tree_code ()) + { + *code1 = c1; + *code2 = c2; + } + if (insn_data[icode1].operand[0].mode == TYPE_MODE (wide_vectype) && insn_data[icode2].operand[0].mode == TYPE_MODE (wide_vectype)) @@ -12339,7 +12375,7 @@ supportable_widening_operation (vec_info *vinfo, prev_type = vectype; prev_mode = vec_mode; - if (!CONVERT_EXPR_CODE_P (code)) + if (!CONVERT_EXPR_CODE_P ((tree_code) code)) return false; /* We assume here that there will not be more than MAX_INTERM_CVT_STEPS @@ -12437,9 +12473,9 @@ supportable_widening_operation (vec_info *vinfo, narrowing operation (short in the above example). */ bool -supportable_narrowing_operation (enum tree_code code, +supportable_narrowing_operation (code_helper code, tree vectype_out, tree vectype_in, - enum tree_code *code1, int *multi_step_cvt, + code_helper *code1, int *multi_step_cvt, vec *interm_types) { machine_mode vec_mode; @@ -12454,8 +12490,11 @@ supportable_narrowing_operation (enum tree_code code, unsigned HOST_WIDE_INT n_elts; bool uns; + if (!code.is_tree_code ()) + return false; + *multi_step_cvt = 0; - switch (code) + switch ((tree_code) code) { CASE_CONVERT: c1 = VEC_PACK_TRUNC_EXPR; diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 9cf2fb23fe397b467d89aa7cc5ebeaa293ed4cce..f215cd0639bcf803c9d0554cfdc57823431991d5 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -2139,13 +2139,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree, enum vect_def_type *, tree *, stmt_vec_info * = NULL); extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree); -extern bool supportable_widening_operation (vec_info *, - enum tree_code, stmt_vec_info, - tree, tree, enum tree_code *, - enum tree_code *, int *, - vec *); -extern bool supportable_narrowing_operation (enum tree_code, tree, tree, - enum tree_code *, int *, +extern bool supportable_widening_operation (vec_info*, code_helper, + stmt_vec_info, tree, tree, + code_helper*, code_helper*, + int*, vec *); +extern bool supportable_narrowing_operation (code_helper, tree, tree, + code_helper *, int *, vec *); extern unsigned record_stmt_cost (stmt_vector_for_cost *, int, @@ -2583,4 +2582,7 @@ vect_is_integer_truncation (stmt_vec_info stmt_info) && TYPE_PRECISION (lhs_type) < TYPE_PRECISION (rhs_type)); } +/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code, + or internal_fn contained in ch, respectively. */ +gimple * vect_gimple_build (tree, code_helper, tree, tree = NULL_TREE); #endif /* GCC_TREE_VECTORIZER_H */ diff --git a/gcc/tree.h b/gcc/tree.h index abcdb5638d49aea4ccc46efa8e540b1fa78aa27a..f6cd528e7d789c3f81fb2da3c1e1a29fa11f6e0f 100644 --- a/gcc/tree.h +++ b/gcc/tree.h @@ -93,6 +93,8 @@ public: bool is_internal_fn () const; bool is_builtin_fn () const; int get_rep () const { return rep; } + enum tree_code safe_as_tree_code () const; + combined_fn safe_as_fn_code () const; bool operator== (const code_helper &other) { return rep == other.rep; } bool operator!= (const code_helper &other) { return rep != other.rep; } bool operator== (tree_code c) { return rep == code_helper (c).rep; } @@ -102,6 +104,17 @@ private: int rep; }; +inline enum tree_code +code_helper::safe_as_tree_code () const +{ + return is_tree_code () ? (tree_code) *this : MAX_TREE_CODES; +} + +inline combined_fn +code_helper::safe_as_fn_code () const { + return is_fn_code () ? (combined_fn) *this : CFN_LAST; +} + inline code_helper::operator internal_fn () const { return as_internal_fn (combined_fn (*this));