From patchwork Fri Apr 28 12:36:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Andre Vieira (lists)" X-Patchwork-Id: 1774956 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=hLAk8xxn; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q7C0d6hHKz23td for ; Fri, 28 Apr 2023 22:41:53 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DE85F38708CF for ; Fri, 28 Apr 2023 12:41:51 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org DE85F38708CF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1682685711; bh=jztMJ+sAiF9uC3du2aIGBs5+PDIP+7NQ3eEcGPaJ5zY=; h=Date:To:Cc:References:Subject:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=hLAk8xxn7IRVqM2714qb1uWLAYf7t2W8e5HGlXxwsrVSYPIZBkctSQ4hfZF9baEmu QA3bTxPIPP2OnF7AOaQZ1SgsqxouYe58P+eVWzIsYt1d+Ak2pQ2VYr1Cu2BeJFBaPK qFlz6fT/Fufw+8Q+xI2DaDgukl+t27iUSlyAWJ2A= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id A2D38385354F for ; Fri, 28 Apr 2023 12:37:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A2D38385354F Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C61812FC; Fri, 28 Apr 2023 05:37:50 -0700 (PDT) Received: from [10.57.70.10] (unknown [10.57.70.10]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 79B483F5A1; Fri, 28 Apr 2023 05:37:05 -0700 (PDT) Message-ID: Date: Fri, 28 Apr 2023 13:36:59 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Content-Language: en-US To: Richard Biener Cc: Richard Biener , Richard Sandiford , "gcc-patches@gcc.gnu.org" References: <51ce8969-3130-452e-092e-f9d91eff2dad@arm.com> Subject: [PATCH 1/3] Refactor to allow internal_fn's In-Reply-To: X-Spam-Status: No, score=-14.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_LOTSOFHASH, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Andre Vieira \(lists\) via Gcc-patches" From: "Andre Vieira (lists)" Reply-To: "Andre Vieira \(lists\)" Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi, I'm posting the patches separately now with ChangeLogs. I made the suggested changes and tried to simplify the code a bit further. Where internal to tree-vect-stmts I changed most functions to use code_helper to avoid having to check at places we didn't need to. I was trying to simplify things further by also modifying supportable_half_widening_operation and supportable_convert_operation but the result of that was that I ended up moving the code to cast to tree code inside them rather than at the call site and it didn't look simpler, so I left those. Though if we did make those changes we'd no longer need to keep around the tc1 variable in vectorizable_conversion... Let me know what you think. gcc/ChangeLog: 2023-04-28 Andre Vieira Joel Hutton * tree-vect-patterns.cc (vect_gimple_build): New Function. (vect_recog_widen_op_pattern): Refactor to use code_helper. * tree-vect-stmts.cc (vect_gen_widened_results_half): Likewise. (vect_create_vectorized_demotion_stmts): Likewise. (vect_create_vectorized_promotion_stmts): Likewise. (vect_create_half_widening_stmts): Likewise. (vectorizable_conversion): Likewise. (vectorizable_call): Likewise. (supportable_widening_operation): Likewise. (supportable_narrowing_operation): Likewise. (simple_integer_narrowing): Likewise. * tree-vectorizer.h (supportable_widening_operation): Likewise. (supportable_narrowing_operation): Likewise. (vect_gimple_build): New function prototype. * tree.h (code_helper::safe_as_tree_code): New function. (code_helper::safe_as_fn_code): New function. diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 8802141cd6edb298866025b8a55843eae1f0eb17..b35023adade94c1996cd076c4b7419560e819c6b 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -25,6 +25,8 @@ along with GCC; see the file COPYING3. If not see #include "rtl.h" #include "tree.h" #include "gimple.h" +#include "gimple-iterator.h" +#include "gimple-fold.h" #include "ssa.h" #include "expmed.h" #include "optabs-tree.h" @@ -1391,7 +1393,7 @@ vect_recog_sad_pattern (vec_info *vinfo, static gimple * vect_recog_widen_op_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info, tree *type_out, - tree_code orig_code, tree_code wide_code, + tree_code orig_code, code_helper wide_code, bool shift_p, const char *name) { gimple *last_stmt = last_stmt_info->stmt; @@ -1434,7 +1436,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo, vecctype = get_vectype_for_scalar_type (vinfo, ctype); } - enum tree_code dummy_code; + code_helper dummy_code; int dummy_int; auto_vec dummy_vec; if (!vectype @@ -1455,8 +1457,7 @@ vect_recog_widen_op_pattern (vec_info *vinfo, 2, oprnd, half_type, unprom, vectype); tree var = vect_recog_temp_ssa_var (itype, NULL); - gimple *pattern_stmt = gimple_build_assign (var, wide_code, - oprnd[0], oprnd[1]); + gimple *pattern_stmt = vect_gimple_build (var, wide_code, oprnd[0], oprnd[1]); if (vecctype != vecitype) pattern_stmt = vect_convert_output (vinfo, last_stmt_info, ctype, @@ -6406,3 +6407,20 @@ vect_pattern_recog (vec_info *vinfo) /* After this no more add_stmt calls are allowed. */ vinfo->stmt_vec_info_ro = true; } + +/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code, + or internal_fn contained in ch, respectively. */ +gimple * +vect_gimple_build (tree lhs, code_helper ch, tree op0, tree op1) +{ + gcc_assert (op0 != NULL_TREE); + if (ch.is_tree_code ()) + return gimple_build_assign (lhs, (tree_code) ch, op0, op1); + + gcc_assert (ch.is_internal_fn ()); + gimple* stmt = gimple_build_call_internal (as_internal_fn ((combined_fn) ch), + op1 == NULL_TREE ? 1 : 2, + op0, op1); + gimple_call_set_lhs (stmt, lhs); + return stmt; +} diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 6b7dbfd4a231baec24e740ffe0ce0b0bf7a1de6b..ce47f4940fa9a1baca4ba1162065cfc3b4072eba 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -3258,13 +3258,13 @@ vectorizable_bswap (vec_info *vinfo, static bool simple_integer_narrowing (tree vectype_out, tree vectype_in, - tree_code *convert_code) + code_helper *convert_code) { if (!INTEGRAL_TYPE_P (TREE_TYPE (vectype_out)) || !INTEGRAL_TYPE_P (TREE_TYPE (vectype_in))) return false; - tree_code code; + code_helper code; int multi_step_cvt = 0; auto_vec interm_types; if (!supportable_narrowing_operation (NOP_EXPR, vectype_out, vectype_in, @@ -3478,7 +3478,7 @@ vectorizable_call (vec_info *vinfo, tree callee = gimple_call_fndecl (stmt); /* First try using an internal function. */ - tree_code convert_code = ERROR_MARK; + code_helper convert_code = MAX_TREE_CODES; if (cfn != CFN_LAST && (modifier == NONE || (modifier == NARROW @@ -3664,8 +3664,8 @@ vectorizable_call (vec_info *vinfo, continue; } new_temp = make_ssa_name (vec_dest); - new_stmt = gimple_build_assign (new_temp, convert_code, - prev_res, half_res); + new_stmt = vect_gimple_build (new_temp, convert_code, + prev_res, half_res); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); } @@ -3755,8 +3755,8 @@ vectorizable_call (vec_info *vinfo, continue; } new_temp = make_ssa_name (vec_dest); - new_stmt = gimple_build_assign (new_temp, convert_code, - prev_res, half_res); + new_stmt = vect_gimple_build (new_temp, convert_code, prev_res, + half_res); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); } else @@ -4768,7 +4768,7 @@ vectorizable_simd_clone_call (vec_info *vinfo, stmt_vec_info stmt_info, STMT_INFO is the original scalar stmt that we are vectorizing. */ static gimple * -vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code, +vect_gen_widened_results_half (vec_info *vinfo, code_helper ch, tree vec_oprnd0, tree vec_oprnd1, int op_type, tree vec_dest, gimple_stmt_iterator *gsi, stmt_vec_info stmt_info) @@ -4777,12 +4777,11 @@ vect_gen_widened_results_half (vec_info *vinfo, enum tree_code code, tree new_temp; /* Generate half of the widened result: */ - gcc_assert (op_type == TREE_CODE_LENGTH (code)); if (op_type != binary_op) vec_oprnd1 = NULL; - new_stmt = gimple_build_assign (vec_dest, code, vec_oprnd0, vec_oprnd1); + new_stmt = vect_gimple_build (vec_dest, ch, vec_oprnd0, vec_oprnd1); new_temp = make_ssa_name (vec_dest, new_stmt); - gimple_assign_set_lhs (new_stmt, new_temp); + gimple_set_lhs (new_stmt, new_temp); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); return new_stmt; @@ -4799,7 +4798,7 @@ vect_create_vectorized_demotion_stmts (vec_info *vinfo, vec *vec_oprnds, stmt_vec_info stmt_info, vec &vec_dsts, gimple_stmt_iterator *gsi, - slp_tree slp_node, enum tree_code code) + slp_tree slp_node, code_helper code) { unsigned int i; tree vop0, vop1, new_tmp, vec_dest; @@ -4811,9 +4810,9 @@ vect_create_vectorized_demotion_stmts (vec_info *vinfo, vec *vec_oprnds, /* Create demotion operation. */ vop0 = (*vec_oprnds)[i]; vop1 = (*vec_oprnds)[i + 1]; - gassign *new_stmt = gimple_build_assign (vec_dest, code, vop0, vop1); + gimple *new_stmt = vect_gimple_build (vec_dest, code, vop0, vop1); new_tmp = make_ssa_name (vec_dest, new_stmt); - gimple_assign_set_lhs (new_stmt, new_tmp); + gimple_set_lhs (new_stmt, new_tmp); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); if (multi_step_cvt) @@ -4861,8 +4860,8 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo, vec *vec_oprnds1, stmt_vec_info stmt_info, tree vec_dest, gimple_stmt_iterator *gsi, - enum tree_code code1, - enum tree_code code2, int op_type) + code_helper ch1, + code_helper ch2, int op_type) { int i; tree vop0, vop1, new_tmp1, new_tmp2; @@ -4878,10 +4877,10 @@ vect_create_vectorized_promotion_stmts (vec_info *vinfo, vop1 = NULL_TREE; /* Generate the two halves of promotion operation. */ - new_stmt1 = vect_gen_widened_results_half (vinfo, code1, vop0, vop1, + new_stmt1 = vect_gen_widened_results_half (vinfo, ch1, vop0, vop1, op_type, vec_dest, gsi, stmt_info); - new_stmt2 = vect_gen_widened_results_half (vinfo, code2, vop0, vop1, + new_stmt2 = vect_gen_widened_results_half (vinfo, ch2, vop0, vop1, op_type, vec_dest, gsi, stmt_info); if (is_gimple_call (new_stmt1)) @@ -4912,7 +4911,7 @@ vect_create_half_widening_stmts (vec_info *vinfo, vec *vec_oprnds1, stmt_vec_info stmt_info, tree vec_dest, gimple_stmt_iterator *gsi, - enum tree_code code1, + code_helper code1, int op_type) { int i; @@ -4942,13 +4941,13 @@ vect_create_half_widening_stmts (vec_info *vinfo, new_stmt2 = gimple_build_assign (new_tmp2, NOP_EXPR, vop1); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt2, gsi); /* Perform the operation. With both vector inputs widened. */ - new_stmt3 = gimple_build_assign (vec_dest, code1, new_tmp1, new_tmp2); + new_stmt3 = vect_gimple_build (vec_dest, code1, new_tmp1, new_tmp2); } else { /* Perform the operation. With the single vector input widened. */ - new_stmt3 = gimple_build_assign (vec_dest, code1, new_tmp1, vop1); - } + new_stmt3 = vect_gimple_build (vec_dest, code1, new_tmp1, vop1); + } new_tmp3 = make_ssa_name (vec_dest, new_stmt3); gimple_assign_set_lhs (new_stmt3, new_tmp3); @@ -4978,8 +4977,9 @@ vectorizable_conversion (vec_info *vinfo, tree scalar_dest; tree op0, op1 = NULL_TREE; loop_vec_info loop_vinfo = dyn_cast (vinfo); - enum tree_code code, code1 = ERROR_MARK, code2 = ERROR_MARK; - enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK; + tree_code tc1; + code_helper code, code1, code2; + code_helper codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK; tree new_temp; enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; int ndts = 2; @@ -5008,31 +5008,43 @@ vectorizable_conversion (vec_info *vinfo, && ! vec_stmt) return false; - gassign *stmt = dyn_cast (stmt_info->stmt); - if (!stmt) + gimple* stmt = stmt_info->stmt; + if (!(is_gimple_assign (stmt) || is_gimple_call (stmt))) return false; - if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME) + if (gimple_get_lhs (stmt) == NULL_TREE + || TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME) return false; - code = gimple_assign_rhs_code (stmt); - if (!CONVERT_EXPR_CODE_P (code) - && code != FIX_TRUNC_EXPR - && code != FLOAT_EXPR - && code != WIDEN_PLUS_EXPR - && code != WIDEN_MINUS_EXPR - && code != WIDEN_MULT_EXPR - && code != WIDEN_LSHIFT_EXPR) + if (TREE_CODE (gimple_get_lhs (stmt)) != SSA_NAME) + return false; + + if (is_gimple_assign (stmt)) + { + code = gimple_assign_rhs_code (stmt); + op_type = TREE_CODE_LENGTH ((tree_code) code); + } + else if (gimple_call_internal_p (stmt)) + { + code = gimple_call_internal_fn (stmt); + op_type = gimple_call_num_args (stmt); + } + else return false; bool widen_arith = (code == WIDEN_PLUS_EXPR - || code == WIDEN_MINUS_EXPR - || code == WIDEN_MULT_EXPR - || code == WIDEN_LSHIFT_EXPR); - op_type = TREE_CODE_LENGTH (code); + || code == WIDEN_MINUS_EXPR + || code == WIDEN_MULT_EXPR + || code == WIDEN_LSHIFT_EXPR); + + if (!widen_arith + && !CONVERT_EXPR_CODE_P (code) + && code != FIX_TRUNC_EXPR + && code != FLOAT_EXPR) + return false; /* Check types of lhs and rhs. */ - scalar_dest = gimple_assign_lhs (stmt); + scalar_dest = gimple_get_lhs (stmt); lhs_type = TREE_TYPE (scalar_dest); vectype_out = STMT_VINFO_VECTYPE (stmt_info); @@ -5070,10 +5082,14 @@ vectorizable_conversion (vec_info *vinfo, if (op_type == binary_op) { - gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR - || code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR); + gcc_assert (code == WIDEN_MULT_EXPR + || code == WIDEN_LSHIFT_EXPR + || code == WIDEN_PLUS_EXPR + || code == WIDEN_MINUS_EXPR); + - op1 = gimple_assign_rhs2 (stmt); + op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) : + gimple_call_arg (stmt, 0); tree vectype1_in; if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 1, &op1, &slp_op1, &dt[1], &vectype1_in)) @@ -5157,8 +5173,13 @@ vectorizable_conversion (vec_info *vinfo, && code != FLOAT_EXPR && !CONVERT_EXPR_CODE_P (code)) return false; - if (supportable_convert_operation (code, vectype_out, vectype_in, &code1)) + gcc_assert (code.is_tree_code ()); + if (supportable_convert_operation ((tree_code) code, vectype_out, + vectype_in, &tc1)) + { + code1 = tc1; break; + } /* FALLTHRU */ unsupported: if (dump_enabled_p ()) @@ -5169,9 +5190,12 @@ vectorizable_conversion (vec_info *vinfo, case WIDEN: if (known_eq (nunits_in, nunits_out)) { - if (!supportable_half_widening_operation (code, vectype_out, - vectype_in, &code1)) + if (!(code.is_tree_code () + && supportable_half_widening_operation ((tree_code) code, + vectype_out, vectype_in, + &tc1))) goto unsupported; + code1 = tc1; gcc_assert (!(multi_step_cvt && op_type == binary_op)); break; } @@ -5205,14 +5229,17 @@ vectorizable_conversion (vec_info *vinfo, if (GET_MODE_SIZE (rhs_mode) == fltsz) { - if (!supportable_convert_operation (code, vectype_out, - cvt_type, &codecvt1)) + tc1 = ERROR_MARK; + gcc_assert (code.is_tree_code ()); + if (!supportable_convert_operation ((tree_code) code, vectype_out, + cvt_type, &tc1)) goto unsupported; + codecvt1 = tc1; } - else if (!supportable_widening_operation (vinfo, code, stmt_info, - vectype_out, cvt_type, - &codecvt1, &codecvt2, - &multi_step_cvt, + else if (!supportable_widening_operation (vinfo, code, + stmt_info, vectype_out, + cvt_type, &codecvt1, + &codecvt2, &multi_step_cvt, &interm_types)) continue; else @@ -5220,8 +5247,9 @@ vectorizable_conversion (vec_info *vinfo, if (supportable_widening_operation (vinfo, NOP_EXPR, stmt_info, cvt_type, - vectype_in, &code1, &code2, - &multi_step_cvt, &interm_types)) + vectype_in, &code1, + &code2, &multi_step_cvt, + &interm_types)) { found_mode = true; break; @@ -5257,9 +5285,11 @@ vectorizable_conversion (vec_info *vinfo, cvt_type = get_same_sized_vectype (cvt_type, vectype_in); if (cvt_type == NULL_TREE) goto unsupported; - if (!supportable_convert_operation (code, cvt_type, vectype_in, - &codecvt1)) + if (!code.is_tree_code () + || !supportable_convert_operation ((tree_code) code, cvt_type, + vectype_in, &tc1)) goto unsupported; + codecvt1 = tc1; if (supportable_narrowing_operation (NOP_EXPR, vectype_out, cvt_type, &code1, &multi_step_cvt, &interm_types)) @@ -5377,10 +5407,9 @@ vectorizable_conversion (vec_info *vinfo, FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) { /* Arguments are ready, create the new vector stmt. */ - gcc_assert (TREE_CODE_LENGTH (code1) == unary_op); - gassign *new_stmt = gimple_build_assign (vec_dest, code1, vop0); + gimple *new_stmt = vect_gimple_build (vec_dest, code1, vop0); new_temp = make_ssa_name (vec_dest, new_stmt); - gimple_assign_set_lhs (new_stmt, new_temp); + gimple_set_lhs (new_stmt, new_temp); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); if (slp_node) @@ -5410,17 +5439,16 @@ vectorizable_conversion (vec_info *vinfo, for (i = multi_step_cvt; i >= 0; i--) { tree this_dest = vec_dsts[i]; - enum tree_code c1 = code1, c2 = code2; + code_helper c1 = code1, c2 = code2; if (i == 0 && codecvt2 != ERROR_MARK) { c1 = codecvt1; c2 = codecvt2; } if (known_eq (nunits_out, nunits_in)) - vect_create_half_widening_stmts (vinfo, &vec_oprnds0, - &vec_oprnds1, stmt_info, - this_dest, gsi, - c1, op_type); + vect_create_half_widening_stmts (vinfo, &vec_oprnds0, &vec_oprnds1, + stmt_info, this_dest, gsi, c1, + op_type); else vect_create_vectorized_promotion_stmts (vinfo, &vec_oprnds0, &vec_oprnds1, stmt_info, @@ -5433,9 +5461,8 @@ vectorizable_conversion (vec_info *vinfo, gimple *new_stmt; if (cvt_type) { - gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op); new_temp = make_ssa_name (vec_dest); - new_stmt = gimple_build_assign (new_temp, codecvt1, vop0); + new_stmt = vect_gimple_build (new_temp, codecvt1, vop0); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); } else @@ -5459,10 +5486,8 @@ vectorizable_conversion (vec_info *vinfo, if (cvt_type) FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) { - gcc_assert (TREE_CODE_LENGTH (codecvt1) == unary_op); new_temp = make_ssa_name (vec_dest); - gassign *new_stmt - = gimple_build_assign (new_temp, codecvt1, vop0); + gimple *new_stmt = vect_gimple_build (new_temp, codecvt1, vop0); vect_finish_stmt_generation (vinfo, stmt_info, new_stmt, gsi); vec_oprnds0[i] = new_temp; } @@ -12151,9 +12176,11 @@ vect_maybe_update_slp_op_vectype (slp_tree op, tree vectype) bool supportable_widening_operation (vec_info *vinfo, - enum tree_code code, stmt_vec_info stmt_info, + code_helper code, + stmt_vec_info stmt_info, tree vectype_out, tree vectype_in, - enum tree_code *code1, enum tree_code *code2, + code_helper *code1, + code_helper *code2, int *multi_step_cvt, vec *interm_types) { @@ -12164,7 +12191,7 @@ supportable_widening_operation (vec_info *vinfo, optab optab1, optab2; tree vectype = vectype_in; tree wide_vectype = vectype_out; - enum tree_code c1, c2; + tree_code c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES; int i; tree prev_type, intermediate_type; machine_mode intermediate_mode, prev_mode; @@ -12174,8 +12201,12 @@ supportable_widening_operation (vec_info *vinfo, if (loop_info) vect_loop = LOOP_VINFO_LOOP (loop_info); - switch (code) + switch (code.safe_as_tree_code ()) { + case MAX_TREE_CODES: + /* Don't set c1 and c2 if code is not a tree_code. */ + break; + case WIDEN_MULT_EXPR: /* The result of a vectorized widening operation usually requires two vectors (because the widened results do not fit into one vector). @@ -12215,8 +12246,9 @@ supportable_widening_operation (vec_info *vinfo, && !nested_in_vect_loop_p (vect_loop, stmt_info) && supportable_widening_operation (vinfo, VEC_WIDEN_MULT_EVEN_EXPR, stmt_info, vectype_out, - vectype_in, code1, code2, - multi_step_cvt, interm_types)) + vectype_in, code1, + code2, multi_step_cvt, + interm_types)) { /* Elements in a vector with vect_used_by_reduction property cannot be reordered if the use chain with this property does not have the @@ -12292,7 +12324,7 @@ supportable_widening_operation (vec_info *vinfo, optab1 = optab_for_tree_code (c1, vectype_out, optab_default); optab2 = optab_for_tree_code (c2, vectype_out, optab_default); } - else if (CONVERT_EXPR_CODE_P (code) + else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ()) && VECTOR_BOOLEAN_TYPE_P (wide_vectype) && VECTOR_BOOLEAN_TYPE_P (vectype) && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype) @@ -12317,8 +12349,12 @@ supportable_widening_operation (vec_info *vinfo, || (icode2 = optab_handler (optab2, vec_mode)) == CODE_FOR_nothing) return false; - *code1 = c1; - *code2 = c2; + if (code.is_tree_code ()) + { + *code1 = c1; + *code2 = c2; + } + if (insn_data[icode1].operand[0].mode == TYPE_MODE (wide_vectype) && insn_data[icode2].operand[0].mode == TYPE_MODE (wide_vectype)) @@ -12339,7 +12375,7 @@ supportable_widening_operation (vec_info *vinfo, prev_type = vectype; prev_mode = vec_mode; - if (!CONVERT_EXPR_CODE_P (code)) + if (!CONVERT_EXPR_CODE_P ((tree_code) code)) return false; /* We assume here that there will not be more than MAX_INTERM_CVT_STEPS @@ -12437,9 +12473,9 @@ supportable_widening_operation (vec_info *vinfo, narrowing operation (short in the above example). */ bool -supportable_narrowing_operation (enum tree_code code, +supportable_narrowing_operation (code_helper code, tree vectype_out, tree vectype_in, - enum tree_code *code1, int *multi_step_cvt, + code_helper *code1, int *multi_step_cvt, vec *interm_types) { machine_mode vec_mode; @@ -12454,8 +12490,11 @@ supportable_narrowing_operation (enum tree_code code, unsigned HOST_WIDE_INT n_elts; bool uns; + if (!code.is_tree_code ()) + return false; + *multi_step_cvt = 0; - switch (code) + switch ((tree_code) code) { CASE_CONVERT: c1 = VEC_PACK_TRUNC_EXPR; diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 9cf2fb23fe397b467d89aa7cc5ebeaa293ed4cce..f215cd0639bcf803c9d0554cfdc57823431991d5 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -2139,13 +2139,12 @@ extern bool vect_is_simple_use (vec_info *, stmt_vec_info, slp_tree, enum vect_def_type *, tree *, stmt_vec_info * = NULL); extern bool vect_maybe_update_slp_op_vectype (slp_tree, tree); -extern bool supportable_widening_operation (vec_info *, - enum tree_code, stmt_vec_info, - tree, tree, enum tree_code *, - enum tree_code *, int *, - vec *); -extern bool supportable_narrowing_operation (enum tree_code, tree, tree, - enum tree_code *, int *, +extern bool supportable_widening_operation (vec_info*, code_helper, + stmt_vec_info, tree, tree, + code_helper*, code_helper*, + int*, vec *); +extern bool supportable_narrowing_operation (code_helper, tree, tree, + code_helper *, int *, vec *); extern unsigned record_stmt_cost (stmt_vector_for_cost *, int, @@ -2583,4 +2582,7 @@ vect_is_integer_truncation (stmt_vec_info stmt_info) && TYPE_PRECISION (lhs_type) < TYPE_PRECISION (rhs_type)); } +/* Build a GIMPLE_ASSIGN or GIMPLE_CALL with the tree_code, + or internal_fn contained in ch, respectively. */ +gimple * vect_gimple_build (tree, code_helper, tree, tree = NULL_TREE); #endif /* GCC_TREE_VECTORIZER_H */ diff --git a/gcc/tree.h b/gcc/tree.h index abcdb5638d49aea4ccc46efa8e540b1fa78aa27a..f6cd528e7d789c3f81fb2da3c1e1a29fa11f6e0f 100644 --- a/gcc/tree.h +++ b/gcc/tree.h @@ -93,6 +93,8 @@ public: bool is_internal_fn () const; bool is_builtin_fn () const; int get_rep () const { return rep; } + enum tree_code safe_as_tree_code () const; + combined_fn safe_as_fn_code () const; bool operator== (const code_helper &other) { return rep == other.rep; } bool operator!= (const code_helper &other) { return rep != other.rep; } bool operator== (tree_code c) { return rep == code_helper (c).rep; } @@ -102,6 +104,17 @@ private: int rep; }; +inline enum tree_code +code_helper::safe_as_tree_code () const +{ + return is_tree_code () ? (tree_code) *this : MAX_TREE_CODES; +} + +inline combined_fn +code_helper::safe_as_fn_code () const { + return is_fn_code () ? (combined_fn) *this : CFN_LAST; +} + inline code_helper::operator internal_fn () const { return as_internal_fn (combined_fn (*this)); From patchwork Fri Apr 28 12:37:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Andre Vieira (lists)" X-Patchwork-Id: 1774963 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=S0tNQWvr; dkim-atps=neutral Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q7C1q38wbz23tl for ; Fri, 28 Apr 2023 22:42:55 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6E6F5385381D for ; Fri, 28 Apr 2023 12:42:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6E6F5385381D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1682685773; bh=GVGdHcNDf3p14oBsgAjJFY5gnNryBKblqZwm1gcn5s0=; h=Date:Subject:To:Cc:References:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=S0tNQWvrdt4819LuY0BOSeZ6W6kfRqfiRiFtRNqCU/eGv+VViD8fM5zIVeW6iGFnk kZvQ+39RRsnONtg/HhX+/LUkmhf3i++M5h82NNIvVIdBgb5NL5olENwx31x3lWva1O 4yJrsOSAk433sxr1yrfE4kpbEv3qGnicxwhPgT3g= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id CB4433881D06 for ; Fri, 28 Apr 2023 12:37:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CB4433881D06 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7C737C14; Fri, 28 Apr 2023 05:38:05 -0700 (PDT) Received: from [10.57.70.10] (unknown [10.57.70.10]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 98A363F5A1; Fri, 28 Apr 2023 05:37:20 -0700 (PDT) Message-ID: Date: Fri, 28 Apr 2023 13:37:14 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: [PATCH 2/3] Refactor widen_plus as internal_fn Content-Language: en-US To: Richard Biener Cc: Richard Biener , Richard Sandiford , "gcc-patches@gcc.gnu.org" References: <51ce8969-3130-452e-092e-f9d91eff2dad@arm.com> In-Reply-To: X-Spam-Status: No, score=-14.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_LOTSOFHASH, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Andre Vieira \(lists\) via Gcc-patches" From: "Andre Vieira (lists)" Reply-To: "Andre Vieira \(lists\)" Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" This patch replaces the existing tree_code widen_plus and widen_minus patterns with internal_fn versions. DEF_INTERNAL_OPTAB_HILO_FN is like DEF_INTERNAL_OPTAB_FN except it provides convenience wrappers for defining conversions that require a hi/lo split, like widening and narrowing operations. Each definition for will require an optab named and two other optabs that you specify for signed and unsigned. The hi/lo pair is necessary because the widening operations take n narrow elements as inputs and return n/2 wide elements as outputs. The 'lo' operation operates on the first n/2 elements of input. The 'hi' operation operates on the second n/2 elements of input. Defining an internal_fn along with hi/lo variations allows a single internal function to be returned from a vect_recog function that will later be expanded to hi/lo. DEF_INTERNAL_OPTAB_HILO_FN is used in internal-fn.def to register a widening internal_fn. It is defined differently in different places and internal-fn.def is sourced from those places so the parameters given can be reused. internal-fn.c: defined to expand to hi/lo signed/unsigned optabs, later defined to generate the 'expand_' functions for the hi/lo versions of the fn. internal-fn.def: defined to invoke DEF_INTERNAL_OPTAB_FN for the original and hi/lo variants of the internal_fn For example: IFN_VEC_WIDEN_PLUS -> IFN_VEC_WIDEN_PLUS_HI, IFN_VEC_WIDEN_PLUS_LO for aarch64: IFN_VEC_WIDEN_PLUS_HI -> vec_widen_addl_hi_ -> (u/s)addl2 IFN_VEC_WIDEN_PLUS_LO -> vec_widen_addl_lo_ -> (u/s)addl This gives the same functionality as the previous WIDEN_PLUS/WIDEN_MINUS tree codes which are expanded into VEC_WIDEN_PLUS_LO, VEC_WIDEN_PLUS_HI. gcc/ChangeLog: 2023-04-28 Andre Vieira Joel Hutton Tamar Christina * internal-fn.cc (INCLUDE_MAP): Include maps for use in optab lookup. (DEF_INTERNAL_OPTAB_HILO_FN): Macro to define an internal_fn that expands into multiple internal_fns (for widening). (ifn_cmp): Function to compare ifn's for sorting/searching. (lookup_hilo_ifn_optab): Add lookup function. (lookup_hilo_internal_fn): Add lookup function. (commutative_binary_fn_p): Add widen_plus fn's. (widening_fn_p): New function. (decomposes_to_hilo_fn_p): New function. * internal-fn.def (DEF_INTERNAL_OPTAB_HILO_FN): Define widening plus,minus functions. (VEC_WIDEN_PLUS): Replacement for VEC_WIDEN_PLUS tree code. (VEC_WIDEN_MINUS): Replacement for VEC_WIDEN_MINUS tree code. * internal-fn.h (GCC_INTERNAL_FN_H): Add headers. (lookup_hilo_ifn_optab): Add prototype. (lookup_hilo_internal_fn): Likewise. (widening_fn_p): Likewise. (decomposes_to_hilo_fn_p): Likewise. * optabs.cc (commutative_optab_p): Add widening plus, minus optabs. * optabs.def (OPTAB_CD): widen add, sub optabs * tree-vect-patterns.cc (vect_recog_widen_op_pattern): Support patterns with a hi/lo split. (vect_recog_widen_plus_pattern): Refactor to return IFN_VECT_WIDEN_PLUS. (vect_recog_widen_minus_pattern): Refactor to return new IFN_VEC_WIDEN_MINUS. * tree-vect-stmts.cc (vectorizable_conversion): Add widen plus/minus ifn support. (supportable_widening_operation): Add widen plus/minus ifn support. gcc/testsuite/ChangeLog: * gcc.target/aarch64/vect-widen-add.c: Test that new IFN_VEC_WIDEN_PLUS is being used. * gcc.target/aarch64/vect-widen-sub.c: Test that new IFN_VEC_WIDEN_MINUS is being used. diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc index 6e81dc05e0e0714256759b0594816df451415a2d..e4d815cd577d266d2bccf6fb68d62aac91a8b4cf 100644 --- a/gcc/internal-fn.cc +++ b/gcc/internal-fn.cc @@ -17,6 +17,7 @@ You should have received a copy of the GNU General Public License along with GCC; see the file COPYING3. If not see . */ +#define INCLUDE_MAP #include "config.h" #include "system.h" #include "coretypes.h" @@ -70,6 +71,26 @@ const int internal_fn_flags_array[] = { 0 }; +const enum internal_fn internal_fn_hilo_keys_array[] = { +#undef DEF_INTERNAL_OPTAB_HILO_FN +#define DEF_INTERNAL_OPTAB_HILO_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \ + IFN_##NAME##_LO, \ + IFN_##NAME##_HI, +#include "internal-fn.def" + IFN_LAST +#undef DEF_INTERNAL_OPTAB_HILO_FN +}; + +const optab internal_fn_hilo_values_array[] = { +#undef DEF_INTERNAL_OPTAB_HILO_FN +#define DEF_INTERNAL_OPTAB_HILO_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \ + SOPTAB##_lo_optab, UOPTAB##_lo_optab, \ + SOPTAB##_hi_optab, UOPTAB##_hi_optab, +#include "internal-fn.def" + unknown_optab, unknown_optab +#undef DEF_INTERNAL_OPTAB_HILO_FN +}; + /* Return the internal function called NAME, or IFN_LAST if there's no such function. */ @@ -90,6 +111,61 @@ lookup_internal_fn (const char *name) return entry ? *entry : IFN_LAST; } +static int +ifn_cmp (const void *a_, const void *b_) +{ + typedef std::pair ifn_pair; + auto *a = (const std::pair *)a_; + auto *b = (const std::pair *)b_; + return (int) (a->first.first) - (b->first.first); +} + +/* Return the optab belonging to the given internal function NAME for the given + SIGN or unknown_optab. */ + +optab +lookup_hilo_ifn_optab (enum internal_fn fn, unsigned sign) +{ + typedef std::pair ifn_pair; + typedef auto_vec >fn_to_optab_map_type; + static fn_to_optab_map_type *fn_to_optab_map; + + if (!fn_to_optab_map) + { + unsigned num + = sizeof (internal_fn_hilo_keys_array) / sizeof (enum internal_fn); + fn_to_optab_map = new fn_to_optab_map_type (); + for (unsigned int i = 0; i < num - 1; ++i) + { + enum internal_fn fn = internal_fn_hilo_keys_array[i]; + optab v1 = internal_fn_hilo_values_array[2*i]; + optab v2 = internal_fn_hilo_values_array[2*i + 1]; + ifn_pair key1 (fn, 0); + fn_to_optab_map->safe_push ({key1, v1}); + ifn_pair key2 (fn, 1); + fn_to_optab_map->safe_push ({key2, v2}); + } + fn_to_optab_map->qsort (ifn_cmp); + } + + ifn_pair new_pair (fn, sign ? 1 : 0); + optab tmp; + std::pair pair_wrap (new_pair, tmp); + auto entry = fn_to_optab_map->bsearch (&pair_wrap, ifn_cmp); + return entry != fn_to_optab_map->end () ? entry->second : unknown_optab; +} + +extern void +lookup_hilo_internal_fn (enum internal_fn ifn, enum internal_fn *lo, + enum internal_fn *hi) +{ + gcc_assert (decomposes_to_hilo_fn_p (ifn)); + + *lo = internal_fn (ifn + 1); + *hi = internal_fn (ifn + 2); +} + + /* Fnspec of each internal function, indexed by function number. */ const_tree internal_fn_fnspec_array[IFN_LAST + 1]; @@ -3970,6 +4046,9 @@ commutative_binary_fn_p (internal_fn fn) case IFN_UBSAN_CHECK_MUL: case IFN_ADD_OVERFLOW: case IFN_MUL_OVERFLOW: + case IFN_VEC_WIDEN_PLUS: + case IFN_VEC_WIDEN_PLUS_LO: + case IFN_VEC_WIDEN_PLUS_HI: return true; default: @@ -4043,6 +4122,42 @@ first_commutative_argument (internal_fn fn) } } +/* Return true if FN has a wider output type than its argument types. */ + +bool +widening_fn_p (internal_fn fn) +{ + switch (fn) + { + case IFN_VEC_WIDEN_PLUS: + case IFN_VEC_WIDEN_MINUS: + return true; + + default: + return false; + } +} + +/* Return true if FN decomposes to _hi and _lo IFN. If true this should also + be a widening function. */ + +bool +decomposes_to_hilo_fn_p (internal_fn fn) +{ + if (!widening_fn_p (fn)) + return false; + + switch (fn) + { + case IFN_VEC_WIDEN_PLUS: + case IFN_VEC_WIDEN_MINUS: + return true; + + default: + return false; + } +} + /* Return true if IFN_SET_EDOM is supported. */ bool @@ -4055,6 +4170,32 @@ set_edom_supported_p (void) #endif } +#undef DEF_INTERNAL_OPTAB_HILO_FN +#define DEF_INTERNAL_OPTAB_HILO_FN(CODE, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \ + static void \ + expand_##CODE (internal_fn, gcall *) \ + { \ + gcc_unreachable (); \ + } \ + static void \ + expand_##CODE##_LO (internal_fn fn, gcall *stmt) \ + { \ + tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \ + if (!TYPE_UNSIGNED (ty)) \ + expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_lo##_optab); \ + else \ + expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_lo##_optab); \ + } \ + static void \ + expand_##CODE##_HI (internal_fn fn, gcall *stmt) \ + { \ + tree ty = TREE_TYPE (gimple_get_lhs (stmt)); \ + if (!TYPE_UNSIGNED (ty)) \ + expand_##TYPE##_optab_fn (fn, stmt, SOPTAB##_hi##_optab); \ + else \ + expand_##TYPE##_optab_fn (fn, stmt, UOPTAB##_hi##_optab); \ + } + #define DEF_INTERNAL_OPTAB_FN(CODE, FLAGS, OPTAB, TYPE) \ static void \ expand_##CODE (internal_fn fn, gcall *stmt) \ @@ -4071,6 +4212,7 @@ set_edom_supported_p (void) expand_##TYPE##_optab_fn (fn, stmt, which_optab); \ } #include "internal-fn.def" +#undef DEF_INTERNAL_OPTAB_HILO_FN /* Routines to expand each internal function, indexed by function number. Each routine has the prototype: diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def index 7fe742c2ae713e7152ab05cfdfba86e4e0aa3456..347ed667d92620e0ee3ea15c58ecac6c242ebe73 100644 --- a/gcc/internal-fn.def +++ b/gcc/internal-fn.def @@ -85,6 +85,13 @@ along with GCC; see the file COPYING3. If not see says that the function extends the C-level BUILT_IN_{,L,LL,IMAX} group of functions to any integral mode (including vector modes). + DEF_INTERNAL_OPTAB_HILO_FN is like DEF_INTERNAL_OPTAB_FN except it + provides convenience wrappers for defining conversions that require a + hi/lo split, like widening and narrowing operations. Each definition + for will require an optab named and two other optabs that + you specify for signed and unsigned. + + Each entry must have a corresponding expander of the form: void expand_NAME (gimple_call stmt) @@ -123,6 +130,14 @@ along with GCC; see the file COPYING3. If not see DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE) #endif +#ifndef DEF_INTERNAL_OPTAB_HILO_FN +#define DEF_INTERNAL_OPTAB_HILO_FN(NAME, FLAGS, OPTAB, SOPTAB, UOPTAB, TYPE) \ + DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE) \ + DEF_INTERNAL_OPTAB_FN (NAME ## _LO, FLAGS, unknown, TYPE) \ + DEF_INTERNAL_OPTAB_FN (NAME ## _HI, FLAGS, unknown, TYPE) +#endif + + DEF_INTERNAL_OPTAB_FN (MASK_LOAD, ECF_PURE, maskload, mask_load) DEF_INTERNAL_OPTAB_FN (LOAD_LANES, ECF_CONST, vec_load_lanes, load_lanes) DEF_INTERNAL_OPTAB_FN (MASK_LOAD_LANES, ECF_PURE, @@ -315,6 +330,14 @@ DEF_INTERNAL_OPTAB_FN (COMPLEX_ADD_ROT270, ECF_CONST, cadd270, binary) DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL, ECF_CONST, cmul, binary) DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL_CONJ, ECF_CONST, cmul_conj, binary) DEF_INTERNAL_OPTAB_FN (VEC_ADDSUB, ECF_CONST, vec_addsub, binary) +DEF_INTERNAL_OPTAB_HILO_FN (VEC_WIDEN_PLUS, + ECF_CONST | ECF_NOTHROW, + vec_widen_add, vec_widen_saddl, vec_widen_uaddl, + binary) +DEF_INTERNAL_OPTAB_HILO_FN (VEC_WIDEN_MINUS, + ECF_CONST | ECF_NOTHROW, + vec_widen_sub, vec_widen_ssubl, vec_widen_usubl, + binary) DEF_INTERNAL_OPTAB_FN (VEC_FMADDSUB, ECF_CONST, vec_fmaddsub, ternary) DEF_INTERNAL_OPTAB_FN (VEC_FMSUBADD, ECF_CONST, vec_fmsubadd, ternary) diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h index 08922ed4254898f5fffca3f33973e96ed9ce772f..6a5f8762e872ad2ef64ce2986a678e3b40622d81 100644 --- a/gcc/internal-fn.h +++ b/gcc/internal-fn.h @@ -20,6 +20,10 @@ along with GCC; see the file COPYING3. If not see #ifndef GCC_INTERNAL_FN_H #define GCC_INTERNAL_FN_H +#include "insn-codes.h" +#include "insn-opinit.h" + + /* INTEGER_CST values for IFN_UNIQUE function arg-0. UNSPEC: Undifferentiated UNIQUE. @@ -112,6 +116,9 @@ internal_fn_name (enum internal_fn fn) } extern internal_fn lookup_internal_fn (const char *); +extern optab lookup_hilo_ifn_optab (enum internal_fn, unsigned); +extern void lookup_hilo_internal_fn (enum internal_fn, enum internal_fn *, + enum internal_fn *); /* Return the ECF_* flags for function FN. */ @@ -210,6 +217,8 @@ extern bool commutative_binary_fn_p (internal_fn); extern bool commutative_ternary_fn_p (internal_fn); extern int first_commutative_argument (internal_fn); extern bool associative_binary_fn_p (internal_fn); +extern bool widening_fn_p (internal_fn); +extern bool decomposes_to_hilo_fn_p (internal_fn); extern bool set_edom_supported_p (void); diff --git a/gcc/optabs.cc b/gcc/optabs.cc index c8e39c82d57a7d726e7da33d247b80f32ec9236c..d4dd7ee3d34d01c32ab432ae4e4ce9e4b522b2f7 100644 --- a/gcc/optabs.cc +++ b/gcc/optabs.cc @@ -1314,7 +1314,12 @@ commutative_optab_p (optab binoptab) || binoptab == smul_widen_optab || binoptab == umul_widen_optab || binoptab == smul_highpart_optab - || binoptab == umul_highpart_optab); + || binoptab == umul_highpart_optab + || binoptab == vec_widen_add_optab + || binoptab == vec_widen_saddl_hi_optab + || binoptab == vec_widen_saddl_lo_optab + || binoptab == vec_widen_uaddl_hi_optab + || binoptab == vec_widen_uaddl_lo_optab); } /* X is to be used in mode MODE as operand OPN to BINOPTAB. If we're diff --git a/gcc/optabs.def b/gcc/optabs.def index 695f5911b300c9ca5737de9be809fa01aabe5e01..e064189103b3be70644468d11f3c91ac45ffe0d0 100644 --- a/gcc/optabs.def +++ b/gcc/optabs.def @@ -78,6 +78,8 @@ OPTAB_CD(smsub_widen_optab, "msub$b$a4") OPTAB_CD(umsub_widen_optab, "umsub$b$a4") OPTAB_CD(ssmsub_widen_optab, "ssmsub$b$a4") OPTAB_CD(usmsub_widen_optab, "usmsub$a$b4") +OPTAB_CD(vec_widen_add_optab, "add$a$b3") +OPTAB_CD(vec_widen_sub_optab, "sub$a$b3") OPTAB_CD(vec_load_lanes_optab, "vec_load_lanes$a$b") OPTAB_CD(vec_store_lanes_optab, "vec_store_lanes$a$b") OPTAB_CD(vec_mask_load_lanes_optab, "vec_mask_load_lanes$a$b") diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c index 220bd9352a4c7acd2e3713e441d74898d3e92b30..7037673d32bd780e1c9b58a51e58e2bac3b30b7e 100644 --- a/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c +++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-add.c @@ -1,5 +1,5 @@ /* { dg-do run } */ -/* { dg-options "-O3 -save-temps" } */ +/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */ #include #include @@ -86,6 +86,8 @@ main() return 0; } +/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_LO" "vect" } } */ +/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_PLUS_HI" "vect" } } */ /* { dg-final { scan-assembler-times {\tuaddl\t} 1} } */ /* { dg-final { scan-assembler-times {\tuaddl2\t} 1} } */ /* { dg-final { scan-assembler-times {\tsaddl\t} 1} } */ diff --git a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c index a2bed63affbd091977df95a126da1f5b8c1d41d2..83bc1edb6105f47114b665e24a13e6194b2179a2 100644 --- a/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c +++ b/gcc/testsuite/gcc.target/aarch64/vect-widen-sub.c @@ -1,5 +1,5 @@ /* { dg-do run } */ -/* { dg-options "-O3 -save-temps" } */ +/* { dg-options "-O3 -save-temps -fdump-tree-vect-all" } */ #include #include @@ -86,6 +86,8 @@ main() return 0; } +/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_LO" "vect" } } */ +/* { dg-final { scan-tree-dump "add new stmt.*VEC_WIDEN_MINUS_HI" "vect" } } */ /* { dg-final { scan-assembler-times {\tusubl\t} 1} } */ /* { dg-final { scan-assembler-times {\tusubl2\t} 1} } */ /* { dg-final { scan-assembler-times {\tssubl\t} 1} } */ diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index b35023adade94c1996cd076c4b7419560e819c6b..3175dd92187c0935f78ebbf2eb476bdcf8b4ccd1 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -1394,14 +1394,16 @@ static gimple * vect_recog_widen_op_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info, tree *type_out, tree_code orig_code, code_helper wide_code, - bool shift_p, const char *name) + bool shift_p, const char *name, + enum optab_subtype *subtype = NULL) { gimple *last_stmt = last_stmt_info->stmt; vect_unpromoted_value unprom[2]; tree half_type; if (!vect_widened_op_tree (vinfo, last_stmt_info, orig_code, orig_code, - shift_p, 2, unprom, &half_type)) + shift_p, 2, unprom, &half_type, subtype)) + return NULL; /* Pattern detected. */ @@ -1467,6 +1469,20 @@ vect_recog_widen_op_pattern (vec_info *vinfo, type, pattern_stmt, vecctype); } +static gimple * +vect_recog_widen_op_pattern (vec_info *vinfo, + stmt_vec_info last_stmt_info, tree *type_out, + tree_code orig_code, internal_fn wide_ifn, + bool shift_p, const char *name, + enum optab_subtype *subtype = NULL) +{ + combined_fn ifn = as_combined_fn (wide_ifn); + return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out, + orig_code, ifn, shift_p, name, + subtype); +} + + /* Try to detect multiplication on widened inputs, converting MULT_EXPR to WIDEN_MULT_EXPR. See vect_recog_widen_op_pattern for details. */ @@ -1480,26 +1496,30 @@ vect_recog_widen_mult_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info, } /* Try to detect addition on widened inputs, converting PLUS_EXPR - to WIDEN_PLUS_EXPR. See vect_recog_widen_op_pattern for details. */ + to IFN_VEC_WIDEN_PLUS. See vect_recog_widen_op_pattern for details. */ static gimple * vect_recog_widen_plus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info, tree *type_out) { + enum optab_subtype subtype; return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out, - PLUS_EXPR, WIDEN_PLUS_EXPR, false, - "vect_recog_widen_plus_pattern"); + PLUS_EXPR, IFN_VEC_WIDEN_PLUS, + false, "vect_recog_widen_plus_pattern", + &subtype); } /* Try to detect subtraction on widened inputs, converting MINUS_EXPR - to WIDEN_MINUS_EXPR. See vect_recog_widen_op_pattern for details. */ + to IFN_VEC_WIDEN_MINUS. See vect_recog_widen_op_pattern for details. */ static gimple * vect_recog_widen_minus_pattern (vec_info *vinfo, stmt_vec_info last_stmt_info, tree *type_out) { + enum optab_subtype subtype; return vect_recog_widen_op_pattern (vinfo, last_stmt_info, type_out, - MINUS_EXPR, WIDEN_MINUS_EXPR, false, - "vect_recog_widen_minus_pattern"); + MINUS_EXPR, IFN_VEC_WIDEN_MINUS, + false, "vect_recog_widen_minus_pattern", + &subtype); } /* Function vect_recog_popcount_pattern @@ -6067,6 +6087,7 @@ static vect_recog_func vect_vect_recog_func_ptrs[] = { { vect_recog_mask_conversion_pattern, "mask_conversion" }, { vect_recog_widen_plus_pattern, "widen_plus" }, { vect_recog_widen_minus_pattern, "widen_minus" }, + /* These must come after the double widening ones. */ }; const unsigned int NUM_PATTERNS = ARRAY_SIZE (vect_vect_recog_func_ptrs); diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index ce47f4940fa9a1baca4ba1162065cfc3b4072eba..2a7ef2439e12d1966e8884433963a3d387a856b7 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -5035,7 +5035,9 @@ vectorizable_conversion (vec_info *vinfo, bool widen_arith = (code == WIDEN_PLUS_EXPR || code == WIDEN_MINUS_EXPR || code == WIDEN_MULT_EXPR - || code == WIDEN_LSHIFT_EXPR); + || code == WIDEN_LSHIFT_EXPR + || code == IFN_VEC_WIDEN_PLUS + || code == IFN_VEC_WIDEN_MINUS); if (!widen_arith && !CONVERT_EXPR_CODE_P (code) @@ -5085,7 +5087,9 @@ vectorizable_conversion (vec_info *vinfo, gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR || code == WIDEN_PLUS_EXPR - || code == WIDEN_MINUS_EXPR); + || code == WIDEN_MINUS_EXPR + || code == IFN_VEC_WIDEN_PLUS + || code == IFN_VEC_WIDEN_MINUS); op1 = is_gimple_assign (stmt) ? gimple_assign_rhs2 (stmt) : @@ -12335,12 +12339,46 @@ supportable_widening_operation (vec_info *vinfo, optab1 = vec_unpacks_sbool_lo_optab; optab2 = vec_unpacks_sbool_hi_optab; } - else - { - optab1 = optab_for_tree_code (c1, vectype, optab_default); - optab2 = optab_for_tree_code (c2, vectype, optab_default); + + if (code.is_fn_code ()) + { + internal_fn ifn = as_internal_fn ((combined_fn) code); + gcc_assert (decomposes_to_hilo_fn_p (ifn)); + + internal_fn lo, hi; + lookup_hilo_internal_fn (ifn, &lo, &hi); + *code1 = as_combined_fn (lo); + *code2 = as_combined_fn (hi); + optab1 = lookup_hilo_ifn_optab (lo, !TYPE_UNSIGNED (vectype)); + optab2 = lookup_hilo_ifn_optab (hi, !TYPE_UNSIGNED (vectype)); } + if (code.is_tree_code ()) + { + if (code == FIX_TRUNC_EXPR) + { + /* The signedness is determined from output operand. */ + optab1 = optab_for_tree_code (c1, vectype_out, optab_default); + optab2 = optab_for_tree_code (c2, vectype_out, optab_default); + } + else if (CONVERT_EXPR_CODE_P (code.safe_as_tree_code ()) + && VECTOR_BOOLEAN_TYPE_P (wide_vectype) + && VECTOR_BOOLEAN_TYPE_P (vectype) + && TYPE_MODE (wide_vectype) == TYPE_MODE (vectype) + && SCALAR_INT_MODE_P (TYPE_MODE (vectype))) + { + /* If the input and result modes are the same, a different optab + is needed where we pass in the number of units in vectype. */ + optab1 = vec_unpacks_sbool_lo_optab; + optab2 = vec_unpacks_sbool_hi_optab; + } + else + { + optab1 = optab_for_tree_code (c1, vectype, optab_default); + optab2 = optab_for_tree_code (c2, vectype, optab_default); + } + } + if (!optab1 || !optab2) return false; From patchwork Fri Apr 28 12:37:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Andre Vieira (lists)" X-Patchwork-Id: 1774948 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=JXW1dx6e; dkim-atps=neutral Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q7BzD3N73z23td for ; Fri, 28 Apr 2023 22:40:40 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 796FC393BA4A for ; Fri, 28 Apr 2023 12:40:38 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 796FC393BA4A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1682685638; bh=H0dEuLN8r6fV07UYgAcjxHIlZJl15W0EUjHtRO4RKmQ=; h=Date:Subject:To:Cc:References:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=JXW1dx6e+i6dv4FA7GiYfn27rGOh3f7Kd06oLMFmT7mjMciivfZKVLw77x1FrHyeV LZHFpdG2Qfgh3lWALjWZpI9/GSRX3kVHcnRBdRhXDA+np0jAoktnCpQ2GdlkO9F8LU HZX1J/cm0d1Htw9ZujCuQD1LzdSxIruCBxaDgKjQ= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 60645385B506 for ; Fri, 28 Apr 2023 12:37:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 60645385B506 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 202ABC14; Fri, 28 Apr 2023 05:38:19 -0700 (PDT) Received: from [10.57.70.10] (unknown [10.57.70.10]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3E22D3F5A1; Fri, 28 Apr 2023 05:37:34 -0700 (PDT) Message-ID: Date: Fri, 28 Apr 2023 13:37:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: [PATCH 3/3] Remove widen_plus/minus_expr tree codes Content-Language: en-US To: Richard Biener Cc: Richard Biener , Richard Sandiford , "gcc-patches@gcc.gnu.org" References: <51ce8969-3130-452e-092e-f9d91eff2dad@arm.com> In-Reply-To: X-Spam-Status: No, score=-14.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_LOTSOFHASH, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Andre Vieira \(lists\) via Gcc-patches" From: "Andre Vieira (lists)" Reply-To: "Andre Vieira \(lists\)" Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" This is a rebase of Joel's previous patch. This patch removes the old widen plus/minus tree codes which have been replaced by internal functions. gcc/ChangeLog: 2023-04-28 Andre Vieira Joel Hutton * doc/generic.texi: Remove old tree codes. * expr.cc (expand_expr_real_2): Remove old tree code cases. * gimple-pretty-print.cc (dump_binary_rhs): Likewise. * optabs-tree.cc (optab_for_tree_code): Likewise. (supportable_half_widening_operation): Likewise. * tree-cfg.cc (verify_gimple_assign_binary): Likewise. * tree-inline.cc (estimate_operator_cost): Likewise. (op_symbol_code): Likewise. * tree-vect-data-refs.cc (vect_get_smallest_scalar_type): Likewise. (vect_analyze_data_ref_accesses): Likewise. * tree-vect-generic.cc (expand_vector_operations_1): Likewise. * cfgexpand.cc (expand_debug_expr): Likewise. * tree-vect-stmts.cc (vectorizable_conversion): Likewise. (supportable_widening_operation): Likewise. * gimple-range-op.cc (gimple_range_op_handler::maybe_non_standard): Likewise. * tree-vect-patterns.cc (vect_widened_op_tree): Refactor to replace usage in vect_recog_sad_pattern. (vect_recog_sad_pattern): Replace tree code widening pattern with internal function. (vect_recog_average_pattern): Likewise. * tree-pretty-print.cc (dump_generic_node): Remove tree code definition. * tree.def (WIDEN_PLUS_EXPR, WIDEN_MINUS_EXPR, VEC_WIDEN_PLUS_HI_EXPR, VEC_WIDEN_PLUS_LO_EXPR, VEC_WIDEN_MINUS_HI_EXPR, VEC_WIDEN_MINUS_LO_EXPR): Likewise diff --git a/gcc/cfgexpand.cc b/gcc/cfgexpand.cc index 1a1b26b1c6c23ce273bcd08dc9a973f777174007..25b1558dcb941ea491a19aeeb2cd8f4d2dbdf7c6 100644 --- a/gcc/cfgexpand.cc +++ b/gcc/cfgexpand.cc @@ -5365,10 +5365,6 @@ expand_debug_expr (tree exp) case VEC_WIDEN_MULT_ODD_EXPR: case VEC_WIDEN_LSHIFT_HI_EXPR: case VEC_WIDEN_LSHIFT_LO_EXPR: - case VEC_WIDEN_PLUS_HI_EXPR: - case VEC_WIDEN_PLUS_LO_EXPR: - case VEC_WIDEN_MINUS_HI_EXPR: - case VEC_WIDEN_MINUS_LO_EXPR: case VEC_PERM_EXPR: case VEC_DUPLICATE_EXPR: case VEC_SERIES_EXPR: @@ -5405,8 +5401,6 @@ expand_debug_expr (tree exp) case WIDEN_MULT_EXPR: case WIDEN_MULT_PLUS_EXPR: case WIDEN_MULT_MINUS_EXPR: - case WIDEN_PLUS_EXPR: - case WIDEN_MINUS_EXPR: if (SCALAR_INT_MODE_P (GET_MODE (op0)) && SCALAR_INT_MODE_P (mode)) { @@ -5419,10 +5413,6 @@ expand_debug_expr (tree exp) op1 = simplify_gen_unary (ZERO_EXTEND, mode, op1, inner_mode); else op1 = simplify_gen_unary (SIGN_EXTEND, mode, op1, inner_mode); - if (TREE_CODE (exp) == WIDEN_PLUS_EXPR) - return simplify_gen_binary (PLUS, mode, op0, op1); - else if (TREE_CODE (exp) == WIDEN_MINUS_EXPR) - return simplify_gen_binary (MINUS, mode, op0, op1); op0 = simplify_gen_binary (MULT, mode, op0, op1); if (TREE_CODE (exp) == WIDEN_MULT_EXPR) return op0; diff --git a/gcc/doc/generic.texi b/gcc/doc/generic.texi index 2c14b7abce2db0a3da0a21e916907947cb56a265..3816abaaf4d364d604a44942317f96f3f303e5b6 100644 --- a/gcc/doc/generic.texi +++ b/gcc/doc/generic.texi @@ -1811,10 +1811,6 @@ a value from @code{enum annot_expr_kind}, the third is an @code{INTEGER_CST}. @tindex VEC_RSHIFT_EXPR @tindex VEC_WIDEN_MULT_HI_EXPR @tindex VEC_WIDEN_MULT_LO_EXPR -@tindex VEC_WIDEN_PLUS_HI_EXPR -@tindex VEC_WIDEN_PLUS_LO_EXPR -@tindex VEC_WIDEN_MINUS_HI_EXPR -@tindex VEC_WIDEN_MINUS_LO_EXPR @tindex VEC_UNPACK_HI_EXPR @tindex VEC_UNPACK_LO_EXPR @tindex VEC_UNPACK_FLOAT_HI_EXPR @@ -1861,33 +1857,6 @@ vector of @code{N/2} products. In the case of @code{VEC_WIDEN_MULT_LO_EXPR} the low @code{N/2} elements of the two vector are multiplied to produce the vector of @code{N/2} products. -@item VEC_WIDEN_PLUS_HI_EXPR -@itemx VEC_WIDEN_PLUS_LO_EXPR -These nodes represent widening vector addition of the high and low parts of -the two input vectors, respectively. Their operands are vectors that contain -the same number of elements (@code{N}) of the same integral type. The result -is a vector that contains half as many elements, of an integral type whose size -is twice as wide. In the case of @code{VEC_WIDEN_PLUS_HI_EXPR} the high -@code{N/2} elements of the two vectors are added to produce the vector of -@code{N/2} products. In the case of @code{VEC_WIDEN_PLUS_LO_EXPR} the low -@code{N/2} elements of the two vectors are added to produce the vector of -@code{N/2} products. - -@item VEC_WIDEN_MINUS_HI_EXPR -@itemx VEC_WIDEN_MINUS_LO_EXPR -These nodes represent widening vector subtraction of the high and low parts of -the two input vectors, respectively. Their operands are vectors that contain -the same number of elements (@code{N}) of the same integral type. The high/low -elements of the second vector are subtracted from the high/low elements of the -first. The result is a vector that contains half as many elements, of an -integral type whose size is twice as wide. In the case of -@code{VEC_WIDEN_MINUS_HI_EXPR} the high @code{N/2} elements of the second -vector are subtracted from the high @code{N/2} of the first to produce the -vector of @code{N/2} products. In the case of -@code{VEC_WIDEN_MINUS_LO_EXPR} the low @code{N/2} elements of the second -vector are subtracted from the low @code{N/2} of the first to produce the -vector of @code{N/2} products. - @item VEC_UNPACK_HI_EXPR @itemx VEC_UNPACK_LO_EXPR These nodes represent unpacking of the high and low parts of the input vector, diff --git a/gcc/expr.cc b/gcc/expr.cc index f8f5cc5a6ca67f291b3c8b7246d593c0be80272f..454d1391b19a7d2aa53f0a88876d1eaf0494de51 100644 --- a/gcc/expr.cc +++ b/gcc/expr.cc @@ -9601,8 +9601,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode, target, unsignedp); return target; - case WIDEN_PLUS_EXPR: - case WIDEN_MINUS_EXPR: case WIDEN_MULT_EXPR: /* If first operand is constant, swap them. Thus the following special case checks need only @@ -10380,10 +10378,6 @@ expand_expr_real_2 (sepops ops, rtx target, machine_mode tmode, return temp; } - case VEC_WIDEN_PLUS_HI_EXPR: - case VEC_WIDEN_PLUS_LO_EXPR: - case VEC_WIDEN_MINUS_HI_EXPR: - case VEC_WIDEN_MINUS_LO_EXPR: case VEC_WIDEN_MULT_HI_EXPR: case VEC_WIDEN_MULT_LO_EXPR: case VEC_WIDEN_MULT_EVEN_EXPR: diff --git a/gcc/gimple-pretty-print.cc b/gcc/gimple-pretty-print.cc index 300e9d7ed1e7be73f30875e08c461a8880c3134e..d903826894e7f0dfd34dc0caad92eea3caa45e05 100644 --- a/gcc/gimple-pretty-print.cc +++ b/gcc/gimple-pretty-print.cc @@ -459,10 +459,6 @@ dump_binary_rhs (pretty_printer *buffer, const gassign *gs, int spc, case VEC_PACK_FLOAT_EXPR: case VEC_WIDEN_LSHIFT_HI_EXPR: case VEC_WIDEN_LSHIFT_LO_EXPR: - case VEC_WIDEN_PLUS_HI_EXPR: - case VEC_WIDEN_PLUS_LO_EXPR: - case VEC_WIDEN_MINUS_HI_EXPR: - case VEC_WIDEN_MINUS_LO_EXPR: case VEC_SERIES_EXPR: for (p = get_tree_code_name (code); *p; p++) pp_character (buffer, TOUPPER (*p)); diff --git a/gcc/gimple-range-op.cc b/gcc/gimple-range-op.cc index 4ca32a7b5d52f8426b09d1446a336650e143b41f..5ae7f7596c6fc6f901e4e47ae44f00185f4602b2 100644 --- a/gcc/gimple-range-op.cc +++ b/gcc/gimple-range-op.cc @@ -797,12 +797,6 @@ gimple_range_op_handler::maybe_non_standard () if (gimple_code (m_stmt) == GIMPLE_ASSIGN) switch (gimple_assign_rhs_code (m_stmt)) { - case WIDEN_PLUS_EXPR: - { - signed_op = ptr_op_widen_plus_signed; - unsigned_op = ptr_op_widen_plus_unsigned; - } - gcc_fallthrough (); case WIDEN_MULT_EXPR: { m_valid = false; diff --git a/gcc/optabs-tree.cc b/gcc/optabs-tree.cc index 8010046c6a8b3e809c989ddef7a06ddaa68ae32a..ee1aa8c9676ee9c67edbf403e6295da391826a62 100644 --- a/gcc/optabs-tree.cc +++ b/gcc/optabs-tree.cc @@ -190,22 +190,6 @@ optab_for_tree_code (enum tree_code code, const_tree type, return (TYPE_UNSIGNED (type) ? vec_widen_ushiftl_lo_optab : vec_widen_sshiftl_lo_optab); - case VEC_WIDEN_PLUS_LO_EXPR: - return (TYPE_UNSIGNED (type) - ? vec_widen_uaddl_lo_optab : vec_widen_saddl_lo_optab); - - case VEC_WIDEN_PLUS_HI_EXPR: - return (TYPE_UNSIGNED (type) - ? vec_widen_uaddl_hi_optab : vec_widen_saddl_hi_optab); - - case VEC_WIDEN_MINUS_LO_EXPR: - return (TYPE_UNSIGNED (type) - ? vec_widen_usubl_lo_optab : vec_widen_ssubl_lo_optab); - - case VEC_WIDEN_MINUS_HI_EXPR: - return (TYPE_UNSIGNED (type) - ? vec_widen_usubl_hi_optab : vec_widen_ssubl_hi_optab); - case VEC_UNPACK_HI_EXPR: return (TYPE_UNSIGNED (type) ? vec_unpacku_hi_optab : vec_unpacks_hi_optab); @@ -312,8 +296,6 @@ optab_for_tree_code (enum tree_code code, const_tree type, 'hi'/'lo' pair using codes such as VEC_WIDEN_MINUS_HI/LO. Supported widening operations: - WIDEN_MINUS_EXPR - WIDEN_PLUS_EXPR WIDEN_MULT_EXPR WIDEN_LSHIFT_EXPR @@ -345,12 +327,6 @@ supportable_half_widening_operation (enum tree_code code, tree vectype_out, case WIDEN_LSHIFT_EXPR: *code1 = LSHIFT_EXPR; break; - case WIDEN_MINUS_EXPR: - *code1 = MINUS_EXPR; - break; - case WIDEN_PLUS_EXPR: - *code1 = PLUS_EXPR; - break; case WIDEN_MULT_EXPR: *code1 = MULT_EXPR; break; diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc index a9fcc7fd050f871437ef336ecfb8d6cc81280ee0..f80cd1465df83b5540492e619e56b9af249e9f31 100644 --- a/gcc/tree-cfg.cc +++ b/gcc/tree-cfg.cc @@ -4017,8 +4017,6 @@ verify_gimple_assign_binary (gassign *stmt) return false; } - case WIDEN_PLUS_EXPR: - case WIDEN_MINUS_EXPR: case PLUS_EXPR: case MINUS_EXPR: { @@ -4139,10 +4137,6 @@ verify_gimple_assign_binary (gassign *stmt) return false; } - case VEC_WIDEN_MINUS_HI_EXPR: - case VEC_WIDEN_MINUS_LO_EXPR: - case VEC_WIDEN_PLUS_HI_EXPR: - case VEC_WIDEN_PLUS_LO_EXPR: case VEC_WIDEN_MULT_HI_EXPR: case VEC_WIDEN_MULT_LO_EXPR: case VEC_WIDEN_MULT_EVEN_EXPR: diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc index c702f0032a19203a7c536a01c1e7f47fc7b77add..6e5fd45a0c2435109dd3d50e8fc8e1d4969a1fd0 100644 --- a/gcc/tree-inline.cc +++ b/gcc/tree-inline.cc @@ -4273,8 +4273,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights, case REALIGN_LOAD_EXPR: - case WIDEN_PLUS_EXPR: - case WIDEN_MINUS_EXPR: case WIDEN_SUM_EXPR: case WIDEN_MULT_EXPR: case DOT_PROD_EXPR: @@ -4283,10 +4281,6 @@ estimate_operator_cost (enum tree_code code, eni_weights *weights, case WIDEN_MULT_MINUS_EXPR: case WIDEN_LSHIFT_EXPR: - case VEC_WIDEN_PLUS_HI_EXPR: - case VEC_WIDEN_PLUS_LO_EXPR: - case VEC_WIDEN_MINUS_HI_EXPR: - case VEC_WIDEN_MINUS_LO_EXPR: case VEC_WIDEN_MULT_HI_EXPR: case VEC_WIDEN_MULT_LO_EXPR: case VEC_WIDEN_MULT_EVEN_EXPR: diff --git a/gcc/tree-pretty-print.cc b/gcc/tree-pretty-print.cc index 7947f9647a15110b52d195643ad7d28ee32d4236..9941d8bf80535a98e647b8928619a6bf08bc434c 100644 --- a/gcc/tree-pretty-print.cc +++ b/gcc/tree-pretty-print.cc @@ -2874,8 +2874,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags, break; /* Binary arithmetic and logic expressions. */ - case WIDEN_PLUS_EXPR: - case WIDEN_MINUS_EXPR: case WIDEN_SUM_EXPR: case WIDEN_MULT_EXPR: case MULT_EXPR: @@ -3831,10 +3829,6 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags, case VEC_SERIES_EXPR: case VEC_WIDEN_MULT_HI_EXPR: case VEC_WIDEN_MULT_LO_EXPR: - case VEC_WIDEN_PLUS_HI_EXPR: - case VEC_WIDEN_PLUS_LO_EXPR: - case VEC_WIDEN_MINUS_HI_EXPR: - case VEC_WIDEN_MINUS_LO_EXPR: case VEC_WIDEN_MULT_EVEN_EXPR: case VEC_WIDEN_MULT_ODD_EXPR: case VEC_WIDEN_LSHIFT_HI_EXPR: @@ -4352,12 +4346,6 @@ op_symbol_code (enum tree_code code) case WIDEN_LSHIFT_EXPR: return "w<<"; - case WIDEN_PLUS_EXPR: - return "w+"; - - case WIDEN_MINUS_EXPR: - return "w-"; - case POINTER_PLUS_EXPR: return "+"; diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc index 8daf7bd7dd34d043b1d7b4cba1779f0ecf9f520a..213a3899a6c145bb057cd118bec1df7a05728aef 100644 --- a/gcc/tree-vect-data-refs.cc +++ b/gcc/tree-vect-data-refs.cc @@ -136,8 +136,6 @@ vect_get_smallest_scalar_type (stmt_vec_info stmt_info, tree scalar_type) || gimple_assign_rhs_code (assign) == WIDEN_SUM_EXPR || gimple_assign_rhs_code (assign) == WIDEN_MULT_EXPR || gimple_assign_rhs_code (assign) == WIDEN_LSHIFT_EXPR - || gimple_assign_rhs_code (assign) == WIDEN_PLUS_EXPR - || gimple_assign_rhs_code (assign) == WIDEN_MINUS_EXPR || gimple_assign_rhs_code (assign) == FLOAT_EXPR) { tree rhs_type = TREE_TYPE (gimple_assign_rhs1 (assign)); diff --git a/gcc/tree-vect-generic.cc b/gcc/tree-vect-generic.cc index 445da53292e9d1d2db62ca962fc017bb0e6c9bbe..342ffc5fa7f3b8f37e6bd4658d2f1fccf1d2c7fa 100644 --- a/gcc/tree-vect-generic.cc +++ b/gcc/tree-vect-generic.cc @@ -2227,10 +2227,6 @@ expand_vector_operations_1 (gimple_stmt_iterator *gsi, arguments, not the widened result. VEC_UNPACK_FLOAT_*_EXPR is calculated in the same way above. */ if (code == WIDEN_SUM_EXPR - || code == VEC_WIDEN_PLUS_HI_EXPR - || code == VEC_WIDEN_PLUS_LO_EXPR - || code == VEC_WIDEN_MINUS_HI_EXPR - || code == VEC_WIDEN_MINUS_LO_EXPR || code == VEC_WIDEN_MULT_HI_EXPR || code == VEC_WIDEN_MULT_LO_EXPR || code == VEC_WIDEN_MULT_EVEN_EXPR diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 3175dd92187c0935f78ebbf2eb476bdcf8b4ccd1..ab3162b5ac66ea8a96c0ea7c45138ca5ee13423f 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -561,21 +561,30 @@ vect_joust_widened_type (tree type, tree new_type, tree *common_type) static unsigned int vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code, - tree_code widened_code, bool shift_p, + code_helper widened_code, bool shift_p, unsigned int max_nops, vect_unpromoted_value *unprom, tree *common_type, enum optab_subtype *subtype = NULL) { /* Check for an integer operation with the right code. */ - gassign *assign = dyn_cast (stmt_info->stmt); - if (!assign) + gimple* stmt = stmt_info->stmt; + if (!(is_gimple_assign (stmt) || is_gimple_call (stmt))) + return 0; + + code_helper rhs_code; + if (is_gimple_assign (stmt)) + rhs_code = gimple_assign_rhs_code (stmt); + else if (is_gimple_call (stmt)) + rhs_code = gimple_call_combined_fn (stmt); + else return 0; - tree_code rhs_code = gimple_assign_rhs_code (assign); - if (rhs_code != code && rhs_code != widened_code) + if (rhs_code != code + && rhs_code != widened_code) return 0; - tree type = TREE_TYPE (gimple_assign_lhs (assign)); + tree lhs = gimple_get_lhs (stmt); + tree type = TREE_TYPE (lhs); if (!INTEGRAL_TYPE_P (type)) return 0; @@ -588,7 +597,7 @@ vect_widened_op_tree (vec_info *vinfo, stmt_vec_info stmt_info, tree_code code, { vect_unpromoted_value *this_unprom = &unprom[next_op]; unsigned int nops = 1; - tree op = gimple_op (assign, i + 1); + tree op = gimple_arg (stmt, i); if (i == 1 && TREE_CODE (op) == INTEGER_CST) { /* We already have a common type from earlier operands. @@ -1342,8 +1351,9 @@ vect_recog_sad_pattern (vec_info *vinfo, /* FORNOW. Can continue analyzing the def-use chain when this stmt in a phi inside the loop (in case we are analyzing an outer-loop). */ vect_unpromoted_value unprom[2]; - if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR, WIDEN_MINUS_EXPR, - false, 2, unprom, &half_type)) + if (!vect_widened_op_tree (vinfo, diff_stmt_vinfo, MINUS_EXPR, + CFN_VEC_WIDEN_MINUS, false, 2, unprom, + &half_type)) return NULL; vect_pattern_detected ("vect_recog_sad_pattern", last_stmt); @@ -2696,9 +2706,10 @@ vect_recog_average_pattern (vec_info *vinfo, internal_fn ifn = IFN_AVG_FLOOR; vect_unpromoted_value unprom[3]; tree new_type; + enum optab_subtype subtype; unsigned int nops = vect_widened_op_tree (vinfo, plus_stmt_info, PLUS_EXPR, - WIDEN_PLUS_EXPR, false, 3, - unprom, &new_type); + CFN_VEC_WIDEN_PLUS, false, 3, + unprom, &new_type, &subtype); if (nops == 0) return NULL; if (nops == 3) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 2a7ef2439e12d1966e8884433963a3d387a856b7..ef3ac551f7fe247893b021d98e43c581e2078dbb 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -5032,9 +5032,7 @@ vectorizable_conversion (vec_info *vinfo, else return false; - bool widen_arith = (code == WIDEN_PLUS_EXPR - || code == WIDEN_MINUS_EXPR - || code == WIDEN_MULT_EXPR + bool widen_arith = (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR || code == IFN_VEC_WIDEN_PLUS || code == IFN_VEC_WIDEN_MINUS); @@ -5086,8 +5084,6 @@ vectorizable_conversion (vec_info *vinfo, { gcc_assert (code == WIDEN_MULT_EXPR || code == WIDEN_LSHIFT_EXPR - || code == WIDEN_PLUS_EXPR - || code == WIDEN_MINUS_EXPR || code == IFN_VEC_WIDEN_PLUS || code == IFN_VEC_WIDEN_MINUS); @@ -12192,7 +12188,7 @@ supportable_widening_operation (vec_info *vinfo, class loop *vect_loop = NULL; machine_mode vec_mode; enum insn_code icode1, icode2; - optab optab1, optab2; + optab optab1 = unknown_optab, optab2 = unknown_optab; tree vectype = vectype_in; tree wide_vectype = vectype_out; tree_code c1 = MAX_TREE_CODES, c2 = MAX_TREE_CODES; @@ -12290,16 +12286,6 @@ supportable_widening_operation (vec_info *vinfo, c2 = VEC_WIDEN_LSHIFT_HI_EXPR; break; - case WIDEN_PLUS_EXPR: - c1 = VEC_WIDEN_PLUS_LO_EXPR; - c2 = VEC_WIDEN_PLUS_HI_EXPR; - break; - - case WIDEN_MINUS_EXPR: - c1 = VEC_WIDEN_MINUS_LO_EXPR; - c2 = VEC_WIDEN_MINUS_HI_EXPR; - break; - CASE_CONVERT: c1 = VEC_UNPACK_LO_EXPR; c2 = VEC_UNPACK_HI_EXPR; diff --git a/gcc/tree.def b/gcc/tree.def index ee02754354f015a16737c7e879d89c3e3be0d5aa..a58e608a90078818a7ade9d1173ac7ec84c48c7a 100644 --- a/gcc/tree.def +++ b/gcc/tree.def @@ -1374,15 +1374,16 @@ DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr", tcc_expression, 3) DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr", tcc_binary, 2) /* Widening sad (sum of absolute differences). - The first two arguments are of type t1 which should be integer. - The third argument and the result are of type t2, such that t2 is at least - twice the size of t1. Like DOT_PROD_EXPR, SAD_EXPR (arg1,arg2,arg3) is + The first two arguments are of type t1 which should be a vector of integers. + The third argument and the result are of type t2, such that the size of + the elements of t2 is at least twice the size of the elements of t1. + Like DOT_PROD_EXPR, SAD_EXPR (arg1,arg2,arg3) is equivalent to: - tmp = WIDEN_MINUS_EXPR (arg1, arg2) + tmp = IFN_VEC_WIDEN_MINUS_EXPR (arg1, arg2) tmp2 = ABS_EXPR (tmp) arg3 = PLUS_EXPR (tmp2, arg3) or: - tmp = WIDEN_MINUS_EXPR (arg1, arg2) + tmp = IFN_VEC_WIDEN_MINUS_EXPR (arg1, arg2) tmp2 = ABS_EXPR (tmp) arg3 = WIDEN_SUM_EXPR (tmp2, arg3) */ @@ -1421,8 +1422,6 @@ DEFTREECODE (WIDEN_MULT_MINUS_EXPR, "widen_mult_minus_expr", tcc_expression, 3) the first argument from type t1 to type t2, and then shifting it by the second argument. */ DEFTREECODE (WIDEN_LSHIFT_EXPR, "widen_lshift_expr", tcc_binary, 2) -DEFTREECODE (WIDEN_PLUS_EXPR, "widen_plus_expr", tcc_binary, 2) -DEFTREECODE (WIDEN_MINUS_EXPR, "widen_minus_expr", tcc_binary, 2) /* Widening vector multiplication. The two operands are vectors with N elements of size S. Multiplying the @@ -1487,10 +1486,6 @@ DEFTREECODE (VEC_PACK_FLOAT_EXPR, "vec_pack_float_expr", tcc_binary, 2) */ DEFTREECODE (VEC_WIDEN_LSHIFT_HI_EXPR, "widen_lshift_hi_expr", tcc_binary, 2) DEFTREECODE (VEC_WIDEN_LSHIFT_LO_EXPR, "widen_lshift_lo_expr", tcc_binary, 2) -DEFTREECODE (VEC_WIDEN_PLUS_HI_EXPR, "widen_plus_hi_expr", tcc_binary, 2) -DEFTREECODE (VEC_WIDEN_PLUS_LO_EXPR, "widen_plus_lo_expr", tcc_binary, 2) -DEFTREECODE (VEC_WIDEN_MINUS_HI_EXPR, "widen_minus_hi_expr", tcc_binary, 2) -DEFTREECODE (VEC_WIDEN_MINUS_LO_EXPR, "widen_minus_lo_expr", tcc_binary, 2) /* PREDICT_EXPR. Specify hint for branch prediction. The PREDICT_EXPR_PREDICTOR specify predictor and PREDICT_EXPR_OUTCOME the