From patchwork Mon Jun 24 13:55:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Pan2" X-Patchwork-Id: 1951588 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=ae9Ye81n; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W78cb3WBsz1ydW for ; Mon, 24 Jun 2024 23:55:43 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 706303858424 for ; Mon, 24 Jun 2024 13:55:41 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by sourceware.org (Postfix) with ESMTPS id EF0AE3858D35 for ; Mon, 24 Jun 2024 13:55:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org EF0AE3858D35 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org EF0AE3858D35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=198.175.65.14 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1719237318; cv=none; b=Z+Cuj/PmRkXqOHNmuWZMio+7XRTdSoisTncjrCPq6DdEBw3S1ckO3SxYs7PAA36B4N90JCjL2d8PWxbyBQPoG92cJYt9BG4+Ro4CBb1UY2+aPcjEIkyB8xQFcZdBjXkF9xrCwFXbHqnPDm1Z7fKxRlUmlLulYAHzto/cc/ifX7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1719237318; c=relaxed/simple; bh=sEH0xLumW3+GCt+4eQxoGbkg3gO1nuoy3GEgV1As2mo=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=WtVEltINPdZG8JJMbqLcBSJbfgM0if5uwFqQnBvTQqXkIlr7ZBVuhRNan9Y/FQJs4ShAc22x3Sb0L4EEGmJZfw4q63agRICL1badtB+jVFld9JGNuYigx4+ESpDAPTdT7mHJp/DXURv8FbkiZ940w45lO33CwqB90mYyNpmZL/k= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1719237317; x=1750773317; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sEH0xLumW3+GCt+4eQxoGbkg3gO1nuoy3GEgV1As2mo=; b=ae9Ye81nspYVDQQRoqylOfzSuLbNerN4gxE2j2w0IEG8jvxpjSKxMhzI 8fHVqVDK63Ci3Q4OwdXXmDSqdY/dq+y4qSYlLykUHwxeJK9W2toFbotus yNRU8rBAe5pkIVGomutSdLYI/FmQcjJ5B5kOgRxgL/66GfgDr/fxjxzpw 8RYAiZJ7S+LxdONVqRXzaYAZlIHHq+DgNn0gr8lLRbEzUDEXtl59+6DbY sTIVPclsOvkO59PNFx2NzgEJ2PEWWy4iGJWidvQOQzny/Tyfk3IC2YNwv XxNJfifbsDIVbPZ3cplexYXJXfr6UzKpKDP1LHDAr95FcRSZdFRTU6oBy Q==; X-CSE-ConnectionGUID: qnRxb6lrSpOpo5UTP1VB4A== X-CSE-MsgGUID: BAVgtGUdTUC0rajNivSKuw== X-IronPort-AV: E=McAfee;i="6700,10204,11112"; a="20020847" X-IronPort-AV: E=Sophos;i="6.08,262,1712646000"; d="scan'208";a="20020847" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jun 2024 06:55:16 -0700 X-CSE-ConnectionGUID: jRpV3roHSc2hgXyQ2PBxsQ== X-CSE-MsgGUID: YubU2HR/SCiowY5rzW8/8A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,262,1712646000"; d="scan'208";a="43979378" Received: from shvmail02.sh.intel.com ([10.239.244.9]) by orviesa007.jf.intel.com with ESMTP; 24 Jun 2024 06:55:13 -0700 Received: from pli-ubuntu.sh.intel.com (pli-ubuntu.sh.intel.com [10.239.159.47]) by shvmail02.sh.intel.com (Postfix) with ESMTP id 2355410071CA; Mon, 24 Jun 2024 21:55:12 +0800 (CST) From: pan2.li@intel.com To: gcc-patches@gcc.gnu.org Cc: juzhe.zhong@rivai.ai, kito.cheng@gmail.com, richard.guenther@gmail.com, jeffreyalaw@gmail.com, pinskia@gmail.com, Pan Li Subject: [PATCH v2] Vect: Support truncate after .SAT_SUB pattern in zip Date: Mon, 24 Jun 2024 21:55:10 +0800 Message-Id: <20240624135510.3509497-1-pan2.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240621035252.742099-1-pan2.li@intel.com> References: <20240621035252.742099-1-pan2.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org From: Pan Li The zip benchmark of coremark-pro have one SAT_SUB like pattern but truncated as below: void test (uint16_t *x, unsigned b, unsigned n) { unsigned a = 0; register uint16_t *p = x; do { a = *--p; *p = (uint16_t)(a >= b ? a - b : 0); // Truncate after .SAT_SUB } while (--n); } It will have gimple before vect pass, it cannot hit any pattern of SAT_SUB and then cannot vectorize to SAT_SUB. _2 = a_11 - b_12(D); iftmp.0_13 = (short unsigned int) _2; _18 = a_11 >= b_12(D); iftmp.0_5 = _18 ? iftmp.0_13 : 0; This patch would like to improve the pattern match to recog above as truncate after .SAT_SUB pattern. Then we will have the pattern similar to below, as well as eliminate the first 3 dead stmt. _2 = a_11 - b_12(D); iftmp.0_13 = (short unsigned int) _2; _18 = a_11 >= b_12(D); iftmp.0_5 = (short unsigned int).SAT_SUB (a_11, b_12(D)); The below tests are passed for this patch. 1. The rv64gcv fully regression tests. 2. The rv64gcv build with glibc. 3. The x86 bootstrap tests. 4. The x86 fully regression tests. gcc/ChangeLog: * match.pd: Add convert description for minus and capture. * tree-vect-patterns.cc (vect_recog_build_binary_gimple_call): Add new logic to handle in_type is incompatibile with out_type, as well as rename from. (vect_recog_build_binary_gimple_stmt): Rename to. (vect_recog_sat_add_pattern): Leverage above renamed func. (vect_recog_sat_sub_pattern): Ditto. Signed-off-by: Pan Li --- gcc/match.pd | 4 +-- gcc/tree-vect-patterns.cc | 51 ++++++++++++++++++++++++--------------- 2 files changed, 33 insertions(+), 22 deletions(-) diff --git a/gcc/match.pd b/gcc/match.pd index 3d0689c9312..4a4b0b2e72f 100644 --- a/gcc/match.pd +++ b/gcc/match.pd @@ -3164,9 +3164,9 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT) /* Unsigned saturation sub, case 2 (branch with ge): SAT_U_SUB = X >= Y ? X - Y : 0. */ (match (unsigned_integer_sat_sub @0 @1) - (cond^ (ge @0 @1) (minus @0 @1) integer_zerop) + (cond^ (ge @0 @1) (convert? (minus (convert1? @0) (convert1? @1))) integer_zerop) (if (INTEGRAL_TYPE_P (type) && TYPE_UNSIGNED (type) - && types_match (type, @0, @1)))) + && TYPE_UNSIGNED (TREE_TYPE (@0)) && types_match (@0, @1)))) /* Unsigned saturation sub, case 3 (branchless with gt): SAT_U_SUB = (X - Y) * (X > Y). */ diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index cef901808eb..3d887d36050 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -4490,26 +4490,37 @@ vect_recog_mult_pattern (vec_info *vinfo, extern bool gimple_unsigned_integer_sat_add (tree, tree*, tree (*)(tree)); extern bool gimple_unsigned_integer_sat_sub (tree, tree*, tree (*)(tree)); -static gcall * -vect_recog_build_binary_gimple_call (vec_info *vinfo, gimple *stmt, +static gimple * +vect_recog_build_binary_gimple_stmt (vec_info *vinfo, stmt_vec_info stmt_info, internal_fn fn, tree *type_out, - tree op_0, tree op_1) + tree lhs, tree op_0, tree op_1) { tree itype = TREE_TYPE (op_0); - tree vtype = get_vectype_for_scalar_type (vinfo, itype); + tree otype = TREE_TYPE (lhs); + tree v_itype = get_vectype_for_scalar_type (vinfo, itype); + tree v_otype = get_vectype_for_scalar_type (vinfo, otype); - if (vtype != NULL_TREE - && direct_internal_fn_supported_p (fn, vtype, OPTIMIZE_FOR_BOTH)) + if (v_itype != NULL_TREE && v_otype != NULL_TREE + && direct_internal_fn_supported_p (fn, v_itype, OPTIMIZE_FOR_BOTH)) { gcall *call = gimple_build_call_internal (fn, 2, op_0, op_1); + tree in_ssa = vect_recog_temp_ssa_var (itype, NULL); - gimple_call_set_lhs (call, vect_recog_temp_ssa_var (itype, NULL)); + gimple_call_set_lhs (call, in_ssa); gimple_call_set_nothrow (call, /* nothrow_p */ false); - gimple_set_location (call, gimple_location (stmt)); + gimple_set_location (call, gimple_location (STMT_VINFO_STMT (stmt_info))); + + *type_out = v_otype; - *type_out = vtype; + if (types_compatible_p (itype, otype)) + return call; + else + { + append_pattern_def_seq (vinfo, stmt_info, call, v_itype); + tree out_ssa = vect_recog_temp_ssa_var (otype, NULL); - return call; + return gimple_build_assign (out_ssa, CONVERT_EXPR, in_ssa); + } } return NULL; @@ -4541,13 +4552,13 @@ vect_recog_sat_add_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, if (gimple_unsigned_integer_sat_add (lhs, ops, NULL)) { - gcall *call = vect_recog_build_binary_gimple_call (vinfo, last_stmt, - IFN_SAT_ADD, type_out, - ops[0], ops[1]); - if (call) + gimple *stmt = vect_recog_build_binary_gimple_stmt (vinfo, stmt_vinfo, + IFN_SAT_ADD, type_out, + lhs, ops[0], ops[1]); + if (stmt) { vect_pattern_detected ("vect_recog_sat_add_pattern", last_stmt); - return call; + return stmt; } } @@ -4579,13 +4590,13 @@ vect_recog_sat_sub_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, if (gimple_unsigned_integer_sat_sub (lhs, ops, NULL)) { - gcall *call = vect_recog_build_binary_gimple_call (vinfo, last_stmt, - IFN_SAT_SUB, type_out, - ops[0], ops[1]); - if (call) + gimple *stmt = vect_recog_build_binary_gimple_stmt (vinfo, stmt_vinfo, + IFN_SAT_SUB, type_out, + lhs, ops[0], ops[1]); + if (stmt) { vect_pattern_detected ("vect_recog_sat_sub_pattern", last_stmt); - return call; + return stmt; } }