From patchwork Tue Jul 9 04:02:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Pan2" X-Patchwork-Id: 1958189 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Jgey9VZb; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WJ6mD1VBSz1xr9 for ; Tue, 9 Jul 2024 14:03:22 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 933DA386182F for ; Tue, 9 Jul 2024 04:03:20 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) by sourceware.org (Postfix) with ESMTPS id 5144E3858D35 for ; Tue, 9 Jul 2024 04:02:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5144E3858D35 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5144E3858D35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720497782; cv=none; b=eNfIYD6MWHKOLHsK+jvnh4HIeHlo1mT8/K0jBoj3k37TaCNuaDCr1VnTKYKaLwezqnyd1Eyixlr8GiuYloZB5DBC/Tw0E6rB/UCskH8T5ArOw958C51FQ8hxWnZMp0javfQ1nOJX4E+kyLfnBLBU6jx6yngirKNZbP3Jz7NdOzs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720497782; c=relaxed/simple; bh=dHwPHRonQLzKXjkALkLJV2KSWaOzpovFiKmNrQBXvv8=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=rJTVRDEcJmyX8qpN/SbzQ2R80lTusTs5NgWLCA1RRWPgZaCYadYkrOx3eXp4eEyFYr3HD2Bj4Llo5ASVc2sBZq1obXJ/p7Rn6e41Wr3L4PJeI0xF95kNqE/tv8Q4569CDnvwdZQ0cId2cpTlUPae7qJH6wucdbxB3SC+VgDjcpQ= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1720497780; x=1752033780; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=dHwPHRonQLzKXjkALkLJV2KSWaOzpovFiKmNrQBXvv8=; b=Jgey9VZb/KDnWxui9B+qQowzjryXKnrMjJN4SF3GeEEPF723sjGya7FU 2EgksKfEv0b9jNKsIUAzNacWH22u+3WFtotd6/lvkh5X3VFesEzlIVqPY LmwOQS50MPGT6whc0zCVvrBmM47rPxMUa76+/pBjMlbsRZCoF9qhkWivV VnRCEeZY0d6m3YNaxWE3YC4vFgn26Jw610VY9IwMzeDWOrB0o3w19vaUw WJBq+JD4WKLIlXwJ1fkr/ariy039KgBcUCpnBs9FbLBJt2b9HRr6dO3Cd BoEB+P7Lwqfip8jVBt9rMbenv1tklm/CIqsLLcp9ocmyDkctOp9f5E7FE A==; X-CSE-ConnectionGUID: 85DPkae/SrWIRDxdLvfEMA== X-CSE-MsgGUID: yFNdgbWrTSe/P2UTufEGPA== X-IronPort-AV: E=McAfee;i="6700,10204,11127"; a="43149803" X-IronPort-AV: E=Sophos;i="6.09,193,1716274800"; d="scan'208";a="43149803" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jul 2024 21:02:50 -0700 X-CSE-ConnectionGUID: rdYeEoywThyE158g52wn2g== X-CSE-MsgGUID: XH2tPyUjRJeKvC7lF/RDgw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,193,1716274800"; d="scan'208";a="48439197" Received: from shvmail02.sh.intel.com ([10.239.244.9]) by orviesa008.jf.intel.com with ESMTP; 08 Jul 2024 21:02:48 -0700 Received: from pli-ubuntu.sh.intel.com (pli-ubuntu.sh.intel.com [10.239.159.47]) by shvmail02.sh.intel.com (Postfix) with ESMTP id 4EF0B1005684; Tue, 9 Jul 2024 12:02:46 +0800 (CST) From: pan2.li@intel.com To: gcc-patches@gcc.gnu.org Cc: juzhe.zhong@rivai.ai, kito.cheng@gmail.com, richard.guenther@gmail.com, tamar.christina@arm.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, hongtao.liu@intel.com, Pan Li Subject: [PATCH v3] Vect: Optimize truncation for .SAT_SUB operands Date: Tue, 9 Jul 2024 12:02:44 +0800 Message-Id: <20240709040244.3051328-1-pan2.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org From: Pan Li To get better vectorized code of .SAT_SUB, we would like to avoid the truncated operation for the assignment. For example, as below. unsigned int _1; unsigned int _2; unsigned short int _4; _9 = (unsigned short int).SAT_SUB (_1, _2); If we make sure that the _1 is in the range of unsigned short int. Such as a def similar to: _1 = (unsigned short int)_4; Then we can do the distribute the truncation operation to: _3 = (unsigned short int) MIN (65535, _2); // aka _3 = .SAT_TRUNC (_2); _9 = .SAT_SUB (_4, _3); Then, we can better vectorized code and avoid the unnecessary narrowing stmt during vectorization with below stmt(s). _3 = .SAT_TRUNC(_2); // SI => HI _9 = .SAT_SUB (_4, _3); Let's take RISC-V vector as example to tell the changes. For below sample code: __attribute__((noinline)) void test (uint16_t *x, unsigned b, unsigned n) { unsigned a = 0; uint16_t *p = x; do { a = *--p; *p = (uint16_t)(a >= b ? a - b : 0); } while (--n); } Before this patch: ... .L3: vle16.v v1,0(a3) vrsub.vx v5,v2,t1 mv t3,a4 addw a4,a4,t5 vrgather.vv v3,v1,v5 vsetvli zero,zero,e32,m1,ta,ma vzext.vf2 v1,v3 vssubu.vx v1,v1,a1 vsetvli zero,zero,e16,mf2,ta,ma vncvt.x.x.w v1,v1 vrgather.vv v3,v1,v5 vse16.v v3,0(a3) sub a3,a3,t4 bgtu t6,a4,.L3 ... After this patch: test: ... .L3: vle16.v v3,0(a3) vrsub.vx v5,v2,a6 mv a7,a4 addw a4,a4,t3 vrgather.vv v1,v3,v5 vssubu.vv v1,v1,v6 vrgather.vv v3,v1,v5 vse16.v v3,0(a3) sub a3,a3,t1 bgtu t4,a4,.L3 ... The below test suites are passed for this patch: 1. The rv64gcv fully regression tests. 2. The rv64gcv build with glibc. 3. The x86 bootstrap tests. 4. The x86 fully regression tests. gcc/ChangeLog: * tree-vect-patterns.cc (vect_recog_sat_sub_pattern_transform): Add new func impl to perform the truncation distribution. (vect_recog_sat_sub_pattern): Perform above optimize before generate .SAT_SUB call. Signed-off-by: Pan Li --- gcc/tree-vect-patterns.cc | 65 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 86e893a1c43..4570c25b664 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -4566,6 +4566,70 @@ vect_recog_sat_add_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, return NULL; } +/* + * Try to transform the truncation for .SAT_SUB pattern, mostly occurs in + * the benchmark zip. Aka: + * + * unsigned int _1; + * unsigned int _2; + * unsigned short int _4; + * _9 = (unsigned short int).SAT_SUB (_1, _2); + * + * if _1 is known to be in the range of unsigned short int. For example + * there is a def _1 = (unsigned short int)_4. Then we can transform the + * truncation to: + * + * _3 = (unsigned short int) MIN (65535, _2); // aka _3 = .SAT_TRUNC (_2); + * _9 = .SAT_SUB (_4, _3); + * + * Then, we can better vectorized code and avoid the unnecessary narrowing + * stmt during vectorization with below stmt(s). + * + * _3 = .SAT_TRUNC(_2); // SI => HI + * _9 = .SAT_SUB (_4, _3); + */ +static void +vect_recog_sat_sub_pattern_transform (vec_info *vinfo, + stmt_vec_info stmt_vinfo, + tree lhs, tree *ops) +{ + tree otype = TREE_TYPE (lhs); + tree itype = TREE_TYPE (ops[0]); + unsigned itype_prec = TYPE_PRECISION (itype); + unsigned otype_prec = TYPE_PRECISION (otype); + + if (types_compatible_p (otype, itype) || otype_prec >= itype_prec) + return; + + tree v_otype = get_vectype_for_scalar_type (vinfo, otype); + tree v_itype = get_vectype_for_scalar_type (vinfo, itype); + tree_pair v_pair = tree_pair (v_otype, v_itype); + + if (v_otype == NULL_TREE || v_itype == NULL_TREE + || !direct_internal_fn_supported_p (IFN_SAT_TRUNC, v_pair, + OPTIMIZE_FOR_BOTH)) + return; + + /* 1. Find the _4 and update ops[0] as above example. */ + vect_unpromoted_value unprom; + tree tmp = vect_look_through_possible_promotion (vinfo, ops[0], &unprom); + + if (tmp == NULL_TREE || TYPE_PRECISION (unprom.type) != otype_prec) + return; + + ops[0] = tmp; + + /* 2. Generate _3 = .SAT_TRUNC (_2) and update ops[1] as above example. */ + tree trunc_lhs_ssa = vect_recog_temp_ssa_var (otype, NULL); + gcall *call = gimple_build_call_internal (IFN_SAT_TRUNC, 1, ops[1]); + + gimple_call_set_lhs (call, trunc_lhs_ssa); + gimple_call_set_nothrow (call, /* nothrow_p */ false); + append_pattern_def_seq (vinfo, stmt_vinfo, call, v_otype); + + ops[1] = trunc_lhs_ssa; +} + /* * Try to detect saturation sub pattern (SAT_ADD), aka below gimple: * _7 = _1 >= _2; @@ -4591,6 +4655,7 @@ vect_recog_sat_sub_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, if (gimple_unsigned_integer_sat_sub (lhs, ops, NULL)) { + vect_recog_sat_sub_pattern_transform (vinfo, stmt_vinfo, lhs, ops); gimple *stmt = vect_recog_build_binary_gimple_stmt (vinfo, stmt_vinfo, IFN_SAT_SUB, type_out, lhs, ops[0], ops[1]);