From patchwork Fri Jul 5 14:20:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Pan2" X-Patchwork-Id: 1957333 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=BRgR1Hvk; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WFwfQ34GYz1xql for ; Sat, 6 Jul 2024 00:20:46 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8952E3885C3F for ; Fri, 5 Jul 2024 14:20:44 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) by sourceware.org (Postfix) with ESMTPS id 4F0FE388459B for ; Fri, 5 Jul 2024 14:20:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4F0FE388459B Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4F0FE388459B Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=192.198.163.16 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720189226; cv=none; b=Q2u+hmzln77jysRCFLIr5YughZhk8VTZRp7//drsPIh54OpaLq41x5bAILWDaFq32ZiO8TK4Q3HCBJEaybgjiDGCIcs/Lz/e2g6RuO3Tjw/o2H2TZxwOhstxsaEuCmtinRlniHN+GVe81tOdyVOHn3hct4TaVWbIXbJe3JQ+M3w= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720189226; c=relaxed/simple; bh=iaw260TmYiCsZqnP/coRKqcQPcj9Ixd53IYe2qdnHRA=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=oyVTzgvLQiBvlZ1in8C5NthJ1NjZ9H4IeL8Obl72FQsAKbv67rTKfhDWC/d3Bu3eu32ilNMZA3Sk4721e6g50Bl2X+QeSVfcfgSI+0pLS3VIx87Kw/74qnzv/SnQEGbU4T71ox4sgmjcU+xvWG4YfNq+DGTBeqIbPUK0b1ChSDk= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1720189216; x=1751725216; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=iaw260TmYiCsZqnP/coRKqcQPcj9Ixd53IYe2qdnHRA=; b=BRgR1Hvk6MWiS0BpaVuKXjklh8qrZcwvJlWkJ3kdCG1t4ZlyDV4x1Z1N 0VFV2VszPeJDTYmzq1CxZYEv1/71jJmTS85WVU3rogcErK7YVwHj9sEwR 2dbrjGPz/bFgWlrwomXjVvazI43bBbVGGfFKOxxmIItdP1lrU6HcWgoVR 9SfbEfeF0DtGbukLceUBn4kaQuFbC1R56uHtZXWImvq3W8SJ8fKo6W4wN 02M7MovV2jYXziOE4HXOTKAzSVcR6C0BuoUm/j87g5QkMqnwnbVXuFXBn +5U3b1US4mVsKGR4GhtHQzbF0Hw1gwH2dp/q7RGgOM5EIg9IGvsLwNWIQ w==; X-CSE-ConnectionGUID: Ab7G8K6uQo6GcCSQzkJ1GA== X-CSE-MsgGUID: 8EyWm0hAReW6A5unGUwH6w== X-IronPort-AV: E=McAfee;i="6700,10204,11123"; a="12408071" X-IronPort-AV: E=Sophos;i="6.09,184,1716274800"; d="scan'208";a="12408071" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2024 07:20:15 -0700 X-CSE-ConnectionGUID: XwaU3CY7Tz2H9O9yORkFQw== X-CSE-MsgGUID: Mm/hs+kTR1W4Z51TyvRCpQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,184,1716274800"; d="scan'208";a="77634341" Received: from shvmail03.sh.intel.com ([10.239.245.20]) by orviesa002.jf.intel.com with ESMTP; 05 Jul 2024 07:20:11 -0700 Received: from pli-ubuntu.sh.intel.com (pli-ubuntu.sh.intel.com [10.239.159.47]) by shvmail03.sh.intel.com (Postfix) with ESMTP id CAE5D100700E; Fri, 5 Jul 2024 22:20:10 +0800 (CST) From: pan2.li@intel.com To: gcc-patches@gcc.gnu.org Cc: juzhe.zhong@rivai.ai, kito.cheng@gmail.com, richard.guenther@gmail.com, tamar.christina@arm.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, hongtao.liu@intel.com, Pan Li Subject: [PATCH v2] Vect: Distribute truncation into .SAT_SUB operands Date: Fri, 5 Jul 2024 22:20:08 +0800 Message-Id: <20240705142008.3965248-1-pan2.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org From: Pan Li To get better vectorized code of .SAT_SUB, we would like to avoid the truncated operation for the assignment. For example, as below. unsigned int _1; unsigned int _2; _9 = (unsigned short int).SAT_SUB (_1, _2); If we make sure that the _1 is in the range of unsigned short int. Such as a def similar to: _1 = (unsigned short int)_4; Then we can do the distribute the truncation operation to: _3 = MIN_EXPR (_2, 65535); _9 = .SAT_SUB ((unsigned short int)_1, (unsigned short int)_3); Let's take RISC-V vector as example to tell the changes. For below sample code: __attribute__((noinline)) void test (uint16_t *x, unsigned b, unsigned n) { unsigned a = 0; uint16_t *p = x; do { a = *--p; *p = (uint16_t)(a >= b ? a - b : 0); } while (--n); } Before this patch: ... .L3: vle16.v v1,0(a3) vrsub.vx v5,v2,t1 mv t3,a4 addw a4,a4,t5 vrgather.vv v3,v1,v5 vsetvli zero,zero,e32,m1,ta,ma vzext.vf2 v1,v3 vssubu.vx v1,v1,a1 vsetvli zero,zero,e16,mf2,ta,ma vncvt.x.x.w v1,v1 vrgather.vv v3,v1,v5 vse16.v v3,0(a3) sub a3,a3,t4 bgtu t6,a4,.L3 ... After this patch: test: ... .L3: vle16.v v3,0(a3) vrsub.vx v5,v2,a6 mv a7,a4 addw a4,a4,t3 vrgather.vv v1,v3,v5 vssubu.vv v1,v1,v6 vrgather.vv v3,v1,v5 vse16.v v3,0(a3) sub a3,a3,t1 bgtu t4,a4,.L3 ... The below test suites are passed for this patch: 1. The rv64gcv fully regression tests. 2. The rv64gcv build with glibc. 3. The x86 bootstrap tests. 4. The x86 fully regression tests. gcc/ChangeLog: * tree-vect-patterns.cc (vect_recog_sat_sub_pattern_distribute): Add new func impl to perform the truncation distribution. (vect_recog_sat_sub_pattern): Perform above optimize before generate .SAT_SUB call. Signed-off-by: Pan Li --- gcc/tree-vect-patterns.cc | 75 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 86e893a1c43..90449bd0ddd 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -4566,6 +4566,79 @@ vect_recog_sat_add_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, return NULL; } +/* + * Try to distribute the truncation for .SAT_SUB pattern, mostly occurs in + * the benchmark zip. Aka: + * + * unsigned int _1; + * unsigned int _2; + * _9 = (unsigned short int).SAT_SUB (_1, _2); + * + * if _1 is known to be in the range of unsigned short int. For example + * there is a def _1 = (unsigned short int)_4. Then we can distribute the + * truncation to: + * + * _3 = MIN (65535, _2); + * _9 = .SAT_SUB (_4, (unsigned short int)_3); + * + * Then, we can better vectorized code and avoid the unnecessary narrowing + * stmt during vectorization. + */ +static void +vect_recog_sat_sub_pattern_distribute (vec_info *vinfo, + stmt_vec_info stmt_vinfo, + gimple *stmt, tree lhs, tree *ops) +{ + tree otype = TREE_TYPE (lhs); + tree itype = TREE_TYPE (ops[0]); + unsigned itype_prec = TYPE_PRECISION (itype); + unsigned otype_prec = TYPE_PRECISION (otype); + + if (types_compatible_p (otype, itype) || otype_prec >= itype_prec) + return; + + int_range_max r; + gimple_ranger granger; + + if (granger.range_of_expr (r, ops[0], stmt) && !r.undefined_p ()) + { + wide_int bound = r.upper_bound (); + wide_int otype_max = wi::mask (otype_prec, /* negate */false, itype_prec); + tree v_otype = get_vectype_for_scalar_type (vinfo, otype); + tree v_itype = get_vectype_for_scalar_type (vinfo, itype); + + if (!wi::leu_p (bound, otype_max) || v_otype == NULL || v_itype == NULL + || !target_supports_op_p (v_itype, MIN_EXPR, optab_vector)) + return; + + /* 1. Build truncated op_0 */ + vect_unpromoted_value unprom; + tree tmp = vect_look_through_possible_promotion (vinfo, ops[0], &unprom); + + if (tmp == NULL_TREE || TYPE_PRECISION (unprom.type) != otype_prec) + { + tmp = vect_recog_temp_ssa_var (otype, NULL); + gimple *op_0_cast = gimple_build_assign (tmp, NOP_EXPR, ops[0]); + append_pattern_def_seq (vinfo, stmt_vinfo, op_0_cast, v_otype); + } + + ops[0] = tmp; + + /* 2. Build MIN_EXPR (op_1, 65536) */ + tree max = wide_int_to_tree (itype, otype_max); + tree op_1_in = vect_recog_temp_ssa_var (itype, NULL); + gimple *op_1_min = gimple_build_assign (op_1_in, MIN_EXPR, ops[1], max); + append_pattern_def_seq (vinfo, stmt_vinfo, op_1_min, v_itype); + + /* 3. Build truncated op_1 */ + tmp = vect_recog_temp_ssa_var (otype, NULL); + gimple *op_1_cast = gimple_build_assign (tmp, NOP_EXPR, op_1_in); + append_pattern_def_seq (vinfo, stmt_vinfo, op_1_cast, v_otype); + + ops[1] = tmp; + } +} + /* * Try to detect saturation sub pattern (SAT_ADD), aka below gimple: * _7 = _1 >= _2; @@ -4591,6 +4664,8 @@ vect_recog_sat_sub_pattern (vec_info *vinfo, stmt_vec_info stmt_vinfo, if (gimple_unsigned_integer_sat_sub (lhs, ops, NULL)) { + vect_recog_sat_sub_pattern_distribute (vinfo, stmt_vinfo, last_stmt, + lhs, ops); gimple *stmt = vect_recog_build_binary_gimple_stmt (vinfo, stmt_vinfo, IFN_SAT_SUB, type_out, lhs, ops[0], ops[1]);