From patchwork Sat Jan 13 01:51:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6ZKf5bGF5ZOy?= X-Patchwork-Id: 1886303 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TBhGz5jT0z1yPf for ; Sat, 13 Jan 2024 12:52:10 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6F5E63858CDA for ; Sat, 13 Jan 2024 01:52:08 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgau2.qq.com (smtpbgau2.qq.com [54.206.34.216]) by sourceware.org (Postfix) with ESMTPS id A3AEB3858D1E for ; Sat, 13 Jan 2024 01:51:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A3AEB3858D1E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A3AEB3858D1E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.206.34.216 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705110706; cv=none; b=vUfEZ7hWH4mPKTu7Ym4mXZZpbRh3CMC/kOzNBLPLlYND/m+o6gu9Nox5WfhS4WDwVrTXlWHnDggJTO29VrafLiB0eRal6e1ZCW3IaaewUP47fTIQpgbxdMHVyfvB66cLWt7hETDHgnvE6fwO9LIV5MgE/HIBBii4bW0xfCVzhGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705110706; c=relaxed/simple; bh=M4N4xK8VLv2hCjjOc5A7F6IdT5leec4Un8r1JxXlB9s=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=Ys+1NW39n3rO48V/+TxNU8r6Droe7PMjttsKwA47+/zoe/SmT29eN5M/2Xsw4T2ykdEtvEYM+3UzPOt8GPcgUpjKURNpaEioGiGiDJ4LSvDT8ILdzUz5cbJk2iNZSm8PcEVqfQXCm5kIA57rUC0FVz2CGjGaW2wa1cKpOBydEaQ= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp62t1705110695tkxcp53o Received: from rios-cad122.hadoop.rioslab.org ( [58.60.1.26]) by bizesmtp.qq.com (ESMTP) with id ; Sat, 13 Jan 2024 09:51:34 +0800 (CST) X-QQ-SSF: 01400000000000G0V000000A0000000 X-QQ-FEAT: 7jw2iSiCazp3UcKTQHIS5xpDFFCkDpTd+3KRWnY6ynLi/WSWff/x8C7fXslMD Vw6edu4koYjRjslXsUO9GAUeWtIHv6+4KnxnDrRcx/KAToguX0gU4UsQrNHrInURN0gOkke Xf2oCoOGIb0QUwcwNQhwKtIdJ3c/cmM1QRcL4PrP50MbvRIwZ3VE/aGyL9BHUan+T8MiFJs oviqy9FhpwoM2Wf/mwf8+dO4/LnmiHYuUkc7XU7m60kFL30OpN+T/j9Whl/lq1BIU5f874a HG3dxpVuANMYbIbP+U4CQwneuB1hZVWU0tM0d6fTfCdPx9Qv1VsE1EB/mfXaN03PKdA+09/ HVbZGpsjQIYFVJ/pMjNMZ9vfPoMUc1whaxDGVOhlTlogxuLFX6Yx5xKFxJbAg== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 17733438947770347045 From: Juzhe-Zhong To: gcc-patches@gcc.gnu.org Cc: kito.cheng@gmail.com, kito.cheng@sifive.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, Juzhe-Zhong Subject: [PATCH] RISC-V: Adjust loop len by costing 1 when NITER < VF [GCC 14 regression] Date: Sat, 13 Jan 2024 09:51:33 +0800 Message-Id: <20240113015133.741517-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org This patch fixes the regression between GCC 13.2.0 and trunk GCC (GCC-14) GCC 13.2.0: lui a5,%hi(a) li a4,19 sb a4,%lo(a)(a5) li a0,0 ret Trunk GCC: vsetvli a5,zero,e8,mf2,ta,ma li a4,-32768 vid.v v1 vsetvli zero,zero,e16,m1,ta,ma addiw a4,a4,104 vmv.v.i v3,15 lui a1,%hi(a) li a0,19 vsetvli zero,zero,e8,mf2,ta,ma vadd.vi v1,v1,1 sb a0,%lo(a)(a1) vsetvli zero,zero,e16,m1,ta,ma vzext.vf2 v2,v1 vmv.v.x v1,a4 vminu.vv v2,v2,v3 vsrl.vv v1,v1,v2 vslidedown.vi v1,v1,17 vmv.x.s a0,v1 snez a0,a0 ret The root cause we are vectorizing the codes inefficiently since we doesn't cost len when NITERS < VF. Leverage loop control of mask targets or rs6000 fixes the regression. Tested no regression. Ok for trunk ? PR target/113281 gcc/ChangeLog: * config/riscv/riscv-vector-costs.cc (costs::adjust_vect_cost_per_loop): New function. (costs::finish_cost): Adjust cost * config/riscv/riscv-vector-costs.h: New function. gcc/testsuite/ChangeLog: * gcc.dg/vect/costmodel/riscv/rvv/pr113281-3.c: New test. * gcc.dg/vect/costmodel/riscv/rvv/pr113281-4.c: New test. --- gcc/config/riscv/riscv-vector-costs.cc | 61 +++++++++++++++++++ gcc/config/riscv/riscv-vector-costs.h | 2 + .../vect/costmodel/riscv/rvv/pr113281-3.c | 18 ++++++ .../vect/costmodel/riscv/rvv/pr113281-4.c | 18 ++++++ 4 files changed, 99 insertions(+) create mode 100644 gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-3.c create mode 100644 gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-4.c diff --git a/gcc/config/riscv/riscv-vector-costs.cc b/gcc/config/riscv/riscv-vector-costs.cc index 1c3708f23a0..9c0b9a874de 100644 --- a/gcc/config/riscv/riscv-vector-costs.cc +++ b/gcc/config/riscv/riscv-vector-costs.cc @@ -1110,9 +1110,70 @@ costs::add_stmt_cost (int count, vect_cost_for_stmt kind, return record_stmt_cost (stmt_info, where, count * stmt_cost); } +/* For some target specific vectorization cost which can't be handled per stmt, + we check the requisite conditions and adjust the vectorization cost + accordingly if satisfied. One typical example is to model model and adjust + loop_len cost for known_lt (NITERS, VF). */ + +void +costs::adjust_vect_cost_per_loop (loop_vec_info loop_vinfo) +{ + if (LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo) + && !LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo) + && m_num_vector_iterations == 1 + && LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) + && known_le (LOOP_VINFO_INT_NITERS (loop_vinfo), + LOOP_VINFO_VECT_FACTOR (loop_vinfo))) + { + /* In middle-end loop vectorizer, we don't count the loop_len cost in + vect_estimate_min_profitable_iters when NITERS < VF, that is, we only + count cost of len that we need to iterate loop more than once with VF + (m_num_vector_iterations > 1). It's correct for most of the cases: + + E.g. VF = [4, 4] + for (int i = 0; i < 3; i ++) + a[i] += b[i]; + + We don't need to cost MIN_EXPR or SELECT_VL for the case above. + + However, for some inefficient vectorized cases, it does use MIN_EXPR + to generate len. + + E.g. VF = [256, 256] + + Loop body: + # loop_len_110 = PHI <18(2), _119(11)> + ... + _117 = MIN_EXPR ; + _118 = 18 - _117; + _119 = MIN_EXPR <_118, POLY_INT_CST [256, 256]>; + ... + + Epilogue: + ... + _112 = .VEC_EXTRACT (vect_patt_27.14_109, _111); + + We cost 1 unconditionally for this situation like other targets which + apply mask as the loop control. */ + rgroup_controls *rgc; + unsigned int num_vectors_m1; + unsigned int body_stmts = 0; + FOR_EACH_VEC_ELT (LOOP_VINFO_LENS (loop_vinfo), num_vectors_m1, rgc) + if (rgc->type) + body_stmts += num_vectors_m1 + 1; + + add_stmt_cost (body_stmts, scalar_stmt, NULL, NULL, NULL_TREE, 0, + vect_body); + } +} + void costs::finish_cost (const vector_costs *scalar_costs) { + if (loop_vec_info loop_vinfo = dyn_cast (m_vinfo)) + { + adjust_vect_cost_per_loop (loop_vinfo); + } vector_costs::finish_cost (scalar_costs); } diff --git a/gcc/config/riscv/riscv-vector-costs.h b/gcc/config/riscv/riscv-vector-costs.h index 9bf041bb65c..3defd45fd4c 100644 --- a/gcc/config/riscv/riscv-vector-costs.h +++ b/gcc/config/riscv/riscv-vector-costs.h @@ -101,6 +101,8 @@ private: V_REGS spills according to the analysis. */ bool m_has_unexpected_spills_p = false; void record_potential_unexpected_spills (loop_vec_info); + + void adjust_vect_cost_per_loop (loop_vec_info); }; } // namespace riscv_vector diff --git a/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-3.c b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-3.c new file mode 100644 index 00000000000..706e19116c9 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-3.c @@ -0,0 +1,18 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv_zvl4096b -mabi=lp64d -O3 -ftree-vectorize --param=riscv-autovec-lmul=m8" } */ + +unsigned char a; + +int main() { + short b = a = 0; + for (; a != 19; a++) + if (a) + b = 32872 >> a; + + if (b == 0) + return 0; + else + return 1; +} + +/* { dg-final { scan-assembler-not {vset} } } */ diff --git a/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-4.c b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-4.c new file mode 100644 index 00000000000..b0305db2d48 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr113281-4.c @@ -0,0 +1,18 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv_zvl4096b -mabi=lp64d -O3 -ftree-vectorize --param=riscv-autovec-lmul=m8 --param=riscv-autovec-preference=fixed-vlmax" } */ + +unsigned char a; + +int main() { + short b = a = 0; + for (; a != 19; a++) + if (a) + b = 32872 >> a; + + if (b == 0) + return 0; + else + return 1; +} + +/* { dg-final { scan-assembler-not {vset} } } */