From patchwork Tue Jul 27 03:20:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kewen.Lin" X-Patchwork-Id: 1510249 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=hg4rHILv; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GYhsz4mbKz9sX5 for ; Tue, 27 Jul 2021 13:21:58 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B080D396ECA5 for ; Tue, 27 Jul 2021 03:21:55 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B080D396ECA5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1627356115; bh=QDiJa7xTYkQDBcyTkv9uGvSl5nLziCXf2AQnHd6x2lA=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=hg4rHILvefpBOGDulFHB3UZNeCQDZBe/VYdhzS21BqMZJodzjDbcLAyENOYwchd7c jFUkDbkVNR5TZmxW1I0FHvbe0hWjJ10oEk4FMNDaK4YnYUg/9KnE8pXRciNNV5XdCb Dwj2rWDQjCXkbTvkYrz4oPcwQqhTY0C3mSRf8bMY= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id C51E9383D001 for ; Tue, 27 Jul 2021 03:21:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C51E9383D001 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16R3FL0D108039; Mon, 26 Jul 2021 23:21:03 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3a2a2t83my-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jul 2021 23:21:02 -0400 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 16R3IQOX127541; Mon, 26 Jul 2021 23:21:02 -0400 Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com with ESMTP id 3a2a2t83ks-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jul 2021 23:21:02 -0400 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 16R2QX3l002933; Tue, 27 Jul 2021 03:20:59 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma02fra.de.ibm.com with ESMTP id 3a235xr39j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 27 Jul 2021 03:20:59 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 16R3KuME21234032 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 27 Jul 2021 03:20:56 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 93FAF4C040; Tue, 27 Jul 2021 03:20:56 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 53AB14C046; Tue, 27 Jul 2021 03:20:55 +0000 (GMT) Received: from KewenLins-MacBook-Pro.local (unknown [9.197.253.22]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 27 Jul 2021 03:20:55 +0000 (GMT) To: GCC Patches Subject: [PATCH] vect: Fix wrong check in vect_recog_mulhs_pattern [PR101596] Message-ID: Date: Tue, 27 Jul 2021 11:20:53 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 Content-Language: en-US X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: ibhfOmFNilqlHpGMThyD-J5L7EnY-Rjh X-Proofpoint-GUID: E_aDyPTiPOiqTknk7mIQUkxdWnZvgEuS X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-07-27_03:2021-07-26, 2021-07-27 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 malwarescore=0 adultscore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 impostorscore=0 mlxscore=0 priorityscore=1501 lowpriorityscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2107270012 X-Spam-Status: No, score=-10.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Kewen.Lin via Gcc-patches" From: "Kewen.Lin" Reply-To: "Kewen.Lin" Cc: Richard Sandiford Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi, As PR101596 showed, vect_recog_mulhs_pattern uses target_precision to check the scale_term is expected or not, it could be wrong when the precision of the actual used new_type larger than target_precision as shown by the example. This patch is to use precision of new_type instead of target_precision for the scale_term matching check. Bootstrapped & regtested on powerpc64le-linux-gnu P10, powerpc64-linux-gnu P8, x86_64-redhat-linux and aarch64-linux-gnu. Is it ok for trunk? BR, Kewen ----- gcc/ChangeLog: PR tree-optimization/100696 * tree-vect-patterns.c (vect_recog_mulhs_pattern): Fix wrong check by using new_type's precision instead. gcc/testsuite/ChangeLog: PR tree-optimization/100696 * gcc.target/powerpc/pr101596-1.c: New test. * gcc.target/powerpc/pr101596-2.c: Likewise. * gcc.target/powerpc/pr101596-3.c: Likewise. --- gcc/testsuite/gcc.target/powerpc/pr101596-1.c | 30 +++++++++ gcc/testsuite/gcc.target/powerpc/pr101596-2.c | 30 +++++++++ gcc/testsuite/gcc.target/powerpc/pr101596-3.c | 58 +++++++++++++++++ gcc/tree-vect-patterns.c | 63 +++++++++++-------- 4 files changed, 154 insertions(+), 27 deletions(-) create mode 100644 gcc/testsuite/gcc.target/powerpc/pr101596-1.c create mode 100644 gcc/testsuite/gcc.target/powerpc/pr101596-2.c create mode 100644 gcc/testsuite/gcc.target/powerpc/pr101596-3.c diff --git a/gcc/testsuite/gcc.target/powerpc/pr101596-1.c b/gcc/testsuite/gcc.target/powerpc/pr101596-1.c new file mode 100644 index 00000000000..ed09e5f96ed --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/pr101596-1.c @@ -0,0 +1,30 @@ +/* { dg-require-effective-target power10_ok } */ +/* { dg-options "-mdejagnu-cpu=power10 -O2 -ftree-vectorize -fno-vect-cost-model -fdump-tree-vect-details" } */ + +/* Check vect_recog_mulhs_pattern can't be detected with shift count 48. */ + +#define N 128 + +typedef signed long long sLL; +typedef unsigned long long uLL; + +signed int si_a[N], si_b[N]; +unsigned int ui_a[N], ui_b[N]; +signed short sh_c[N]; +unsigned short uh_c[N]; + +void +test1 () +{ + for (int i = 0; i < N; i++) + sh_c[i] = ((sLL) si_a[i] * (sLL) si_b[i]) >> 48; +} + +void +test2 () +{ + for (int i = 0; i < N; i++) + uh_c[i] = ((uLL) ui_a[i] * (uLL) ui_b[i]) >> 48; +} + +/* { dg-final { scan-tree-dump-not "vect_recog_mulhs_pattern: detected" "vect" } } */ diff --git a/gcc/testsuite/gcc.target/powerpc/pr101596-2.c b/gcc/testsuite/gcc.target/powerpc/pr101596-2.c new file mode 100644 index 00000000000..5caa7ce4189 --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/pr101596-2.c @@ -0,0 +1,30 @@ +/* { dg-require-effective-target power10_ok } */ +/* { dg-options "-mdejagnu-cpu=power10 -O2 -ftree-vectorize -fno-vect-cost-model -fdump-tree-vect-details" } */ + +/* Check vect_recog_mulhs_pattern can be detected with shift count 32. */ + +#define N 128 + +typedef signed long long sLL; +typedef unsigned long long uLL; + +signed int si_a[N], si_b[N]; +unsigned int ui_a[N], ui_b[N]; +signed short sh_c[N]; +unsigned short uh_c[N]; + +void +test1 () +{ + for (int i = 0; i < N; i++) + sh_c[i] = ((sLL) si_a[i] * (sLL) si_b[i]) >> 32; +} + +void +test2 () +{ + for (int i = 0; i < N; i++) + uh_c[i] = ((uLL) ui_a[i] * (uLL) ui_b[i]) >> 32; +} + +/* { dg-final { scan-tree-dump-times "vect_recog_mulhs_pattern: detected" 2 "vect" } } */ diff --git a/gcc/testsuite/gcc.target/powerpc/pr101596-3.c b/gcc/testsuite/gcc.target/powerpc/pr101596-3.c new file mode 100644 index 00000000000..4b92661412d --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/pr101596-3.c @@ -0,0 +1,58 @@ +/* { dg-do run } */ +/* { dg-require-effective-target power10_hw } */ +/* { dg-options "-mdejagnu-cpu=power10 -O2 -ftree-vectorize -fno-vect-cost-model" } */ + +/* Verify the execution goes well with shift count either 32 or 48. */ + +#define N 128 + +typedef signed int si; +typedef signed short sh; +typedef signed long long sll; +typedef unsigned int ui; +typedef unsigned short uh; +typedef unsigned long long ull; + +si si_a[N], si_b[N]; +ui ui_a[N], ui_b[N]; +sh sh_c[N]; +uh uh_c[N]; + +#define TEST(NTYPE, TYPE, WTYPE, CNT) \ + void __attribute__ ((noipa)) test_##TYPE##CNT () \ + { \ + for (int i = 0; i < N; i++) \ + NTYPE##_c[i] = ((WTYPE) TYPE##_a[i] * (WTYPE) TYPE##_b[i]) >> CNT; \ + } \ + \ + void __attribute__ ((noipa, optimize ("O1"))) check_##TYPE##CNT () \ + { \ + test_##TYPE##CNT (); \ + for (int i = 0; i < N; i++) \ + { \ + NTYPE exp = ((WTYPE) TYPE##_a[i] * (WTYPE) TYPE##_b[i]) >> CNT; \ + if (NTYPE##_c[i] != exp) \ + __builtin_abort (); \ + } \ + } + +TEST (sh, si, sll, 32) +TEST (sh, si, sll, 48) +TEST (uh, ui, ull, 32) +TEST (uh, ui, ull, 48) + +int +main () +{ + + for (int i = 0; i < N; i++) + { + ui_a[i] = si_a[i] = 0x12345678ULL + 0x1000ULL * (i * 3 - 1); + ui_b[i] = si_b[i] = 0x87654321ULL - 0x500000ULL * (i * 5 + 1); + } + + check_si32 (); + check_si48 (); + check_ui32 (); + check_ui48 (); +} diff --git a/gcc/tree-vect-patterns.c b/gcc/tree-vect-patterns.c index 70bb751865b..743fd3f5414 100644 --- a/gcc/tree-vect-patterns.c +++ b/gcc/tree-vect-patterns.c @@ -1986,7 +1986,7 @@ vect_recog_mulhs_pattern (vec_info *vinfo, stmt_vec_info mulh_stmt_info; tree scale_term; - internal_fn ifn; + bool rounding_p = false; /* Check for the presence of the rounding term. */ if (gimple_assign_rhs_code (rshift_input_stmt) == PLUS_EXPR) @@ -2035,37 +2035,18 @@ vect_recog_mulhs_pattern (vec_info *vinfo, /* Get the scaling term. */ scale_term = gimple_assign_rhs2 (plus_input_stmt); - /* Check that the scaling factor is correct. */ - if (TREE_CODE (scale_term) != INTEGER_CST) - return NULL; - - /* Check pattern 2). */ - if (wi::to_widest (scale_term) + target_precision + 2 - != TYPE_PRECISION (lhs_type)) - return NULL; - - ifn = IFN_MULHRS; + rounding_p = true; } else { mulh_stmt_info = rshift_input_stmt_info; scale_term = gimple_assign_rhs2 (last_stmt); - /* Check that the scaling factor is correct. */ - if (TREE_CODE (scale_term) != INTEGER_CST) - return NULL; - - /* Check for pattern 1). */ - if (wi::to_widest (scale_term) + target_precision + 1 - == TYPE_PRECISION (lhs_type)) - ifn = IFN_MULHS; - /* Check for pattern 3). */ - else if (wi::to_widest (scale_term) + target_precision - == TYPE_PRECISION (lhs_type)) - ifn = IFN_MULH; - else - return NULL; } + /* Check that the scaling factor is constant. */ + if (TREE_CODE (scale_term) != INTEGER_CST) + return NULL; + /* Check whether the scaling input term can be seen as two widened inputs multiplied together. */ vect_unpromoted_value unprom_mult[2]; @@ -2076,13 +2057,41 @@ vect_recog_mulhs_pattern (vec_info *vinfo, if (nops != 2) return NULL; - vect_pattern_detected ("vect_recog_mulhs_pattern", last_stmt); - /* Adjust output precision. */ if (TYPE_PRECISION (new_type) < target_precision) new_type = build_nonstandard_integer_type (target_precision, TYPE_UNSIGNED (new_type)); + unsigned mult_precision = TYPE_PRECISION (new_type); + internal_fn ifn; + /* Check that the scaling factor is expected. Instead of + target_precision, we should use the one that we actually + use for internal function. */ + if (rounding_p) + { + /* Check pattern 2). */ + if (wi::to_widest (scale_term) + mult_precision + 2 + != TYPE_PRECISION (lhs_type)) + return NULL; + + ifn = IFN_MULHRS; + } + else + { + /* Check for pattern 1). */ + if (wi::to_widest (scale_term) + mult_precision + 1 + == TYPE_PRECISION (lhs_type)) + ifn = IFN_MULHS; + /* Check for pattern 3). */ + else if (wi::to_widest (scale_term) + mult_precision + == TYPE_PRECISION (lhs_type)) + ifn = IFN_MULH; + else + return NULL; + } + + vect_pattern_detected ("vect_recog_mulhs_pattern", last_stmt); + /* Check for target support. */ tree new_vectype = get_vectype_for_scalar_type (vinfo, new_type); if (!new_vectype