From patchwork Wed Nov 12 17:52:45 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Lawrence X-Patchwork-Id: 410066 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 989EA1400E7 for ; Thu, 13 Nov 2014 04:52:57 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :message-id:date:from:mime-version:to:subject:in-reply-to :content-type; q=dns; s=default; b=bwi6r5jsGnqezr+7wPWHi65/Mt1m9 su0vxZl/czORqp+kvFy9hShSz8kOG/XFv6Ao+kpl24JuZGMek/F/5NiiuPMYD5+9 QRVY5x2U3PngIkqe1QOc1Z3EmXOoP+oDHR9TpoE2mBshMFpwINY1i3UY+SujrFDT 7djVw6pGjXfxaA= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :message-id:date:from:mime-version:to:subject:in-reply-to :content-type; s=default; bh=YUm4I9C+HPfSaJJAM3VAfkz/5vU=; b=qE3 0ihoTTfppo8Y2gBDTNOT7rOmGP5BLUIeBBD3KSFNoHUs1QTG03OjHIMaqUJvsycv A/kU3m5q8tbn9RLIN3vDPLpriXkBySr8xsAi91d2e9pXjUo+bWWN2sbiOKHn3Ycd y4Q5a5Ojg0LizvX7ZGJVAtLm36zgo3dFf7gy5tX8= Received: (qmail 1551 invoked by alias); 12 Nov 2014 17:52:50 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 1541 invoked by uid 89); 12 Nov 2014 17:52:50 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.8 required=5.0 tests=AWL, BAYES_00, SPF_PASS autolearn=ham version=3.3.2 X-HELO: service87.mimecast.com Received: from service87.mimecast.com (HELO service87.mimecast.com) (91.220.42.44) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 12 Nov 2014 17:52:48 +0000 Received: from cam-owa1.Emea.Arm.com (fw-tnat.cambridge.arm.com [217.140.96.21]) by service87.mimecast.com; Wed, 12 Nov 2014 17:52:46 +0000 Received: from [10.1.209.51] ([10.1.255.212]) by cam-owa1.Emea.Arm.com with Microsoft SMTPSVC(6.0.3790.3959); Wed, 12 Nov 2014 17:52:46 +0000 Message-ID: <54639E6D.3000402@arm.com> Date: Wed, 12 Nov 2014 17:52:45 +0000 From: Alan Lawrence User-Agent: Thunderbird 2.0.0.24 (X11/20101213) MIME-Version: 1.0 To: "gcc-patches@gcc.gnu.org" Subject: [PATCH 1/4][Vectorizer] Split vect_gen_perm_mask into _checked and _any variants In-Reply-To: <54639D83.7090205@arm.com> X-MC-Unique: 114111217524600801 X-IsSubscribed: yes This is a preliminary to patch 2, which wants functionality equivalent to vect_gen_perm_mask (converting a char* to an RTL const_vector) but without the check of can_vec_perm_p. All existing calls to vect_gen_perm_mask barring that in perm_mask_for_reverse, assert the return value is non-null. Hence, this patch splits the existing vect_gen_perm_mask into two: a checked variant which makes the equivalent assertion; and an unchecked variant, which just turns any char* into an equivalent const_vector. (An) alternative strategy would be merely to remove the check from vect_gen_perm_mask (so equivalent to this patch's vect_gen_perm_mask_any), and add a preceding assert at all callsites (i.e. changing the many "gen_vect_perm_mask (...); assert (... != NULL);" into "assert (can_vec_perm_p (...)); gen_vect_perm_mask (...);" - that would work just as well as far as this patch series is concerned. On its own, bootstrapped on x86-none-linux-gnu (more testing with patches 2+3). gcc/ChangeLog: * tree-vectorizer.h (vect_gen_perm_mask): Remove. (vect_gen_perm_mask_checked, vect_gen_perm_mask_any): New. tree_vec_data_refs.c (vect_permute_load_chain, vec_permute_store_chain, vec_shift_permute_load_chain): Replace vect_gen_perm_mask & assert with vect_gen_perm_mask_checked. * tree-vect-stmts.c (vectorizable_mask_load_store, vectorizable_load): Likewise. (vect_gen_perm_mask_checked): New. (vect_gen_perm_mask): Remove can_vec_perm_p check, rename to... (vect_gen_perm_mask_any): ...this. (perm_mask_for_reverse): Call can_vec_perm_p and vect_gen_perm_mask_checked. diff --git a/gcc/tree-vect-data-refs.c b/gcc/tree-vect-data-refs.c index 1d30bdf78cbd66874906c5f7444ce30c4c391feb..4c5b1394517bfcf5c4226143e53cbbe79e0c68b1 100644 --- a/gcc/tree-vect-data-refs.c +++ b/gcc/tree-vect-data-refs.c @@ -4623,8 +4623,7 @@ vect_permute_store_chain (vec dr_chain, if (3 * i + nelt2 < nelt) sel[3 * i + nelt2] = 0; } - perm3_mask_low = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm3_mask_low != NULL); + perm3_mask_low = vect_gen_perm_mask_checked (vectype, sel); for (i = 0; i < nelt; i++) { @@ -4635,8 +4634,7 @@ vect_permute_store_chain (vec dr_chain, if (3 * i + nelt2 < nelt) sel[3 * i + nelt2] = nelt + j2++; } - perm3_mask_high = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm3_mask_high != NULL); + perm3_mask_high = vect_gen_perm_mask_checked (vectype, sel); vect1 = dr_chain[0]; vect2 = dr_chain[1]; @@ -4675,13 +4673,11 @@ vect_permute_store_chain (vec dr_chain, sel[i * 2] = i; sel[i * 2 + 1] = i + nelt; } - perm_mask_high = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm_mask_high != NULL); + perm_mask_high = vect_gen_perm_mask_checked (vectype, sel); for (i = 0; i < nelt; i++) sel[i] += nelt / 2; - perm_mask_low = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm_mask_low != NULL); + perm_mask_low = vect_gen_perm_mask_checked (vectype, sel); for (i = 0, n = log_length; i < n; i++) { @@ -5184,8 +5180,7 @@ vect_permute_load_chain (vec dr_chain, sel[i] = 3 * i + k; else sel[i] = 0; - perm3_mask_low = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm3_mask_low != NULL); + perm3_mask_low = vect_gen_perm_mask_checked (vectype, sel); for (i = 0, j = 0; i < nelt; i++) if (3 * i + k < 2 * nelt) @@ -5193,8 +5188,7 @@ vect_permute_load_chain (vec dr_chain, else sel[i] = nelt + ((nelt + k) % 3) + 3 * (j++); - perm3_mask_high = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm3_mask_high != NULL); + perm3_mask_high = vect_gen_perm_mask_checked (vectype, sel); first_vect = dr_chain[0]; second_vect = dr_chain[1]; @@ -5228,13 +5222,11 @@ vect_permute_load_chain (vec dr_chain, for (i = 0; i < nelt; ++i) sel[i] = i * 2; - perm_mask_even = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm_mask_even != NULL); + perm_mask_even = vect_gen_perm_mask_checked (vectype, sel); for (i = 0; i < nelt; ++i) sel[i] = i * 2 + 1; - perm_mask_odd = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm_mask_odd != NULL); + perm_mask_odd = vect_gen_perm_mask_checked (vectype, sel); for (i = 0; i < log_length; i++) { @@ -5389,8 +5381,7 @@ vect_shift_permute_load_chain (vec dr_chain, supported by target\n"); return false; } - perm2_mask1 = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm2_mask1 != NULL); + perm2_mask1 = vect_gen_perm_mask_checked (vectype, sel); for (i = 0; i < nelt / 2; ++i) sel[i] = i * 2 + 1; @@ -5404,8 +5395,7 @@ vect_shift_permute_load_chain (vec dr_chain, supported by target\n"); return false; } - perm2_mask2 = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm2_mask2 != NULL); + perm2_mask2 = vect_gen_perm_mask_checked (vectype, sel); /* Generating permutation constant to shift all elements. For vector length 8 it is {4 5 6 7 8 9 10 11}. */ @@ -5418,8 +5408,7 @@ vect_shift_permute_load_chain (vec dr_chain, "shift permutation is not supported by target\n"); return false; } - shift1_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (shift1_mask != NULL); + shift1_mask = vect_gen_perm_mask_checked (vectype, sel); /* Generating permutation constant to select vector from 2. For vector length 8 it is {0 1 2 3 12 13 14 15}. */ @@ -5434,8 +5423,7 @@ vect_shift_permute_load_chain (vec dr_chain, "select is not supported by target\n"); return false; } - select_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (select_mask != NULL); + select_mask = vect_gen_perm_mask_checked (vectype, sel); first_vect = dr_chain[0]; second_vect = dr_chain[1]; @@ -5494,8 +5482,7 @@ vect_shift_permute_load_chain (vec dr_chain, supported by target\n"); return false; } - perm3_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm3_mask != NULL); + perm3_mask = vect_gen_perm_mask_checked (vectype, sel); /* Generating permutation constant to shift all elements. For vector length 8 it is {6 7 8 9 10 11 12 13}. */ @@ -5508,8 +5495,7 @@ vect_shift_permute_load_chain (vec dr_chain, "shift permutation is not supported by target\n"); return false; } - shift1_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (shift1_mask != NULL); + shift1_mask = vect_gen_perm_mask_checked (vectype, sel); /* Generating permutation constant to shift all elements. For vector length 8 it is {5 6 7 8 9 10 11 12}. */ @@ -5522,8 +5508,7 @@ vect_shift_permute_load_chain (vec dr_chain, "shift permutation is not supported by target\n"); return false; } - shift2_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (shift2_mask != NULL); + shift2_mask = vect_gen_perm_mask_checked (vectype, sel); /* Generating permutation constant to shift all elements. For vector length 8 it is {3 4 5 6 7 8 9 10}. */ @@ -5536,8 +5521,7 @@ vect_shift_permute_load_chain (vec dr_chain, "shift permutation is not supported by target\n"); return false; } - shift3_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (shift3_mask != NULL); + shift3_mask = vect_gen_perm_mask_checked (vectype, sel); /* Generating permutation constant to shift all elements. For vector length 8 it is {5 6 7 8 9 10 11 12}. */ @@ -5550,8 +5534,7 @@ vect_shift_permute_load_chain (vec dr_chain, "shift permutation is not supported by target\n"); return false; } - shift4_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (shift4_mask != NULL); + shift4_mask = vect_gen_perm_mask_checked (vectype, sel); for (k = 0; k < 3; k++) { diff --git a/gcc/tree-vect-stmts.c b/gcc/tree-vect-stmts.c index 681479ed911e9ff0fadc8d52576086ed3bbaca5a..3c2d7d7926e7aece6c0aca63210a2d6d591ac1fb 100644 --- a/gcc/tree-vect-stmts.c +++ b/gcc/tree-vect-stmts.c @@ -1913,8 +1913,7 @@ vectorizable_mask_load_store (gimple stmt, gimple_stmt_iterator *gsi, for (i = 0; i < gather_off_nunits; ++i) sel[i] = i | nunits; - perm_mask = vect_gen_perm_mask (gather_off_vectype, sel); - gcc_assert (perm_mask != NULL_TREE); + perm_mask = vect_gen_perm_mask_checked (gather_off_vectype, sel); } else if (nunits == gather_off_nunits * 2) { @@ -1925,13 +1924,11 @@ vectorizable_mask_load_store (gimple stmt, gimple_stmt_iterator *gsi, sel[i] = i < gather_off_nunits ? i : i + nunits - gather_off_nunits; - perm_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm_mask != NULL_TREE); + perm_mask = vect_gen_perm_mask_checked (vectype, sel); ncopies *= 2; for (i = 0; i < nunits; ++i) sel[i] = i | gather_off_nunits; - mask_perm_mask = vect_gen_perm_mask (masktype, sel); - gcc_assert (mask_perm_mask != NULL_TREE); + mask_perm_mask = vect_gen_perm_mask_checked (masktype, sel); } else gcc_unreachable (); @@ -4936,7 +4933,9 @@ perm_mask_for_reverse (tree vectype) for (i = 0; i < nunits; ++i) sel[i] = nunits - 1 - i; - return vect_gen_perm_mask (vectype, sel); + if (!can_vec_perm_p (TYPE_MODE (vectype), false, sel)) + return NULL_TREE; + return vect_gen_perm_mask_checked (vectype, sel); } /* Function vectorizable_store. @@ -5467,21 +5466,19 @@ vectorizable_store (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt, return true; } -/* Given a vector type VECTYPE and permutation SEL returns - the VECTOR_CST mask that implements the permutation of the - vector elements. If that is impossible to do, returns NULL. */ +/* Given a vector type VECTYPE, turns permutation SEL into the equivalent + VECTOR_CST mask. No checks are made that the target platform supports the + mask, so callers may wish to test can_vec_perm_p separately, or use + vect_gen_perm_mask_checked. */ tree -vect_gen_perm_mask (tree vectype, unsigned char *sel) +vect_gen_perm_mask_any (tree vectype, const unsigned char *sel) { tree mask_elt_type, mask_type, mask_vec, *mask_elts; int i, nunits; nunits = TYPE_VECTOR_SUBPARTS (vectype); - if (!can_vec_perm_p (TYPE_MODE (vectype), false, sel)) - return NULL; - mask_elt_type = lang_hooks.types.type_for_mode (int_mode_for_mode (TYPE_MODE (TREE_TYPE (vectype))), 1); mask_type = get_vectype_for_scalar_type (mask_elt_type); @@ -5494,6 +5491,15 @@ vect_gen_perm_mask (tree vectype, unsigned char *sel) return mask_vec; } +/* Checked version of vect_gen_perm_mask_any. Asserts can_vec_perm_p. */ + +tree +vect_gen_perm_mask_checked (tree vectype, const unsigned char *sel) +{ + gcc_assert (can_vec_perm_p (TYPE_MODE (vectype), false, sel)); + return vect_gen_perm_mask_any (vectype, sel); +} + /* Given a vector variable X and Y, that was generated for the scalar STMT, generate instructions to permute the vector elements of X and Y using permutation mask MASK_VEC, insert them at *GSI and return the @@ -5849,8 +5855,7 @@ vectorizable_load (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt, for (i = 0; i < gather_off_nunits; ++i) sel[i] = i | nunits; - perm_mask = vect_gen_perm_mask (gather_off_vectype, sel); - gcc_assert (perm_mask != NULL_TREE); + perm_mask = vect_gen_perm_mask_checked (gather_off_vectype, sel); } else if (nunits == gather_off_nunits * 2) { @@ -5861,8 +5866,7 @@ vectorizable_load (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt, sel[i] = i < gather_off_nunits ? i : i + nunits - gather_off_nunits; - perm_mask = vect_gen_perm_mask (vectype, sel); - gcc_assert (perm_mask != NULL_TREE); + perm_mask = vect_gen_perm_mask_checked (vectype, sel); ncopies *= 2; } else diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 93aa73e59c6079dc95288979e15c1d47725391b7..d817f9f53a6ed8d88a305207f6f482ef7a30fea3 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -1040,7 +1040,8 @@ extern void vect_get_store_cost (struct data_reference *, int, extern bool vect_supportable_shift (enum tree_code, tree); extern void vect_get_vec_defs (tree, tree, gimple, vec *, vec *, slp_tree, int); -extern tree vect_gen_perm_mask (tree, unsigned char *); +extern tree vect_gen_perm_mask_any (tree, const unsigned char *); +extern tree vect_gen_perm_mask_checked (tree, const unsigned char *); /* In tree-vect-data-refs.c. */ extern bool vect_can_force_dr_alignment_p (const_tree, unsigned int);