From patchwork Thu May 19 19:38:11 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Enkovich X-Patchwork-Id: 624214 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3r9hGX4jwVz9ssM for ; Fri, 20 May 2016 05:39:32 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b=Oist6O4z; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:mime-version:content-type; q=dns; s= default; b=spUvZNFZWBKqSRYFMTMTjyNSSU9lFElsFi1RiVs+Gq+Gk5+nGRZF6 IyZEyS1lMHdD3l/4AAH2LpHiokRklbF7Or5tP17j4v1xchvTL7KYfToEIpaAoFCq jVAgDXTEPhBl9KusQB9lF73K0d5DF8WjqlJJMvBPrpB9FzEvq1jrEk= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:mime-version:content-type; s= default; bh=J7S6rtjqNQoQRJLmp4wY6zIHsu8=; b=Oist6O4zDwPrp/CH/+To 1wj/wt5XWf767fdHWQadSUL3Tgs2zxdQKmCmdPD5wh3hOaVpflg3xRWrYPjpKv1N EcY39oPhhskVcDEHE6Pes0TqW3ls3dEm7SSqtLeuaYHWTkZZVHzROwcF6G2KCbMr eCQVc93kil1E63RjJCAPjSM= Received: (qmail 122362 invoked by alias); 19 May 2016 19:39:23 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 122352 invoked by uid 89); 19 May 2016 19:39:22 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.6 required=5.0 tests=BAYES_00, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, SPF_PASS autolearn=ham version=3.3.2 spammy=ilya.enkovich@intel.com, ilyaenkovichintelcom, 336, 6, 3366 X-HELO: mail-wm0-f67.google.com Received: from mail-wm0-f67.google.com (HELO mail-wm0-f67.google.com) (74.125.82.67) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Thu, 19 May 2016 19:39:12 +0000 Received: by mail-wm0-f67.google.com with SMTP id n129so23773720wmn.1 for ; Thu, 19 May 2016 12:39:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=1dnEO8zMAnGv6fB/ISFWKrxzJKMPYjfgHDXCfAPYfLE=; b=kdQiD94yrLjqw20dV2OKzKVpJwy1VWl2nbO8+W0JG0mUaUs6Wr5lui7kammtaNnikJ Jmv/XKyvvDFIh4HchO09D7t0TIF/clR6JenYaSJY5RFuJUYxssCJRw6B/2/g4xUGbnCM hGpPjrBfLlarzkoJB28AmnhajBM9GVXD37NBbaRZkqqJ/d2V0j7gfMqr19sQkoxEs7d1 BC9gC0X04zum+eusPhSPd4PyfDJvZXKue3cS1mSh8mTlX6umy2iHEkBGEQXQheDszq+N uf7w7G+QlHoIi+yZoMHSCMldkMeo6kJQbgayVT6P4ykxFuDGMriOCH0IpakGn7J3dd16 oNwQ== X-Gm-Message-State: AOPr4FWvzZAkfgq+G1AQDJB11ZEf3cdSGpNp0XoXsRikVXfk5cihjMQr2kkN+WEqWMnzCg== X-Received: by 10.28.230.137 with SMTP id e9mr37927116wmi.90.1463686749639; Thu, 19 May 2016 12:39:09 -0700 (PDT) Received: from msticlxl57.ims.intel.com (irdmzpr01-ext.ir.intel.com. [192.198.151.36]) by smtp.gmail.com with ESMTPSA id jp2sm16010190wjc.16.2016.05.19.12.39.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 May 2016 12:39:09 -0700 (PDT) Date: Thu, 19 May 2016 22:38:11 +0300 From: Ilya Enkovich To: gcc-patches@gcc.gnu.org Subject: [PATCH, vec-tails 02/10] Extend _loop_vec_info structure with epilogue related fields Message-ID: <20160519193811.GC40563@msticlxl57.ims.intel.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-IsSubscribed: yes Hi, This patch adds new fields to _loop_vec_info structure to support loop epilogue vectorization. Thanks, Ilya --- gcc/ 2016-05-19 Ilya Enkovich * tree-vectorizer.h (struct _loop_vec_info): Add new fields can_be_masked, required_masks, mask_epilogue, combine_epilogue, need_masking, orig_loop_info. (LOOP_VINFO_CAN_BE_MASKED): New. (LOOP_VINFO_REQUIRED_MASKS): New. (LOOP_VINFO_COMBINE_EPILOGUE): New. (LOOP_VINFO_MASK_EPILOGUE): New. (LOOP_VINFO_NEED_MASKING): New. (LOOP_VINFO_ORIG_LOOP_INFO): New. (LOOP_VINFO_EPILOGUE_P): New. (LOOP_VINFO_ORIG_MASK_EPILOGUE): New. (LOOP_VINFO_ORIG_VECT_FACTOR): New. * tree-vect-loop.c (new_loop_vec_info): Initialize new _loop_vec_info fields. diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c index da98211..aac0df9 100644 --- a/gcc/tree-vect-loop.c +++ b/gcc/tree-vect-loop.c @@ -1125,6 +1125,12 @@ new_loop_vec_info (struct loop *loop) LOOP_VINFO_PEELING_FOR_GAPS (res) = false; LOOP_VINFO_PEELING_FOR_NITER (res) = false; LOOP_VINFO_OPERANDS_SWAPPED (res) = false; + LOOP_VINFO_CAN_BE_MASKED (res) = false; + LOOP_VINFO_REQUIRED_MASKS (res) = 0; + LOOP_VINFO_COMBINE_EPILOGUE (res) = false; + LOOP_VINFO_MASK_EPILOGUE (res) = false; + LOOP_VINFO_NEED_MASKING (res) = false; + LOOP_VINFO_ORIG_LOOP_INFO (res) = NULL; return res; } diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index bd1d55a..4c19317 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -336,6 +336,23 @@ typedef struct _loop_vec_info : public vec_info { /* Mark loops having masked stores. */ bool has_mask_store; + /* True if vectorized loop can be masked. */ + bool can_be_masked; + /* If vector mask with 2^N elements is required to mask the loop + then N-th bit of this field is set to 1. */ + unsigned required_masks; + + /* True if we should vectorize loop epilogue with masking. */ + bool mask_epilogue; + /* True if we should combine main loop with epilogue using masking. */ + bool combine_epilogue; + /* True if loop vectorization requires masking. E.g. we want to + vectorize loop with low trip count. */ + bool need_masking; + /* For loops being epilogues of already vectorized loops + this points to the original vectorized loop. Otherwise NULL. */ + _loop_vec_info *orig_loop_info; + } *loop_vec_info; /* Access Functions. */ @@ -374,6 +391,12 @@ typedef struct _loop_vec_info : public vec_info { #define LOOP_VINFO_HAS_MASK_STORE(L) (L)->has_mask_store #define LOOP_VINFO_SCALAR_ITERATION_COST(L) (L)->scalar_cost_vec #define LOOP_VINFO_SINGLE_SCALAR_ITERATION_COST(L) (L)->single_scalar_iteration_cost +#define LOOP_VINFO_CAN_BE_MASKED(L) (L)->can_be_masked +#define LOOP_VINFO_REQUIRED_MASKS(L) (L)->required_masks +#define LOOP_VINFO_COMBINE_EPILOGUE(L) (L)->combine_epilogue +#define LOOP_VINFO_MASK_EPILOGUE(L) (L)->mask_epilogue +#define LOOP_VINFO_NEED_MASKING(L) (L)->need_masking +#define LOOP_VINFO_ORIG_LOOP_INFO(L) (L)->orig_loop_info #define LOOP_REQUIRES_VERSIONING_FOR_ALIGNMENT(L) \ ((L)->may_misalign_stmts.length () > 0) @@ -383,6 +406,14 @@ typedef struct _loop_vec_info : public vec_info { #define LOOP_VINFO_NITERS_KNOWN_P(L) \ (tree_fits_shwi_p ((L)->num_iters) && tree_to_shwi ((L)->num_iters) > 0) +#define LOOP_VINFO_EPILOGUE_P(L) \ + (LOOP_VINFO_ORIG_LOOP_INFO(L) != NULL) + +#define LOOP_VINFO_ORIG_MASK_EPILOGUE(L) \ + (LOOP_VINFO_MASK_EPILOGUE (LOOP_VINFO_ORIG_LOOP_INFO(L))) +#define LOOP_VINFO_ORIG_VECT_FACTOR(L) \ + (LOOP_VINFO_VECT_FACTOR (LOOP_VINFO_ORIG_LOOP_INFO(L))) + static inline loop_vec_info loop_vec_info_for_loop (struct loop *loop) {