From patchwork Tue Jun 4 10:58:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Andre Vieira (lists)" X-Patchwork-Id: 1943270 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VtnfP0pVRz20Tb for ; Tue, 4 Jun 2024 20:59:23 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id F1C523827371 for ; Tue, 4 Jun 2024 10:59:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id D8DC23875462 for ; Tue, 4 Jun 2024 10:58:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D8DC23875462 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D8DC23875462 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717498733; cv=none; b=fvgnLlJTvPPlEkwNN2c9F/hru267XAl7j9c9sbUWlGe2OQ2DnLxYrKPUnlEeh+VH7wxs0BDVBzNOadwWtMkIwJTAKIKMd7eR6uZQnFnSx0Uv3g9V4VstzsswFfP9uf/LYO/kDMr7OkpX5My8JuzEWHqtfEWWAQuFGo1Pn94gI6k= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1717498733; c=relaxed/simple; bh=lqOdQtSqoa6Ihq3nSjUELdMAqtHLm7x6IY1EpVKHGHo=; h=Message-ID:Date:MIME-Version:To:From:Subject; b=lYdy6bKDgI1ZzG1MpAi7Ss/aOmalYmeud9nhka+zx65bMPQQ1L5ndWj9vMgaB0bbZT4kbqNpfBYL4sbxyUbxpxNKiBA6ngtO2evqYiO9njLRWkVTWgjiuWfa2R2oC5nNTNZ/eFysFvL7MrkA0EQS+u/T7CKxV7BEAOHh6qB8tdQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B5D0D1042; Tue, 4 Jun 2024 03:59:12 -0700 (PDT) Received: from [10.1.31.189] (E121495.arm.com [10.1.31.189]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 954B13F762; Tue, 4 Jun 2024 03:58:47 -0700 (PDT) Message-ID: <24a64801-3f5a-4244-acb7-e8159d160152@arm.com> Date: Tue, 4 Jun 2024 11:58:43 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: "gcc-patches@gcc.gnu.org" Cc: Richard Sandiford , Richard Biener , "jakub@redhat.com" From: "Andre Vieira (lists)" Subject: RFC: Support for pragma clang loop interleave_count(N) X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_LOTSOFHASH, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Hi, We got a question as to whether GCC had something similar to llvm's pragma clang loop interleave_count(N), see https://clang.llvm.org/docs/LanguageExtensions.html#extensions-for-loop-hint-optimizations I did a quick hack, using 'GCC interleaves N', just as a proof of concept, to see whether we could connect this to the suggested_unroll_factor in the vectorizer and to test the waters regarding having something like this upstream. For the real thing I'd suggest we use the same pragma syntax as clang's so its easier to port code. It is my understanding that the main use for this is for doing performance tuning of HPC kernels and performance tuning of CPU's cost models. This seems to work (TM), though with the move to slp-only I guess this will stop working? Though I suspect we will want to have similar capabilities in SLP, or maybe we have already and I didn't look hard enough. Also only implemented it for C and C++, have not looked at Fortran. WDYT? Kind regards, Andre diff --git a/gcc/c-family/c-pragma.h b/gcc/c-family/c-pragma.h index ce93a52fa578127f1eade05dbafdf52021fd61fe..945c314d31c715522e56141ef9b616e52b466261 100644 --- a/gcc/c-family/c-pragma.h +++ b/gcc/c-family/c-pragma.h @@ -87,6 +87,7 @@ enum pragma_kind { PRAGMA_GCC_PCH_PREPROCESS, PRAGMA_IVDEP, PRAGMA_UNROLL, + PRAGMA_INTERLEAVES, PRAGMA_NOVECTOR, PRAGMA_FIRST_EXTERNAL diff --git a/gcc/c-family/c-pragma.cc b/gcc/c-family/c-pragma.cc index 1237ee6e62b9d501a7f9ad1cc267061ed068b920..facfb75c1eb9f184f05f4476a3aa04b45c21fd9a 100644 --- a/gcc/c-family/c-pragma.cc +++ b/gcc/c-family/c-pragma.cc @@ -1828,6 +1828,10 @@ init_pragma (void) cpp_register_deferred_pragma (parse_in, "GCC", "unroll", PRAGMA_UNROLL, false, false); + if (!flag_preprocess_only) + cpp_register_deferred_pragma (parse_in, "GCC", "interleaves", PRAGMA_INTERLEAVES, + false, false); + if (!flag_preprocess_only) cpp_register_deferred_pragma (parse_in, "GCC", "novector", PRAGMA_NOVECTOR, false, false); diff --git a/gcc/c/c-parser.cc b/gcc/c/c-parser.cc index 00f8bf4376e537e04ea8e468a05dade3c7212d8b..69b36d196bd74b494ffb6186be5240e3d2431407 100644 --- a/gcc/c/c-parser.cc +++ b/gcc/c/c-parser.cc @@ -1665,11 +1665,12 @@ static tree c_parser_c99_block_statement (c_parser *, bool *, location_t * = NULL); static void c_parser_if_statement (c_parser *, bool *, vec *); static void c_parser_switch_statement (c_parser *, bool *); -static void c_parser_while_statement (c_parser *, bool, unsigned short, bool, - bool *); -static void c_parser_do_statement (c_parser *, bool, unsigned short, bool); -static void c_parser_for_statement (c_parser *, bool, unsigned short, bool, - bool *); +static void c_parser_while_statement (c_parser *, bool, unsigned short, + unsigned short, bool, bool *); +static void c_parser_do_statement (c_parser *, bool, unsigned short, + unsigned short,bool); +static void c_parser_for_statement (c_parser *, bool, unsigned short, + unsigned short, bool, bool *); static tree c_parser_asm_statement (c_parser *); static tree c_parser_asm_operands (c_parser *); static tree c_parser_asm_goto_operands (c_parser *); @@ -7603,13 +7604,13 @@ c_parser_statement_after_labels (c_parser *parser, bool *if_p, c_parser_switch_statement (parser, if_p); break; case RID_WHILE: - c_parser_while_statement (parser, false, 0, false, if_p); + c_parser_while_statement (parser, false, 0, 0, false, if_p); break; case RID_DO: - c_parser_do_statement (parser, false, 0, false); + c_parser_do_statement (parser, false, 0, 0, false); break; case RID_FOR: - c_parser_for_statement (parser, false, 0, false, if_p); + c_parser_for_statement (parser, false, 0, 0, false, if_p); break; case RID_GOTO: c_parser_consume_token (parser); @@ -8105,7 +8106,7 @@ c_parser_switch_statement (c_parser *parser, bool *if_p) static void c_parser_while_statement (c_parser *parser, bool ivdep, unsigned short unroll, - bool novector, bool *if_p) + unsigned short interleaves, bool novector, bool *if_p) { tree block, cond, body; unsigned char save_in_statement; @@ -8135,6 +8136,11 @@ c_parser_while_statement (c_parser *parser, bool ivdep, unsigned short unroll, build_int_cst (integer_type_node, annot_expr_unroll_kind), build_int_cst (integer_type_node, unroll)); + if (interleaves && cond != error_mark_node) + cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, + build_int_cst (integer_type_node, + annot_expr_interleaves_kind), + build_int_cst (integer_type_node, interleaves)); if (novector && cond != error_mark_node) cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, build_int_cst (integer_type_node, @@ -8172,7 +8178,7 @@ c_parser_while_statement (c_parser *parser, bool ivdep, unsigned short unroll, static void c_parser_do_statement (c_parser *parser, bool ivdep, unsigned short unroll, - bool novector) + unsigned short interleaves, bool novector) { tree block, cond, body; unsigned char save_in_statement; @@ -8209,6 +8215,11 @@ c_parser_do_statement (c_parser *parser, bool ivdep, unsigned short unroll, build_int_cst (integer_type_node, annot_expr_unroll_kind), build_int_cst (integer_type_node, unroll)); + if (interleaves && cond != error_mark_node) + cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, + build_int_cst (integer_type_node, + annot_expr_interleaves_kind), + build_int_cst (integer_type_node, interleaves)); if (novector && cond != error_mark_node) cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, build_int_cst (integer_type_node, @@ -8282,7 +8293,7 @@ c_parser_do_statement (c_parser *parser, bool ivdep, unsigned short unroll, static void c_parser_for_statement (c_parser *parser, bool ivdep, unsigned short unroll, - bool novector, bool *if_p) + unsigned short interleaves, bool novector, bool *if_p) { tree block, cond, incr, body; unsigned char save_in_statement; @@ -8424,6 +8435,12 @@ c_parser_for_statement (c_parser *parser, bool ivdep, unsigned short unroll, "with % pragma"); cond = error_mark_node; } + else if (interleaves) + { + c_parser_error (parser, "missing loop condition in loop " + "with % pragma"); + cond = error_mark_node; + } else { c_parser_consume_token (parser); @@ -8446,6 +8463,11 @@ c_parser_for_statement (c_parser *parser, bool ivdep, unsigned short unroll, build_int_cst (integer_type_node, annot_expr_unroll_kind), build_int_cst (integer_type_node, unroll)); + if (interleaves && cond != error_mark_node) + cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, + build_int_cst (integer_type_node, + annot_expr_interleaves_kind), + build_int_cst (integer_type_node, interleaves)); if (novector && cond && cond != error_mark_node) cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, build_int_cst (integer_type_node, @@ -14466,6 +14488,37 @@ c_parser_pragma_unroll (c_parser *parser) return unroll; } +static unsigned short +c_parser_pragma_interleaves (c_parser *parser) +{ + unsigned short interleaves; + c_parser_consume_pragma (parser); + location_t location = c_parser_peek_token (parser)->location; + tree expr = c_parser_expr_no_commas (parser, NULL).value; + mark_exp_read (expr); + expr = c_fully_fold (expr, false, NULL); + HOST_WIDE_INT linterleaves = 0; + if (!INTEGRAL_TYPE_P (TREE_TYPE (expr)) + || TREE_CODE (expr) != INTEGER_CST + || (linterleaves = tree_to_shwi (expr)) < 0 + || linterleaves >= USHRT_MAX) + { + error_at (location, "%<#pragma GCC interleaves%> requires an" + " assignment-expression that evaluates to a non-negative" + " integral constant less than %u", USHRT_MAX); + interleaves = 0; + } + else + { + interleaves = (unsigned short)linterleaves; + if (interleaves == 0) + interleaves = 1; + } + + c_parser_skip_to_pragma_eol (parser); + return interleaves; +} + /* Handle pragmas. Some OpenMP pragmas are associated with, and therefore should be considered, statements. ALLOW_STMT is true if we're within the context of a function and such pragmas are to be allowed. Returns @@ -14674,10 +14727,12 @@ c_parser_pragma (c_parser *parser, enum pragma_context context, bool *if_p) case PRAGMA_NOVECTOR: case PRAGMA_UNROLL: + case PRAGMA_INTERLEAVES: case PRAGMA_IVDEP: { bool novector = false; unsigned short unroll = 0; + unsigned short interleaves = 0; bool ivdep = false; switch (id) @@ -14688,6 +14743,9 @@ c_parser_pragma (c_parser *parser, enum pragma_context context, bool *if_p) case PRAGMA_UNROLL: unroll = c_parser_pragma_unroll (parser); break; + case PRAGMA_INTERLEAVES: + interleaves = c_parser_pragma_interleaves (parser); + break; case PRAGMA_IVDEP: ivdep = c_parse_pragma_ivdep (parser); break; @@ -14707,6 +14765,9 @@ c_parser_pragma (c_parser *parser, enum pragma_context context, bool *if_p) case PRAGMA_UNROLL: unroll = c_parser_pragma_unroll (parser); break; + case PRAGMA_INTERLEAVES: + interleaves = c_parser_pragma_interleaves (parser); + break; case PRAGMA_NOVECTOR: novector = c_parse_pragma_novector (parser); break; @@ -14725,11 +14786,13 @@ c_parser_pragma (c_parser *parser, enum pragma_context context, bool *if_p) return false; } if (c_parser_next_token_is_keyword (parser, RID_FOR)) - c_parser_for_statement (parser, ivdep, unroll, novector, if_p); + c_parser_for_statement (parser, ivdep, unroll, interleaves, novector, + if_p); else if (c_parser_next_token_is_keyword (parser, RID_WHILE)) - c_parser_while_statement (parser, ivdep, unroll, novector, if_p); + c_parser_while_statement (parser, ivdep, unroll, interleaves, + novector, if_p); else - c_parser_do_statement (parser, ivdep, unroll, novector); + c_parser_do_statement (parser, ivdep, unroll, interleaves, novector); } return true; diff --git a/gcc/cfgloop.h b/gcc/cfgloop.h index 30b5e40d0d999ef285d44fe4e2303969229e3498..50cb39f684979eadd1a62cffc766d2771c4f68e9 100644 --- a/gcc/cfgloop.h +++ b/gcc/cfgloop.h @@ -239,6 +239,8 @@ public: Other values means unroll with the given unrolling factor. */ unsigned short unroll; + unsigned short interleaves; + /* If this loop was inlined the main clique of the callee which does not need remapping when copying the loop body. */ unsigned short owned_clique; diff --git a/gcc/cfgloopmanip.cc b/gcc/cfgloopmanip.cc index 3707db2fdb39cea26ee3ec5372eccc5d28be660c..d9450dfe53c509382d378f2cb4f54f4f89274a14 100644 --- a/gcc/cfgloopmanip.cc +++ b/gcc/cfgloopmanip.cc @@ -1095,6 +1095,7 @@ copy_loop_info (class loop *loop, class loop *target) target->in_oacc_kernels_region = loop->in_oacc_kernels_region; target->finite_p = loop->finite_p; target->unroll = loop->unroll; + target->interleaves = loop->interleaves; target->owned_clique = loop->owned_clique; } diff --git a/gcc/cp/cp-tree.h b/gcc/cp/cp-tree.h index 6206482c602dddfdb92187f90324d24d6700078b..1ef605655a164f98867a57532e12b967e62e57c6 100644 --- a/gcc/cp/cp-tree.h +++ b/gcc/cp/cp-tree.h @@ -7779,16 +7779,16 @@ extern void begin_else_clause (tree); extern void finish_else_clause (tree); extern void finish_if_stmt (tree); extern tree begin_while_stmt (void); -extern void finish_while_stmt_cond (tree, tree, bool, tree, bool); +extern void finish_while_stmt_cond (tree, tree, bool, tree, tree, bool); extern void finish_while_stmt (tree); extern tree begin_do_stmt (void); extern void finish_do_body (tree); -extern void finish_do_stmt (tree, tree, bool, tree, bool); +extern void finish_do_stmt (tree, tree, bool, tree, tree, bool); extern tree finish_return_stmt (tree); extern tree begin_for_scope (tree *); extern tree begin_for_stmt (tree, tree); extern void finish_init_stmt (tree); -extern void finish_for_cond (tree, tree, bool, tree, bool); +extern void finish_for_cond (tree, tree, bool, tree, tree, bool); extern void finish_for_expr (tree, tree); extern void finish_for_stmt (tree); extern tree begin_range_for_stmt (tree, tree); @@ -7990,6 +7990,7 @@ extern tree finish_omp_target (location_t, tree, tree, bool); extern void finish_omp_target_clauses (location_t, tree, tree *); extern void maybe_warn_unparenthesized_assignment (tree, bool, tsubst_flags_t); extern tree cp_check_pragma_unroll (location_t, tree); +extern tree cp_check_pragma_interleaves (location_t, tree); /* in tree.cc */ extern int cp_tree_operand_length (const_tree); diff --git a/gcc/cp/init.cc b/gcc/cp/init.cc index 4a7ed7f53027516174bfc948360591fa3703ee18..6960b40e14d189ce4593d29d3020994250526f3e 100644 --- a/gcc/cp/init.cc +++ b/gcc/cp/init.cc @@ -4886,7 +4886,7 @@ build_vec_init (tree base, tree maxindex, tree init, finish_init_stmt (for_stmt); finish_for_cond (build2 (GT_EXPR, boolean_type_node, iterator, build_int_cst (TREE_TYPE (iterator), -1)), - for_stmt, false, 0, false); + for_stmt, false, 0, 0, false); /* We used to pass this decrement to finish_for_expr; now we add it to elt_init below so it's part of the same full-expression as the initialization, and thus happens before any potentially throwing diff --git a/gcc/cp/method.cc b/gcc/cp/method.cc index 0b21656ed612ebca5fd32f140783645c7da69824..ecf609585896df3cb5de2ec07b6526d6f1fa8fb5 100644 --- a/gcc/cp/method.cc +++ b/gcc/cp/method.cc @@ -1656,7 +1656,7 @@ build_comparison_op (tree fndecl, bool defining, tsubst_flags_t complain) add_stmt (idx); finish_init_stmt (for_stmt); finish_for_cond (build2 (LE_EXPR, boolean_type_node, idx, - maxval), for_stmt, false, 0, + maxval), for_stmt, false, 0, 0, false); finish_for_expr (cp_build_unary_op (PREINCREMENT_EXPR, TARGET_EXPR_SLOT (idx), diff --git a/gcc/cp/parser.cc b/gcc/cp/parser.cc index 779625144db405561e62da8c7553c16234324c79..67828431e74cac887fd8fe439d2b44233ba6d21e 100644 --- a/gcc/cp/parser.cc +++ b/gcc/cp/parser.cc @@ -2451,13 +2451,13 @@ static tree cp_parser_selection_statement static tree cp_parser_condition (cp_parser *); static tree cp_parser_iteration_statement - (cp_parser *, bool *, bool, tree, bool); + (cp_parser *, bool *, bool, tree, tree, bool); static bool cp_parser_init_statement (cp_parser *, tree *decl); static tree cp_parser_for - (cp_parser *, bool, tree, bool); + (cp_parser *, bool, tree, tree, tree, bool); static tree cp_parser_c_for - (cp_parser *, tree, tree, bool, tree, bool); + (cp_parser *, tree, tree, bool, tree, tree, bool); static tree cp_parser_range_for (cp_parser *, tree, tree, tree, bool, tree, bool, bool); static void do_range_for_auto_deduction @@ -12744,7 +12744,8 @@ cp_parser_statement (cp_parser* parser, tree in_statement_expr, case RID_FOR: std_attrs = process_stmt_hotness_attribute (std_attrs, attrs_loc); statement = cp_parser_iteration_statement (parser, if_p, false, - NULL_TREE, false); + NULL_TREE, NULL_TREE, + false); break; case RID_BREAK: @@ -14034,7 +14035,8 @@ cp_parser_condition (cp_parser* parser) not included. */ static tree -cp_parser_for (cp_parser *parser, bool ivdep, tree unroll, bool novector) +cp_parser_for (cp_parser *parser, bool ivdep, tree interleaves, tree unroll, + bool novector) { tree init, scope, decl; bool is_range_for; @@ -14062,16 +14064,19 @@ cp_parser_for (cp_parser *parser, bool ivdep, tree unroll, bool novector) /* Parse the initialization. */ is_range_for = cp_parser_init_statement (parser, &decl); + /* TODO: should probably warn when interleaves is being used for a range_for? + or should we support that too? */ if (is_range_for) return cp_parser_range_for (parser, scope, init, decl, ivdep, unroll, novector, false); else - return cp_parser_c_for (parser, scope, init, ivdep, unroll, novector); + return cp_parser_c_for (parser, scope, init, ivdep, interleaves, unroll, + novector); } static tree cp_parser_c_for (cp_parser *parser, tree scope, tree init, bool ivdep, - tree unroll, bool novector) + tree interleaves, tree unroll, bool novector) { /* Normal for loop */ tree condition = NULL_TREE; @@ -14098,7 +14103,13 @@ cp_parser_c_for (cp_parser *parser, tree scope, tree init, bool ivdep, "% pragma"); condition = error_mark_node; } - finish_for_cond (condition, stmt, ivdep, unroll, novector); + else if (interleaves) + { + cp_parser_error (parser, "missing loop condition in loop with " + "% pragma"); + condition = error_mark_node; + } + finish_for_cond (condition, stmt, ivdep, interleaves, unroll, novector); /* Look for the `;'. */ cp_parser_require (parser, CPP_SEMICOLON, RT_SEMICOLON); @@ -14444,7 +14455,7 @@ cp_convert_range_for (tree statement, tree range_decl, tree range_expr, begin, ERROR_MARK, end, ERROR_MARK, NULL_TREE, NULL, tf_warning_or_error); - finish_for_cond (condition, statement, ivdep, unroll, novector); + finish_for_cond (condition, statement, ivdep, NULL_TREE, unroll, novector); /* The new increment expression. */ expression = finish_unary_op_expr (input_location, @@ -14608,7 +14619,7 @@ cp_parser_range_for_member_function (tree range, tree identifier) static tree cp_parser_iteration_statement (cp_parser* parser, bool *if_p, bool ivdep, - tree unroll, bool novector) + tree interleaves, tree unroll, bool novector) { cp_token *token; enum rid keyword; @@ -14651,7 +14662,8 @@ cp_parser_iteration_statement (cp_parser* parser, bool *if_p, bool ivdep, parens.require_open (parser); /* Parse the condition. */ condition = cp_parser_condition (parser); - finish_while_stmt_cond (condition, statement, ivdep, unroll, novector); + finish_while_stmt_cond (condition, statement, ivdep, interleaves, + unroll, novector); /* Look for the `)'. */ parens.require_close (parser); /* Parse the dependent statement. */ @@ -14686,7 +14698,8 @@ cp_parser_iteration_statement (cp_parser* parser, bool *if_p, bool ivdep, /* Parse the expression. */ expression = cp_parser_expression (parser); /* We're done with the do-statement. */ - finish_do_stmt (expression, statement, ivdep, unroll, novector); + finish_do_stmt (expression, statement, ivdep, interleaves, unroll, + novector); /* Look for the `)'. */ parens.require_close (parser); /* Look for the `;'. */ @@ -14700,7 +14713,8 @@ cp_parser_iteration_statement (cp_parser* parser, bool *if_p, bool ivdep, matching_parens parens; parens.require_open (parser); - statement = cp_parser_for (parser, ivdep, unroll, novector); + statement = cp_parser_for (parser, ivdep, interleaves, unroll, + novector); /* Look for the `)'. */ parens.require_close (parser); @@ -51015,6 +51029,20 @@ cp_parser_pragma_unroll (cp_parser *parser, cp_token *pragma_tok) return unroll; } +/* Parse a pragma GCC interleaves. */ + +static tree +cp_parser_pragma_interleaves (cp_parser *parser, cp_token *pragma_tok) +{ + location_t location = cp_lexer_peek_token (parser->lexer)->location; + tree interleaves = cp_parser_constant_expression (parser); + interleaves + = cp_check_pragma_interleaves (location, + fold_non_dependent_expr (interleaves)); + cp_parser_skip_to_pragma_eol (parser, pragma_tok); + return interleaves; +} + /* Parse a pragma GCC novector. */ static bool @@ -51341,10 +51369,12 @@ cp_parser_pragma (cp_parser *parser, enum pragma_context context, bool *if_p) case PRAGMA_IVDEP: case PRAGMA_UNROLL: + case PRAGMA_INTERLEAVES: case PRAGMA_NOVECTOR: { bool ivdep = false; tree unroll = NULL_TREE; + tree interleaves = NULL_TREE; bool novector = false; const char *pragma_str; @@ -51356,6 +51386,9 @@ cp_parser_pragma (cp_parser *parser, enum pragma_context context, bool *if_p) case PRAGMA_UNROLL: pragma_str = "unroll"; break; + case PRAGMA_INTERLEAVES: + pragma_str = "interleaves"; + break; case PRAGMA_NOVECTOR: pragma_str = "novector"; break; @@ -51391,6 +51424,13 @@ cp_parser_pragma (cp_parser *parser, enum pragma_context context, bool *if_p) unroll = cp_parser_pragma_unroll (parser, tok); break; } + case PRAGMA_INTERLEAVES: + { + if (tok != pragma_tok) + tok = cp_lexer_consume_token (parser->lexer); + interleaves = cp_parser_pragma_interleaves (parser, tok); + break; + } case PRAGMA_NOVECTOR: { if (tok != pragma_tok) @@ -51415,7 +51455,8 @@ cp_parser_pragma (cp_parser *parser, enum pragma_context context, bool *if_p) cp_parser_error (parser, "for, while or do statement expected"); return false; } - cp_parser_iteration_statement (parser, if_p, ivdep, unroll, novector); + cp_parser_iteration_statement (parser, if_p, ivdep, interleaves, unroll, + novector); return true; } diff --git a/gcc/cp/pt.cc b/gcc/cp/pt.cc index dfce1b3c35910d8afe19ccde89b9cb11183b2cbc..66649d284a395f4a28db6cbab43071b66b2a9c62 100644 --- a/gcc/cp/pt.cc +++ b/gcc/cp/pt.cc @@ -18638,7 +18638,7 @@ tsubst_stmt (tree t, tree args, tsubst_flags_t complain, tree in_decl) RECUR (FOR_INIT_STMT (t)); finish_init_stmt (stmt); tmp = RECUR (FOR_COND (t)); - finish_for_cond (tmp, stmt, false, 0, false); + finish_for_cond (tmp, stmt, false, 0, 0, false); tmp = RECUR (FOR_EXPR (t)); finish_for_expr (tmp, stmt); { @@ -18701,7 +18701,7 @@ tsubst_stmt (tree t, tree args, tsubst_flags_t complain, tree in_decl) case WHILE_STMT: stmt = begin_while_stmt (); tmp = RECUR (WHILE_COND (t)); - finish_while_stmt_cond (tmp, stmt, false, 0, false); + finish_while_stmt_cond (tmp, stmt, false, 0, 0, false); { bool prev = note_iteration_stmt_body_start (); RECUR (WHILE_BODY (t)); @@ -18719,7 +18719,7 @@ tsubst_stmt (tree t, tree args, tsubst_flags_t complain, tree in_decl) } finish_do_body (stmt); tmp = RECUR (DO_COND (t)); - finish_do_stmt (tmp, stmt, false, 0, false); + finish_do_stmt (tmp, stmt, false, 0, 0, false); break; case IF_STMT: @@ -19470,10 +19470,18 @@ tsubst_stmt (tree t, tree args, tsubst_flags_t complain, tree in_decl) tree op1 = RECUR (TREE_OPERAND (t, 0)); tree op2 = tsubst_expr (TREE_OPERAND (t, 1), args, complain, in_decl); tree op3 = tsubst_expr (TREE_OPERAND (t, 2), args, complain, in_decl); - if (TREE_CODE (op2) == INTEGER_CST - && wi::to_widest (op2) == (int) annot_expr_unroll_kind) - op3 = cp_check_pragma_unroll (EXPR_LOCATION (TREE_OPERAND (t, 2)), - op3); + + if (TREE_CODE (op2) == INTEGER_CST) + { + if (wi::to_widest (op2) == (int) annot_expr_unroll_kind) + op3 = cp_check_pragma_unroll (EXPR_LOCATION (TREE_OPERAND (t, 2)), + op3); + else if (wi::to_widest (op2) == (int) annot_expr_interleaves_kind) + { + location_t loc = EXPR_LOCATION (TREE_OPERAND (t, 2)); + op3 = cp_check_pragma_interleaves (loc, op3); + } + } RETURN (build3_loc (EXPR_LOCATION (t), ANNOTATE_EXPR, TREE_TYPE (op1), op1, op2, op3)); } diff --git a/gcc/cp/semantics.cc b/gcc/cp/semantics.cc index f90c304a65b7709e2ab1447b94b8dd3a551a279a..ec853cd0e75f58357ba163a01d8353797793d387 100644 --- a/gcc/cp/semantics.cc +++ b/gcc/cp/semantics.cc @@ -1284,7 +1284,7 @@ begin_while_stmt (void) void finish_while_stmt_cond (tree cond, tree while_stmt, bool ivdep, - tree unroll, bool novector) + tree interleaves, tree unroll, bool novector) { cond = maybe_convert_cond (cond); finish_cond (&WHILE_COND (while_stmt), cond); @@ -1303,6 +1303,13 @@ finish_while_stmt_cond (tree cond, tree while_stmt, bool ivdep, build_int_cst (integer_type_node, annot_expr_unroll_kind), unroll); + if (interleaves && cond != error_mark_node) + WHILE_COND (while_stmt) + = build3 (ANNOTATE_EXPR, + TREE_TYPE (WHILE_COND (while_stmt)), + WHILE_COND (while_stmt), + build_int_cst (integer_type_node, annot_expr_interleaves_kind), + interleaves); if (novector && cond != error_mark_node) WHILE_COND (while_stmt) = build3 (ANNOTATE_EXPR, TREE_TYPE (WHILE_COND (while_stmt)), @@ -1356,7 +1363,7 @@ finish_do_body (tree do_stmt) void finish_do_stmt (tree cond, tree do_stmt, bool ivdep, tree unroll, - bool novector) + tree interleaves, bool novector) { cond = maybe_convert_cond (cond); end_maybe_infinite_loop (cond); @@ -1373,6 +1380,10 @@ finish_do_stmt (tree cond, tree do_stmt, bool ivdep, tree unroll, cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, build_int_cst (integer_type_node, annot_expr_unroll_kind), unroll); + if (interleaves && cond != error_mark_node) + cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, + build_int_cst (integer_type_node, annot_expr_interleaves_kind), + interleaves); if (novector && cond != error_mark_node) cond = build3 (ANNOTATE_EXPR, TREE_TYPE (cond), cond, build_int_cst (integer_type_node, annot_expr_no_vector_kind), @@ -1481,8 +1492,8 @@ finish_init_stmt (tree for_stmt) FOR_STMT. */ void -finish_for_cond (tree cond, tree for_stmt, bool ivdep, tree unroll, - bool novector) +finish_for_cond (tree cond, tree for_stmt, bool ivdep, tree interleaves, + tree unroll, bool novector) { cond = maybe_convert_cond (cond); finish_cond (&FOR_COND (for_stmt), cond); @@ -1494,6 +1505,13 @@ finish_for_cond (tree cond, tree for_stmt, bool ivdep, tree unroll, build_int_cst (integer_type_node, annot_expr_ivdep_kind), integer_zero_node); + if (interleaves && cond != error_mark_node) + FOR_COND (for_stmt) = build3 (ANNOTATE_EXPR, + TREE_TYPE (FOR_COND (for_stmt)), + FOR_COND (for_stmt), + build_int_cst (integer_type_node, + annot_expr_interleaves_kind), + interleaves); if (unroll && cond != error_mark_node) FOR_COND (for_stmt) = build3 (ANNOTATE_EXPR, TREE_TYPE (FOR_COND (for_stmt)), @@ -13342,6 +13360,36 @@ cp_build_bit_cast (location_t loc, tree type, tree arg, return ret; } +/* Diagnose invalid #pragma GCC interleaves argument and adjust + it if needed. */ + +tree +cp_check_pragma_interleaves (location_t loc, tree value) +{ + HOST_WIDE_INT lvalue = 0; + if (type_dependent_expression_p (value)) + ; + else if (!INTEGRAL_TYPE_P (TREE_TYPE (value)) + || (!value_dependent_expression_p (value) + && (!tree_fits_shwi_p (value) + || (lvalue = tree_to_shwi (value)) < 0 + || lvalue >= USHRT_MAX + || (lvalue % 2 != 0)))) + { + error_at (loc, "%<#pragma GCC interleaves%> requires an" + " assignment-expression that evaluates to a non-negative" + " power of two integral constant less than %u", USHRT_MAX); + value = NULL_TREE; + } + else if (TREE_CODE (value) == INTEGER_CST) + { + value = fold_convert (integer_type_node, value); + if (integer_zerop (value)) + value = integer_one_node; + } + return value; +} + /* Diagnose invalid #pragma GCC unroll argument and adjust it if needed. */ diff --git a/gcc/gimplify.cc b/gcc/gimplify.cc index b0ed58ed0f917bf694b34d90c7f53f7196a8e791..be24e5399aa6aa0fd8df2473abebb0a2994bd2f6 100644 --- a/gcc/gimplify.cc +++ b/gcc/gimplify.cc @@ -4582,6 +4582,7 @@ gimple_boolify (tree expr) { case annot_expr_ivdep_kind: case annot_expr_unroll_kind: + case annot_expr_interleaves_kind: case annot_expr_no_vector_kind: case annot_expr_vector_kind: case annot_expr_parallel_kind: diff --git a/gcc/ipa-icf-gimple.cc b/gcc/ipa-icf-gimple.cc index c25eb24710f6a785d2c739e5d5ad7c066c3e0f3d..f9c8456a7ea2d82114944824c1b2ce3d81dbb19f 100644 --- a/gcc/ipa-icf-gimple.cc +++ b/gcc/ipa-icf-gimple.cc @@ -541,6 +541,8 @@ func_checker::compare_loops (basic_block bb1, basic_block bb2) return return_false_with_msg ("finite_p"); if (l1->unroll != l2->unroll) return return_false_with_msg ("unroll"); + if (l1->interleaves != l2->interleaves) + return return_false_with_msg ("interleaves"); if (!compare_variable_decl (l1->simduid, l2->simduid)) return return_false_with_msg ("simduid"); diff --git a/gcc/tree-cfg.cc b/gcc/tree-cfg.cc index 7fb7b92966be0e54824079b193d53a92c3fb28c6..b474a0971cddf7f486a524b52af2f70bd7e56580 100644 --- a/gcc/tree-cfg.cc +++ b/gcc/tree-cfg.cc @@ -286,6 +286,10 @@ replace_loop_annotate_in_block (basic_block bb, class loop *loop) = (unsigned short) tree_to_shwi (gimple_call_arg (stmt, 2)); cfun->has_unroll = true; break; + case annot_expr_interleaves_kind: + loop->interleaves + = (unsigned short) tree_to_shwi (gimple_call_arg (stmt, 2)); + break; case annot_expr_no_vector_kind: loop->dont_vectorize = true; break; @@ -347,6 +351,7 @@ replace_loop_annotate (void) { case annot_expr_ivdep_kind: case annot_expr_unroll_kind: + case annot_expr_interleaves_kind: case annot_expr_no_vector_kind: case annot_expr_vector_kind: case annot_expr_parallel_kind: diff --git a/gcc/tree-core.h b/gcc/tree-core.h index 9fa7434291914ce02927938441463b220ec832a7..9c7978699e33739e2e009f794927834450884108 100644 --- a/gcc/tree-core.h +++ b/gcc/tree-core.h @@ -980,6 +980,7 @@ enum tree_node_kind { enum annot_expr_kind { annot_expr_ivdep_kind, annot_expr_unroll_kind, + annot_expr_interleaves_kind, annot_expr_no_vector_kind, annot_expr_vector_kind, annot_expr_parallel_kind, diff --git a/gcc/tree-pretty-print.cc b/gcc/tree-pretty-print.cc index f9ad8562078157572a999abd93ce259c3a8d4b64..2a3feb777acd44865031c25310851d9ba6061850 100644 --- a/gcc/tree-pretty-print.cc +++ b/gcc/tree-pretty-print.cc @@ -3454,8 +3454,12 @@ dump_generic_node (pretty_printer *pp, tree node, int spc, dump_flags_t flags, pp_string (pp, ", ivdep"); break; case annot_expr_unroll_kind: + case annot_expr_interleaves_kind: { - pp_string (pp, ", unroll "); + pp_string (pp, + annot_expr_unroll_kind + ? ", unroll " + : ", interleaves "); pp_decimal_int (pp, (int) TREE_INT_CST_LOW (TREE_OPERAND (node, 2))); break; diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 04a9ac64df7401d3f4646e518bc41b73b3f2d036..59c6ee81f1ea225b243e9096890f751a69fe7141 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -5008,6 +5008,9 @@ vect_estimate_min_profitable_iters (loop_vec_info loop_vinfo, &vec_prologue_cost, &vec_inside_cost, &vec_epilogue_cost, suggested_unroll_factor); + if (suggested_unroll_factor && loop_vinfo->loop->interleaves > 0) + *suggested_unroll_factor = loop_vinfo->loop->interleaves; + if (suggested_unroll_factor && *suggested_unroll_factor > 1 && LOOP_VINFO_MAX_VECT_FACTOR (loop_vinfo) != MAX_VECTORIZATION_FACTOR && !known_le (LOOP_VINFO_VECT_FACTOR (loop_vinfo) *