From patchwork Sat Aug 3 18:36:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 1968751 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bc6QhV75; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WbryZ5YkHz1yZl for ; Sun, 4 Aug 2024 04:36:54 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DBA383858C39 for ; Sat, 3 Aug 2024 18:36:52 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTP id 940813858D29 for ; Sat, 3 Aug 2024 18:36:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 940813858D29 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 940813858D29 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1722710192; cv=none; b=k1QsnyX2hXLJp4KoXrkn7qtEGb99YFWONFVU038cxKrP/LZvwPHG6AfH0BrB+20tgMCznC41KexC/Q7kpdk4D/p6sq7VTF7T8TPHo+ysYguZtg7JSGSzu9SqYlDwpUDPbvPnPkMrWEcBe2PeGYD1VboytvqswWxxPyXkI+qMYc4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1722710192; c=relaxed/simple; bh=sSMa+he5aCeP4dNU/6p6zDIPGlQHWCKN3UASSbFlh5I=; h=DKIM-Signature:Date:From:To:Subject:Message-ID:MIME-Version; b=lJzwsVfQLI2mA/j7YNIieAaqjvtAN7vRsyOSK/AeaJzWnQ0v2i9ZIjrh05nve2DKpdqFk7ttj3V5f3xt1n7jJrqOQ9gq8TIXYXAvkZVQNYuOzwqVZWcsW2igxNWS9/vP1NFRQRRFV8vlZi60vx9xFrCb+86tyf5uFxbu5nnNIrc= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1722710190; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=IzxOfgy+DvRmyku9qYs0BD3PIywqWYc0w5t86rgvvlw=; b=bc6QhV75AY0fR8mVuOcYUrp2PvZmklo8ZQEP1QVSaTlt/pyA5fqlNtN17svtAybtIfR8Ph QsokmMQcgTcTYxffqBWd1fUwrV5B+gPOrF/aIf2fz79cVwDfbdEpF+msRQtukoYClxXeut 8Ctkc0TB/V4XC1B7ulRNJKEsFxst2to= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-450-T3T7kw0mOMGwgzdLCwxM7w-1; Sat, 03 Aug 2024 14:36:27 -0400 X-MC-Unique: T3T7kw0mOMGwgzdLCwxM7w-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 068FD1955F0D for ; Sat, 3 Aug 2024 18:36:27 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.45.224.25]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 527861955D42; Sat, 3 Aug 2024 18:36:24 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.17.1/8.17.1) with ESMTPS id 473IaLsQ944438 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Sat, 3 Aug 2024 20:36:21 +0200 Received: (from jakub@localhost) by tucnak.zalov.cz (8.17.1/8.17.1/Submit) id 473IaLIc944437; Sat, 3 Aug 2024 20:36:21 +0200 Date: Sat, 3 Aug 2024 20:36:20 +0200 From: Jakub Jelinek To: Jason Merrill , David Malcolm Cc: gcc-patches@gcc.gnu.org Subject: [PATCH] c++: Speed up compilation of large char array initializers when not using #embed Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Jakub Jelinek Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org Hi! The following patch (again, on top of the #embed patchset, so https://gcc.gnu.org/pipermail/gcc-patches/2024-June/655012.html https://gcc.gnu.org/pipermail/gcc-patches/2024-June/655013.html https://gcc.gnu.org/pipermail/gcc-patches/2024-July/657049.html https://gcc.gnu.org/pipermail/gcc-patches/2024-July/657053.html https://gcc.gnu.org/pipermail/gcc-patches/2024-July/658137.html https://gcc.gnu.org/pipermail/gcc-patches/2024-August/659331.html ) attempts to optimize compilation of large {{{,un}signed ,}char,std::byte} array initializers when not using #embed in the source. Unlike the C patch which is done during the parsing of initializers this is done when lexing tokens into an array, because C++ lexes all tokens upfront and so by the time we parse the initializers we already have 16 bytes per token allocated (i.e. 32 extra compile time memory bytes per one byte in the array). The drawback is again that it can result in worse locations for diagnostics (-Wnarrowing, -Wconversion) when initializing signed char arrays with values 128..255. Not really sure what to do about this though unlike the C case, the locations would need to be preserved through reshape_init* and perhaps till template instantiation. For #embed, there is just a single location_t (could be range of the directive), for diagnostics perhaps we could extend it to say byte xyz of the file embedded here or something like that, but the optimization done by this patch, either we'd need to bump the minimum limit at which to try it, or say temporarily allocate a location_t array for each byte and then clear it when we no longer need it or something. I've been using the same testcases as for C, with #embed of 100'000'000 bytes: time ./cc1plus -quiet -O2 -o test4a.s2 test4a.c real 0m0.972s user 0m0.578s sys 0m0.195s with xxd -i alternative of the same data without this patch it consumed around 13.2GB of RAM and time ./cc1plus -quiet -O2 -o test4b.s4 test4b.c real 3m47.968s user 3m41.907s sys 0m5.015s and the same with this patch it consumed around 3.7GB of RAM and time ./cc1plus -quiet -O2 -o test4b.s3 test4b.c real 0m24.772s user 0m23.118s sys 0m1.495s Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? 2024-08-03 Jakub Jelinek * parser.cc (cp_lexer_new_main): Attempt to optimize large sequences of CPP_NUMBER with int type and values 0-255 separated by CPP_COMMA into CPP_EMBED with RAW_DATA_CST u.value. Jakub --- gcc/cp/parser.cc.jj 2024-08-02 11:34:20.343813689 +0200 +++ gcc/cp/parser.cc 2024-08-02 19:35:12.380611787 +0200 @@ -734,6 +734,12 @@ cp_lexer_new_main (void) gcc_assert (!the_parser); the_parser = cp_parser_new (lexer); + unsigned raw_data_tokens = 0; + char *raw_data_buf = NULL; + const unsigned int raw_data_max_len + = 131072 - offsetof (struct tree_string, str) - 1; + const unsigned int raw_data_min_len = 128; + /* Get the remaining tokens from the preprocessor. */ while (tok->type != CPP_EOF) { @@ -742,6 +748,72 @@ cp_lexer_new_main (void) module_token_lang (tok->type, tok->keyword, tok->u.value, tok->location, filter); + /* Attempt to optimize long lists of 0-255 integers + separated by commas into CPP_EMBED. */ + recheck: + if (tok->type == CPP_NUMBER + && (raw_data_tokens & 1) == 0 + && TREE_CODE (tok->u.value) == INTEGER_CST + && TREE_TYPE (tok->u.value) == integer_type_node + && !wi::neg_p (wi::to_wide (tok->u.value)) + && wi::to_widest (tok->u.value) <= UCHAR_MAX + && raw_data_tokens < raw_data_max_len * 2) + { + raw_data_tokens++; + if (raw_data_tokens >= raw_data_min_len * 2 + 3) + { + unsigned int len = lexer->buffer->length (); + unsigned int new_len; + if (raw_data_tokens == raw_data_min_len * 2 + 3) + { + if (raw_data_buf == NULL) + raw_data_buf = XNEWVEC (char, raw_data_max_len); + for (unsigned i = len - raw_data_tokens, j = 0; + i < len; i += 2, ++j) + raw_data_buf[j] + = (char) tree_to_uhwi ((*lexer->buffer)[i].u.value); + new_len = len - raw_data_tokens + 5; + } + else + { + raw_data_buf[raw_data_tokens / 2] + = (char) tree_to_uhwi (tok->u.value); + new_len = len - 2; + } + (*lexer->buffer)[new_len - 2] = (*lexer->buffer)[len - 2]; + (*lexer->buffer)[new_len - 1] = (*lexer->buffer)[len - 1]; + lexer->buffer->truncate (new_len); + } + } + else if (tok->type == CPP_COMMA + && (raw_data_tokens & 1) == 1) + raw_data_tokens++; + else if (raw_data_tokens >= raw_data_min_len * 2 + 3) + { + unsigned last_number = (raw_data_tokens & 1); + unsigned int idx = lexer->buffer->length () - 4 - !last_number; + if (!last_number) + --raw_data_tokens; + raw_data_tokens = raw_data_tokens / 2 + 1; + tree raw = make_node (RAW_DATA_CST); + TREE_TYPE (raw) = integer_type_node; + RAW_DATA_LENGTH (raw) = raw_data_tokens - 2; + tree owner = build_string (raw_data_tokens, raw_data_buf); + TREE_TYPE (owner) = build_array_type_nelts (unsigned_char_type_node, + raw_data_tokens); + RAW_DATA_OWNER (raw) = owner; + RAW_DATA_POINTER (raw) = TREE_STRING_POINTER (owner) + 1; + (*lexer->buffer)[idx].type = CPP_EMBED; + (*lexer->buffer)[idx].u.value = raw; + raw_data_tokens = 0; + goto recheck; + } + else if (raw_data_tokens) + { + raw_data_tokens = 0; + goto recheck; + } + /* Check for early pragmas that need to be handled now. */ if (tok->type == CPP_PRAGMA_EOL) cp_lexer_handle_early_pragma (lexer); @@ -750,6 +822,8 @@ cp_lexer_new_main (void) cp_lexer_get_preprocessor_token (C_LEX_STRING_NO_JOIN, tok); } + XDELETEVEC (raw_data_buf); + lexer->next_token = lexer->buffer->address (); lexer->last_token = lexer->next_token + lexer->buffer->length ()