From patchwork Fri Jul 16 02:13:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 1505969 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=b5Lo0a+K; dkim-atps=neutral Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GQvvG74Fbz9sS8 for ; Fri, 16 Jul 2021 12:14:33 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id AF0C939A0845 for ; Fri, 16 Jul 2021 02:14:30 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org AF0C939A0845 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1626401670; bh=PCq+hs7gYOm6LQLEVkJo5SuWb4KA0NRVX3bvH9Je3NU=; h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=b5Lo0a+KIyJqr4wbCEfheqlUrHoKbnPXhzP+I2ztFmfuC9NZgsdoqvrbl/vV34scd NEHu6dJDE2WxB4UQxxETDNjIzxkFHe3bL9nOltrzmMf3Ozh3cS8Uw2P/MBNTYlZcxJ CZp1cmbqwXtQ9+MSq0Os7Q6R1xW9Y9xGMiT1n0TM= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by sourceware.org (Postfix) with ESMTPS id D1FD0385DC10 for ; Fri, 16 Jul 2021 02:13:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D1FD0385DC10 Received: by mail-pj1-x1033.google.com with SMTP id p4-20020a17090a9304b029016f3020d867so6092488pjo.3 for ; Thu, 15 Jul 2021 19:13:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PCq+hs7gYOm6LQLEVkJo5SuWb4KA0NRVX3bvH9Je3NU=; b=PMgR1phk0l1iSHfj/jYNrmj42TGb8QYTdwquZOI2tBlgOw5qyTUJcmwPk9J1Az9gp0 Bj8N3kArHlwIJXbmdDiPWQMo7tN/TElACwoR8iLG8+2ZswAJy4nva8T+FRI+HZHzhij/ K/yI62cjWowPe/wFlD7GuWmMNGvdKz6mgrxQGOLaM89AtlQFse8cPkS9PGrRnK1ogIcR A9w83Z3vmgj5UszKaDbU5cQM9dGaEWP52jCwVWnBt2T1UlEQokhf9+Ay2WgbgYwpZFbQ TAbi29OUFzXoPZTlrYBn20UOkfmkgc/Ezfk0kSxWEedGNmMgVaYBW9Rejyl3ND3GteI9 6FQg== X-Gm-Message-State: AOAM531pM8K57r0Vqf6t3XhgcvYbZf1HtZRALi1DofGcEZyjAeYFN/K/ m9nF2Vy3ZK8zf9y3GUQqgN04NuVPsOeR8z2zZlo= X-Google-Smtp-Source: ABdhPJw8lPSWPWgVdSAQrMTf1ecrVBYwwkMjHz/aCbJPIIs1vroNwVJXKizE/iTG7tIvI0Y089oejNk0iErpdItSY9E= X-Received: by 2002:a17:903:2282:b029:12b:f6e:fc52 with SMTP id b2-20020a1709032282b029012b0f6efc52mr5713201plh.79.1626401625786; Thu, 15 Jul 2021 19:13:45 -0700 (PDT) MIME-Version: 1.0 References: <20210715173009.558698-1-hjl.tools@gmail.com> In-Reply-To: Date: Thu, 15 Jul 2021 19:13:09 -0700 Message-ID: Subject: [PATCH v2] x86: Don't set AVX_U128_DIRTY when all bits are zero To: Hongtao Liu X-Spam-Status: No, score=-3031.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Gcc-patches" From: "H.J. Lu" Reply-To: "H.J. Lu" Cc: GCC Patches Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" On Thu, Jul 15, 2021 at 6:36 PM Hongtao Liu wrote: > > On Fri, Jul 16, 2021 at 1:30 AM H.J. Lu via Gcc-patches > wrote: > > > > In a single SET, all bits of the source YMM/ZMM register are zero when > > > > 1. The source is contant zero. > > 2. The source YMM/ZMM operand are defined from contant zero. > > > > and we don't set AVX_U128_DIRTY. > > > > gcc/ > > > > PR target/101456 > > * config/i386/i386.c (ix86_avx_u128_mode_needed): Don't set > > AVX_U128_DIRTY when all bits are zero. > > > > gcc/testsuite/ > > > > PR target/101456 > > * gcc.target/i386/pr101456-1.c: New test. > > --- > > gcc/config/i386/i386.c | 47 ++++++++++++++++++++++ > > gcc/testsuite/gcc.target/i386/pr101456-1.c | 28 +++++++++++++ > > 2 files changed, 75 insertions(+) > > create mode 100644 gcc/testsuite/gcc.target/i386/pr101456-1.c > > > > diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c > > index cff26909292..c2b06934053 100644 > > --- a/gcc/config/i386/i386.c > > +++ b/gcc/config/i386/i386.c > > @@ -14129,6 +14129,53 @@ ix86_avx_u128_mode_needed (rtx_insn *insn) > > return AVX_U128_CLEAN; > > } > > > > + rtx set = single_set (insn); > > + if (set) > > + { > > + rtx dest = SET_DEST (set); > > + rtx src = SET_SRC (set); > > + if (ix86_check_avx_upper_register (dest)) > > + { > > + /* It is not dirty if the source is known zero. */ > > + if (standard_sse_constant_p (src, GET_MODE (dest)) == 1) > > + return AVX_U128_ANY; > > + else > > + return AVX_U128_DIRTY; > > + } > > + else if (ix86_check_avx_upper_register (src)) > > + { > > + /* Check for the source operand with all DEFs from constant > > + zero. */ > > + df_ref def = DF_REG_DEF_CHAIN (REGNO (src)); > > + if (!def) > > + return AVX_U128_DIRTY; > > + > > + for (; def; def = DF_REF_NEXT_REG (def)) > > + if (DF_REF_REG_DEF_P (def) > > + && !DF_REF_IS_ARTIFICIAL (def)) > > + { > > + rtx_insn *def_insn = DF_REF_INSN (def); > > + set = single_set (def_insn); > > + if (!set) > > + return AVX_U128_DIRTY; > > + > > + dest = SET_DEST (set); > > + if (ix86_check_avx_upper_register (dest)) > > + { > > + src = SET_SRC (set); > > + /* It is dirty if the source operand isn't constant > > + zero. */ > > + if (standard_sse_constant_p (src, GET_MODE (dest)) > > + != 1) > > + return AVX_U128_DIRTY; > > + } > > + } > > + > > + /* It is not dirty only if all sources are known zero. */ > > + return AVX_U128_ANY; > > + } > > + } > > + > > /* Require DIRTY mode if a 256bit or 512bit AVX register is referenced. > > Hardware changes state only when a 256bit register is written to, > > but we need to prevent the compiler from moving optimal insertion > > diff --git a/gcc/testsuite/gcc.target/i386/pr101456-1.c b/gcc/testsuite/gcc.target/i386/pr101456-1.c > > new file mode 100644 > > index 00000000000..6a0f6ccd756 > > --- /dev/null > > +++ b/gcc/testsuite/gcc.target/i386/pr101456-1.c > > @@ -0,0 +1,28 @@ > > +/* { dg-do compile } */ > > +/* { dg-options "-O2 -march=skylake" } */ > > + > > +#include > > + > > +extern __m256 x1; > > +extern __m256d x2; > > +extern __m256i x3; > > + > > +void > > +foo1 (void) > > +{ > > + x1 = _mm256_setzero_ps (); > > +} > > + > > +void > > +foo2 (void) > > +{ > > + x2 = _mm256_setzero_pd (); > > +} > > + > > +void > > +foo3 (void) > > +{ > > + x3 = _mm256_setzero_si256 (); > > +} > > + > > +/* { dg-final { scan-assembler-not "vzeroupper" } } */ > > -- > > 2.31.1 > > > > LGTM. > Here is the v2 patch to handle calls. From 4bd6aba8326eee9fa3c5310086fc5b76fc090795 Mon Sep 17 00:00:00 2001 From: "H.J. Lu" Date: Wed, 14 Jul 2021 17:03:15 -0700 Subject: [PATCH v2] x86: Don't set AVX_U128_DIRTY when all bits are zero In a single SET, all bits of the source YMM/ZMM register are zero when 1. The source is contant zero. 2. The source YMM/ZMM operand are defined from contant zero. and we don't set AVX_U128_DIRTY. gcc/ PR target/101456 * config/i386/i386.c (ix86_avx_u128_mode_needed): Don't set AVX_U128_DIRTY when all bits are zero. gcc/testsuite/ PR target/101456 * gcc.target/i386/pr101456-1.c: New test. * gcc.target/i386/pr101456-2.c: Likewise. --- gcc/config/i386/i386.c | 63 ++++++++++++++++++++++ gcc/testsuite/gcc.target/i386/pr101456-1.c | 33 ++++++++++++ gcc/testsuite/gcc.target/i386/pr101456-2.c | 33 ++++++++++++ 3 files changed, 129 insertions(+) create mode 100644 gcc/testsuite/gcc.target/i386/pr101456-1.c create mode 100644 gcc/testsuite/gcc.target/i386/pr101456-2.c diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c index 9d74b7a191b..8df099351f3 100644 --- a/gcc/config/i386/i386.c +++ b/gcc/config/i386/i386.c @@ -14093,6 +14093,8 @@ ix86_check_avx_upper_register (const_rtx exp) && GET_MODE_BITSIZE (GET_MODE (exp)) > 128); } +static void ix86_check_avx_upper_stores (rtx, const_rtx, void *); + /* Return needed mode for entity in optimize_mode_switching pass. */ static int @@ -14129,6 +14131,67 @@ ix86_avx_u128_mode_needed (rtx_insn *insn) return AVX_U128_CLEAN; } + rtx set = single_set (insn); + if (set) + { + rtx dest = SET_DEST (set); + rtx src = SET_SRC (set); + if (ix86_check_avx_upper_register (dest)) + { + /* It is not dirty if the source is known zero. */ + if (standard_sse_constant_p (src, GET_MODE (dest)) == 1) + return AVX_U128_ANY; + else + return AVX_U128_DIRTY; + } + else if (ix86_check_avx_upper_register (src)) + { + /* Check for the source operand with all DEFs from constant + zero. */ + df_ref def = DF_REG_DEF_CHAIN (REGNO (src)); + if (!def) + return AVX_U128_DIRTY; + + for (; def; def = DF_REF_NEXT_REG (def)) + if (DF_REF_REG_DEF_P (def) + && !DF_REF_IS_ARTIFICIAL (def)) + { + rtx_insn *def_insn = DF_REF_INSN (def); + + if (CALL_P (def_insn)) + { + bool avx_upper_reg_found = false; + note_stores (def_insn, ix86_check_avx_upper_stores, + &avx_upper_reg_found); + + /* It is dirty if call is dirty. */ + if (avx_upper_reg_found) + return AVX_U128_DIRTY; + + continue; + } + + set = single_set (def_insn); + if (!set) + return AVX_U128_DIRTY; + + dest = SET_DEST (set); + if (ix86_check_avx_upper_register (dest)) + { + src = SET_SRC (set); + /* It is dirty if the source operand isn't constant + zero. */ + if (standard_sse_constant_p (src, GET_MODE (dest)) + != 1) + return AVX_U128_DIRTY; + } + } + + /* It is not dirty only if all sources are known zero. */ + return AVX_U128_ANY; + } + } + /* Require DIRTY mode if a 256bit or 512bit AVX register is referenced. Hardware changes state only when a 256bit register is written to, but we need to prevent the compiler from moving optimal insertion diff --git a/gcc/testsuite/gcc.target/i386/pr101456-1.c b/gcc/testsuite/gcc.target/i386/pr101456-1.c new file mode 100644 index 00000000000..803fc6e0207 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr101456-1.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=skylake" } */ + +#include + +extern __m256 x1; +extern __m256d x2; +extern __m256i x3; + +extern void bar (void); + +void +foo1 (void) +{ + x1 = _mm256_setzero_ps (); + bar (); +} + +void +foo2 (void) +{ + x2 = _mm256_setzero_pd (); + bar (); +} + +void +foo3 (void) +{ + x3 = _mm256_setzero_si256 (); + bar (); +} + +/* { dg-final { scan-assembler-not "vzeroupper" } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr101456-2.c b/gcc/testsuite/gcc.target/i386/pr101456-2.c new file mode 100644 index 00000000000..71318368645 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr101456-2.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=skylake" } */ + +#include + +extern __m256 x1; +extern __m256d x2; +extern __m256i x3; + +extern __m256 bar (void); + +void +foo1 (void) +{ + x1 = _mm256_setzero_ps (); + bar (); +} + +void +foo2 (void) +{ + x2 = _mm256_setzero_pd (); + bar (); +} + +void +foo3 (void) +{ + x3 = _mm256_setzero_si256 (); + bar (); +} + +/* { dg-final { scan-assembler-times "vzeroupper" 3 } } */ -- 2.31.1