From patchwork Thu Jul 1 06:16:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: liuhongt X-Patchwork-Id: 1499399 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=hNAUlBfq; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GFq9r6Rdcz9sVb for ; Thu, 1 Jul 2021 17:10:40 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 025153951809 for ; Thu, 1 Jul 2021 07:10:38 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 025153951809 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1625123438; bh=ejNVV22lywvhz3wNTvZ/0m2nH0Saq4TiOXxA80gBYTE=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=hNAUlBfq6uofT3A3m5JzJYl4Z9DcP8kBgL0/dZzq8z4+3F7hIUPaItncitq9uJ9gW PVmZr6+pBmQW8n9Jx1VGFUVkJHyGyb+TyGlBAoDQRmBV2Sx+lbh+rcohRZTUxzvt2o REtkT+f1byYoxmQ3Poe2aCB3zsiv6K/Gk+LCs7YM= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by sourceware.org (Postfix) with ESMTPS id 480863848024 for ; Thu, 1 Jul 2021 06:18:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 480863848024 X-IronPort-AV: E=McAfee;i="6200,9189,10031"; a="294115040" X-IronPort-AV: E=Sophos;i="5.83,313,1616482800"; d="scan'208";a="294115040" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2021 23:18:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,313,1616482800"; d="scan'208";a="409038891" Received: from scymds01.sc.intel.com ([10.148.94.138]) by orsmga003.jf.intel.com with ESMTP; 30 Jun 2021 23:18:06 -0700 Received: from shliclel320.sh.intel.com (shliclel320.sh.intel.com [10.239.236.50]) by scymds01.sc.intel.com with ESMTP id 1616GmfQ031625; Wed, 30 Jun 2021 23:18:05 -0700 To: gcc-patches@gcc.gnu.org Subject: [PATCH 47/62] AVX512FP16: Add scalar fma instructions. Date: Thu, 1 Jul 2021 14:16:33 +0800 Message-Id: <20210701061648.9447-48-hongtao.liu@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20210701061648.9447-1-hongtao.liu@intel.com> References: <20210701061648.9447-1-hongtao.liu@intel.com> X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: liuhongt via Gcc-patches From: liuhongt Reply-To: liuhongt Cc: jakub@redhat.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Add vfmadd[132,213,231]sh/vfnmadd[132,213,231]sh/ vfmsub[132,213,231]sh/vfnmsub[132,213,231]sh. gcc/ChangeLog: * config/i386/avx512fp16intrin.h (_mm_fmadd_sh): New intrinsic. (_mm_mask_fmadd_sh): Likewise. (_mm_mask3_fmadd_sh): Likewise. (_mm_maskz_fmadd_sh): Likewise. (_mm_fmadd_round_sh): Likewise. (_mm_mask_fmadd_round_sh): Likewise. (_mm_mask3_fmadd_round_sh): Likewise. (_mm_maskz_fmadd_round_sh): Likewise. (_mm_fnmadd_sh): Likewise. (_mm_mask_fnmadd_sh): Likewise. (_mm_mask3_fnmadd_sh): Likewise. (_mm_maskz_fnmadd_sh): Likewise. (_mm_fnmadd_round_sh): Likewise. (_mm_mask_fnmadd_round_sh): Likewise. (_mm_mask3_fnmadd_round_sh): Likewise. (_mm_maskz_fnmadd_round_sh): Likewise. (_mm_fmsub_sh): Likewise. (_mm_mask_fmsub_sh): Likewise. (_mm_mask3_fmsub_sh): Likewise. (_mm_maskz_fmsub_sh): Likewise. (_mm_fmsub_round_sh): Likewise. (_mm_mask_fmsub_round_sh): Likewise. (_mm_mask3_fmsub_round_sh): Likewise. (_mm_maskz_fmsub_round_sh): Likewise. (_mm_fnmsub_sh): Likewise. (_mm_mask_fnmsub_sh): Likewise. (_mm_mask3_fnmsub_sh): Likewise. (_mm_maskz_fnmsub_sh): Likewise. (_mm_fnmsub_round_sh): Likewise. (_mm_mask_fnmsub_round_sh): Likewise. (_mm_mask3_fnmsub_round_sh): Likewise. (_mm_maskz_fnmsub_round_sh): Likewise. * config/i386/i386-builtin-types.def (V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT): New builtin type. * config/i386/i386-builtin.def: Add new builtins. * config/i386/i386-expand.c: Handle new builtin type. * config/i386/sse.md (fmai_vmfmadd_): Ajdust to support FP16. (fmai_vmfmsub_): Ditto. (fmai_vmfnmadd_): Ditto. (fmai_vmfnmsub_): Ditto. (*fmai_fmadd_): Ditto. (*fmai_fmsub_): Ditto. (*fmai_fnmadd_): Ditto. (*fmai_fnmsub_): Ditto. (avx512f_vmfmadd__mask): Ditto. (avx512f_vmfmadd__mask3): Ditto. (avx512f_vmfmadd__maskz): Ditto. (avx512f_vmfmadd__maskz_1): Ditto. (*avx512f_vmfmsub__mask): Ditto. (avx512f_vmfmsub__mask3): Ditto. (*avx512f_vmfmsub__maskz_1): Ditto. (*avx512f_vmfnmsub__mask): Ditto. (*avx512f_vmfnmsub__mask3): Ditto. (*avx512f_vmfnmsub__mask): Ditto. (*avx512f_vmfnmadd__mask): Renamed to ... (avx512f_vmfnmadd__mask) ... this, and adjust to support FP16. (avx512f_vmfnmadd__mask3): Ditto. (avx512f_vmfnmadd__maskz_1): Ditto. (avx512f_vmfnmadd__maskz): New expander. gcc/testsuite/ChangeLog: * gcc.target/i386/avx-1.c: Add test for new builtins. * gcc.target/i386/sse-13.c: Ditto. * gcc.target/i386/sse-23.c: Ditto. * gcc.target/i386/sse-14.c: Add test for new intrinsics. * gcc.target/i386/sse-22.c: Ditto. --- gcc/config/i386/avx512fp16intrin.h | 412 +++++++++++++++++++++++++ gcc/config/i386/i386-builtin-types.def | 1 + gcc/config/i386/i386-builtin.def | 7 + gcc/config/i386/i386-expand.c | 1 + gcc/config/i386/sse.md | 340 ++++++++++---------- gcc/testsuite/gcc.target/i386/avx-1.c | 12 + gcc/testsuite/gcc.target/i386/sse-13.c | 12 + gcc/testsuite/gcc.target/i386/sse-14.c | 16 + gcc/testsuite/gcc.target/i386/sse-22.c | 16 + gcc/testsuite/gcc.target/i386/sse-23.c | 12 + 10 files changed, 666 insertions(+), 163 deletions(-) diff --git a/gcc/config/i386/avx512fp16intrin.h b/gcc/config/i386/avx512fp16intrin.h index f246bab5159..5c85ec15b22 100644 --- a/gcc/config/i386/avx512fp16intrin.h +++ b/gcc/config/i386/avx512fp16intrin.h @@ -5697,6 +5697,418 @@ _mm512_maskz_fnmsub_round_ph (__mmask32 __U, __m512h __A, __m512h __B, #endif /* __OPTIMIZE__ */ +/* Intrinsics vfmadd[132,213,231]sh. */ +extern __inline __m128h + __attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fmadd_sh (__m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) -1, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fmadd_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fmadd_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask3 ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fmadd_sh (__mmask8 __U, __m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_maskz ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + + +#ifdef __OPTIMIZE__ +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fmadd_round_sh (__m128h __W, __m128h __A, __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) -1, + __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fmadd_round_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B, + const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fmadd_round_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U, + const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask3 ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fmadd_round_sh (__mmask8 __U, __m128h __W, __m128h __A, + __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_maskz ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +#else +#define _mm_fmadd_round_sh(A, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_mask ((A), (B), (C), (-1), (R))) +#define _mm_mask_fmadd_round_sh(A, U, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_mask ((A), (B), (C), (U), (R))) +#define _mm_mask3_fmadd_round_sh(A, B, C, U, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_mask3 ((A), (B), (C), (U), (R))) +#define _mm_maskz_fmadd_round_sh(U, A, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_maskz ((A), (B), (C), (U), (R))) + +#endif /* __OPTIMIZE__ */ + +/* Intrinsics vfnmadd[132,213,231]sh. */ +extern __inline __m128h + __attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fnmadd_sh (__m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) -1, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fnmadd_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fnmadd_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_mask3 ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fnmadd_sh (__mmask8 __U, __m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_maskz ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + + +#ifdef __OPTIMIZE__ +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fnmadd_round_sh (__m128h __W, __m128h __A, __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) -1, + __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fnmadd_round_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B, + const int __R) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fnmadd_round_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U, + const int __R) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_mask3 ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fnmadd_round_sh (__mmask8 __U, __m128h __W, __m128h __A, + __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfnmaddsh3_maskz ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +#else +#define _mm_fnmadd_round_sh(A, B, C, R) \ + ((__m128h) __builtin_ia32_vfnmaddsh3_mask ((A), (B), (C), (-1), (R))) +#define _mm_mask_fnmadd_round_sh(A, U, B, C, R) \ + ((__m128h) __builtin_ia32_vfnmaddsh3_mask ((A), (B), (C), (U), (R))) +#define _mm_mask3_fnmadd_round_sh(A, B, C, U, R) \ + ((__m128h) __builtin_ia32_vfnmaddsh3_mask3 ((A), (B), (C), (U), (R))) +#define _mm_maskz_fnmadd_round_sh(U, A, B, C, R) \ + ((__m128h) __builtin_ia32_vfnmaddsh3_maskz ((A), (B), (C), (U), (R))) + +#endif /* __OPTIMIZE__ */ + +/* Intrinsics vfmsub[132,213,231]sh. */ +extern __inline __m128h + __attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fmsub_sh (__m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + -(__v8hf) __B, + (__mmask8) -1, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fmsub_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fmsub_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U) +{ + return (__m128h) __builtin_ia32_vfmsubsh3_mask3 ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fmsub_sh (__mmask8 __U, __m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_maskz ((__v8hf) __W, + (__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + + +#ifdef __OPTIMIZE__ +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fmsub_round_sh (__m128h __W, __m128h __A, __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + -(__v8hf) __B, + (__mmask8) -1, + __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fmsub_round_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B, + const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + (__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fmsub_round_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U, + const int __R) +{ + return (__m128h) __builtin_ia32_vfmsubsh3_mask3 ((__v8hf) __W, + (__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fmsub_round_sh (__mmask8 __U, __m128h __W, __m128h __A, + __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_maskz ((__v8hf) __W, + (__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, __R); +} + +#else +#define _mm_fmsub_round_sh(A, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_mask ((A), (B), -(C), (-1), (R))) +#define _mm_mask_fmsub_round_sh(A, U, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_mask ((A), (B), -(C), (U), (R))) +#define _mm_mask3_fmsub_round_sh(A, B, C, U, R) \ + ((__m128h) __builtin_ia32_vfmsubsh3_mask3 ((A), (B), (C), (U), (R))) +#define _mm_maskz_fmsub_round_sh(U, A, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_maskz ((A), (B), -(C), (U), (R))) + +#endif /* __OPTIMIZE__ */ + +/* Intrinsics vfnmsub[132,213,231]sh. */ +extern __inline __m128h + __attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fnmsub_sh (__m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + -(__v8hf) __A, + -(__v8hf) __B, + (__mmask8) -1, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fnmsub_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + -(__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fnmsub_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U) +{ + return (__m128h) __builtin_ia32_vfmsubsh3_mask3 ((__v8hf) __W, + -(__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fnmsub_sh (__mmask8 __U, __m128h __W, __m128h __A, __m128h __B) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_maskz ((__v8hf) __W, + -(__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, + _MM_FROUND_CUR_DIRECTION); +} + + +#ifdef __OPTIMIZE__ +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_fnmsub_round_sh (__m128h __W, __m128h __A, __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + -(__v8hf) __A, + -(__v8hf) __B, + (__mmask8) -1, + __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask_fnmsub_round_sh (__m128h __W, __mmask8 __U, __m128h __A, __m128h __B, + const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_mask ((__v8hf) __W, + -(__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_mask3_fnmsub_round_sh (__m128h __W, __m128h __A, __m128h __B, __mmask8 __U, + const int __R) +{ + return (__m128h) __builtin_ia32_vfmsubsh3_mask3 ((__v8hf) __W, + -(__v8hf) __A, + (__v8hf) __B, + (__mmask8) __U, __R); +} + +extern __inline __m128h +__attribute__ ((__gnu_inline__, __always_inline__, __artificial__)) +_mm_maskz_fnmsub_round_sh (__mmask8 __U, __m128h __W, __m128h __A, + __m128h __B, const int __R) +{ + return (__m128h) __builtin_ia32_vfmaddsh3_maskz ((__v8hf) __W, + -(__v8hf) __A, + -(__v8hf) __B, + (__mmask8) __U, __R); +} + +#else +#define _mm_fnmsub_round_sh(A, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_mask ((A), -(B), -(C), (-1), (R))) +#define _mm_mask_fnmsub_round_sh(A, U, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_mask ((A), -(B), -(C), (U), (R))) +#define _mm_mask3_fnmsub_round_sh(A, B, C, U, R) \ + ((__m128h) __builtin_ia32_vfmsubsh3_mask3 ((A), -(B), (C), (U), (R))) +#define _mm_maskz_fnmsub_round_sh(U, A, B, C, R) \ + ((__m128h) __builtin_ia32_vfmaddsh3_maskz ((A), -(B), -(C), (U), (R))) + +#endif /* __OPTIMIZE__ */ + #ifdef __DISABLE_AVX512FP16__ #undef __DISABLE_AVX512FP16__ #pragma GCC pop_options diff --git a/gcc/config/i386/i386-builtin-types.def b/gcc/config/i386/i386-builtin-types.def index 0cdbf1bc0c0..22b924bf98d 100644 --- a/gcc/config/i386/i386-builtin-types.def +++ b/gcc/config/i386/i386-builtin-types.def @@ -1342,6 +1342,7 @@ DEF_FUNCTION_TYPE (V8HF, V8HF, V8HF, INT) DEF_FUNCTION_TYPE (V8HF, V8HF, INT, V8HF, UQI) DEF_FUNCTION_TYPE (UQI, V8HF, V8HF, INT, UQI) DEF_FUNCTION_TYPE (V8HF, V8HF, V8HF, V8HF, UQI) +DEF_FUNCTION_TYPE (V8HF, V8HF, V8HF, V8HF, INT) DEF_FUNCTION_TYPE (UQI, V8HF, V8HF, INT, UQI, INT) DEF_FUNCTION_TYPE (V8DI, V8HF, V8DI, UQI, INT) DEF_FUNCTION_TYPE (V8DF, V8HF, V8DF, UQI, INT) diff --git a/gcc/config/i386/i386-builtin.def b/gcc/config/i386/i386-builtin.def index cf0259843cc..f446a6ce5d3 100644 --- a/gcc/config/i386/i386-builtin.def +++ b/gcc/config/i386/i386-builtin.def @@ -3194,6 +3194,13 @@ BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512bw_fmsub_v32hf_maskz_round BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512bw_fnmsub_v32hf_mask_round, "__builtin_ia32_vfnmsubph512_mask", IX86_BUILTIN_VFNMSUBPH512_MASK, UNKNOWN, (int) V32HF_FTYPE_V32HF_V32HF_V32HF_USI_INT) BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512bw_fnmsub_v32hf_mask3_round, "__builtin_ia32_vfnmsubph512_mask3", IX86_BUILTIN_VFNMSUBPH512_MASK3, UNKNOWN, (int) V32HF_FTYPE_V32HF_V32HF_V32HF_USI_INT) BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512bw_fnmsub_v32hf_maskz_round, "__builtin_ia32_vfnmsubph512_maskz", IX86_BUILTIN_VFNMSUBPH512_MASKZ, UNKNOWN, (int) V32HF_FTYPE_V32HF_V32HF_V32HF_USI_INT) +BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512f_vmfmadd_v8hf_mask_round, "__builtin_ia32_vfmaddsh3_mask", IX86_BUILTIN_VFMADDSH3_MASK, UNKNOWN, (int) V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT) +BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512f_vmfmadd_v8hf_mask3_round, "__builtin_ia32_vfmaddsh3_mask3", IX86_BUILTIN_VFMADDSH3_MASK3, UNKNOWN, (int) V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT) +BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512f_vmfmadd_v8hf_maskz_round, "__builtin_ia32_vfmaddsh3_maskz", IX86_BUILTIN_VFMADDSH3_MASKZ, UNKNOWN, (int) V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT) +BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512f_vmfnmadd_v8hf_mask_round, "__builtin_ia32_vfnmaddsh3_mask", IX86_BUILTIN_VFNMADDSH3_MASK, UNKNOWN, (int) V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT) +BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512f_vmfnmadd_v8hf_mask3_round, "__builtin_ia32_vfnmaddsh3_mask3", IX86_BUILTIN_VFNMADDSH3_MASK3, UNKNOWN, (int) V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT) +BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512f_vmfnmadd_v8hf_maskz_round, "__builtin_ia32_vfnmaddsh3_maskz", IX86_BUILTIN_VFNMADDSH3_MASKZ, UNKNOWN, (int) V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT) +BDESC (0, OPTION_MASK_ISA2_AVX512FP16, CODE_FOR_avx512f_vmfmsub_v8hf_mask3_round, "__builtin_ia32_vfmsubsh3_mask3", IX86_BUILTIN_VFMSUBSH3_MASK3, UNKNOWN, (int) V8HF_FTYPE_V8HF_V8HF_V8HF_UQI_INT) BDESC_END (ROUND_ARGS, MULTI_ARG) diff --git a/gcc/config/i386/i386-expand.c b/gcc/config/i386/i386-expand.c index 006f4bec8db..f6de05c769a 100644 --- a/gcc/config/i386/i386-expand.c +++ b/gcc/config/i386/i386-expand.c @@ -10558,6 +10558,7 @@ ix86_expand_round_builtin (const struct builtin_description *d, case V8HF_FTYPE_V8DI_V8HF_UQI_INT: case V8HF_FTYPE_V8DF_V8HF_UQI_INT: case V16HF_FTYPE_V16SF_V16HF_UHI_INT: + case V8HF_FTYPE_V8HF_V8HF_V8HF_INT: nargs = 4; break; case V4SF_FTYPE_V4SF_V4SF_INT_INT: diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md index cbf1e75c0b2..31f8fc68c65 100644 --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -5049,60 +5049,60 @@ (define_insn "_fmsubadd__mask3" ;; high-order elements from the destination register. (define_expand "fmai_vmfmadd_" - [(set (match_operand:VF_128 0 "register_operand") - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand") - (match_operand:VF_128 2 "") - (match_operand:VF_128 3 "")) + [(set (match_operand:VFH_128 0 "register_operand") + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand") + (match_operand:VFH_128 2 "") + (match_operand:VFH_128 3 "")) (match_dup 1) (const_int 1)))] "TARGET_FMA") (define_expand "fmai_vmfmsub_" - [(set (match_operand:VF_128 0 "register_operand") - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand") - (match_operand:VF_128 2 "") - (neg:VF_128 - (match_operand:VF_128 3 ""))) + [(set (match_operand:VFH_128 0 "register_operand") + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand") + (match_operand:VFH_128 2 "") + (neg:VFH_128 + (match_operand:VFH_128 3 ""))) (match_dup 1) (const_int 1)))] "TARGET_FMA") (define_expand "fmai_vmfnmadd_" - [(set (match_operand:VF_128 0 "register_operand") - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "")) - (match_operand:VF_128 1 "register_operand") - (match_operand:VF_128 3 "")) + [(set (match_operand:VFH_128 0 "register_operand") + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "")) + (match_operand:VFH_128 1 "register_operand") + (match_operand:VFH_128 3 "")) (match_dup 1) (const_int 1)))] "TARGET_FMA") (define_expand "fmai_vmfnmsub_" - [(set (match_operand:VF_128 0 "register_operand") - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "")) - (match_operand:VF_128 1 "register_operand") - (neg:VF_128 - (match_operand:VF_128 3 ""))) + [(set (match_operand:VFH_128 0 "register_operand") + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "")) + (match_operand:VFH_128 1 "register_operand") + (neg:VFH_128 + (match_operand:VFH_128 3 ""))) (match_dup 1) (const_int 1)))] "TARGET_FMA") (define_insn "*fmai_fmadd_" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 2 "" ", v") - (match_operand:VF_128 3 "" "v,")) + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 2 "" ", v") + (match_operand:VFH_128 3 "" "v,")) (match_dup 1) (const_int 1)))] "TARGET_FMA || TARGET_AVX512F" @@ -5113,13 +5113,13 @@ (define_insn "*fmai_fmadd_" (set_attr "mode" "")]) (define_insn "*fmai_fmsub_" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 2 "" ",v") - (neg:VF_128 - (match_operand:VF_128 3 "" "v,"))) + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 2 "" ",v") + (neg:VFH_128 + (match_operand:VFH_128 3 "" "v,"))) (match_dup 1) (const_int 1)))] "TARGET_FMA || TARGET_AVX512F" @@ -5130,13 +5130,13 @@ (define_insn "*fmai_fmsub_" (set_attr "mode" "")]) (define_insn "*fmai_fnmadd_" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" ",v")) - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 3 "" "v,")) + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" ",v")) + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 3 "" "v,")) (match_dup 1) (const_int 1)))] "TARGET_FMA || TARGET_AVX512F" @@ -5147,14 +5147,14 @@ (define_insn "*fmai_fnmadd_" (set_attr "mode" "")]) (define_insn "*fmai_fnmsub_" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" ",v")) - (match_operand:VF_128 1 "register_operand" "0,0") - (neg:VF_128 - (match_operand:VF_128 3 "" "v,"))) + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" ",v")) + (match_operand:VFH_128 1 "register_operand" "0,0") + (neg:VFH_128 + (match_operand:VFH_128 3 "" "v,"))) (match_dup 1) (const_int 1)))] "TARGET_FMA || TARGET_AVX512F" @@ -5165,13 +5165,13 @@ (define_insn "*fmai_fnmsub_" (set_attr "mode" "")]) (define_insn "avx512f_vmfmadd__mask" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 2 "" ",v") - (match_operand:VF_128 3 "" "v,")) + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 2 "" ",v") + (match_operand:VFH_128 3 "" "v,")) (match_dup 1) (match_operand:QI 4 "register_operand" "Yk,Yk")) (match_dup 1) @@ -5184,13 +5184,13 @@ (define_insn "avx512f_vmfmadd__mask" (set_attr "mode" "")]) (define_insn "avx512f_vmfmadd__mask3" - [(set (match_operand:VF_128 0 "register_operand" "=v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "" "%v") - (match_operand:VF_128 2 "" "") - (match_operand:VF_128 3 "register_operand" "0")) + [(set (match_operand:VFH_128 0 "register_operand" "=v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "" "%v") + (match_operand:VFH_128 2 "" "") + (match_operand:VFH_128 3 "register_operand" "0")) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -5201,10 +5201,10 @@ (define_insn "avx512f_vmfmadd__mask3" (set_attr "mode" "")]) (define_expand "avx512f_vmfmadd__maskz" - [(match_operand:VF_128 0 "register_operand") - (match_operand:VF_128 1 "") - (match_operand:VF_128 2 "") - (match_operand:VF_128 3 "") + [(match_operand:VFH_128 0 "register_operand") + (match_operand:VFH_128 1 "") + (match_operand:VFH_128 2 "") + (match_operand:VFH_128 3 "") (match_operand:QI 4 "register_operand")] "TARGET_AVX512F" { @@ -5215,14 +5215,14 @@ (define_expand "avx512f_vmfmadd__maskz" }) (define_insn "avx512f_vmfmadd__maskz_1" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 2 "" ",v") - (match_operand:VF_128 3 "" "v,")) - (match_operand:VF_128 4 "const0_operand" "C,C") + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 2 "" ",v") + (match_operand:VFH_128 3 "" "v,")) + (match_operand:VFH_128 4 "const0_operand" "C,C") (match_operand:QI 5 "register_operand" "Yk,Yk")) (match_dup 1) (const_int 1)))] @@ -5234,14 +5234,14 @@ (define_insn "avx512f_vmfmadd__maskz_1" (set_attr "mode" "")]) (define_insn "*avx512f_vmfmsub__mask" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 2 "" ",v") - (neg:VF_128 - (match_operand:VF_128 3 "" "v,"))) + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 2 "" ",v") + (neg:VFH_128 + (match_operand:VFH_128 3 "" "v,"))) (match_dup 1) (match_operand:QI 4 "register_operand" "Yk,Yk")) (match_dup 1) @@ -5254,14 +5254,14 @@ (define_insn "*avx512f_vmfmsub__mask" (set_attr "mode" "")]) (define_insn "avx512f_vmfmsub__mask3" - [(set (match_operand:VF_128 0 "register_operand" "=v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "" "%v") - (match_operand:VF_128 2 "" "") - (neg:VF_128 - (match_operand:VF_128 3 "register_operand" "0"))) + [(set (match_operand:VFH_128 0 "register_operand" "=v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "" "%v") + (match_operand:VFH_128 2 "" "") + (neg:VFH_128 + (match_operand:VFH_128 3 "register_operand" "0"))) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -5272,15 +5272,15 @@ (define_insn "avx512f_vmfmsub__mask3" (set_attr "mode" "")]) (define_insn "*avx512f_vmfmsub__maskz_1" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 2 "" ",v") - (neg:VF_128 - (match_operand:VF_128 3 "" "v,"))) - (match_operand:VF_128 4 "const0_operand" "C,C") + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 2 "" ",v") + (neg:VFH_128 + (match_operand:VFH_128 3 "" "v,"))) + (match_operand:VFH_128 4 "const0_operand" "C,C") (match_operand:QI 5 "register_operand" "Yk,Yk")) (match_dup 1) (const_int 1)))] @@ -5291,15 +5291,15 @@ (define_insn "*avx512f_vmfmsub__maskz_1" [(set_attr "type" "ssemuladd") (set_attr "mode" "")]) -(define_insn "*avx512f_vmfnmadd__mask" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" ",v")) - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 3 "" "v,")) +(define_insn "avx512f_vmfnmadd__mask" + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" ",v")) + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 3 "" "v,")) (match_dup 1) (match_operand:QI 4 "register_operand" "Yk,Yk")) (match_dup 1) @@ -5311,15 +5311,15 @@ (define_insn "*avx512f_vmfnmadd__mask" [(set_attr "type" "ssemuladd") (set_attr "mode" "")]) -(define_insn "*avx512f_vmfnmadd__mask3" - [(set (match_operand:VF_128 0 "register_operand" "=v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" "")) - (match_operand:VF_128 1 "" "%v") - (match_operand:VF_128 3 "register_operand" "0")) +(define_insn "avx512f_vmfnmadd__mask3" + [(set (match_operand:VFH_128 0 "register_operand" "=v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" "")) + (match_operand:VFH_128 1 "" "%v") + (match_operand:VFH_128 3 "register_operand" "0")) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -5329,16 +5329,30 @@ (define_insn "*avx512f_vmfnmadd__mask3" [(set_attr "type" "ssemuladd") (set_attr "mode" "")]) -(define_insn "*avx512f_vmfnmadd__maskz_1" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" ",v")) - (match_operand:VF_128 1 "register_operand" "0,0") - (match_operand:VF_128 3 "" "v,")) - (match_operand:VF_128 4 "const0_operand" "C,C") +(define_expand "avx512f_vmfnmadd__maskz" + [(match_operand:VFH_128 0 "register_operand") + (match_operand:VFH_128 1 "") + (match_operand:VFH_128 2 "") + (match_operand:VFH_128 3 "") + (match_operand:QI 4 "register_operand")] + "TARGET_AVX512F" +{ + emit_insn (gen_avx512f_vmfnmadd__maskz_1 ( + operands[0], operands[1], operands[2], operands[3], + CONST0_RTX (mode), operands[4])); + DONE; +}) + +(define_insn "avx512f_vmfnmadd__maskz_1" + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" ",v")) + (match_operand:VFH_128 1 "register_operand" "0,0") + (match_operand:VFH_128 3 "" "v,")) + (match_operand:VFH_128 4 "const0_operand" "C,C") (match_operand:QI 5 "register_operand" "Yk,Yk")) (match_dup 1) (const_int 1)))] @@ -5350,15 +5364,15 @@ (define_insn "*avx512f_vmfnmadd__maskz_1" (set_attr "mode" "")]) (define_insn "*avx512f_vmfnmsub__mask" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" ",v")) - (match_operand:VF_128 1 "register_operand" "0,0") - (neg:VF_128 - (match_operand:VF_128 3 "" "v,"))) + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" ",v")) + (match_operand:VFH_128 1 "register_operand" "0,0") + (neg:VFH_128 + (match_operand:VFH_128 3 "" "v,"))) (match_dup 1) (match_operand:QI 4 "register_operand" "Yk,Yk")) (match_dup 1) @@ -5371,15 +5385,15 @@ (define_insn "*avx512f_vmfnmsub__mask" (set_attr "mode" "")]) (define_insn "*avx512f_vmfnmsub__mask3" - [(set (match_operand:VF_128 0 "register_operand" "=v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" "")) - (match_operand:VF_128 1 "" "%v") - (neg:VF_128 - (match_operand:VF_128 3 "register_operand" "0"))) + [(set (match_operand:VFH_128 0 "register_operand" "=v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" "")) + (match_operand:VFH_128 1 "" "%v") + (neg:VFH_128 + (match_operand:VFH_128 3 "register_operand" "0"))) (match_dup 3) (match_operand:QI 4 "register_operand" "Yk")) (match_dup 3) @@ -5390,16 +5404,16 @@ (define_insn "*avx512f_vmfnmsub__mask3" (set_attr "mode" "")]) (define_insn "*avx512f_vmfnmsub__maskz_1" - [(set (match_operand:VF_128 0 "register_operand" "=v,v") - (vec_merge:VF_128 - (vec_merge:VF_128 - (fma:VF_128 - (neg:VF_128 - (match_operand:VF_128 2 "" ",v")) - (match_operand:VF_128 1 "register_operand" "0,0") - (neg:VF_128 - (match_operand:VF_128 3 "" "v,"))) - (match_operand:VF_128 4 "const0_operand" "C,C") + [(set (match_operand:VFH_128 0 "register_operand" "=v,v") + (vec_merge:VFH_128 + (vec_merge:VFH_128 + (fma:VFH_128 + (neg:VFH_128 + (match_operand:VFH_128 2 "" ",v")) + (match_operand:VFH_128 1 "register_operand" "0,0") + (neg:VFH_128 + (match_operand:VFH_128 3 "" "v,"))) + (match_operand:VFH_128 4 "const0_operand" "C,C") (match_operand:QI 5 "register_operand" "Yk,Yk")) (match_dup 1) (const_int 1)))] diff --git a/gcc/testsuite/gcc.target/i386/avx-1.c b/gcc/testsuite/gcc.target/i386/avx-1.c index d2ab16538d8..6c2d1dc3df4 100644 --- a/gcc/testsuite/gcc.target/i386/avx-1.c +++ b/gcc/testsuite/gcc.target/i386/avx-1.c @@ -775,6 +775,18 @@ #define __builtin_ia32_vfnmsubph512_mask(A, B, C, D, E) __builtin_ia32_vfnmsubph512_mask(A, B, C, D, 8) #define __builtin_ia32_vfnmsubph512_mask3(A, B, C, D, E) __builtin_ia32_vfnmsubph512_mask3(A, B, C, D, 8) #define __builtin_ia32_vfnmsubph512_maskz(A, B, C, D, E) __builtin_ia32_vfnmsubph512_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_mask(A, B, C, D, E) __builtin_ia32_vfmaddsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_mask3(A, B, C, D, E) __builtin_ia32_vfmaddsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_maskz(A, B, C, D, E) __builtin_ia32_vfmaddsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_mask(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_mask3(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_maskz(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_mask(A, B, C, D, E) __builtin_ia32_vfmsubsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_mask3(A, B, C, D, E) __builtin_ia32_vfmsubsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_maskz(A, B, C, D, E) __builtin_ia32_vfmsubsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_mask(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_mask3(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_maskz(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_maskz(A, B, C, D, 8) /* avx512fp16vlintrin.h */ #define __builtin_ia32_vcmpph_v8hf_mask(A, B, C, D) __builtin_ia32_vcmpph_v8hf_mask(A, B, 1, D) diff --git a/gcc/testsuite/gcc.target/i386/sse-13.c b/gcc/testsuite/gcc.target/i386/sse-13.c index 49c72f6fcef..f16be008909 100644 --- a/gcc/testsuite/gcc.target/i386/sse-13.c +++ b/gcc/testsuite/gcc.target/i386/sse-13.c @@ -792,6 +792,18 @@ #define __builtin_ia32_vfnmsubph512_mask(A, B, C, D, E) __builtin_ia32_vfnmsubph512_mask(A, B, C, D, 8) #define __builtin_ia32_vfnmsubph512_mask3(A, B, C, D, E) __builtin_ia32_vfnmsubph512_mask3(A, B, C, D, 8) #define __builtin_ia32_vfnmsubph512_maskz(A, B, C, D, E) __builtin_ia32_vfnmsubph512_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_mask(A, B, C, D, E) __builtin_ia32_vfmaddsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_mask3(A, B, C, D, E) __builtin_ia32_vfmaddsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_maskz(A, B, C, D, E) __builtin_ia32_vfmaddsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_mask(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_mask3(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_maskz(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_mask(A, B, C, D, E) __builtin_ia32_vfmsubsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_mask3(A, B, C, D, E) __builtin_ia32_vfmsubsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_maskz(A, B, C, D, E) __builtin_ia32_vfmsubsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_mask(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_mask3(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_maskz(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_maskz(A, B, C, D, 8) /* avx512fp16vlintrin.h */ #define __builtin_ia32_vcmpph_v8hf_mask(A, B, C, D) __builtin_ia32_vcmpph_v8hf_mask(A, B, 1, D) diff --git a/gcc/testsuite/gcc.target/i386/sse-14.c b/gcc/testsuite/gcc.target/i386/sse-14.c index 9151e50afd2..01ac4e04173 100644 --- a/gcc/testsuite/gcc.target/i386/sse-14.c +++ b/gcc/testsuite/gcc.target/i386/sse-14.c @@ -842,6 +842,10 @@ test_3 (_mm512_fmadd_round_ph, __m512h, __m512h, __m512h, __m512h, 9) test_3 (_mm512_fnmadd_round_ph, __m512h, __m512h, __m512h, __m512h, 9) test_3 (_mm512_fmsub_round_ph, __m512h, __m512h, __m512h, __m512h, 9) test_3 (_mm512_fnmsub_round_ph, __m512h, __m512h, __m512h, __m512h, 9) +test_3 (_mm_fmadd_round_sh, __m128h, __m128h, __m128h, __m128h, 9) +test_3 (_mm_fnmadd_round_sh, __m128h, __m128h, __m128h, __m128h, 9) +test_3 (_mm_fmsub_round_sh, __m128h, __m128h, __m128h, __m128h, 9) +test_3 (_mm_fnmsub_round_sh, __m128h, __m128h, __m128h, __m128h, 9) test_3x (_mm512_mask_cmp_round_ph_mask, __mmask32, __mmask32, __m512h, __m512h, 1, 8) test_3x (_mm_mask_cmp_round_sh_mask, __mmask8, __mmask8, __m128h, __m128h, 1, 8) test_3x (_mm512_mask_reduce_round_ph, __m512h, __m512h, __mmask32, __m512h, 123, 8) @@ -892,6 +896,18 @@ test_4 (_mm512_maskz_fmsub_round_ph, __m512h, __mmask32, __m512h, __m512h, __m51 test_4 (_mm512_mask_fnmsub_round_ph, __m512h, __m512h, __mmask32, __m512h, __m512h, 9) test_4 (_mm512_mask3_fnmsub_round_ph, __m512h, __m512h, __m512h, __m512h, __mmask32, 9) test_4 (_mm512_maskz_fnmsub_round_ph, __m512h, __mmask32, __m512h, __m512h, __m512h, 9) +test_4 (_mm_mask_fmadd_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fmadd_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fmadd_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) +test_4 (_mm_mask_fnmadd_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fnmadd_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fnmadd_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) +test_4 (_mm_mask_fmsub_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fmsub_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fmsub_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) +test_4 (_mm_mask_fnmsub_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fnmsub_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fnmsub_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) test_4x (_mm_mask_reduce_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 123, 8) test_4x (_mm_mask_roundscale_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 123, 8) test_4x (_mm_mask_getmant_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 1, 1) diff --git a/gcc/testsuite/gcc.target/i386/sse-22.c b/gcc/testsuite/gcc.target/i386/sse-22.c index 892b6334ae2..79e3f35ab86 100644 --- a/gcc/testsuite/gcc.target/i386/sse-22.c +++ b/gcc/testsuite/gcc.target/i386/sse-22.c @@ -945,6 +945,10 @@ test_3 (_mm512_fmadd_round_ph, __m512h, __m512h, __m512h, __m512h, 9) test_3 (_mm512_fnmadd_round_ph, __m512h, __m512h, __m512h, __m512h, 9) test_3 (_mm512_fmsub_round_ph, __m512h, __m512h, __m512h, __m512h, 9) test_3 (_mm512_fnmsub_round_ph, __m512h, __m512h, __m512h, __m512h, 9) +test_3 (_mm_fmadd_round_sh, __m128h, __m128h, __m128h, __m128h, 9) +test_3 (_mm_fnmadd_round_sh, __m128h, __m128h, __m128h, __m128h, 9) +test_3 (_mm_fmsub_round_sh, __m128h, __m128h, __m128h, __m128h, 9) +test_3 (_mm_fnmsub_round_sh, __m128h, __m128h, __m128h, __m128h, 9) test_3x (_mm512_mask_cmp_round_ph_mask, __mmask32, __mmask32, __m512h, __m512h, 1, 8) test_3x (_mm_mask_cmp_round_sh_mask, __mmask8, __mmask8, __m128h, __m128h, 1, 8) test_3x (_mm512_mask_reduce_round_ph, __m512h, __m512h, __mmask32, __m512h, 123, 8) @@ -994,6 +998,18 @@ test_4 (_mm512_maskz_fmsub_round_ph, __m512h, __mmask32, __m512h, __m512h, __m51 test_4 (_mm512_mask_fnmsub_round_ph, __m512h, __m512h, __mmask32, __m512h, __m512h, 9) test_4 (_mm512_mask3_fnmsub_round_ph, __m512h, __m512h, __m512h, __m512h, __mmask32, 9) test_4 (_mm512_maskz_fnmsub_round_ph, __m512h, __mmask32, __m512h, __m512h, __m512h, 9) +test_4 (_mm_mask_fmadd_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fmadd_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fmadd_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) +test_4 (_mm_mask_fnmadd_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fnmadd_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fnmadd_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) +test_4 (_mm_mask_fmsub_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fmsub_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fmsub_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) +test_4 (_mm_mask_fnmsub_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 9) +test_4 (_mm_mask3_fnmsub_round_sh, __m128h, __m128h, __m128h, __m128h, __mmask8, 9) +test_4 (_mm_maskz_fnmsub_round_sh, __m128h, __mmask8, __m128h, __m128h, __m128h, 9) test_4x (_mm_mask_reduce_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 123, 8) test_4x (_mm_mask_roundscale_round_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 123, 8) test_4x (_mm_mask_getmant_sh, __m128h, __m128h, __mmask8, __m128h, __m128h, 1, 1) diff --git a/gcc/testsuite/gcc.target/i386/sse-23.c b/gcc/testsuite/gcc.target/i386/sse-23.c index 447b83829f3..caf14408b91 100644 --- a/gcc/testsuite/gcc.target/i386/sse-23.c +++ b/gcc/testsuite/gcc.target/i386/sse-23.c @@ -793,6 +793,18 @@ #define __builtin_ia32_vfnmsubph512_mask(A, B, C, D, E) __builtin_ia32_vfnmsubph512_mask(A, B, C, D, 8) #define __builtin_ia32_vfnmsubph512_mask3(A, B, C, D, E) __builtin_ia32_vfnmsubph512_mask3(A, B, C, D, 8) #define __builtin_ia32_vfnmsubph512_maskz(A, B, C, D, E) __builtin_ia32_vfnmsubph512_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_mask(A, B, C, D, E) __builtin_ia32_vfmaddsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_mask3(A, B, C, D, E) __builtin_ia32_vfmaddsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfmaddsh3_maskz(A, B, C, D, E) __builtin_ia32_vfmaddsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_mask(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_mask3(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfnmaddsh3_maskz(A, B, C, D, E) __builtin_ia32_vfnmaddsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_mask(A, B, C, D, E) __builtin_ia32_vfmsubsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_mask3(A, B, C, D, E) __builtin_ia32_vfmsubsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfmsubsh3_maskz(A, B, C, D, E) __builtin_ia32_vfmsubsh3_maskz(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_mask(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_mask(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_mask3(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_mask3(A, B, C, D, 8) +#define __builtin_ia32_vfnmsubsh3_maskz(A, B, C, D, E) __builtin_ia32_vfnmsubsh3_maskz(A, B, C, D, 8) /* avx512fp16vlintrin.h */ #define __builtin_ia32_vcmpph_v8hf_mask(A, B, C, D) __builtin_ia32_vcmpph_v8hf_mask(A, B, 1, D)