diff mbox series

[pushed] aarch64: Fix gather x32/x64 selection

Message ID mptikvkk5y9.fsf@arm.com
State New
Headers show
Series [pushed] aarch64: Fix gather x32/x64 selection | expand

Commit Message

Richard Sandiford Aug. 28, 2024, 3:44 p.m. UTC 
The SVE gather and scatter costs are classified based on whether
they do 4 loads per 128 bits (x32) or 2 loads per 128 bits (x64).
The number after the "x" refers to the number of bits in each
"container".

However, the test for which to use was based on the element size
rather than the container size.  This meant that we'd use the
overly conservative x32 costs for VNx2SI gathers.  VNx2SI gathers
are really .D gathers in which the upper half of each extension
result is ignored.

This patch is necessary to switch -mtune=generic over to the
"new" vector costs.

Tested on aarch64-linux-gnu.  Pushed as previously agreed with
Tamar and Kyrill.

Richard


gcc/
	* config/aarch64/aarch64.cc (aarch64_detect_vector_stmt_subtype)
	(aarch64_vector_costs::add_stmt_cost): Use the x64 cost rather
	than x32 cost for all VNx2 modes.
---
 gcc/config/aarch64/aarch64.cc | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
diff mbox series

Patch

diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index 40dacfcf2e7..033ea61d3a8 100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -16819,7 +16819,8 @@  aarch64_detect_vector_stmt_subtype (vec_info *vinfo, vect_cost_for_stmt kind,
       && STMT_VINFO_MEMORY_ACCESS_TYPE (stmt_info) == VMAT_GATHER_SCATTER)
     {
       unsigned int nunits = vect_nunits_for_cost (vectype);
-      if (GET_MODE_UNIT_BITSIZE (TYPE_MODE (vectype)) == 64)
+      /* Test for VNx2 modes, which have 64-bit containers.  */
+      if (known_eq (GET_MODE_NUNITS (TYPE_MODE (vectype)), aarch64_sve_vg))
 	return { sve_costs->gather_load_x64_cost, nunits };
       return { sve_costs->gather_load_x32_cost, nunits };
     }
@@ -17309,7 +17310,9 @@  aarch64_vector_costs::add_stmt_cost (int count, vect_cost_for_stmt kind,
 	  const sve_vec_cost *sve_costs = aarch64_tune_params.vec_costs->sve;
 	  if (sve_costs)
 	    {
-	      if (GET_MODE_UNIT_BITSIZE (TYPE_MODE (vectype)) == 64)
+	      /* Test for VNx2 modes, which have 64-bit containers.  */
+	      if (known_eq (GET_MODE_NUNITS (TYPE_MODE (vectype)),
+			    aarch64_sve_vg))
 		m_sve_gather_scatter_init_cost
 		  += sve_costs->gather_load_x64_init_cost;
 	      else