| Message ID | ZUiYCIg0KPALrH50@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | None | expand |
On Mon, 6 Nov 2023, Tamar Christina wrote: > Hi All, > > This has loop versioning use the vectorizer's IV exit edge when it's available > since single_exit (..) fails with multiple exits. > > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. > > Ok for master? > > Thanks, > Tamar > > gcc/ChangeLog: > > * tree-vect-loop-manip.cc (vect_loop_versioning): Support multiple > exits. > > --- inline copy of patch -- > diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc > index 3d59119787d6afdc5a6465a547d1ea2d3d940373..58b4b9c11d8b844ee86156cdfcba7f838030a7c2 100644 > --- a/gcc/tree-vect-loop-manip.cc > +++ b/gcc/tree-vect-loop-manip.cc > @@ -4180,12 +4180,24 @@ vect_loop_versioning (loop_vec_info loop_vinfo, > If loop versioning wasn't done from loop, but scalar_loop instead, > merge_bb will have already just a single successor. */ > > - merge_bb = single_exit (loop_to_version)->dest; > + /* Due to the single_exit check above we should only get here when > + loop == loop_to_version, that means we can use loop_vinfo to get the > + exits. */ You mean LOOP_VINFO_EARLY_BREAKS can only ever version the loop itself? That's correct. All inner loops of loop_to_version have a single exit unless it's loop itself. Please reword a bit and instead do edge exit_edge; if (loop_to_version == loop) exit_edge = LOOP_VINFO_IV_EXIT (loop_vinfo); else exit_edge = single_exit (loop_to_version); OK with that change. Richard. > + edge exit_edge = single_exit (loop_to_version); > + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) > + { > + /* In early exits the main exit will fail into the merge block of the > + alternative exits. So we need the single successor of the main > + exit here to find the merge block. */ > + exit_edge = LOOP_VINFO_IV_EXIT (loop_vinfo); > + } > + gcc_assert (exit_edge); > + merge_bb = exit_edge->dest; > if (EDGE_COUNT (merge_bb->preds) >= 2) > { > gcc_assert (EDGE_COUNT (merge_bb->preds) >= 2); > - new_exit_bb = split_edge (single_exit (loop_to_version)); > - new_exit_e = single_exit (loop_to_version); > + new_exit_bb = split_edge (exit_edge); > + new_exit_e = exit_edge; > e = EDGE_SUCC (new_exit_bb, 0); > > for (gsi = gsi_start_phis (merge_bb); !gsi_end_p (gsi);
--- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -4180,12 +4180,24 @@ vect_loop_versioning (loop_vec_info loop_vinfo, If loop versioning wasn't done from loop, but scalar_loop instead, merge_bb will have already just a single successor. */ - merge_bb = single_exit (loop_to_version)->dest; + /* Due to the single_exit check above we should only get here when + loop == loop_to_version, that means we can use loop_vinfo to get the + exits. */ + edge exit_edge = single_exit (loop_to_version); + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) + { + /* In early exits the main exit will fail into the merge block of the + alternative exits. So we need the single successor of the main + exit here to find the merge block. */ + exit_edge = LOOP_VINFO_IV_EXIT (loop_vinfo); + } + gcc_assert (exit_edge); + merge_bb = exit_edge->dest; if (EDGE_COUNT (merge_bb->preds) >= 2) { gcc_assert (EDGE_COUNT (merge_bb->preds) >= 2); - new_exit_bb = split_edge (single_exit (loop_to_version)); - new_exit_e = single_exit (loop_to_version); + new_exit_bb = split_edge (exit_edge); + new_exit_e = exit_edge; e = EDGE_SUCC (new_exit_bb, 0); for (gsi = gsi_start_phis (merge_bb); !gsi_end_p (gsi);
Hi All, This has loop versioning use the vectorizer's IV exit edge when it's available since single_exit (..) fails with multiple exits. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-loop-manip.cc (vect_loop_versioning): Support multiple exits. --- inline copy of patch -- diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 3d59119787d6afdc5a6465a547d1ea2d3d940373..58b4b9c11d8b844ee86156cdfcba7f838030a7c2 100644 -- diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 3d59119787d6afdc5a6465a547d1ea2d3d940373..58b4b9c11d8b844ee86156cdfcba7f838030a7c2 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -4180,12 +4180,24 @@ vect_loop_versioning (loop_vec_info loop_vinfo, If loop versioning wasn't done from loop, but scalar_loop instead, merge_bb will have already just a single successor. */ - merge_bb = single_exit (loop_to_version)->dest; + /* Due to the single_exit check above we should only get here when + loop == loop_to_version, that means we can use loop_vinfo to get the + exits. */ + edge exit_edge = single_exit (loop_to_version); + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) + { + /* In early exits the main exit will fail into the merge block of the + alternative exits. So we need the single successor of the main + exit here to find the merge block. */ + exit_edge = LOOP_VINFO_IV_EXIT (loop_vinfo); + } + gcc_assert (exit_edge); + merge_bb = exit_edge->dest; if (EDGE_COUNT (merge_bb->preds) >= 2) { gcc_assert (EDGE_COUNT (merge_bb->preds) >= 2); - new_exit_bb = split_edge (single_exit (loop_to_version)); - new_exit_e = single_exit (loop_to_version); + new_exit_bb = split_edge (exit_edge); + new_exit_e = exit_edge; e = EDGE_SUCC (new_exit_bb, 0); for (gsi = gsi_start_phis (merge_bb); !gsi_end_p (gsi);