| Message ID | patch-18055-tamar@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | middle-end: correct loop bounds for early breaks and peeled vector loops | expand |
On Wed, 6 Dec 2023, Tamar Christina wrote: > Hi All, > > While waiting for reviews I've continued to run more test. > In this case this was one found running 32-bit systems. > > While we calculate the right latch count for the epilog, > the vectorizer overrides SCEV and so unrolling goes wrong. > > This updates the bounds for the case where we've peeled a > vector iteration. > > Testcase in early break testsuite adjusted to test for this > and I'll merge this commit in the main one. > > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. > > Ok for master? OK. > Thanks, > Tamar > > gcc/ChangeLog: > > * tree-vect-loop-manip.cc (vect_do_peeling): Adjust bounds. > > --- inline copy of patch -- > diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc > index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644 > --- a/gcc/tree-vect-loop-manip.cc > +++ b/gcc/tree-vect-loop-manip.cc > @@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, > if (bound_scalar.is_constant (&bound)) > { > gcc_assert (bound != 0); > + /* Adjust the upper bound by the extra peeled vector iteration if we > + are an epilogue of an peeled vect loop and not VLA. For VLA the > + loop bounds are unknown. */ > + if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo) > + && vf.is_constant ()) > + bound += vf.to_constant (); > /* -1 to convert loop iterations to latch iterations. */ > record_niter_bound (epilog, bound - 1, false, true); > scale_loop_profile (epilog, profile_probability::always (), > > > > >
--- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, if (bound_scalar.is_constant (&bound)) { gcc_assert (bound != 0); + /* Adjust the upper bound by the extra peeled vector iteration if we + are an epilogue of an peeled vect loop and not VLA. For VLA the + loop bounds are unknown. */ + if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo) + && vf.is_constant ()) + bound += vf.to_constant (); /* -1 to convert loop iterations to latch iterations. */ record_niter_bound (epilog, bound - 1, false, true); scale_loop_profile (epilog, profile_probability::always (),
Hi All, While waiting for reviews I've continued to run more test. In this case this was one found running 32-bit systems. While we calculate the right latch count for the epilog, the vectorizer overrides SCEV and so unrolling goes wrong. This updates the bounds for the case where we've peeled a vector iteration. Testcase in early break testsuite adjusted to test for this and I'll merge this commit in the main one. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-loop-manip.cc (vect_do_peeling): Adjust bounds. --- inline copy of patch -- diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644 -- diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, if (bound_scalar.is_constant (&bound)) { gcc_assert (bound != 0); + /* Adjust the upper bound by the extra peeled vector iteration if we + are an epilogue of an peeled vect loop and not VLA. For VLA the + loop bounds are unknown. */ + if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo) + && vf.is_constant ()) + bound += vf.to_constant (); /* -1 to convert loop iterations to latch iterations. */ record_niter_bound (epilog, bound - 1, false, true); scale_loop_profile (epilog, profile_probability::always (),