| Message ID | 20230801130403.164060-1-berrange@redhat.com |
|---|---|
| Headers | show |
| Series | gitlab: speed up msys windows jobs with GCC | expand |
Daniel P. Berrangé <berrange@redhat.com> writes: > This is an alternative and/or complementary to Thomas' proposal > to use CLang with msys: > > https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.html > > First of all, the current msys installer we're using is over 12 > months out of date. Thus after running the install, pacman then > replaces most of what we've just installed with new downloaded > content. Using the most update installer cuts 3+1/2 minutes off > the msys install time - 7 minutes becomes 3+1/2. > > Secondly, QEMU defaults to compiling with -O2 and this is more > computationally expensive for GCC. Switching to -O0 drops the > build time from 60 minutes down to 45 minutes. From the fine manual[*]: "The effectiveness of some warnings depends on optimizations also being enabled. For example '-Wsuggest-final-types' is more effective with link-time optimization and some instances of other warnings may not be issued at all unless optimization is enabled. While optimization in general improves the efficacy of control and data flow sensitive warnings, in some cases it may also cause false positives." Do we care? [...] [*] https://gcc.gnu.org/onlinedocs/gcc-13.2.0/gcc/Warning-Options.html
On Tue, Aug 01, 2023 at 03:53:22PM +0200, Markus Armbruster wrote: > Daniel P. Berrangé <berrange@redhat.com> writes: > > > This is an alternative and/or complementary to Thomas' proposal > > to use CLang with msys: > > > > https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.html > > > > First of all, the current msys installer we're using is over 12 > > months out of date. Thus after running the install, pacman then > > replaces most of what we've just installed with new downloaded > > content. Using the most update installer cuts 3+1/2 minutes off > > the msys install time - 7 minutes becomes 3+1/2. > > > > Secondly, QEMU defaults to compiling with -O2 and this is more > > computationally expensive for GCC. Switching to -O0 drops the > > build time from 60 minutes down to 45 minutes. > > From the fine manual[*]: "The effectiveness of some warnings depends on > optimizations also being enabled. For example '-Wsuggest-final-types' > is more effective with link-time optimization and some instances of > other warnings may not be issued at all unless optimization is enabled. > While optimization in general improves the efficacy of control and data > flow sensitive warnings, in some cases it may also cause false > positives." Do we care? In general, yes, we do care. In this specific case though, we're battling to figure out the lesser of multiple evils. Right now we configure with: --target-list=x86_64-softmmu --without-default-devices and so with optimization enabled, we'll get good warning coverage of a small amount of code, except we don't because people started ignoring the msys jobs as they timeout too frequently. If we can use Thomas' clang switch or my -O0 patches, we can get within the timeout, so people can trust the job once again. If we can do both ideas and cut the time even more, then we can enable more features (perhaps drop --without-default-devices). So the warnings might not be quite as good, but we'll have the warnings across a larger amount of code. Alot of the warnings from the Linux/macOS builds will also apply in the Windows builds. I think on balance I'd probably prefer us to build a larger amount of code for Windows. This is in context of free shared runners at least. As a more drastic option, we might need to consider using the Azure credits for Windows runners too. If we could have bigger VMs for Windows CI, we can build more and have better warnings at the same time. With regards, Daniel
On 01/08/2023 16.35, Daniel P. Berrangé wrote: > On Tue, Aug 01, 2023 at 03:53:22PM +0200, Markus Armbruster wrote: >> Daniel P. Berrangé <berrange@redhat.com> writes: >> >>> This is an alternative and/or complementary to Thomas' proposal >>> to use CLang with msys: >>> >>> https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.html >>> >>> First of all, the current msys installer we're using is over 12 >>> months out of date. Thus after running the install, pacman then >>> replaces most of what we've just installed with new downloaded >>> content. Using the most update installer cuts 3+1/2 minutes off >>> the msys install time - 7 minutes becomes 3+1/2. >>> >>> Secondly, QEMU defaults to compiling with -O2 and this is more >>> computationally expensive for GCC. Switching to -O0 drops the >>> build time from 60 minutes down to 45 minutes. >> >> From the fine manual[*]: "The effectiveness of some warnings depends on >> optimizations also being enabled. For example '-Wsuggest-final-types' >> is more effective with link-time optimization and some instances of >> other warnings may not be issued at all unless optimization is enabled. >> While optimization in general improves the efficacy of control and data >> flow sensitive warnings, in some cases it may also cause false >> positives." Do we care? > > In general, yes, we do care. > > In this specific case though, we're battling to figure out the lesser > of multiple evils. I agree. Additionally, we also test compiling for Windows with the MinGW cross compiler suite in a Fedora container, and we still use the default optimization there, so we should have that covered. Thomas
On 01/08/2023 15.03, Daniel P. Berrangé wrote: > This is an alternative and/or complementary to Thomas' proposal > to use CLang with msys: > > https://lists.gnu.org/archive/html/qemu-devel/2023-07/msg05402.html > > First of all, the current msys installer we're using is over 12 > months out of date. Thus after running the install, pacman then > replaces most of what we've just installed with new downloaded > content. Using the most update installer cuts 3+1/2 minutes off > the msys install time - 7 minutes becomes 3+1/2. > > Secondly, QEMU defaults to compiling with -O2 and this is more > computationally expensive for GCC. Switching to -O0 drops the > build time from 60 minutes down to 45 minutes. > > Thirdly, including debug symbols also has an overhead, and turning > that off reduces time still further down to 38 minutes. > > IOW, between all three changes, we can cut approx 25-26 minutes > off the job execution time, bringing it nicely within the job > timeout. > > The actually phase of installing the mingw deps still accounts > for about 10 minutes and has not been optimized. > > Possibly the same trick of -O0 and skipping -g would also help > the clang alternative Thomas' proposed. If so, that could be > enough to let us enable more features / targets during the > msys build. I really like the idea! And I guess my idea with Clang needs some more work 'til it is acceptable, so let's go with your idea for now to fix the timeout problem in the CI ... we can still optimize later with Clang in case we found a good solution for that ms_struct problem... Thomas