| Message ID | 20240625075606.1517262-1-j@lambda.is |
|---|---|
| State | New |
| Headers | show |
| Series | Add function filtering to gcov | expand |
On 6/25/24 1:56 AM, Jørgen Kvalsvik wrote: > Add the --include and --exclude flags to gcov to control what functions > to report on. This is meant to make gcov more practical as an when > writing test suites or performing other coverage experiments, which > tends to focus on a few functions at the time. This really shines in > combination with the -t/--stdout flag. With support for more expansive > metrics in gcov like modified condition/decision coverage (MC/DC) and > path coverage, output quickly gets overwhelming without filtering. > > The approach is quite simple: filters are egrep regexes and are > evaluated left-to-right, and the last filter "wins", that is, if a > function matches an --include and a subsequent --exclude, it should not > be included in the output. All of the output machinery works on the > function table, so by optionally (not) adding function makes the even > the json output work as expected, and only minor changes are needed to > suppress the filtered-out functions. > > Demo: math.c > > int mul (int a, int b) { > return a * b; > } > > int sub (int a, int b) { > return a - b; > } > > int sum (int a, int b) { > return a + b; > } > > Plain matches: > > $ gcov -t math --include=sum > -: 0:Source:filter.c > -: 0:Graph:filter.gcno > -: 0:Data:- > -: 0:Runs:0 > #####: 9:int sum (int a, int b) { > #####: 10: return a + b; > > $ gcov -t math --include=mul > -: 0:Source:filter.c > -: 0:Graph:filter.gcno > -: 0:Data:- > -: 0:Runs:0 > #####: 1:int mul (int a, int b) { > #####: 2: return a * b; > > Regex match: > > $ gcov -t math --include=su > -: 0:Source:filter.c > -: 0:Graph:filter.gcno > -: 0:Data:- > -: 0:Runs:0 > #####: 5:int sub (int a, int b) { > #####: 6: return a - b; > -: 7:} > #####: 9:int sum (int a, int b) { > #####: 10: return a + b; > > And similar for exclude: > > $ gcov -t math --exclude=sum > -: 0:Source:filter.c > -: 0:Graph:filter.gcno > -: 0:Data:- > -: 0:Runs:0 > #####: 1:int mul (int a, int b) { > #####: 2: return a * b; > -: 3:} > #####: 5:int sub (int a, int b) { > #####: 6: return a - b; > > And json, for good measure: > > $ gcov -t math --include=sum --json | jq ".files[].lines[]" > { > "line_number": 9, > "function_name": "sum", > "count": 0, > "unexecuted_block": true, > "block_ids": [], > "branches": [], > "calls": [] > } > { > "line_number": 10, > "function_name": "sum", > "count": 0, > "unexecuted_block": true, > "block_ids": [ > 2 > ], > "branches": [], > "calls": [] > } > > Note that the last function gets "clipped" when lines are associated to > functions, which means the closing brace is dropped from the report. I > hope this can be fixed, but considering it is not really a part of the > function body, the gcov report is "complete". > > Matching generally work well for mangled names, as the mangled names > also have the base symbol name in it. By default, functions are matched > by the mangled name, which means matching on base names always work as > expected. The -M flag makes the matching work on the demangled name > which is quite useful when you only want to report on specific > overloads and can use the full type names. > > Why not just use grep? grep is not really sufficient as grep is very > line oriented, and the reports that benefit the most from filtering > often unpredictably span multiple lines based on the state of coverage. > For example, a condition coverage report for 3 terms/6 outcomes only > outputs 1 line when all conditions are covered, and 7 with no lines > covered. > > gcc/ChangeLog: > > * doc/gcov.texi: Add --include, --exclude, --match-on-demangled > documentation. > * gcov.cc (struct fnfilter): New. > (print_usage): Add --include, --exclude, -M, > --match-on-demangled. > (process_args): Likewise. > (release_structures): Release filters. > (read_graph_file): Only add function_infos matching filters. > (output_lines): Likewise. > > gcc/testsuite/ChangeLog: > > * lib/gcov.exp: Add filtering test function. > * g++.dg/gcov/gcov-19.C: New test. > * g++.dg/gcov/gcov-20.C: New test. > * g++.dg/gcov/gcov-21.C: New test. > * gcc.misc-tests/gcov-25.c: New test. > * gcc.misc-tests/gcov-26.c: New test. > * gcc.misc-tests/gcov-27.c: New test. > * gcc.misc-tests/gcov-28.c: New test. I've likely mentioned it before, but the same capability with PGO would be useful. Hopefully if something like this ever gets implemented in the PGO space it'd utilize some of the same concepts. OK for the trunk. Thanks for your patience. jeff
On 7/10/24 01:50, Jeff Law wrote: > > > On 6/25/24 1:56 AM, Jørgen Kvalsvik wrote: >> Add the --include and --exclude flags to gcov to control what functions >> to report on. This is meant to make gcov more practical as an when >> writing test suites or performing other coverage experiments, which >> tends to focus on a few functions at the time. This really shines in >> combination with the -t/--stdout flag. With support for more expansive >> metrics in gcov like modified condition/decision coverage (MC/DC) and >> path coverage, output quickly gets overwhelming without filtering. >> >> The approach is quite simple: filters are egrep regexes and are >> evaluated left-to-right, and the last filter "wins", that is, if a >> function matches an --include and a subsequent --exclude, it should not >> be included in the output. All of the output machinery works on the >> function table, so by optionally (not) adding function makes the even >> the json output work as expected, and only minor changes are needed to >> suppress the filtered-out functions. >> >> Demo: math.c >> >> int mul (int a, int b) { >> return a * b; >> } >> >> int sub (int a, int b) { >> return a - b; >> } >> >> int sum (int a, int b) { >> return a + b; >> } >> >> Plain matches: >> >> $ gcov -t math --include=sum >> -: 0:Source:filter.c >> -: 0:Graph:filter.gcno >> -: 0:Data:- >> -: 0:Runs:0 >> #####: 9:int sum (int a, int b) { >> #####: 10: return a + b; >> >> $ gcov -t math --include=mul >> -: 0:Source:filter.c >> -: 0:Graph:filter.gcno >> -: 0:Data:- >> -: 0:Runs:0 >> #####: 1:int mul (int a, int b) { >> #####: 2: return a * b; >> >> Regex match: >> >> $ gcov -t math --include=su >> -: 0:Source:filter.c >> -: 0:Graph:filter.gcno >> -: 0:Data:- >> -: 0:Runs:0 >> #####: 5:int sub (int a, int b) { >> #####: 6: return a - b; >> -: 7:} >> #####: 9:int sum (int a, int b) { >> #####: 10: return a + b; >> >> And similar for exclude: >> >> $ gcov -t math --exclude=sum >> -: 0:Source:filter.c >> -: 0:Graph:filter.gcno >> -: 0:Data:- >> -: 0:Runs:0 >> #####: 1:int mul (int a, int b) { >> #####: 2: return a * b; >> -: 3:} >> #####: 5:int sub (int a, int b) { >> #####: 6: return a - b; >> >> And json, for good measure: >> >> $ gcov -t math --include=sum --json | jq ".files[].lines[]" >> { >> "line_number": 9, >> "function_name": "sum", >> "count": 0, >> "unexecuted_block": true, >> "block_ids": [], >> "branches": [], >> "calls": [] >> } >> { >> "line_number": 10, >> "function_name": "sum", >> "count": 0, >> "unexecuted_block": true, >> "block_ids": [ >> 2 >> ], >> "branches": [], >> "calls": [] >> } >> >> Note that the last function gets "clipped" when lines are associated to >> functions, which means the closing brace is dropped from the report. I >> hope this can be fixed, but considering it is not really a part of the >> function body, the gcov report is "complete". >> >> Matching generally work well for mangled names, as the mangled names >> also have the base symbol name in it. By default, functions are matched >> by the mangled name, which means matching on base names always work as >> expected. The -M flag makes the matching work on the demangled name >> which is quite useful when you only want to report on specific >> overloads and can use the full type names. >> >> Why not just use grep? grep is not really sufficient as grep is very >> line oriented, and the reports that benefit the most from filtering >> often unpredictably span multiple lines based on the state of coverage. >> For example, a condition coverage report for 3 terms/6 outcomes only >> outputs 1 line when all conditions are covered, and 7 with no lines >> covered. >> >> gcc/ChangeLog: >> >> * doc/gcov.texi: Add --include, --exclude, --match-on-demangled >> documentation. >> * gcov.cc (struct fnfilter): New. >> (print_usage): Add --include, --exclude, -M, >> --match-on-demangled. >> (process_args): Likewise. >> (release_structures): Release filters. >> (read_graph_file): Only add function_infos matching filters. >> (output_lines): Likewise. >> >> gcc/testsuite/ChangeLog: >> >> * lib/gcov.exp: Add filtering test function. >> * g++.dg/gcov/gcov-19.C: New test. >> * g++.dg/gcov/gcov-20.C: New test. >> * g++.dg/gcov/gcov-21.C: New test. >> * gcc.misc-tests/gcov-25.c: New test. >> * gcc.misc-tests/gcov-26.c: New test. >> * gcc.misc-tests/gcov-27.c: New test. >> * gcc.misc-tests/gcov-28.c: New test. > I've likely mentioned it before, but the same capability with PGO would > be useful. Hopefully if something like this ever gets implemented in > the PGO space it'd utilize some of the same concepts. > > > OK for the trunk. Thanks for your patience. > > jeff Thanks you -- before I move forward, I would like some feedback on adding this diff to the patch: diff --git a/gcc/gcov.cc b/gcc/gcov.cc index 9cdef19c461..055fa7e78ba 100644 --- a/gcc/gcov.cc +++ b/gcc/gcov.cc @@ -1613,6 +1613,12 @@ process_all_functions (void) } } + /* Make sure to include the last line for this function even when it + is not directly covered by a basic block, for example when } is on + its own line. */ + if (sources[fn->src].lines.size () <= fn->end_line) + sources[fn->src].lines.resize (fn->end_line + 1); + All gcov.exp still pass (for C, C++), and it doesn't seem to have any negative effects. The point is to include the last line of the function, even if it is not touched by a basic block (which it won't be, for the most part, because the function return happens before }). For filtering, the effect is positive because the full function is actually included, note the final }: $ gcov -t sum --include=sub sum.gcda:cannot open data file, assuming not executed -: 0:Source:sum.cc -: 0:Graph:sum.gcno -: 0:Data:- -: 0:Runs:0 #####: 5:int sub (int a, int b) { #####: 6: return a - b; -: 7:} Thoughts? Thanks, Jørgen
On 7/10/24 1:30 PM, Jørgen Kvalsvik wrote: > > Thanks you -- before I move forward, I would like some feedback on > adding this diff to the patch: > > diff --git a/gcc/gcov.cc b/gcc/gcov.cc > index 9cdef19c461..055fa7e78ba 100644 > --- a/gcc/gcov.cc > +++ b/gcc/gcov.cc > @@ -1613,6 +1613,12 @@ process_all_functions (void) > } > } > > + /* Make sure to include the last line for this function even > when it > + is not directly covered by a basic block, for example > when } is on > + its own line. */ > + if (sources[fn->src].lines.size () <= fn->end_line) > + sources[fn->src].lines.resize (fn->end_line + 1); > + > > All gcov.exp still pass (for C, C++), and it doesn't seem to have any > negative effects. The point is to include the last line of the function, > even if it is not touched by a basic block (which it won't be, for the > most part, because the function return happens before }). For filtering, > the effect is positive because the full function is actually included, > note the final }: > > $ gcov -t sum --include=sub > sum.gcda:cannot open data file, assuming not executed > -: 0:Source:sum.cc > -: 0:Graph:sum.gcno > -: 0:Data:- > -: 0:Runs:0 > #####: 5:int sub (int a, int b) { > #####: 6: return a - b; > -: 7:} > > Thoughts? It seems reasonable. This is all a bit fuzzy. You could return via the epilogue and we could conceptually mark the epilogue as being on the close curley from a line number standpoint. But then what do we do with sibling calls which are both calls and returns at the same time. Those we probably wouldn't want to associate with the close curley and that could be the only return path. Point being including that line even though it's not always associated with a basic block seems reasonable to me. jeff > > Thanks, > Jørgen
On 7/10/24 21:54, Jeff Law wrote: > > > On 7/10/24 1:30 PM, Jørgen Kvalsvik wrote: > >> >> Thanks you -- before I move forward, I would like some feedback on >> adding this diff to the patch: >> >> diff --git a/gcc/gcov.cc b/gcc/gcov.cc >> index 9cdef19c461..055fa7e78ba 100644 >> --- a/gcc/gcov.cc >> +++ b/gcc/gcov.cc >> @@ -1613,6 +1613,12 @@ process_all_functions (void) >> } >> } >> >> + /* Make sure to include the last line for this function even >> when it >> + is not directly covered by a basic block, for example >> when } is on >> + its own line. */ >> + if (sources[fn->src].lines.size () <= fn->end_line) >> + sources[fn->src].lines.resize (fn->end_line + 1); >> + >> >> All gcov.exp still pass (for C, C++), and it doesn't seem to have any >> negative effects. The point is to include the last line of the >> function, even if it is not touched by a basic block (which it won't >> be, for the most part, because the function return happens before }). >> For filtering, the effect is positive because the full function is >> actually included, note the final }: >> >> $ gcov -t sum --include=sub >> sum.gcda:cannot open data file, assuming not executed >> -: 0:Source:sum.cc >> -: 0:Graph:sum.gcno >> -: 0:Data:- >> -: 0:Runs:0 >> #####: 5:int sub (int a, int b) { >> #####: 6: return a - b; >> -: 7:} >> >> Thoughts? > It seems reasonable. This is all a bit fuzzy. > > You could return via the epilogue and we could conceptually mark the > epilogue as being on the close curley from a line number standpoint. But > then what do we do with sibling calls which are both calls and returns > at the same time. Those we probably wouldn't want to associate with the > close curley and that could be the only return path. Point being > including that line even though it's not always associated with a basic > block seems reasonable to me. I might be misunderstanding, but I don't think a sibling call would be affected by this, or rather, I don't think there's a change in line association here, and it should not affect the line counts. Line 7, for example, is not executed even though it is printed with --include. A (sibling) call would still be associated with a block anyway, right? Thanks, Jørgen > > jeff >> >> Thanks, >> Jørgen >
On 7/10/24 3:18 PM, Jørgen Kvalsvik wrote: > > > On 7/10/24 21:54, Jeff Law wrote: >> >> >> On 7/10/24 1:30 PM, Jørgen Kvalsvik wrote: >> >>> >>> Thanks you -- before I move forward, I would like some feedback on >>> adding this diff to the patch: >>> >>> diff --git a/gcc/gcov.cc b/gcc/gcov.cc >>> index 9cdef19c461..055fa7e78ba 100644 >>> --- a/gcc/gcov.cc >>> +++ b/gcc/gcov.cc >>> @@ -1613,6 +1613,12 @@ process_all_functions (void) >>> } >>> } >>> >>> + /* Make sure to include the last line for this function >>> even when it >>> + is not directly covered by a basic block, for example >>> when } is on >>> + its own line. */ >>> + if (sources[fn->src].lines.size () <= fn->end_line) >>> + sources[fn->src].lines.resize (fn->end_line + 1); >>> + >>> >>> All gcov.exp still pass (for C, C++), and it doesn't seem to have any >>> negative effects. The point is to include the last line of the >>> function, even if it is not touched by a basic block (which it won't >>> be, for the most part, because the function return happens before }). >>> For filtering, the effect is positive because the full function is >>> actually included, note the final }: >>> >>> $ gcov -t sum --include=sub >>> sum.gcda:cannot open data file, assuming not executed >>> -: 0:Source:sum.cc >>> -: 0:Graph:sum.gcno >>> -: 0:Data:- >>> -: 0:Runs:0 >>> #####: 5:int sub (int a, int b) { >>> #####: 6: return a - b; >>> -: 7:} >>> >>> Thoughts? >> It seems reasonable. This is all a bit fuzzy. >> >> You could return via the epilogue and we could conceptually mark the >> epilogue as being on the close curley from a line number standpoint. >> But then what do we do with sibling calls which are both calls and >> returns at the same time. Those we probably wouldn't want to >> associate with the close curley and that could be the only return >> path. Point being including that line even though it's not always >> associated with a basic block seems reasonable to me. > > I might be misunderstanding, but I don't think a sibling call would be > affected by this, or rather, I don't think there's a change in line > association here, and it should not affect the line counts. Line 7, for > example, is not executed even though it is printed with --include. A > (sibling) call would still be associated with a block anyway, right? Sorry I wasn't clear. My point was that that there's not a hard and fast rule we can follow for handling of the final close curley in gcov. So, if including the last line even if it's not covered by a block works well in practice, then it seems fine to me. jeff
On 7/10/24 23:59, Jeff Law wrote: > > > On 7/10/24 3:18 PM, Jørgen Kvalsvik wrote: >> >> >> On 7/10/24 21:54, Jeff Law wrote: >>> >>> >>> On 7/10/24 1:30 PM, Jørgen Kvalsvik wrote: >>> >>>> >>>> Thanks you -- before I move forward, I would like some feedback on >>>> adding this diff to the patch: >>>> >>>> diff --git a/gcc/gcov.cc b/gcc/gcov.cc >>>> index 9cdef19c461..055fa7e78ba 100644 >>>> --- a/gcc/gcov.cc >>>> +++ b/gcc/gcov.cc >>>> @@ -1613,6 +1613,12 @@ process_all_functions (void) >>>> } >>>> } >>>> >>>> + /* Make sure to include the last line for this function >>>> even when it >>>> + is not directly covered by a basic block, for example >>>> when } is on >>>> + its own line. */ >>>> + if (sources[fn->src].lines.size () <= fn->end_line) >>>> + sources[fn->src].lines.resize (fn->end_line + 1); >>>> + >>>> >>>> All gcov.exp still pass (for C, C++), and it doesn't seem to have >>>> any negative effects. The point is to include the last line of the >>>> function, even if it is not touched by a basic block (which it won't >>>> be, for the most part, because the function return happens before >>>> }). For filtering, the effect is positive because the full function >>>> is actually included, note the final }: >>>> >>>> $ gcov -t sum --include=sub >>>> sum.gcda:cannot open data file, assuming not executed >>>> -: 0:Source:sum.cc >>>> -: 0:Graph:sum.gcno >>>> -: 0:Data:- >>>> -: 0:Runs:0 >>>> #####: 5:int sub (int a, int b) { >>>> #####: 6: return a - b; >>>> -: 7:} >>>> >>>> Thoughts? >>> It seems reasonable. This is all a bit fuzzy. >>> >>> You could return via the epilogue and we could conceptually mark the >>> epilogue as being on the close curley from a line number standpoint. >>> But then what do we do with sibling calls which are both calls and >>> returns at the same time. Those we probably wouldn't want to >>> associate with the close curley and that could be the only return >>> path. Point being including that line even though it's not always >>> associated with a basic block seems reasonable to me. >> >> I might be misunderstanding, but I don't think a sibling call would be >> affected by this, or rather, I don't think there's a change in line >> association here, and it should not affect the line counts. Line 7, >> for example, is not executed even though it is printed with --include. >> A (sibling) call would still be associated with a block anyway, right? > Sorry I wasn't clear. My point was that that there's not a hard and > fast rule we can follow for handling of the final close curley in gcov. > So, if including the last line even if it's not covered by a block works > well in practice, then it seems fine to me. I see, that makes sense. I'll install it as two commits just to make sure bisecting and reverting is easier, just in case it has unintended consequences, before applying. Thanks, Jørgen
diff --git a/gcc/doc/gcov.texi b/gcc/doc/gcov.texi index c118061aed5..dc79bccb8cf 100644 --- a/gcc/doc/gcov.texi +++ b/gcc/doc/gcov.texi @@ -127,11 +127,14 @@ gcov [@option{-v}|@option{--version}] [@option{-h}|@option{--help}] [@option{-g}|@option{--conditions}] [@option{-d}|@option{--display-progress}] [@option{-f}|@option{--function-summaries}] + [@option{--include} @var{regex}] + [@option{--exclude} @var{regex}] [@option{-j}|@option{--json-format}] [@option{-H}|@option{--human-readable}] [@option{-k}|@option{--use-colors}] [@option{-l}|@option{--long-file-names}] [@option{-m}|@option{--demangled-names}] + [@option{-M}|@option{--filter-on-demangled}] [@option{-n}|@option{--no-output}] [@option{-o}|@option{--object-directory} @var{directory|file}] [@option{-p}|@option{--preserve-paths}] @@ -186,6 +189,24 @@ Display the progress on the standard output. @itemx --function-summaries Output summaries for each function in addition to the file level summary. +@item --include @var{regex} +Include functions matching @var{regex}. This option makes +@command{gcov} only report on functions that match the extended +regular expression @var{regex}. This flag can be combined with +@option{--exclude}. If a function matches both includes and excludes, +the last include/exclude applies. By default @command{gcov} reports +on all functions, but if a @command{--include} is used then only +functions matching the include will be reported. + +@item --exclude @var{regex} +Exclude functions matching @var{regex}. This option makes +@command{gcov} not report on functions that match the extended regular +expression @var{regex}. This flag can be combined with +@option{--include}. If a function matches both includes and excludes, +the last include/exclude applies. By default @command{gcov} reports +on all functions, and if @option{--exclude} is used then functions +matching it will be omitted. + @item -h @itemx --help Display help about using @command{gcov} (on the standard output), and @@ -449,6 +470,12 @@ and included file names will be complete path names. Display demangled function names in output. The default is to show mangled function names. +@item -M +@itemx --filter-on-demangled +Make @option{--include} and @option{--exclude} match demangled names. +This does only affects the matching and does not imply +@option{--demangled-names}, but it can safely be combined with it. + @item -n @itemx --no-output Do not create the @command{gcov} output file. @@ -906,6 +933,92 @@ the file doesn't match the executable (differing number of basic block counts) it will ignore the contents of the file. It then adds in the new execution counts and finally writes the data to the file. +You can report on a subset of functions by using @option{--include} +and @option{--exclude}. This is very useful when combined with +@option{--stdout} trying to understand behavior and coverage for a +particular function by running a test, looking at @command{gcov} +output, testing another input, and running @command{gcov} again. + +@smallexample +$ gcov -m --stdout --include inc tmp + -: 0:Source:tmp.cpp + -: 0:Graph:tmp.gcno + -: 0:Data:tmp.gcda + -: 0:Runs:1 + 2*: 8: void inc () @{ b++; @} +------------------ +Foo<char>::inc(): + #####: 8: void inc () @{ b++; @} +------------------ +Foo<int>::inc(): + 2: 8: void inc () @{ b++; @} +------------------ +@end smallexample + +@command{gcov} will match on mangled names by default, which you can +control with the @option{-M} flag. Note that matching and reporting +are independent, so you can match on mangled names while printing +demangled names, and vice versa. To report on the @code{int} +instantiation of @code{Foo} matching on mangled and demangled names: + +@smallexample +$ gcov -t -m -M tmp --include 'Foo<int>' + -: 0:Source:tmp.cpp + -: 0:Graph:tmp.gcno + -: 0:Data:tmp.gcda + -: 0:Runs:1 + 1: 7: Foo(): b (1000) @{@} + 2: 8: void inc () @{ b++; @} +@end smallexample + +@smallexample +$ gcov -t -m tmp --include 'FooIi' + -: 0:Source:tmp.cpp + -: 0:Graph:tmp.gcno + -: 0:Data:tmp.gcda + -: 0:Runs:1 + 1: 7: Foo(): b (1000) @{@} + 2: 8: void inc () @{ b++; @} +@end smallexample + +The arguments to @option{--include} and @option{--exclude} are +extended regular expressions (like @command{grep -E}), so the pattern +@code{in.?} matches both @code{inc} and @code{main}. If used with +@option{-M} then all @code{int} instantiations of @code{Foo} would +match too. @option{--include} and @option{--exclude} can be used +multiple times, and if a name matches multiple filters it is the last +one to match which takes preference. For example, to match +@code{main} and the @code{int} instatiation of @code{inc}, while +omitting the @code{Foo} constructor: + +@smallexample +$ gcov -t -m -M --include in --exclude Foo --include '<int>::inc' tmp + -: 0:Source:tmp.cpp + -: 0:Graph:tmp.gcno + -: 0:Data:tmp.gcda + -: 0:Runs:1 + 2: 8: void inc () @{ b++; @} + 1: 18:main (void) + -: 19:@{ + -: 20: int i, total; + 1: 21: Foo<int> counter; + -: 22: + 1: 23: counter.inc(); + 1: 24: counter.inc(); + 1: 25: total = 0; + -: 26: + 11: 27: for (i = 0; i < 10; i++) + 10: 28: total += i; + -: 29: + 1*: 30: int v = total > 100 ? 1 : 2; + -: 31: + 1: 32: if (total != 45) + #####: 33: printf ("Failure\n"); + -: 34: else + 1: 35: printf ("Success\n"); + 1: 36: return 0; +@end smallexample + @node Gcov and Optimization @section Using @command{gcov} with GCC Optimization diff --git a/gcc/gcov.cc b/gcc/gcov.cc index 0d4ef14e6c9..f6787f0be8f 100644 --- a/gcc/gcov.cc +++ b/gcc/gcov.cc @@ -47,10 +47,10 @@ along with Gcov; see the file COPYING3. If not see #include "pretty-print.h" #include "json.h" #include "hwint.h" +#include "xregex.h" #include <zlib.h> #include <getopt.h> - #include "md5.h" using namespace std; @@ -683,6 +683,48 @@ static int flag_counts = 0; /* Return code of the tool invocation. */ static int return_code = 0; +/* "Keep policy" when adding functions to the global function table. This will + be set to false when --include is used, otherwise every function should be + added to the table. Used for --include/exclude. */ +static bool default_keep = true; + +/* Include/exclude filters function based on matching the (de)mangled name. + The default is to match the mangled name. Note that flag_demangled_names + does not affect this. */ +static bool flag_filter_on_demangled = false; + +/* A 'function filter', a filter and action for determining if a function + should be included in the output or not. Used for --include/--exclude + filtering. */ +struct fnfilter +{ + /* The (extended) compiled regex for this filter. */ + regex_t regex; + + /* The action when this filter (regex) matches - if true, the function should + be kept, otherwise discarded. */ + bool keep; + + /* Compile the regex EXPR, or exit if pattern is malformed. */ + void compile (const char *expr) + { + int err = regcomp (®ex, expr, REG_NOSUB | REG_EXTENDED); + if (err) + { + size_t len = regerror (err, ®ex, nullptr, 0); + char *msg = XNEWVEC (char, len); + regerror (err, ®ex, msg, len); + fprintf (stderr, "Bad regular expression: %s\n", msg); + free (msg); + exit (EXIT_FAILURE); + } + } +}; + +/* A collection of filter functions for including/exclude functions in the + output. This is empty unless --include/--exclude is used. */ +static vector<fnfilter> filters; + /* Forward declarations. */ static int process_args (int, char **); static void print_usage (int) ATTRIBUTE_NORETURN; @@ -977,6 +1019,8 @@ print_usage (int error_p) fnotice (file, " -d, --display-progress Display progress information\n"); fnotice (file, " -D, --debug Display debugging dumps\n"); fnotice (file, " -f, --function-summaries Output summaries for each function\n"); + fnotice (file, " --include Include functions matching this regex\n"); + fnotice (file, " --exclude Exclude functions matching this regex\n"); fnotice (file, " -h, --help Print this help, then exit\n"); fnotice (file, " -j, --json-format Output JSON intermediate format\n\ into .gcov.json.gz file\n"); @@ -985,6 +1029,8 @@ print_usage (int error_p) fnotice (file, " -l, --long-file-names Use long output file names for included\n\ source files\n"); fnotice (file, " -m, --demangled-names Output demangled function names\n"); + fnotice (file, " -M, --filter-on-demangled Make --include/--exclude match on demangled\n\ + names. This does not imply -m\n"); fnotice (file, " -n, --no-output Do not create an output file\n"); fnotice (file, " -o, --object-directory DIR|FILE Search for object files in DIR or called FILE\n"); fnotice (file, " -p, --preserve-paths Preserve all pathname components\n"); @@ -1029,11 +1075,14 @@ static const struct option options[] = { "branch-counts", no_argument, NULL, 'c' }, { "conditions", no_argument, NULL, 'g' }, { "json-format", no_argument, NULL, 'j' }, + { "include", required_argument, NULL, 'I' }, + { "exclude", required_argument, NULL, 'E' }, { "human-readable", no_argument, NULL, 'H' }, { "no-output", no_argument, NULL, 'n' }, { "long-file-names", no_argument, NULL, 'l' }, { "function-summaries", no_argument, NULL, 'f' }, { "demangled-names", no_argument, NULL, 'm' }, + { "filter-on-demangled", no_argument, NULL, 'M' }, { "preserve-paths", no_argument, NULL, 'p' }, { "relative-only", no_argument, NULL, 'r' }, { "object-directory", required_argument, NULL, 'o' }, @@ -1056,7 +1105,7 @@ process_args (int argc, char **argv) { int opt; - const char *opts = "abcdDfghHijklmno:pqrs:tuvwx"; + const char *opts = "abcdDfghHijklmMno:pqrs:tuvwx"; while ((opt = getopt_long (argc, argv, opts, options, NULL)) != -1) { switch (opt) @@ -1082,6 +1131,17 @@ process_args (int argc, char **argv) case 'l': flag_long_names = 1; break; + case 'I': + default_keep = false; + filters.push_back (fnfilter {}); + filters.back ().keep = true; + filters.back ().compile (optarg); + break; + case 'E': + filters.push_back (fnfilter {}); + filters.back ().keep = false; + filters.back ().compile (optarg); + break; case 'H': flag_human_readable_numbers = 1; break; @@ -1094,6 +1154,9 @@ process_args (int argc, char **argv) case 'm': flag_demangled_names = 1; break; + case 'M': + flag_filter_on_demangled = true; + break; case 'n': flag_gcov_file = 0; break; @@ -1728,9 +1791,13 @@ release_structures (void) it != functions.end (); it++) delete (*it); + for (fnfilter &filter : filters) + regfree (&filter.regex); + sources.resize (0); names.resize (0); functions.resize (0); + filters.resize (0); ident_to_fn.clear (); } @@ -1955,8 +2022,6 @@ read_graph_file (void) unsigned end_column = gcov_read_unsigned (); fn = new function_info (); - functions.push_back (fn); - ident_to_fn[ident] = fn; fn->m_name = function_name; fn->ident = ident; @@ -1970,6 +2035,26 @@ read_graph_file (void) fn->artificial = artificial; current_tag = tag; + + /* This is separate from flag_demangled_names to support filtering on + mangled names while printing demangled names, or filtering on + demangled names while printing mangled names. An independent flag + makes sure the function selection does not change even if + demangling is turned on/off. */ + const char *fname = function_name; + if (flag_filter_on_demangled) + fname = fn->get_demangled_name (); + + bool keep = default_keep; + for (const fnfilter &fn : filters) + if (regexec (&fn.regex, fname, 0, nullptr, 0) == 0) + keep = fn.keep; + + if (keep) + { + functions.push_back (fn); + ident_to_fn[ident] = fn; + } } else if (fn && tag == GCOV_TAG_BLOCKS) { @@ -3377,11 +3462,17 @@ output_lines (FILE *gcov_file, const source_info *src) unsigned line_start_group = 0; vector<function_info *> *fns; + unsigned filtered_line_end = !filters.empty () ? 0 : source_lines.size (); for (unsigned line_num = 1; line_num <= source_lines.size (); line_num++) { if (line_num >= src->lines.size ()) { + /* If the src->lines is truncated because the rest of the functions + are filtered out we must stop here, and not fall back to printing + the rest of the file. */ + if (!filters.empty ()) + break; fprintf (gcov_file, "%9s:%5u", "-", line_num); print_source_line (gcov_file, source_lines, line_num); continue; @@ -3400,11 +3491,26 @@ output_lines (FILE *gcov_file, const source_info *src) for (unsigned i = 0; i < fns->size (); i++) if ((*fns)[i]->end_line > line_start_group) line_start_group = (*fns)[i]->end_line; + + /* When filtering, src->lines will be cut short for the last + selected function. To make sure the "overlapping function" + section is printed too, adjust the end so that it is within + src->lines. */ + if (line_start_group >= src->lines.size ()) + line_start_group = src->lines.size () - 1; + + if (!filters.empty ()) + filtered_line_end = line_start_group; } else if (fns != NULL && fns->size () == 1) { function_info *fn = (*fns)[0]; output_function_details (gcov_file, fn); + + /* If functions are filtered, only the matching functions will be in + fns and there is no need for extra checking. */ + if (!filters.empty ()) + filtered_line_end = fn->end_line; } } @@ -3414,12 +3520,16 @@ output_lines (FILE *gcov_file, const source_info *src) Otherwise, print the execution count before the source line. There are 16 spaces of indentation added before the source line so that tabs won't be messed up. */ - output_line_beginning (gcov_file, line->exists, line->unexceptional, - line->has_unexecuted_block, line->count, - line_num, "=====", "#####", src->maximum_count); + if (line_num <= filtered_line_end) + { + output_line_beginning (gcov_file, line->exists, line->unexceptional, + line->has_unexecuted_block, line->count, + line_num, "=====", "#####", + src->maximum_count); - print_source_line (gcov_file, source_lines, line_num); - output_line_details (gcov_file, line, line_num); + print_source_line (gcov_file, source_lines, line_num); + output_line_details (gcov_file, line, line_num); + } if (line_start_group == line_num) { diff --git a/gcc/testsuite/g++.dg/gcov/gcov-19.C b/gcc/testsuite/g++.dg/gcov/gcov-19.C new file mode 100644 index 00000000000..3f898cfae4b --- /dev/null +++ b/gcc/testsuite/g++.dg/gcov/gcov-19.C @@ -0,0 +1,35 @@ +/* { dg-options "-fprofile-arcs -ftest-coverage" } */ +/* { dg-do run { target native } } */ + +/* Filtering on the function base name generally works well, because it becomes + an unadultered part of the symbol. */ + +template <typename T> +T +fn1 (T x) +{ + /* fst */ + return x; +} + +template <typename T> +T +fn2 (T x) +{ + /* snd */ + return 2 * x; +} + +int +main () +{ + fn1 (2); + fn1 (2.0); + fn1 (2.0f); + + fn2 (2); + fn2 (2.0); + fn2 (2.0f); +} + +/* { dg-final { run-gcov { filters { fn1 } { fn2 } } { --include=fn1 gcov-19.C } } } */ diff --git a/gcc/testsuite/g++.dg/gcov/gcov-20.C b/gcc/testsuite/g++.dg/gcov/gcov-20.C new file mode 100644 index 00000000000..fb33f7ecbdd --- /dev/null +++ b/gcc/testsuite/g++.dg/gcov/gcov-20.C @@ -0,0 +1,38 @@ +/* { dg-options "-fprofile-arcs -ftest-coverage" } */ +/* { dg-do run { target native } } */ + +/* Filtering also works by targeting the mangled symbol directly, but the + subtlety is not really caught by the test framework. Matching on fn1I[df] + prints the "overlapping blocks" of both the float and double instantiation, + but *not* the int instantiation. The extra {} around the --include argument + is quoting for tcl/dejagnu. */ + +template <typename T> +T +fn1 (T x) +{ + /* fst */ + return x; +} + +template <typename T> +T +fn2 (T x) +{ + /* snd */ + return 2 * x; +} + +int +main () +{ + fn1 (2); + fn1 (2.0); + fn1 (2.0f); + + fn2 (2); + fn2 (2.0); + fn2 (2.0f); +} + +/* { dg-final { run-gcov { filters { fst } { snd } } { {--include=fn1I[fd]} gcov-20.C } } } */ diff --git a/gcc/testsuite/g++.dg/gcov/gcov-21.C b/gcc/testsuite/g++.dg/gcov/gcov-21.C new file mode 100644 index 00000000000..67f6d79dbf1 --- /dev/null +++ b/gcc/testsuite/g++.dg/gcov/gcov-21.C @@ -0,0 +1,32 @@ +/* { dg-options "-fprofile-arcs -ftest-coverage" } */ +/* { dg-do run { target native } } */ + +/* Filters can be applied to demangled names. This support matching on + types and class hierarchies as well as function names. */ + +template <typename T> +T +fn1 (T x) +{ + /* fst */ + return x; +} + +template <typename T> +T +fn2 (T x) +{ + /* snd */ + return 2 * x; +} + +int +main () +{ + fn1 (2); + + fn2 (2.0); + fn2 (2.0f); +} + +/* { dg-final { run-gcov { filters { fst } } { --filter-on-demangled --include int gcov-21.C } } } */ diff --git a/gcc/testsuite/gcc.misc-tests/gcov-25.c b/gcc/testsuite/gcc.misc-tests/gcov-25.c new file mode 100644 index 00000000000..658f375e055 --- /dev/null +++ b/gcc/testsuite/gcc.misc-tests/gcov-25.c @@ -0,0 +1,25 @@ +/* { dg-options "-fprofile-arcs -ftest-coverage" } */ +/* { dg-do run { target native } } */ + +/* Filters are considered in order with latest-wins, so if a function is + included and later excluded it should not show up. */ + +int +fn1 (int x) +{ + /* fst */ + return x; +} + +int +fn2 (int x) +{ + /* snd */ + return x * 2; +} + +int +main () +{} + +/* { dg-final { run-gcov { filters { snd } { fst main } } { --include=fn --exclude=1 gcov-25.c } } } */ diff --git a/gcc/testsuite/gcc.misc-tests/gcov-26.c b/gcc/testsuite/gcc.misc-tests/gcov-26.c new file mode 100644 index 00000000000..d204cd3c72b --- /dev/null +++ b/gcc/testsuite/gcc.misc-tests/gcov-26.c @@ -0,0 +1,25 @@ +/* { dg-options "-fprofile-arcs -ftest-coverage" } */ +/* { dg-do run { target native } } */ + +/* Filters are considered in order with latest-wins, so if a function is + excluded and later included it should show up. */ + +int +fn1 (int x) +{ + /* fst */ + return x; +} + +int +fn2 (int x) +{ + /* snd */ + return x * 2; +} + +int +main () +{} + +/* { dg-final { run-gcov { filters { fst snd } { main } } { --exclude=1 --include=fn gcov-26.c } } } */ diff --git a/gcc/testsuite/gcc.misc-tests/gcov-27.c b/gcc/testsuite/gcc.misc-tests/gcov-27.c new file mode 100644 index 00000000000..e01a2c794a7 --- /dev/null +++ b/gcc/testsuite/gcc.misc-tests/gcov-27.c @@ -0,0 +1,24 @@ +/* { dg-options "-fprofile-arcs -ftest-coverage" } */ +/* { dg-do run { target native } } */ + +/* If only --exclude is used, other functions should be used by default. */ + +int +fn1 (int x) +{ + /* fst */ + return x; +} + +int +fn2 (int x) +{ + /* snd */ + return x * 2; +} + +int +main () +{} + +/* { dg-final { run-gcov { filters { fst snd } { main } } { --exclude=main gcov-27.c } } } */ diff --git a/gcc/testsuite/gcc.misc-tests/gcov-28.c b/gcc/testsuite/gcc.misc-tests/gcov-28.c new file mode 100644 index 00000000000..26f5b15e06f --- /dev/null +++ b/gcc/testsuite/gcc.misc-tests/gcov-28.c @@ -0,0 +1,22 @@ +/* { dg-options "-fprofile-arcs -ftest-coverage" } */ +/* { dg-do run { target native } } */ + +int +once (int x) +{ + /* fst */ + return x; +} + +int +twice (int x) +{ + /* snd */ + return x * 2; +} + +int +main () +{} + +/* { dg-final { run-gcov { filters { fst } { snd main } } { --include=once gcov-28.c } } } */ diff --git a/gcc/testsuite/lib/gcov.exp b/gcc/testsuite/lib/gcov.exp index dd47d66d1b2..e49f1011884 100644 --- a/gcc/testsuite/lib/gcov.exp +++ b/gcc/testsuite/lib/gcov.exp @@ -495,6 +495,44 @@ proc verify-calls { testname testcase file } { return $failed } +# Verify that report filtering includes and excludes the right functions. +proc verify-filters { testname testcase file expected unexpected } { + set fd [open $file r] + + set seen {} + set ex [concat $expected $unexpected] + + while { [gets $fd line] >= 0 } { + foreach sym $ex { + if [regexp "$sym" "$line"] { + lappend seen $sym + } + } + } + + set seen [lsort -unique $seen] + + set expected [lmap key $expected { + if { $key in $seen } continue + set key + }] + set unexpected [lmap key $unexpected { + if { $key ni $seen } continue + set key + }] + + foreach sym $expected { + fail "Did not see expected symbol '$sym'" + } + + foreach sym $unexpected { + fail "Found unexpected symbol '$sym'" + } + + close $fd + return [expr [llength $expected] + [llength $unexpected]] +} + proc gcov-pytest-format-line { args } { global subdir @@ -570,6 +608,7 @@ proc run-gcov { args } { set gcov_verify_conditions 0 set gcov_verify_lines 1 set gcov_verify_intermediate 0 + set gcov_verify_filters 0 set gcov_remove_gcda 0 set xfailed 0 @@ -580,12 +619,17 @@ proc run-gcov { args } { set gcov_verify_branches 1 } elseif { $a == "conditions" } { set gcov_verify_conditions 1 + } elseif { [lindex $a 0] == "filters" } { + set gcov_verify_filters 1 + set verify_filters_expected [lindex $a 1] + set verify_filters_unexpected [lindex $a 2] } elseif { $a == "intermediate" } { set gcov_verify_intermediate 1 set gcov_verify_calls 0 set gcov_verify_branches 0 set gcov_verify_conditions 0 set gcov_verify_lines 0 + set gcov_verify_filters 0 } elseif { $a == "remove-gcda" } { set gcov_remove_gcda 1 } elseif { $gcov_args == "" } { @@ -670,15 +714,20 @@ proc run-gcov { args } { } else { set ifailed 0 } + if { $gcov_verify_filters } { + set ffailed [verify-filters $testname $testcase $testcase.gcov $verify_filters_expected $verify_filters_unexpected] + } else { + set ffailed 0 + } # Report whether the gcov test passed or failed. If there were # multiple failures then the message is a summary. - set tfailed [expr $lfailed + $bfailed + $cdfailed + $cfailed + $ifailed] + set tfailed [expr $lfailed + $bfailed + $cdfailed + $cfailed + $ifailed + $ffailed] if { $xfailed } { setup_xfail "*-*-*" } if { $tfailed > 0 } { - fail "$testname gcov: $lfailed failures in line counts, $bfailed in branch percentages, $cdfailed in condition/decision, $cfailed in return percentages, $ifailed in intermediate format" + fail "$testname gcov: $lfailed failures in line counts, $bfailed in branch percentages, $cdfailed in condition/decision, $cfailed in return percentages, $ifailed in intermediate format, $ffailed failed in filters" if { $xfailed } { clean-gcov $testcase }
Add the --include and --exclude flags to gcov to control what functions to report on. This is meant to make gcov more practical as an when writing test suites or performing other coverage experiments, which tends to focus on a few functions at the time. This really shines in combination with the -t/--stdout flag. With support for more expansive metrics in gcov like modified condition/decision coverage (MC/DC) and path coverage, output quickly gets overwhelming without filtering. The approach is quite simple: filters are egrep regexes and are evaluated left-to-right, and the last filter "wins", that is, if a function matches an --include and a subsequent --exclude, it should not be included in the output. All of the output machinery works on the function table, so by optionally (not) adding function makes the even the json output work as expected, and only minor changes are needed to suppress the filtered-out functions. Demo: math.c int mul (int a, int b) { return a * b; } int sub (int a, int b) { return a - b; } int sum (int a, int b) { return a + b; } Plain matches: $ gcov -t math --include=sum -: 0:Source:filter.c -: 0:Graph:filter.gcno -: 0:Data:- -: 0:Runs:0 #####: 9:int sum (int a, int b) { #####: 10: return a + b; $ gcov -t math --include=mul -: 0:Source:filter.c -: 0:Graph:filter.gcno -: 0:Data:- -: 0:Runs:0 #####: 1:int mul (int a, int b) { #####: 2: return a * b; Regex match: $ gcov -t math --include=su -: 0:Source:filter.c -: 0:Graph:filter.gcno -: 0:Data:- -: 0:Runs:0 #####: 5:int sub (int a, int b) { #####: 6: return a - b; -: 7:} #####: 9:int sum (int a, int b) { #####: 10: return a + b; And similar for exclude: $ gcov -t math --exclude=sum -: 0:Source:filter.c -: 0:Graph:filter.gcno -: 0:Data:- -: 0:Runs:0 #####: 1:int mul (int a, int b) { #####: 2: return a * b; -: 3:} #####: 5:int sub (int a, int b) { #####: 6: return a - b; And json, for good measure: $ gcov -t math --include=sum --json | jq ".files[].lines[]" { "line_number": 9, "function_name": "sum", "count": 0, "unexecuted_block": true, "block_ids": [], "branches": [], "calls": [] } { "line_number": 10, "function_name": "sum", "count": 0, "unexecuted_block": true, "block_ids": [ 2 ], "branches": [], "calls": [] } Note that the last function gets "clipped" when lines are associated to functions, which means the closing brace is dropped from the report. I hope this can be fixed, but considering it is not really a part of the function body, the gcov report is "complete". Matching generally work well for mangled names, as the mangled names also have the base symbol name in it. By default, functions are matched by the mangled name, which means matching on base names always work as expected. The -M flag makes the matching work on the demangled name which is quite useful when you only want to report on specific overloads and can use the full type names. Why not just use grep? grep is not really sufficient as grep is very line oriented, and the reports that benefit the most from filtering often unpredictably span multiple lines based on the state of coverage. For example, a condition coverage report for 3 terms/6 outcomes only outputs 1 line when all conditions are covered, and 7 with no lines covered. gcc/ChangeLog: * doc/gcov.texi: Add --include, --exclude, --match-on-demangled documentation. * gcov.cc (struct fnfilter): New. (print_usage): Add --include, --exclude, -M, --match-on-demangled. (process_args): Likewise. (release_structures): Release filters. (read_graph_file): Only add function_infos matching filters. (output_lines): Likewise. gcc/testsuite/ChangeLog: * lib/gcov.exp: Add filtering test function. * g++.dg/gcov/gcov-19.C: New test. * g++.dg/gcov/gcov-20.C: New test. * g++.dg/gcov/gcov-21.C: New test. * gcc.misc-tests/gcov-25.c: New test. * gcc.misc-tests/gcov-26.c: New test. * gcc.misc-tests/gcov-27.c: New test. * gcc.misc-tests/gcov-28.c: New test. --- gcc/doc/gcov.texi | 113 ++++++++++++++++++++++ gcc/gcov.cc | 128 +++++++++++++++++++++++-- gcc/testsuite/g++.dg/gcov/gcov-19.C | 35 +++++++ gcc/testsuite/g++.dg/gcov/gcov-20.C | 38 ++++++++ gcc/testsuite/g++.dg/gcov/gcov-21.C | 32 +++++++ gcc/testsuite/gcc.misc-tests/gcov-25.c | 25 +++++ gcc/testsuite/gcc.misc-tests/gcov-26.c | 25 +++++ gcc/testsuite/gcc.misc-tests/gcov-27.c | 24 +++++ gcc/testsuite/gcc.misc-tests/gcov-28.c | 22 +++++ gcc/testsuite/lib/gcov.exp | 53 +++++++++- 10 files changed, 484 insertions(+), 11 deletions(-) create mode 100644 gcc/testsuite/g++.dg/gcov/gcov-19.C create mode 100644 gcc/testsuite/g++.dg/gcov/gcov-20.C create mode 100644 gcc/testsuite/g++.dg/gcov/gcov-21.C create mode 100644 gcc/testsuite/gcc.misc-tests/gcov-25.c create mode 100644 gcc/testsuite/gcc.misc-tests/gcov-26.c create mode 100644 gcc/testsuite/gcc.misc-tests/gcov-27.c create mode 100644 gcc/testsuite/gcc.misc-tests/gcov-28.c