diff options
Diffstat (limited to 'classification_output/04/assembly/48245039')
| -rw-r--r-- | classification_output/04/assembly/48245039 | 538 |
1 files changed, 0 insertions, 538 deletions
diff --git a/classification_output/04/assembly/48245039 b/classification_output/04/assembly/48245039 deleted file mode 100644 index b1a9e6510..000000000 --- a/classification_output/04/assembly/48245039 +++ /dev/null @@ -1,538 +0,0 @@ -assembly: 0.956 -device: 0.953 -other: 0.953 -instruction: 0.951 -semantic: 0.939 -graphic: 0.935 -socket: 0.932 -boot: 0.932 -vnc: 0.926 -mistranslation: 0.888 -KVM: 0.855 -network: 0.818 - -[Qemu-devel] [BUG] gcov support appears to be broken - -Hello, according to out docs, here is the procedure that should produce -coverage report for execution of the complete "make check": - -#./configure --enable-gcov -#make -#make check -#make coverage-report - -It seems that first three commands execute as expected. (For example, there are -plenty of files generated by "make check" that would've not been generated if -"enable-gcov" hadn't been chosen.) However, the last command complains about -some missing files related to FP support. If those files are added (for -example, artificially, using "touch <missing-file"), that it starts complaining -about missing some decodetree-generated files. Other kinds of files are -involved too. - -It would be nice to have coverage support working. Please somebody take a look, -or explain if I make a mistake or misunderstood our gcov support. - -Yours, -Aleksandar - -On Mon, 5 Aug 2019 at 11:39, Aleksandar Markovic <address@hidden> wrote: -> -> -Hello, according to out docs, here is the procedure that should produce -> -coverage report for execution of the complete "make check": -> -> -#./configure --enable-gcov -> -#make -> -#make check -> -#make coverage-report -> -> -It seems that first three commands execute as expected. (For example, there -> -are plenty of files generated by "make check" that would've not been -> -generated if "enable-gcov" hadn't been chosen.) However, the last command -> -complains about some missing files related to FP support. If those files are -> -added (for example, artificially, using "touch <missing-file"), that it -> -starts complaining about missing some decodetree-generated files. Other kinds -> -of files are involved too. -> -> -It would be nice to have coverage support working. Please somebody take a -> -look, or explain if I make a mistake or misunderstood our gcov support. -Cc'ing Alex who's probably the closest we have to a gcov expert. - -(make/make check of a --enable-gcov build is in the set of things our -Travis CI setup runs, so we do defend that part against regressions.) - -thanks --- PMM - -Peter Maydell <address@hidden> writes: - -> -On Mon, 5 Aug 2019 at 11:39, Aleksandar Markovic <address@hidden> wrote: -> -> -> -> Hello, according to out docs, here is the procedure that should produce -> -> coverage report for execution of the complete "make check": -> -> -> -> #./configure --enable-gcov -> -> #make -> -> #make check -> -> #make coverage-report -> -> -> -> It seems that first three commands execute as expected. (For example, -> -> there are plenty of files generated by "make check" that would've not -> -> been generated if "enable-gcov" hadn't been chosen.) However, the -> -> last command complains about some missing files related to FP -> -> support. If those files are added (for example, artificially, using -> -> "touch <missing-file"), that it starts complaining about missing some -> -> decodetree-generated files. Other kinds of files are involved too. -The gcov tool is fairly noisy about missing files but that just -indicates the tests haven't exercised those code paths. "make check" -especially doesn't touch much of the TCG code and a chunk of floating -point. - -> -> -> -> It would be nice to have coverage support working. Please somebody -> -> take a look, or explain if I make a mistake or misunderstood our gcov -> -> support. -So your failure mode is no report is generated at all? It's working for -me here. - -> -> -Cc'ing Alex who's probably the closest we have to a gcov expert. -> -> -(make/make check of a --enable-gcov build is in the set of things our -> -Travis CI setup runs, so we do defend that part against regressions.) -We defend the build but I have just checked and it seems our -check_coverage script is currently failing: -https://travis-ci.org/stsquad/qemu/jobs/567809808#L10328 -But as it's an after_success script it doesn't fail the build. - -> -> -thanks -> --- PMM --- -Alex Bennée - -> -> #./configure --enable-gcov -> -> #make -> -> #make check -> -> #make coverage-report -> -> -> -> It seems that first three commands execute as expected. (For example, -> -> there are plenty of files generated by "make check" that would've not -> -> been generated if "enable-gcov" hadn't been chosen.) However, the -> -> last command complains about some missing files related to FP -> -So your failure mode is no report is generated at all? It's working for -> -me here. -Alex, no report is generated for my test setups - in fact, "make -coverage-report" even says that it explicitly deletes what appears to be the -main coverage report html file). - -This is the terminal output of an unsuccessful executions of "make -coverage-report" for recent ToT: - -~/Build/qemu-TOT-TEST$ make coverage-report -make[1]: Entering directory '/home/user/Build/qemu-TOT-TEST/slirp' -make[1]: Nothing to be done for 'all'. -make[1]: Leaving directory '/home/user/Build/qemu-TOT-TEST/slirp' - CHK version_gen.h - GEN coverage-report.html -Traceback (most recent call last): - File "/usr/bin/gcovr", line 1970, in <module> - print_html_report(covdata, options.html_details) - File "/usr/bin/gcovr", line 1473, in print_html_report - INPUT = open(data['FILENAME'], 'r') -IOError: [Errno 2] No such file or directory: 'wrap.inc.c' -Makefile:1048: recipe for target -'/home/user/Build/qemu-TOT-TEST/reports/coverage/coverage-report.html' failed -make: *** -[/home/user/Build/qemu-TOT-TEST/reports/coverage/coverage-report.html] Error 1 -make: *** Deleting file -'/home/user/Build/qemu-TOT-TEST/reports/coverage/coverage-report.html' - -This instance is executed in QEMU 3.0 source tree: (so, it looks the problem -existed for quite some time) - -~/Build/qemu-3.0$ make coverage-report - CHK version_gen.h - GEN coverage-report.html -Traceback (most recent call last): - File "/usr/bin/gcovr", line 1970, in <module> - print_html_report(covdata, options.html_details) - File "/usr/bin/gcovr", line 1473, in print_html_report - INPUT = open(data['FILENAME'], 'r') -IOError: [Errno 2] No such file or directory: -'/home/user/Build/qemu-3.0/target/openrisc/decode.inc.c' -Makefile:992: recipe for target -'/home/user/Build/qemu-3.0/reports/coverage/coverage-report.html' failed -make: *** [/home/user/Build/qemu-3.0/reports/coverage/coverage-report.html] -Error 1 -make: *** Deleting file -'/home/user/Build/qemu-3.0/reports/coverage/coverage-report.html' - -Fond regards, -Aleksandar - - -> -Alex Bennée - -> -> #./configure --enable-gcov -> -> #make -> -> #make check -> -> #make coverage-report -> -> -> -> It seems that first three commands execute as expected. (For example, -> -> there are plenty of files generated by "make check" that would've not -> -> been generated if "enable-gcov" hadn't been chosen.) However, the -> -> last command complains about some missing files related to FP -> -So your failure mode is no report is generated at all? It's working for -> -me here. -Another piece of info: - -~/Build/qemu-TOT-TEST$ gcov --version -gcov (Ubuntu 5.5.0-12ubuntu1~16.04) 5.5.0 20171010 -Copyright (C) 2015 Free Software Foundation, Inc. -This is free software; see the source for copying conditions. -There is NO warranty; not even for MERCHANTABILITY or -FITNESS FOR A PARTICULAR PURPOSE. - -:~/Build/qemu-TOT-TEST$ gcc --version -gcc (Ubuntu 7.2.0-1ubuntu1~16.04) 7.2.0 -Copyright (C) 2017 Free Software Foundation, Inc. -This is free software; see the source for copying conditions. There is NO -warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. - - - - -Alex, no report is generated for my test setups - in fact, "make -coverage-report" even says that it explicitly deletes what appears to be the -main coverage report html file). - -This is the terminal output of an unsuccessful executions of "make -coverage-report" for recent ToT: - -~/Build/qemu-TOT-TEST$ make coverage-report -make[1]: Entering directory '/home/user/Build/qemu-TOT-TEST/slirp' -make[1]: Nothing to be done for 'all'. -make[1]: Leaving directory '/home/user/Build/qemu-TOT-TEST/slirp' - CHK version_gen.h - GEN coverage-report.html -Traceback (most recent call last): - File "/usr/bin/gcovr", line 1970, in <module> - print_html_report(covdata, options.html_details) - File "/usr/bin/gcovr", line 1473, in print_html_report - INPUT = open(data['FILENAME'], 'r') -IOError: [Errno 2] No such file or directory: 'wrap.inc.c' -Makefile:1048: recipe for target -'/home/user/Build/qemu-TOT-TEST/reports/coverage/coverage-report.html' failed -make: *** -[/home/user/Build/qemu-TOT-TEST/reports/coverage/coverage-report.html] Error 1 -make: *** Deleting file -'/home/user/Build/qemu-TOT-TEST/reports/coverage/coverage-report.html' - -This instance is executed in QEMU 3.0 source tree: (so, it looks the problem -existed for quite some time) - -~/Build/qemu-3.0$ make coverage-report - CHK version_gen.h - GEN coverage-report.html -Traceback (most recent call last): - File "/usr/bin/gcovr", line 1970, in <module> - print_html_report(covdata, options.html_details) - File "/usr/bin/gcovr", line 1473, in print_html_report - INPUT = open(data['FILENAME'], 'r') -IOError: [Errno 2] No such file or directory: -'/home/user/Build/qemu-3.0/target/openrisc/decode.inc.c' -Makefile:992: recipe for target -'/home/user/Build/qemu-3.0/reports/coverage/coverage-report.html' failed -make: *** [/home/user/Build/qemu-3.0/reports/coverage/coverage-report.html] -Error 1 -make: *** Deleting file -'/home/user/Build/qemu-3.0/reports/coverage/coverage-report.html' - -Fond regards, -Aleksandar - - -> -Alex Bennée - -> -> #./configure --enable-gcov -> -> #make -> -> #make check -> -> #make coverage-report -> -> -> -> It seems that first three commands execute as expected. (For example, -> -> there are plenty of files generated by "make check" that would've not -> -> been generated if "enable-gcov" hadn't been chosen.) However, the -> -> last command complains about some missing files related to FP -> -So your failure mode is no report is generated at all? It's working for -> -me here. -Alex, here is the thing: - -Seeing that my gcovr is relatively old (2014) 3.2 version, I upgraded it from -git repo to the most recent 4.1 (actually, to a dev version, from the very tip -of the tree), and "make coverage-report" started generating coverage reports. -It did emit some error messages (totally different than previous), but still it -did not stop like it used to do with gcovr 3.2. - -Perhaps you would want to add some gcov/gcovr minimal version info in our docs. -(or at least a statement "this was tested with such and such gcc, gcov and -gcovr", etc.?) - -Coverage report looked fine at first glance, but it a kind of disappointed me -when I digged deeper into its content - for example, it shows very low coverage -for our FP code (softfloat), while, in fact, we know that "make check" contains -detailed tests on FP functionalities. But this is most likely a separate -problem of a very different nature, perhaps the issue of separate git repo for -FP tests (testfloat) that our FP tests use as a mid-layer. - -I'll try how everything works with my test examples, and will let you know. - -Your help is greatly appreciated, -Aleksandar - -Fond regards, -Aleksandar - - -> -Alex Bennée - -Aleksandar Markovic <address@hidden> writes: - -> ->> #./configure --enable-gcov -> ->> #make -> ->> #make check -> ->> #make coverage-report -> ->> -> ->> It seems that first three commands execute as expected. (For example, -> ->> there are plenty of files generated by "make check" that would've not -> ->> been generated if "enable-gcov" hadn't been chosen.) However, the -> ->> last command complains about some missing files related to FP -> -> -> So your failure mode is no report is generated at all? It's working for -> -> me here. -> -> -Alex, here is the thing: -> -> -Seeing that my gcovr is relatively old (2014) 3.2 version, I upgraded it from -> -git repo to the most recent 4.1 (actually, to a dev version, from the very -> -tip of the tree), and "make coverage-report" started generating coverage -> -reports. It did emit some error messages (totally different than previous), -> -but still it did not stop like it used to do with gcovr 3.2. -> -> -Perhaps you would want to add some gcov/gcovr minimal version info in our -> -docs. (or at least a statement "this was tested with such and such gcc, gcov -> -and gcovr", etc.?) -> -> -Coverage report looked fine at first glance, but it a kind of -> -disappointed me when I digged deeper into its content - for example, -> -it shows very low coverage for our FP code (softfloat), while, in -> -fact, we know that "make check" contains detailed tests on FP -> -functionalities. But this is most likely a separate problem of a very -> -different nature, perhaps the issue of separate git repo for FP tests -> -(testfloat) that our FP tests use as a mid-layer. -I get: - -68.6 % 2593 / 3782 62.2 % 1690 / 2718 - -Which is not bad considering we don't exercise the 80 and 128 bit -softfloat code at all (which is not shared by the re-factored 16/32/64 -bit code). - -> -> -I'll try how everything works with my test examples, and will let you know. -> -> -Your help is greatly appreciated, -> -Aleksandar -> -> -Fond regards, -> -Aleksandar -> -> -> -> Alex Bennée --- -Alex Bennée - -> -> it shows very low coverage for our FP code (softfloat), while, in -> -> fact, we know that "make check" contains detailed tests on FP -> -> functionalities. But this is most likely a separate problem of a very -> -> different nature, perhaps the issue of separate git repo for FP tests -> -> (testfloat) that our FP tests use as a mid-layer. -> -> -I get: -> -> -68.6 % 2593 / 3782 62.2 % 1690 / 2718 -> -I would expect that kind of result too. - -However, I get: - -File: fpu/softfloat.c Lines: 8 3334 0.2 % -Date: 2019-08-05 19:56:58 Branches: 3 2376 0.1 % - -:( - -OK, I'll try to figure that out, and most likely I could live with it if it is -an isolated problem. - -Thank you for your assistance in this matter, -Aleksandar - -> -Which is not bad considering we don't exercise the 80 and 128 bit -> -softfloat code at all (which is not shared by the re-factored 16/32/64 -> -bit code). -> -> -Alex Bennée - -> -> it shows very low coverage for our FP code (softfloat), while, in -> -> fact, we know that "make check" contains detailed tests on FP -> -> functionalities. But this is most likely a separate problem of a very -> -> different nature, perhaps the issue of separate git repo for FP tests -> -> (testfloat) that our FP tests use as a mid-layer. -> -> -I get: -> -> -68.6 % 2593 / 3782 62.2 % 1690 / 2718 -> -This problem is solved too. (and it is my fault) - -I worked with multiple versions of QEMU, and my previous low-coverage results -were for QEMU 3.0, and for that version the directory tests/fp did not even -exist. :D (<blush>) - -For QEMU ToT, I get now: - -fpu/softfloat.c - 68.8 % 2592 / 3770 62.3 % 1693 / 2718 - -which is identical for all intents and purposes to your result. - -Yours cordially, -Aleksandar - |