Our toolchain that we use for providing up to date and bug-reduced versions of various tools for our appliances have a number of internal testing suites. These suites do a pretty good job of exercising code. When you build Perl, and the internal modules and tools, tests are done right then and there, as part of the module installation.
Sadly not many languages do this yet, I think Julia, R, and a few others might. I’d like to see this as part of Python and other tools.
There is also a strange interaction between the gcc 4.7.2 and Perl 5.20.2. In this, if we use an optimization higher than none, one of the test cases fails.
This isn’t a perl issue per se, it works well with the 4.9.x compilers, and some others. I’ve not yet tried it with clang/LLVM, but should (if I ever get the time).
What I am thankful for are these code builds with the testing. I can see the failure, and have a good concept of what it is, not where it is. Had I more time, I’d see if I can work around the specific code that gcc 4.7.2 is mis-generating. But its easy to use -O0 for now, and not worry about it. I have bigger fish to fry.
I’ve had to work around some pretty insane compiler-language bugs in the past with all manner of interesting parsing errors that only showed up in specific compilation cases.
Since I drive my builds with a makefile, and capture all the output, its pretty easy to see what failed. I’ve been meaning to set up a Jenkins CI system in-house, and have even more async aspect to the process, but I find that being able to see the builds in real time sometimes helps me.
So I let them crank off on the side in a window, while I work on other stuff. That way I can get my iterative work done, while remaining quite productive.
[Update] I tried clang/LLVM and it worked (and was very fast). But the issue for the moment is the size of the ramdisk for the appliance, and adding another compiler runtime toolchain is going to make this larger. So this will take more time to correctly study.
I did note that 5.22.0 was released, so I grabbed that. Seems to build/test properly, and I am not getting the errors I was getting with 5.20.2. Sort of a meta-debugging … I am not digging into why I was getting the errors, bumping to 5.22.0 looks like it solves the build problem.