The virtues of test-driven methodology in program development

Short post, directly related to HPC. Short version: When developing new features for a program, routine, method, whatever, it is a good idea to test it how you think you will use it.

Ok. This is a dumb one. Working on our Tiburon installer. Will help with managing clusters of things. Like Linux/other compute clusters. And JackRabbits (storage clusters). Part of the installer makes use of a Perl module I wrote named DBIx::SimplePerl. This module makes using databases, at least the simplified use of databases, almost trivial in Perl in that you don’t have to write/use/see SQL. I consider this to be a good thing, as SQL varies from DB to DB (standards? we don’t need no steenkeen standards :( ). I don’t want to deal with writing it, as that means I have to debug it. So, I let the module write it for me. This approach works. I don’t have to think about the database very much. This lets me focus on my application. We add lots of intelligence to a bootp/pxe/dns server. Allows us to have very smart loading, diagnostics, yadda yadda yadda. Ok. So I built this method called “db_next” as an iterator. It would return a reference to a data structure if there was more data to return, or return a null. This works fine, as long as you use it this way. The problem is when you start to try to get fancy, and use it in a different context. Which I decided to do in the application, as it made expressing the code cleaner. Of course I didn’t build support for this into the method. And worse, I didn’t build a test for this mode of operation. I just used it. And it failed. Miserably. Took a quick-n-dirty test to show me the failure. Said test is now part of the module testing suite. In the past, whenever developing a new feature, I usually build a test case, and write the code that framework. I find that this works quite well in terms of building good code building blocks. Sometimes languages don’t do what they say (yeah, thats my story and I am sticking to it), so this methodology lets me capture what will happen versus what the spec says will happen. Also helps me understand failure modes, and how to detect them and deal with them. In this module, or at least the previous version, I had forgotten that point. Fixed, new version uploaded. Our Tiburon installer (used in a number of places, we finally put a name to the project we had been developing) now works better, without an embarrassing failure, due in large part to a missing test case which would have alerted me to the issue.