On bottlenecks

At BiO Jeff linked to a story from earlier this year on where the bottlenecks really are in computing. The article he linked to was posted in the American Scientist online magazine.

The major thesis of the article is that performance is not the only, or as the title implies, real bottleneck, in scientific computing. I might suggest reading the article if you get the chance. I don’t agree with their major thesis implied in the title. Since they don’t spend much time on it either, I get the sense that this is an editorially derived title. The basic premise of the article is that scientists don’t use good development techniques. They don’t use source control, or other systems. They use poor editors. No symbolic debuggers. Buried in one of the pages is a point I do agree with, that it is imperative to be able to write and test good code. The subsequent pages talk about the efforts to educate scientists on needed basic tools. They focus on Python as an automation tool. Ok … I am a big believe in using tools that work, and that you can make work without much effort. That means tools which function well, and are well understood by the user. The authors take issue with vi. Likely they would not use emacs. Turns out they may not have seen vim and some of the nice bits with it. I like vim in that it works the same on every platform (linux, windows, …). It is not an IDE. It is a text editor. I admit to ignoring emacs, as I do not believe in the one-monolithic-huge-tool-to-rule-them-all approach. I have used vi and vim on everything from Crays through tiny systems. It works, well, consistently across a huge range of platforms. I also use nedit. Again, with the cygwin toolset on windows, it works the same on Linux and windows. If you don’t know what cygwin is, and you use windows, and have a need to build and run unix/linux apps on windows, have a good hard look at it. It should be included by Microsoft with windows, or at least pointed to as a download of significant value. Microsoft would gain much by funding a few engineers working on it to improve it. The authors decry the lack of use of symbolic debuggers. Such tools do have their place. However, a careful developer will be fully cognizant that symbolic debuggers can themselves have bugs. Bugs in your debuggers are not fun. Not hypothetical. I have been bitten by this in the past. Under Linux, you can use the DDD frontend to gdb and others. It is nice for C/Fortran code. Very useful in conjunction with core dumps to help you figure out what and where your code died. Not fool proof though. A helpful –debug switch with a smattering of carefully placed printf equivalents is very helpful. For my dynamic language work (Perl mostly, but will be playing with Ruby), I use Komodo IDE. Is it faster than printf’s and other techniques? Well, possibly. IDEs and integrated debuggers slow things down tremendously. This isn’t wrong, this is simply what they do in order to present the best information to you on actual program state. I personally use a mix of IDE, printf, and various tests that I call sanity checks. These are unit tests that I design/build to work with my code. And that brings up the next point. The author’s claim that little testing is done on code. I can’t speak to others actions. Our code is built along with the test case(s). This can be dangerous as well though, as unit tests could test code paths that are not executed, or that are, but in a different state than you test. This usually means you need to increase the range of your tests. Finally they decry the lack of use of source code control systems. In the open source world, we have had SCCS/RCS for a long time. CVS has been around for a while. It was not fun to use. Couldn’t handle things very well. SVN is a step up, but still not great. New ones have emerged including git. I have used CVS and SVN. Likely we will keep using SVN for a while. Then again, not everyone uses such systems. Usually they require additional maintainence that is beyond that which a scientist can or is willing to do. Very little was mentioned of the thesis suggested by the title though. I would argue that the question imputed by the title is a very important one. How do you maximize productivity of a computational scientist or engineer while minimizing errors? Does higher productivity translate into higher program performance? Does a more productive scientist or engineer need more cycles? Fewer? My thoughts are that it is best to give such a user an excellent and flexible set of tools for their development. Help them learn the ropes of which ever source management and debugging systems are available. Provide them with excellent portable languages and facilities to use these languages. Enable them to think abstractly, and easily, with minimal error, translate these abstract thoughts and symbology into operational programs. Allow them to inspect aspects of the calculations. Let the computer do the hard work of writing great code (note: as of this writing compilers don’t do a great job of this now with a not-so-abstract computer language, so this is a “hard” problem for a more abstract language). Really you need a solution that allows you to go half-way… express an algorithm succinctly, while minimizing the coding and enabling maximum clarity. No, this means eschewing overly verbose languages that force you to adapt a problem to their whims. It also should mean eschewing overly terse languages that force you to build a language parser in your head to read your own code. Let the computers read/write the terse languages. It means that you need powerful means of mapping algorithms onto your machine, and would allow you to get as coarse grained or fine grained as you need. I look at a language like Chapel as a good example. This is something I think I could use to simply express some of the problems out there. Fortran90 and derivatives were a good step in that direction. As are OO languages. But OO is anathema to high performance, but it makes programming easier as you can define interfaces better. And you can integrate sanity checks better. Its just that some of the programming practices that OO often enables are things to be avoided for performance. But this is a a solvable problem. A language that allows you to specify your algorithm in a meaningful fashion without requiring you to write a hundred lines of code to implement it or the interface is something good. And this gets into parallel programming as well. Make these things as easy as possible. Across platforms. I like the OpenMP semantics. It makes programming parallel code very simple, with restrictions. MPI is harder. You have more flexibility with MPI. You have more comfort and ease of use with OpenMP. Everything has a cost. Better productivity comes with easier to use tools. Easier tools rarely perform as well as harder to use more flexible tools. As we advance in the multi-core age, and as APUs (Accelerator Processor Units) start appearing more and more, we are going to need better and simpler interfaces to use these things. As someone recently noted on the Beowulf list, you want your program to strive to be a library. Simple to use, with a well defined interface. Wouldn’t it be great to drop an APU in to a machine, install a library and a driver, and just enjoy faster results? With no coding changes? That would be nice. :) And easy to use. Adding additional cores won’t make your current serial programs faster. It might in some cases make your parallel programs faster. However, you will have hierarchies of parallelism and resources, which you have to manage in your program to effectively use. A good interface and a good portable platform would go a long way to improving productivity and performance. In my opinion, the real bottleneck is the wetware. And this is exactly where you want it. Not in the code development. Or in the processing. You want to follow Hamming’s idea and compute for insight. That is, after all, what computational science and engineering is all about.