… and it can talk …

[root@skunkworks-prototype-n2 ~]# ifinfo device: address/netmask MTU Tx (MB) Rx (MB) eth0: addr not set/mask not set 1500 0.000 0.000 eth1: addr not set/mask not set 1500 0.000 0.000 eth10: addr not set/mask not set 1500 0.000 0.000 eth11: addr not set/mask not set 1500 0.000 0.000 eth12: addr not set/mask not set 1500 0.000 0.000 … Read more… and it can talk …

Its … alive ….

Our little skunkworks project boots!!! Mwahahahaha! Must check off on our list design build boot ??? profit (or something)! Note to self: work on eeeeevul laughter …. And get step 4 ironed out too.

After 4 years, our deskside JackRabbit unit decided to shrug off its mortal coil

икона за подарък… and in the process, take down a drive, 5 of its friends, and our RAID card. We have backups from before the move (15+ days old … sigh). We’ve decided to go full monty on the new unit. Its a JackRabbit JR4 with 12x 2TB drives, 2 hot spares, and 10 disk … Read moreAfter 4 years, our deskside JackRabbit unit decided to shrug off its mortal coil

Updating a design to modern concepts …

So in order to (really) bring my monitoring app into the modern age, I want to change its flow from a synchronous on-demand event driven analysis and reporting tool, to an asynchronous monitoring and analysis tool, with an on-demand “report” function which is basically a presentation core atop the data set.
There are many reasons for this. Not the least of which is that this should be far more efficient at handling what I want to do … not to mention more responsive. I also don’t really want to do this as many independent processes … past history with debugging many independent but functionally interdependent processes.
What we are fundamentally doing is parsing logs. Right now, its apache logs, but a well designed system should be able to parse any logs, with the addition of a basic parser code (no, not a grammar … but something nice and simple).
So what if we wanted to run the parser when the log gets updated? Ok, I know … there are some codes that are smart enough to trigger an event upon an action. Assume for the moment that we are dealing with something where this isn’t true.
Let me go far afield from Apache. And look at Gluster. Its logs are (at best) a horrible … horrible mess. Extracting anything useful from them is very hard. And unfortunately, with many more people depending upon it, we have to parse the output, and at generate some sort of signal when dejecta impacts the rotating air movement system.
But the same is true of other servers as well. The issue is that there really is no good standard for this right now. Something with one of the message queues and a nice standard format? Would be nice. Until then, we have to ()*&*(&^%&% parse ()*&*&^%$%$%$ logs.
Apache is my stand in for a good test case.
So, rather than wait for an external query event to look at stuff, why not set up a nice asynchonous inotify based log reader? Maintain local state only during program execution. Read till the end of the file on startup, calculate the offset, turn on an inotify listener, and only scan the changes from the offset to the end of the file on the write event … updating the internal offset, and doing whatever needful thing we need to do after parsing the data?
Yeah, its more complex. But it gives us far more power.

Read moreUpdating a design to modern concepts …

Every now and then, the truth leaks out

Good article from Matt Asay in The Register today.

While there are some stand-out success stories in Silicon Valley, there is also a raft of startups pushing product features masquerading as companies. Some of these will be acquired by the likes of Zynga and Amazon, and some will go public. But most won’t. In fact, building in the shadow of the giants in the hopes of getting acquired, or of beating them, is a losing strategy for the vast majority of companies.

This is about as truthful as it gets. There are many tiny startups, pulling in various fractions of $1M to more than $10M to develop … product features.
Is this really the right approach for VCs?
And this opens up some interesting new questions on startups and their product offerings themselves. Take Netflix. Running on Amazon S3. And what does Amazon do? Decide they want that market and go after it. To wit

Read moreEvery now and then, the truth leaks out

On my fun end of a week …

(this was actually a while ago, just getting to publishing it now).
Friday, I drove up to a local University to drop off our bid. I sent a note beforehand to let them know I might be a few minutes late, there was construction. Sure enough, got caught in a 30 minute slowdown.
I was 13 minutes late.
They said, “hey thats great. We won’t look at it”
Then on the way back, the old landlord refused to acknowledge that we were tenants, so they refused to refund our deposit.
As I told my wife later in the day, had I been kicked in the groin, my day would have improved some.

Read moreOn my fun end of a week …

2 out of 3 ain't bad

No, not Meatloaf lyrics. A few years ago, I guessed that the HPC market was going to bifurcate or possibly trifurcate. Well, its about 3 years on, and bifurcate it did.
Accelerators (in the form of GPUs) are everywhere. I was dead on correct in almost every aspect of what I had predicted (privately to VCs, from whom we couldn’t raise a cent in the early/mid 2000’s for this market).
Remote cluster/clouds with dropping prices per CPU hour are taking over sections of HPC, and we see some impact upon purchase decisions made by people buying clusters. Buy what you will use day to day, and buy the extra cycles you need when you need em. Just in time cycle acquisition.
Got these right. And yes, we even tried raising money for the cloud bit in 2005/2006. This time from a (short sighted) state program and VCs. Had a large customer lined up, had a VC willing to chip in, just needed the state program to agree to this.
That state program is now generally seen as an abject failure in its previous incarnation … it was supposed to help start up companies with good ideas, VCs, and likely customers. Go figure.
Of course, I got some things wrong.
I guessed that “muscular desktops” and “personal supercomputers” would become the norm.
Boy was I wrong.
Desktops, the ones that people bought, were cheap units for the most part. The big powerful supercomputer in a deskside chassis? Not selling so much. More than 8 processor cores and more than 16 GB ram? Not so interesting to people.
I had bet they were, and we built the Pegasus deskside units around them. These are basically very powerful computers with many cores, huge amounts of ram, accelerators, IO, networking, and graphics.

Read more2 out of 3 ain't bad

Good programming tools and good program implementation

Way back in my early days at web programming stuff, I started out with HTML::Mason as a templating engine. There is nothing wrong with Mason, its actually quite good. But it encourages the same sort of “code-in-page” designs that the entire language of PHP was built around.
I’m mostly a Perl guy for application level stuff these days … have done my time with Fortran, Python, x86 assembly, C/C++, and many others. I have my biases, and I understand them. I simply don’t have time to let a language get in the way of expressing what I want to do.
So I wrote a web log analyzer a while ago. It was a Mason application, with the logic and the page intertwined. It was a mess (just like most PHP turns out). Very hard to understand, but it worked, and darn it, if I wasn’t using it for the last ~8 years or so.
About 5 years ago, I started working with the Catalyst framework for web sites. And I tried rewriting this app in it. But it was, unfortunately, not easy. So I gave up and left it alone.
With my recent EC2 foray, I thought about revisiting this code. I’ve since switched all of our development over to Mojolicious, which is very easy to write code for. And easy to separate out the view and controller functions (PHP and Mason encourage a fused view and controller).
Mojolicious is better in that its very easy to write and deploy … its installation dependency radius is minimal (e.g. how much extra crap you have to install to make it work). Catalyst is huge in this regard, and Jifty, another framework I’ve liked in the past is Ginormous. There are some other similar to Mojolicious frameworks out there, like Dancer. But Dancer (and Catalyst) all use Moose, and Moose has a fairly huge dependency radius (and it is slow).

Read moreGood programming tools and good program implementation