Did distributed memory really win?

If you asked me years ago, I would have said, yes, of course it did.

Now I am having second thoughts.

Our processors have 4, 6, 8, 12 cores, and soon more like 16 and up. All sharing a set of pipes to RAM. Programming these can be done either with a distributed memory interface like MPI, or a much simpler interface like OpenMP.

Vector processors ala GPU, Knights Ferry/Bridge are coming out which are little more than massive numbers of PEs and shared memory.

As you get 48+ processors in your deskside, the refrain I am hearing from customers is, why get a cluster when you can get almost 50 processors next to your desk? The latter are much easier to manage than the former.

I dunno … I think shared memory simply took time to regroup. Some things you will only be able to do with distributed memory coding, but I suspect more interest will be given to shared memory and multi/many core programming going forward.

Viewed 8876 times by 2143 viewers

2 thoughts on “Did distributed memory really win?

  1. Shared memory did not take time to regroup. There simply weren’t enough transistors to pack dozens of cores on a chip earlier. So the only way to scale was to gang together many cores over Ethernet.

    Personally, I couldn’t care less about MPI fading away. That API is a disaster. I am surprised something like BSP didn’t take off in that domain.

  2. I think shared-memory is going to come back heavily as multicore processors are becoming cheaper. I can think of many applications that can benefit from the added power of GPGPUs and multicore processors, but that simply wasn’t a mass concern several years ago. As the Beowulf cluster started to become the de facto standard for most users, it was inevitable for distributed memory-based programs to rise.

    The cluster I’m working on at my university has 8-core systems. This gives us a great deal of flexibility, and with multicore processors becoming cheaper and cheaper I think the one that is going to win is a hybrid approach.

Comments are closed.