Given that I am starting a new position tomorrow, I wanted to build a new deskside machine to use. One of my older cluster machines died a few weeks ago, and I needed to replace it. My deskside machine at the time, was an AMD Epyc Rome unit with 16 physical cores, 128 GB RAM, NVidia 3060Ti , 8 TB ZFS mirror for /home, and some NVMe for boot.
AMD Epyc Rome is a Zen1 based machine. Nothing wrong with it, it makes for a nice server. Not that fast as a desktop. So I figured I'd see what I could build for about $1500 USD.
I shopped around, and eventually got an AMD 7950x with 16 physical cores at 4.5GHz, 128 GB RAM, 2x 1TB NVMe, 2x PCIe gen5 x16 slots (will be running x8/x8), a water block CPU cooler/fan unit, and an ASUS Creator motherboard with 4x NVMe slots. I also picked up a used AMD MI50 Instinct accelerator on Ebay for ~$200 from China. Probably a knockoff, but at least I should be able to run ROCm on it.
Yeah, I wish I had budget for a ThreadRipper with more PCIe lanes. The CPU portion of that seems to start around $4k USD or so, and the motherboards aren't cheap. Plenty of PCIe lanes though ...
I built the machine last week, and have been testing it/tweaking it for the past few days. I wound up pulling out the mobile disk rack after building a new ZFS volume and copying the relevant data over. There was only 660GB of data on the volume, and it fit easily into the 1TB mirror. I trimmed it after copying. Now the unit is silent until I wind up the CPU with computational load.
I've been thinking what to do with the bulk (SRD or Spinning Rust Disks) storage for this. I'd noticed that new chassis don't leave very much room for mobile disk racks with hot swap capability. Most have physical internal holders for 3.5 inch disks, but no infra for hot swap. I've looked at the various alternatives and concluded that I likely need to make the disk rack an external USB 3.2 connected one. The old disk rack had a broken connector, so it is getting tossed, but the disks are still good.
So that is in my future, though I am looking for a (nearly) silent version to attach to this. Loud fans are out.
All this said, the unit is up. Zen4 cores. And it is bloody fast. Easily the fastest box I've (ever) built. Not simply due to the clock speed, though that is a factor. Its nice building/running AVX512f code on this.
Currently running a linux 6.2 kernel via Linux Mint 21.2. AMD ROCm installed, and ... and ...
Still doesn't recognize the built in Radeon as a HIP device. According to the specs in 5.6.1 ROCm, it will recognize the MI50 though. So in a few weeks, hopefully, I'll have a somewhat accelerated deskside unit to play with. I wish I could do this with the ThreadRipper, though I am trying to keep the budget reasonable.
And yeah, I think my 850W supply will need an upgrade. I've got a 1250W sitting on the side I need to test, so if that works, I'll swap it out when the MI50 arrives.
I plan on doing significant amounts of playing/personal development projects around these things. Experiments/code to be pushed to github.
Ad astra!