Cool asymmetric network performance happened to mess up a customer benchmark

A bunch of Unison systems, a 40 GbE network interconnecting them, and a bunch of client nodes on 40GbE -> 4x 10GbE links (to accomodate enough clients for the load testing).

40GbE < -> 40GbE works great. Full bandwidth, only minor oddities (single thread performance around 27Gb/s, need multiple threads to hit 40).

10GbE < -> 10GbE works great. Full bandwidth, nothing odd.

10GbE -> 40GbE works great, get about the expected bandwidth (10GbE). Writes work well.

40GbE -> 10GbE is sorta kinda broken. We get anywhere from 170Mb/s to 1.5Gb/s.

My switch FW is badly out of date, so its time to upgrade. Checking in to see if this is a known problem, but … wow … wasn’t expecting that.

Might need to see if I can drop a few clients and go all 40GbE. If so, we’ll do that for the moment.

Viewed 21123 times by 2108 viewers

One thought on “Cool asymmetric network performance happened to mess up a customer benchmark

  1. Hi Joe, last time I saw such a strange performance drop in one direction was with a switch that had something called “storm control” or “flood control” enabled and disabling it helped. Just in case your switch has something similar…
    – Sven

Comments are closed.