# SRP joy

ok, not really. Late last night, while benchmarking some alternative mechanisms to connect {MD,OS}S to their respective {MD,OS}T for a Lustre design we are proposing for an RFP, I decided to revisit SRP. I liked SRP in the past, it was a simple protocol, SCSI over RDMA. How could you go wrong with this?
Well, I found out last night.
I put our stack on a DeltaV connected with a 10GbE and QDR IB ports to our respective switches. Lit up the target, iSCSI over 10GbE was nice, wanted to try IB. iSCSI (not the iSER) over IB isn’t so nice, about 1/2 the performance of 10GbE. Yeah, iSER will be tried next, but I wanted to see how SRP behaved.
So the target was set up, and it is visible.

-bash-4.1# scstadmin -list_target
Collecting current configuration: done.
Driver  Target
---------------------------------------------------
ib_srpt ib_srpt_target_0
iscsi   iqn.2011-10.com.scalableinformatics:blockio
iqn.2012-10.com.scalableinformatics:fileio


Connect to it … and …
Hey, the dv4 isn’t responding … why? It was 1am, I figured “pilot error” so I’d look this morning.
Get in, fire up the console. Unit is up. Try connecting to this again and …
KERNEL PANIC on dv4
Damn. I liked SRP. No real time to debug for this, onto iSER.
As a rule of thumb, I think its … I dunno … a bad idea to kernel panic on a connection attempt. It might put users off, a bit.

### 2 thoughts on “SRP joy”

1. Did you report it upstream? What does the trace look like?
With recent kernel versions we also had some issues.I tested 3.1-rcX and could fix a serious IPoIB issue before 3.1 was released, but first started to test 3.2 when it was released. Traces were a bit confusing first, as it looked like FhGFS was failing, but then it was an unitialized variable in the IB stack…
I guess linux-rdma needs a bit more manual or automatic testing before making a release… And then I’m not sure how many people are using ib_srpt at all. If a hardware vendor has issues, they might simply fix or workaround it inhouse, without releasing the patch… (which is probably even allowed, as OFED is BSD/GPL dual licensed).
Cheers,
Bernd