Here's performance results. Max latency is 0.31 msec.
####################################################
min_latency_ns: 708
avg_latency_ns: 63723
max_latency_ns: 310795
jitter_ns: 30940
####################################################
Regarding L3 PFC support. In my understanding it's a ROCE requirement. I view RoCE as east-west traffic, since its not leaving the boundaries of the switch fabric, I think non-blocking leaf/spine fabric should be sufficient there. Please correct me if'm missing something.
SoftGate, the Linux router where we use DPDK, is designed for North/South traffic, sitting on the border of the data center, so we don't expect RoCE traffic there, but again, please let me know if you see it differently.
You are right about RoCE traffic. It is mainly East-West traffic. However NTT has a project called IOWN and in the scope of that project as I know they are trying to build North-South traffic based on the RoCE technology. In the case when you will decide to use your SoftGate routers inside the datacenter it would make sense to have a multipath routing capabilities and also solution for "PFC spreading blocking problem". This problem is not particularly bound with RoCE transport layer but instead for a MPI use cases. A several big infrastructure companies built their own network transport layer and in house hardware to solve those problems.