[gpfsug-discuss] alternate path between ESS Servers for Datamigration

[Jonathan Buzzard]

On 09/12/2021 16:04, Douglas O'flaherty wrote:
> 
> Though not directly about your design, our work with NVIDIA on GPUdirect 
> Storage and SuperPOD has shown how sensitive RDMA (IB & RoCE) to both 
> MOFED and Firmware version compatibility can be.
> 
> I would suggest anyone debugging RDMA issues should look at those closely.
> 
May I ask what are the alleged benefits of using RDMA in GPFS?

I can see there would be lower latency over a plain IP Ethernet or IPoIB 
solution but surely disk latency is going to swamp that?

I guess SSD drives might change that calculation but I have never seen 
proper benchmarks comparing the two, or even better yet all four 
connection options.

Just seems a lot of complexity and fragility for very little gain to me.


JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG