[gpfsug-discuss] DSS-G100 and networking

[Ryan Novosielski]

We have never bought any smaller than a DSS-G220, and they’ve always had the 2U servers (so SR650, in our gen2 systems), so I imagine it’s a little different, but we have 2x 2 port cards, all in use for Infiniband, and then often another Connect-X card for high-speed ethernet (which we would bond for bandwidth/redundancy), so it would stand to reason that you’d have some form of that as an option on the G100.

It’s configurable, within reason (eg. I’d imagine you can’t take out the minimum required, etc.).

--
#BlackLivesMatter
____
|| \\UTGERS,     |---------------------------*O*---------------------------
||_// the State  |         Ryan Novosielski - novosirj at rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ  | Office of Advanced Research Computing - MSB A555B, Newark
     `'

On Jul 30, 2025, at 11:41, Jonathan Buzzard <jonathan.buzzard at strath.ac.uk> wrote:


So these servers (at least in their SR630 V2 variant) come with three single port ConnectX-6 Dx cards capable of 200Gbps.

The question is why do I need three? My thinking is that a LACP bond into a couple of switches doing MLAG (or insert whatever your vendor calls it) is ample connectivity? Noting that these are going into a pair of SN3700M switches at the full 200Gbps.

So I don't get the purpose of the third card? What am I missing? Extra connectivity if you want to do Infiniband as well? Direct link between two of them?


JAB.

--
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG


_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20250730/728e17a8/attachment-0001.htm>