<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
We have never bought any smaller than a DSS-G220, and they’ve always had the 2U servers (so SR650, in our gen2 systems), so I imagine it’s a little different, but we have 2x 2 port cards, all in use for Infiniband, and then often another Connect-X card for
high-speed ethernet (which we would bond for bandwidth/redundancy), so it would stand to reason that you’d have some form of that as an option on the G100.
<div><br>
</div>
<div>It’s configurable, within reason (eg. I’d imagine you can’t take out the minimum required, etc.).<br>
<div><br id="lineBreakAtBeginningOfMessage">
<div>
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
-- <br>
#BlackLivesMatter</div>
<div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
____<br>
|| \\UTGERS, |---------------------------*O*---------------------------<br>
||_// the State<span class="Apple-tab-span" style="white-space: pre;"> </span> | Ryan Novosielski - novosirj@rutgers.edu<br>
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus<br>
|| \\ of NJ<span class="Apple-tab-span" style="white-space: pre;"> </span> | Office of Advanced Research Computing - MSB A555B, Newark<br>
`'</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div><br>
<blockquote type="cite">
<div>On Jul 30, 2025, at 11:41, Jonathan Buzzard <jonathan.buzzard@strath.ac.uk> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div><br>
So these servers (at least in their SR630 V2 variant) come with three single port ConnectX-6 Dx cards capable of 200Gbps.<br>
<br>
The question is why do I need three? My thinking is that a LACP bond into a couple of switches doing MLAG (or insert whatever your vendor calls it) is ample connectivity? Noting that these are going into a pair of SN3700M switches at the full 200Gbps.<br>
<br>
So I don't get the purpose of the third card? What am I missing? Extra connectivity if you want to do Infiniband as well? Direct link between two of them?<br>
<br>
<br>
JAB.<br>
<br>
-- <br>
Jonathan A. Buzzard Tel: +44141-5483420<br>
HPC System Administrator, ARCHIE-WeSt.<br>
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG<br>
<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at gpfsug.org<br>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</body>
</html>