<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" >IBM ESS building blocks can have up to 3 dual port 10GBEth, 40GB Eth, 56GB IB, 100GB IB Mlx adapater cards, because we have 2 IO nodes this is up to a total of 12 ports per building block</div>
<div dir="ltr" >so there should not be any reason for this to fail.</div>
<div dir="ltr" > </div>
<div dir="ltr" >I regularly see a Mix of 10GB / 40GB or 10GB / IB configurations</div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<div dir="ltr" >Regards</div>
<div dir="ltr" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div dir="ltr" style="margin-top: 20px;" ><div style="font-size: 12pt; font-weight: bold; font-family: sans-serif; color: #7C7C5F;" >Andrew Beattie</div>
<div style="font-size: 10pt; font-weight: bold; font-family: sans-serif;" >Software Defined Storage - IT Specialist</div>
<div style="font-size: 8pt; font-family: sans-serif; margin-top: 10px;" ><div><span style="font-weight: bold; color: #336699;" >Phone: </span>614-2133-7927</div>
<div><span style="font-weight: bold; color: #336699;" >E-mail: </span><a href="mailto:abeattie@au1.ibm.com" style="color: #555" >abeattie@au1.ibm.com</a></div></div></div></div></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: "J. Eric Wonderley" <eric.wonderley@vt.edu><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: Re: [gpfsug-discuss] more than one mlx connectx-4 adapter in same host<br>Date: Thu, Dec 21, 2017 6:37 AM<br>
<div dir="ltr" >Just plain tcpip.
<div> </div>
<div>We have dual port connectx4s in our nsd servers. Upon adding a second connectx4 hba...no links go up or show "up". I have one port on each hba configured for eth and ibv_devinfo looks sane.</div>
<div> </div>
<div>I cannot find anything indicating that this should not work. I have a ticket opened with mellanox.</div></div>
<div>
<div>On Wed, Dec 20, 2017 at 3:25 PM, Knister, Aaron S. (GSFC-606.2)[COMPUTER SCIENCE CORP] <span dir="ltr" ><<a href="mailto:aaron.s.knister@nasa.gov" target="_blank" >aaron.s.knister@nasa.gov</a>></span> wrote:
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" ><div><div dir="ltr" > </div><br><br>We’ve done a fair amount of VPI work but admittedly not with connectx4. Is it possible the cards are trying to talk IB rather than Eth? I figured you’re Ethernet based because of the mention of Juniper.
<div dir="ltr" > </div>
<div dir="ltr" >Are you attempting to do RoCE or just plain TCP/IP?</div>
<div><div>
<div><div dir="auto" >On December 20, 2017 at 14:40:48 EST, J. Eric Wonderley <<a href="mailto:eric.wonderley@vt.edu" target="_blank" >eric.wonderley@vt.edu</a>> wrote:</div>
<blockquote style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px" type="cite" ><div dir="auto" ><div><div><div dir="ltr" >Hello:
<div> </div>
<div>Does anyone have this type of config?
<div> </div>
<div>The host configuration looks sane but we seem to observe link-down on all mlx adapters no matter what we do.</div></div>
<div> </div>
<div>Big picture is that we are attempting to do mc(multichassis)-lags to a core switch. I'm somewhat fearful as to how this is implemented in the juniper switch we are about to test. </div></div></div></div></div></blockquote></div></div></div></div><br>______________________________<wbr>_________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=STXkGEO2XATS_s2pRCAAh2wXtuUgwVcx1XjUX7ELNdk&m=Olk0lQk7rek9IplOjJ_2Vcd7P1LgVbnrSupC7O0hJHQ&s=I5Dq2T7aYvC87Wp12fsz6CRLw4uo2-RVnrnpxRYfYuA&e=" rel="noreferrer" target="_blank" >spectrumscale.org</a><br><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=STXkGEO2XATS_s2pRCAAh2wXtuUgwVcx1XjUX7ELNdk&m=Olk0lQk7rek9IplOjJ_2Vcd7P1LgVbnrSupC7O0hJHQ&s=hxNNpOkwGQ9zRmTnM3FEo5hgnPSUsPG0FNqZbK6eA6Q&e=" rel="noreferrer" target="_blank" >http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</a><br> </blockquote></div></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=STXkGEO2XATS_s2pRCAAh2wXtuUgwVcx1XjUX7ELNdk&m=Olk0lQk7rek9IplOjJ_2Vcd7P1LgVbnrSupC7O0hJHQ&s=hxNNpOkwGQ9zRmTnM3FEo5hgnPSUsPG0FNqZbK6eA6Q&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=STXkGEO2XATS_s2pRCAAh2wXtuUgwVcx1XjUX7ELNdk&m=Olk0lQk7rek9IplOjJ_2Vcd7P1LgVbnrSupC7O0hJHQ&s=hxNNpOkwGQ9zRmTnM3FEo5hgnPSUsPG0FNqZbK6eA6Q&e=</a></font></div></blockquote>
<div dir="ltr" > </div></div><BR>