<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style id="owaParaStyle" type="text/css">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1" style="word-wrap: break-word;">
<div style="direction: ltr;font-family: Arial;color: #000000;font-size: 10pt;">Here's the stack:<br>
<br>
<a href="https://www.openfabrics.org/resources/ofed-for-linux-ofed-for-windows/ofed-overview.html" target="_blank">https://www.openfabrics.org/resources/ofed-for-linux-ofed-for-windows/ofed-overview.html</a><br>
<br>
VERBS is supported over 10GbE. It should work if OFED VERBS == IBM VERBS.<br>
<div><br>
<div style="font-family: Tahoma; font-size: 13px;"><font face="Arial" size="2"><span style="font-size: 10pt;">
<div>---</div>
<div>Jez Tucker</div>
<div>Senior SysAdmin</div>
<div>Rushes<br>
<a href="http://www.rushes.co.uk">www.rushes.co.uk</a><br>
</div>
</span></font></div>
</div>
<div style="font-family: Times New Roman; color: rgb(0, 0, 0); font-size: 16px;">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF573737"><font color="#000000" face="Tahoma" size="2"><b>From:</b> gpfsug-discuss-bounces@gpfsug.org [gpfsug-discuss-bounces@gpfsug.org] on behalf of Barry Evans [bevans@canditmedia.co.uk]<br>
<b>Sent:</b> 02 July 2012 22:16<br>
<b>To:</b> gpfsug main discussion list<br>
<b>Subject:</b> Re: [gpfsug-discuss] HPC people - interconnects<br>
</font><br>
</div>
<div></div>
<div>It's verbs support you need - not sure where OFED is up to with verbs over 10G, if that's even on the cards
<div><br>
</div>
<div>Cheers,</div>
<div>B<br>
<div><br>
</div>
<div><br>
<div>
<div><br>
<div>
<div>On 2 Jul 2012, at 15:05, Jez Tucker wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium;">
<div bgcolor="white" lang="EN-GB">
<div class="WordSection1" style="">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);">Theoretically using the OFED stack with a supporting card (E.G. Myrinet 10GbE) then you should be able to leverage RDMA.</span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);">That said, I’ve not tried it yet. It’s on my list of things to R&D.</span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);"> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);">OFED/ROCE/iWARP:</span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);"> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);"><a href="http://voidreflections.blogspot.co.uk/2011/08/soft-roce-alternative-to-soft-iwarp.html" style="color: blue; text-decoration: underline;" target="_blank">http://voidreflections.blogspot.co.uk/2011/08/soft-roce-alternative-to-soft-iwarp.html</a></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);"> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="color: rgb(31, 73, 125);"> </span></div>
<div style="border-style: none none none solid; border-left: 1.5pt solid blue; padding: 0cm 0cm 0cm 4pt;">
<div>
<div style="border-style: solid none none; border-top: 1pt solid rgb(181, 196, 223); padding: 3pt 0cm 0cm;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<b><span style="font-size: 10pt; font-family: Tahoma,sans-serif; color: windowtext;" lang="EN-US">From:</span></b><span style="font-size: 10pt; font-family: Tahoma,sans-serif; color: windowtext;" lang="EN-US"><span class="Apple-converted-space"> </span><a href="mailto:gpfsug-discuss-bounces@gpfsug.org" target="_blank">gpfsug-discuss-bounces@gpfsug.org</a>
[mailto:gpfsug-discuss-bounces@gpfsug.org]<span class="Apple-converted-space"> </span><b>On Behalf Of<span class="Apple-converted-space"> </span></b>Arif Ali<br>
<b>Sent:</b><span class="Apple-converted-space"> </span>25 June 2012 15:14<br>
<b>To:</b><span class="Apple-converted-space"> </span><a href="mailto:gpfsug-discuss@gpfsug.org" target="_blank">gpfsug-discuss@gpfsug.org</a><br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [gpfsug-discuss] HPC people - interconnects</span></div>
</div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
On 25/06/12 15:08, Jez Tucker wrote:</div>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
Do you all use IB?</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
Has anyone tried RDMA over 10G via the OFED stack?</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="font-size: 12pt; font-family: 'Times New Roman',serif;"> </span></div>
</blockquote>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri,sans-serif; color: black;">
<span style="font-size: 12pt; font-family: 'Times New Roman',serif;">Most of our customers we use RDMA over verbs<br>
<br>
Is this the same thing you mentioned a few weeks ago with respect to ROCE. Does gpfs even support this?<br>
<br>
</span></div>
<pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; color: black;">-- </pre>
<pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; color: black;">regards,</pre>
<pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; color: black;"> </pre>
<pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; color: black;">Arif</pre>
</div>
</div>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
<a href="mailto:gpfsug-discuss@gpfsug.org" target="_blank">gpfsug-discuss@gpfsug.org</a><br>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss</div>
</span></blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>