<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 30/05/12 08:28, Jez Tucker wrote:
<blockquote
cite="mid:39571EA9316BE44899D59C7A640C13F53059D725@WARVWEXC1.uk.deluxe-eu.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hello all<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> I’ve been having a pootle around ye olde
Internet in a coffee break and noticed that RDMA over Ethernet
exists.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a moz-do-not-send="true"
href="http://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet">http://en.wikipedia.org/wiki/RDMA_over_Converged_Ethernet</a><o:p></o:p></p>
<p class="MsoNormal"><a moz-do-not-send="true"
href="http://www.hpcwire.com/hpcwire/2010-04-22/roce_an_ethernet-infiniband_love_story.html">http://www.hpcwire.com/hpcwire/2010-04-22/roce_an_ethernet-infiniband_love_story.html</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Has anyone had any experience of using
this? (even outside GPFS)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I know GPFS supports RDMA with Infiniband,
but unsure as to RoCE / iWARP support.<o:p></o:p></p>
<p class="MsoNormal">It suddenly occurred to me that I have 10Gb
Brocade VDX switches with DCB & PFC and making things go
faster is great.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Perhaps the HPC crowd do this, but only via
IB?<o:p></o:p></p>
</div>
</blockquote>
I did have a look at this about a year ago, and thought it would be
great. But never thought people would be interested. and didn't find
anything within the GPFS docs or secret configs that indicated that
this is supported<br>
<br>
In most of our setups we do tend to stick with verbs-rdma, and that
is where most of our customer's are working with. <br>
<br>
It would be very interesting to see if it was ever supported, and to
see what kind of performance improvement we would get by taking the
tcp layer away<br>
<br>
maybe one of the devs could shed some light on this.<br>
<pre class="moz-signature" cols="72">--
regards,
Arif</pre>
</body>
</html>