<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
If I understand what you’re asking correctly, we used to have a cluster that did this. GPFS was on Infininiband, some of the compute nodes were too, and the rest were on Omnipath. There were routers in between with both types.<br>
<br>
<div dir="ltr">Sent from my iPhone</div>
<div dir="ltr"><br>
<blockquote type="cite">On Aug 21, 2023, at 13:55, Kidger, Daniel <daniel.kidger@hpe.com> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style>@font-face { font-family: "Cambria Math"; }
@font-face { font-family: Calibri; }
p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; }
span.EmailStyle17 { font-family: Calibri, sans-serif; color: windowtext; }
.MsoChpDefault { font-family: Calibri, sans-serif; }
@page WordSection1 { size: 612pt 792pt; margin: 72pt; }
div.WordSection1 { page: WordSection1; }</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I know in the Lustre world that LNET routers are used to provide RDMA over heterogeneous networks.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Is there an equivalent for Storage Scale?<o:p></o:p></p>
<p class="MsoNormal">eg if an ESS uses Infiniband to connect directly to Cluster A, could that InfiniBand RDMA fabric be “routed” to ClusterB that has RoCE connecting all its nodes together and hence the filesystem mounted?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">ps. The same question would apply to other usually incompatible RDMA networks like Omnipath, Slingshot, Cornelis, … ?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Daniel<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="line-height:12.0pt"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-GB">Daniel Kidger<br>
</span></b><span style="color:black;mso-fareast-language:EN-GB">HPC Storage Solutions Architect, EMEA</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-GB"><br>
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#040400;mso-fareast-language:EN-GB"><a href="mailto:daniel.kidger@hpe.com"><span style="color:#040400">daniel.kidger@hpe.com</span></a><br>
<br>
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black;mso-fareast-language:EN-GB">+44 (0)7818 522266 <br>
<br>
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#040400;mso-fareast-language:EN-GB"><a href="http://www.hpe.com/"><b><span style="color:black;text-decoration:none">hpe.com</span></b></a><br>
<br>
<br>
</span><span style="mso-fareast-language:EN-GB"></p>
<div><image001.png></div>
<o:p></o:p></span>
<p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<span>_______________________________________________</span><br>
<span>gpfsug-discuss mailing list</span><br>
<span>gpfsug-discuss at gpfsug.org</span><br>
<span>http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</span><br>
</div>
</blockquote>
</body>
</html>