<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.E-MailFormatvorlage18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:70.85pt 70.85pt 56.7pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="DE">Hi Jan!<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="DE">That great to hear <o:p></o:p></span></p>
<p class="MsoNormal">So we will try this <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="DE-AT" style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black;mso-fareast-language:DE-AT">Mit freundlichen Grüßen<br>
<b><i>Walter Sklenka</i></b><br>
</span><b><i><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:DE-AT">Technical Consultant</span></i></b><span lang="DE-AT" style="color:black;mso-fareast-language:DE-AT">
</span><span lang="DE-AT" style="color:#212121;mso-fareast-language:DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="DE-AT" style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black;mso-fareast-language:DE-AT"> </span><span lang="DE-AT" style="color:#212121;mso-fareast-language:DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black;mso-fareast-language:DE-AT">EDV-Design Informationstechnologie GmbH<br>
Giefinggasse 6/1/2, A-1210 Wien<br>
Tel: +43 1 29 22 165-31<br>
Fax: +43 1 29 22 165-90<br>
E-Mail: </span><a href="mailto:sklenka@edv-design.at" target="_blank"><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:#0563C1;mso-fareast-language:DE-AT">sklenka@edv-design.at</span></a><span lang="DE-AT" style="font-size:8.0pt;color:black;mso-fareast-language:DE-AT"><br>
</span><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black;mso-fareast-language:DE-AT">Internet:
</span><a href="http://www.edv-design.at/" target="_blank"><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:#0563C1;mso-fareast-language:DE-AT">www.edv-design.at</span></a><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black;mso-fareast-language:DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="DE"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>Von:</b> Jan-Frode Myklebust <janfrode@tanso.net> <br>
<b>Gesendet:</b> Thursday, December 9, 2021 11:25 AM<br>
<b>An:</b> Walter Sklenka <Walter.Sklenka@EDV-Design.at><br>
<b>Cc:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Betreff:</b> Re: [gpfsug-discuss] alternate path between ESS Servers for Datamigration<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">I believe this should be a fully working solution. I see no problem enabling RDMA between a subset of nodes -- just disable verbsRdma on the nodes you want to use plain IP.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> -jf<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Dec 9, 2021 at 11:04 AM Walter Sklenka <<a href="mailto:Walter.Sklenka@edv-design.at">Walter.Sklenka@edv-design.at</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear spectrum scale users!<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">May I ask you a design question?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We have an IB environment which is very mixed at the moment ( connecX3 … connect-X6 with FDR , even FDR10 and with arrive of ESS5000SC7 now also HDR100 and HDR switches. We still
have some big troubles in this fabric when using RDMA , a case at Mellanox and IBM is open .
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The environment has 3 old Building blocks 2xESSGL6 and 1x GL4 , from where we want to migrate the data to ess5000 , ( mmdelvdisk +qos)
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Due to the current problems with RDMA we though eventually we could try a workaround :<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If you are interested there is Maybe you can find the attachment ?
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We build 2 separate fabrics , the ess-IO servers attached to both blue and green and all other cluster members and all remote clusters only to fabric blue
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The daemon interfaces (IPoIP) are on fabric blue<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">It is the aim to setup rdma only on the ess-ioServers in the fabric green , in the blue we must use IPoIB (tcp)
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Do you think datamigration would work between ess01,ess02,… to ess07,ess08 via RDMA ?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Or is it principally not possible to make a rdma network only for a subset of a cluster (though this subset would be reachable via other fabric) ?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thank you very much for any input !<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE">Best regards walter
</span><span lang="DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE"> </span><span lang="DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE"> </span><span lang="DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE"> </span><span lang="DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE-AT" style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Mit freundlichen Grüßen<br>
<b><i>Walter Sklenka</i></b><br>
</span><b><i><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray">Technical Consultant</span></i></b><span lang="DE-AT" style="color:black">
</span><span lang="DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE-AT" style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black"> </span><span lang="DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black">EDV-Design Informationstechnologie GmbH<br>
Giefinggasse 6/1/2, A-1210 Wien<br>
Tel: +43 1 29 22 165-31<br>
Fax: +43 1 29 22 165-90<br>
E-Mail: </span><a href="mailto:sklenka@edv-design.at" target="_blank"><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:#0563C1">sklenka@edv-design.at</span></a><span lang="DE-AT" style="font-size:8.0pt;color:black"><br>
</span><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black">Internet:
</span><a href="http://www.edv-design.at/" target="_blank"><span lang="DE-AT" style="font-size:8.0pt;font-family:"Arial",sans-serif;color:#0563C1">www.edv-design.at</span></a><span lang="DE-AT"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE-AT"> <o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>