<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE-AT" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi Olaf!!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Many thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">OK well we will do mmvdisk vs delete
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"> So <o:p>
</o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">#mmvdisk vs delete … -N ess01,ess02….. would be correct , or?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Best regards walter
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org>
<b>On Behalf Of </b>Olaf Weiser<br>
<b>Sent:</b> Donnerstag, 9. Dezember 2021 13:04<br>
<b>To:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Cc:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Subject:</b> Re: [gpfsug-discuss] alternate path between ESS Servers for Datamigration<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Hallo Walter,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">;-)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">yes !AND! no ..<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">for sure , you can specifiy a subset of nodes to use RDMA and other nodes just communicating TCPIP<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">But that's only half of the truth .<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">The other half is.. who and how , you are going to migrate/copy the data<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">in case you 'll use mmrestripe .... you will have to make sure , that only nodes, connected(green) and configured for RDMA doing the work<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">otherwise.. if will also work to migrate the data, but then data is send throught the Ethernet as well , (as long all those nodes are in the same cluster)<o:p></o:p></span></p>
</div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">laff<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #AAAAAA 1.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">----- Ursprüngliche Nachricht -----<br>
Von: "Walter Sklenka" <<a href="mailto:Walter.Sklenka@EDV-Design.at">Walter.Sklenka@EDV-Design.at</a>><br>
Gesendet von: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@spectrumscale.org</a><br>
An: "'gpfsug-discuss@spectrumscale.org'" <<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.org</a>><br>
CC:<br>
Betreff: [EXTERNAL] [gpfsug-discuss] alternate path between ESS Servers for Datamigration<br>
Datum: Do, 9. Dez 2021 11:04<br>
<o:p></o:p></span></p>
<div>
<p style="margin:0cm"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Dear spectrum scale users!<o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">May I ask you a design question?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">We have an IB environment which is very mixed at the moment ( connecX3 … connect-X6 with FDR , even FDR10 and with arrive of ESS5000SC7 now also HDR100 and HDR
switches. We still have some big troubles in this fabric when using RDMA , a case at Mellanox and IBM is open .
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">The environment has 3 old Building blocks 2xESSGL6 and 1x GL4 , from where we want to migrate the data to ess5000 , ( mmdelvdisk +qos)
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Due to the current problems with RDMA we though eventually we could try a workaround :</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">If you are interested there is Maybe you can find the attachment ?
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">We build 2 separate fabrics , the ess-IO servers attached to both blue and green and all other cluster members and all remote clusters only to fabric blue
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">The daemon interfaces (IPoIP) are on fabric blue</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">It is the aim to setup rdma only on the ess-ioServers in the fabric green , in the blue we must use IPoIB (tcp)
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Do you think datamigration would work between ess01,ess02,… to ess07,ess08 via RDMA ?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Or is it principally not possible to make a rdma network only for a subset of a cluster (though this subset would be reachable via other fabric) ?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Thank you very much for any input !</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Best regards walter
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif"> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Mit freundlichen Grüßen<br>
<b><i>Walter Sklenka</i></b></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><b><i><span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray">Technical Consultant</span></i></b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black"> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black">EDV-Design Informationstechnologie GmbH<br>
Giefinggasse 6/1/2, A-1210 Wien<br>
Tel: +43 1 29 22 165-31<br>
Fax: +43 1 29 22 165-90<br>
E-Mail: <a href="mailto:sklenka@edv-design.at" target="_blank"><span style="color:#0563C1">sklenka@edv-design.at</span></a></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:black">Internet:
<a href="http://www.edv-design.at/" target="_blank"><span style="color:#0563C1">www.edv-design.at</span></a></span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
<p style="margin:0cm"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div id="MIMEAttachInfoDiv">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </span><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> <o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
</div>
</body>
</html>