<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">AFM doesn’t work well if you have dependent filesets though .. which we did for quota purposes.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Simon<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black"><gpfsug-discuss-bounces@spectrumscale.org> on behalf of "YARD@il.ibm.com" <YARD@il.ibm.com><br>
<b>Reply-To: </b>"gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>
<b>Date: </b>Wednesday, 6 March 2019 at 09:01<br>
<b>To: </b>"gpfsug-discuss@spectrumscale.org" <gpfsug-discuss@spectrumscale.org><br>
<b>Subject: </b>Re: [gpfsug-discuss] Migrating billions of files?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Hi</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">What permissions you have ? Do u have only Posix , or also SMB attributes ?</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">If only posix attributes you can do the following:</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">- rsync (which will work on different filesets/directories in parallel.</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">- AFM (but in case you need rollback - it will be problematic)
<br>
</span><br>
<span style="font-size:9.0pt;font-family:"Arial",sans-serif"> </span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">Regards</span><br>
<span style="font-size:9.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="780" style="width:585.0pt;border-collapse:collapse">
<tbody>
<tr style="height:6.0pt">
<td width="780" colspan="4" style="width:585.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" noshade="" style="color:#A0A0A0" align="center">
</div>
<p class="MsoNormal"><br>
<span style="font-size:1.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="780" colspan="4" style="width:585.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:1.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="516" colspan="2" style="width:387.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:blue">Yaron Daniel</span></b><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"> 94 Em Ha'Moshavot Rd</span><o:p></o:p></p>
</td>
<td width="96" rowspan="3" style="width:72.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal" align="right" style="text-align:right"><img width="87" height="30" style="width:.9062in;height:.3125in" id="_x0000_i1033" src="cid:image001.gif@01D4D3FC.251D6610" alt="cid:_1_0FC36C500FC3669C00318CDBC22583B5"><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="516" colspan="2" style="width:387.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:blue">Storage Architect – IL Lab Services (Storage)</span></b><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"> Petach Tiqva, 49527</span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="516" colspan="2" style="width:387.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:blue">IBM Global Markets, Systems HW Sales</span></b><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"> Israel</span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="516" colspan="2" style="width:387.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:blue"> </span></b><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"> </span><o:p></o:p></p>
</td>
<td width="96" style="width:72.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="90" style="width:67.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">Phone:</span><o:p></o:p></p>
</td>
<td width="426" style="width:319.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">+972-3-916-5672</span><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"> </span><o:p></o:p></p>
</td>
<td width="96" style="width:72.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="90" style="width:67.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">Fax:</span><o:p></o:p></p>
</td>
<td width="426" style="width:319.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">+972-3-916-5672</span><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">
</span><o:p></o:p></p>
</td>
<td width="96" style="width:72.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="90" style="width:67.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">Mobile:</span><o:p></o:p></p>
</td>
<td width="426" style="width:319.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">+972-52-8395593</span><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">
</span><o:p></o:p></p>
</td>
<td width="96" style="width:72.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="90" style="width:67.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">e-mail:</span><o:p></o:p></p>
</td>
<td width="426" style="width:319.5pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">yard@il.ibm.com</span><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">
</span><o:p></o:p></p>
</td>
<td width="96" style="width:72.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="516" colspan="2" style="width:387.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><a href="http://www.ibm.com/il/he/"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">IBM Israel</span></a><o:p></o:p></p>
</td>
<td width="168" style="width:126.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">
</span><o:p></o:p></p>
</td>
<td width="96" style="width:72.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:10.0pt"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="780" colspan="4" style="width:585.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"> </span><o:p></o:p></p>
</td>
</tr>
<tr style="height:6.0pt">
<td width="780" colspan="4" style="width:585.0pt;padding:0cm 0cm 0cm 0cm;height:6.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#5F5F5F"> </span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><br>
<img border="0" width="137" height="137" style="width:1.427in;height:1.427in" id="_x0000_i1032" src="cid:image002.gif@01D4D3FC.251D6610" alt="IBM Storage Strategy and Solutions v1"><img border="0" width="138" height="138" style="width:1.4375in;height:1.4375in" id="_x0000_i1031" src="cid:image003.gif@01D4D3FC.251D6610" alt="IBM Storage Management and Data Protection v1"><img border="0" width="138" height="136" style="width:1.4375in;height:1.4166in" id="_x0000_i1030" src="cid:image004.gif@01D4D3FC.251D6610" alt="cid:_1_0FA0428C0FA03A6C00318CDBC22583B5"><img border="0" width="138" height="137" style="width:1.4375in;height:1.427in" id="_x0000_i1029" src="cid:image005.gif@01D4D3FC.251D6610" alt="cid:_1_0FA044940FA03A6C00318CDBC22583B5"><span style="font-size:12.0pt"> </span><img border="0" width="146" height="134" style="width:1.5208in;height:1.3958in" id="_x0000_i1028" src="cid:image006.gif@01D4D3FC.251D6610" alt="https://acclaim-production-app.s3.amazonaws.com/images/6c2c3858-6df8-45be-ac2b-f93b8da74e20/Data%2BDriven%2BMulti%2BCloud%2BStrategy%2BV1%2Bver%2B4.png"><span style="font-size:12.0pt"> </span><img border="0" width="139" height="139" style="width:1.4479in;height:1.4479in" id="_x0000_i1027" src="cid:image007.gif@01D4D3FC.251D6610" alt="FlashSystem A9000/R Foundations"><img border="0" width="140" height="140" style="width:1.4583in;height:1.4583in" id="_x0000_i1026" src="cid:image008.gif@01D4D3FC.251D6610" alt="All Flash Storage Foundations V2"><br>
<br>
<br>
<br>
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">From: </span><span style="font-size:9.0pt;font-family:"Arial",sans-serif">"Oesterlin, Robert" <Robert.Oesterlin@nuance.com></span><br>
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">To: </span><span style="font-size:9.0pt;font-family:"Arial",sans-serif">gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></span><br>
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">Date: </span><span style="font-size:9.0pt;font-family:"Arial",sans-serif">03/05/2019 11:57 PM</span><br>
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">Subject: </span><span style="font-size:9.0pt;font-family:"Arial",sans-serif">[gpfsug-discuss] Migrating billions of files?</span><br>
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#5F5F5F">Sent by: </span><span style="font-size:9.0pt;font-family:"Arial",sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" noshade="" style="color:#A0A0A0" align="center">
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt">I’m looking at migration 3-4 Billion files, maybe 3PB of data between GPFS clusters. Most of the files are small - 60% 8K or less. Ideally I’d like to copy at least 15-20M files per day - ideally 50M.<o:p></o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt"> <o:p></o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt">Any thoughts on how achievable this is? Or what to use? Either with AFM, mpifileutils, rsync.. other? Many of these files would be in 4k inodes. Destination is ESS.<o:p></o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt"> <o:p></o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt"> <o:p></o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span style="font-size:12.0pt">Bob Oesterlin</span><o:p></o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span style="font-size:12.0pt">Sr Principal Storage Engineer, Nuance</span><o:p></o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span style="font-size:12.0pt"> </span><tt><span style="font-size:10.0pt">_______________________________________________</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>gpfsug-discuss mailing list</tt><br>
<tt>gpfsug-discuss at spectrumscale.org</tt><br>
</span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style="font-size:10.0pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><o:p></o:p></p>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>