<html><body><p><font size="2" face="Verdana">Hi,</font><br><br><font size="2" face="Verdana">I agree, a tool with proper wrapping delivered in samples would be the right approach. </font><br><font size="2" face="Verdana"><br>No warranty, no support - below a prototype I documented 2 years ago (prior to mmfind availability). The BP used an alternate approach, so its not tested at scale, but the principle was tested and works.</font><br><font size="2" face="Verdana">Reading through it right now I'd re-test the 'deleted files on destination that were deleted on the source' scenario, that might now require some fixing.</font><br><font size="2" face="Verdana"><br># Use 'GPFS patched' rsync on both ends to keep GPFS attributes</font><ul><ul><a href="https://github.com/gpfsug/gpfsug-tools/tree/master/bin/rsync" target="_blank"><u><font size="2" color="#0000CC" face="Verdana">https://github.com/gpfsug/gpfsug-tools/tree/master/bin/rsync</font></u></a></ul></ul><font size="2" face="Verdana"><br># Policy - initial & differential (add mod_time > .. for incremental runs. Use MOD_TIME < .. to have a defined start for the next incremental rsync, remove it for the 'final' rsync)</font><ul><ul><font size="2" face="Verdana"># </font><a href="http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adv.doc/bl1adv_usngfileattrbts.htm" target="_blank"><u><font size="2" color="#0000CC" face="Verdana">http://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adv.doc/bl1adv_usngfileattrbts.htm</font></u></a><font size="2" face="Verdana"><br>cat /tmp/policy.pol<br>RULE 'mmfind'<br> LIST 'mmfindList'<br> DIRECTORIES_PLUS<br> SHOW( <br> VARCHAR(MODE) || ' ' ||<br> VARCHAR(NLINK) || ' ' ||<br> VARCHAR(USER_ID) || ' ' ||<br> VARCHAR(GROUP_ID) || ' ' ||<br> VARCHAR(FILE_SIZE) || ' ' ||<br> VARCHAR(KB_ALLOCATED) || ' ' ||<br> VARCHAR(POOL_NAME) || ' ' ||<br> VARCHAR(MISC_ATTRIBUTES) || ' ' ||<br> VARCHAR(ACCESS_TIME) || ' ' ||<br> VARCHAR(CREATION_TIME) || ' ' || <br> VARCHAR(MODIFICATION_TIME)<br> ) </font><br><font size="2" face="Verdana"># First run <br> WHERE MODIFICATION_TIME < TIMESTAMP('2016-08-10 00:00:00')</font><br><font size="2" face="Verdana"># Incremental runs</font><br><font size="2" face="Verdana"> WHERE MODIFICATION_TIME > TIMESTAMP('2016-08-10 00:00:00') and MODIFICATION_TIME < TIMESTAMP('2016-08-20 00:00:00')</font><br><font size="2" face="Verdana"># Final run during maintenance, should also do deletes, ensure you to call rsync the proper way (--delete)</font><br><font size="2" face="Verdana"> WHERE TRUE<br></font></ul></ul><font size="2" face="Verdana"><br># Apply policy, defer will ensure the result file(s) are not deleted</font><ul><ul><font size="2" face="Verdana">mmapplypolicy group3fs -P /tmp/policy.pol -f /ibm/group3fs/pol.txt -I defer</font></ul></ul><font size="2" face="Verdana"><br># FYI only - look at results, ... not required</font><ul><ul><font size="2" face="Verdana"># cat /ibm/group3fs/pol.txt.list.mmfindList <br>3 1 0 drwxr-xr-x 4 0 0 262144 512 system D2u 2016-08-25 08:30:35.053057 -- /ibm/group3fs<br>41472 1077291531 0 drwxr-xr-x 5 0 0 4096 0 system D2u 2016-08-18 21:07:36.996777 -- /ibm/group3fs/ces<br>60416 842873924 0 drwxr-xr-x 4 0 0 4096 0 system D2u 2016-08-18 21:07:45.947920 -- /ibm/group3fs/ces/ha<br>60417 2062486126 0 -rw-r--r-- 1 0 0 0 0 system FAu 2016-08-19 15:17:57.428922 -- /ibm/group3fs/ces/ha/.dummy<br>60418 436745294 0 drwxr-xr-x 4 0 0 4096 0 system D2u 2016-08-18 21:05:54.482094 -- /ibm/group3fs/ces/ces<br>60419 647668346 0 -rw-r--r-- 1 0 0 0 0 system FAu 2016-08-19 15:17:57.484923 -- /ibm/group3fs/ces/ces/.dummy<br>60420 1474765985 0 -rw-r--r-- 1 0 0 0 0 system FAu 2016-08-18 21:06:43.133640 -- /ibm/group3fs/ces/ces/addrs/1471554403-node0-9.155.118.69<br>60421 1020724013 0 drwxr-xr-x 2 0 0 4096 0 system D2um 2016-08-18 21:07:37.000695 -- /ibm/group3fs/ces/ganesha<br>cat /ibm/group3fs/pol.txt.list.mmfindList |awk ' { print $19}' <br><br>/ibm/group3fs/ces/ha/.dummy<br>/ibm/group3fs/ces/ces/.dummy<br>/ibm/group3fs/ces/ha/nfs/ganesha/v4recov/node3<br>/ibm/group3fs/ces/ha/nfs/ganesha/v4old/node3<br>/ibm/group3fs/pol.txt.list.mmfindList<br>/ibm/group3fs/ces/ces/connections<br>/ibm/group3fs/ces/ha/nfs/ganesha/gpfs-epoch<br>/ibm/group3fs/ces/ha/nfs/ganesha/v4recov<br>/ibm/group3fs/ces/ha/nfs/ganesha/v4old</font></ul></ul><font size="2" face="Verdana"><br># Start rsync - could split up single result file into multiple ones for parallel / multi node runs</font><ul><ul><font size="2" face="Verdana">rsync -av --gpfs-attrs --progress --files-from $( cat /ibm/group3fs/pol.txt.list.mmfindList ) 10.10.10.10:/path</font></ul></ul><font size="2" face="Verdana"><br>Be sure you verify that extended attributes are properly replicated. I have in mind that you need to ensure the 'remote' rsync is not the default one, but the one with GPFS capabilities (rsync -e "remoteshell"). </font><br><font size="2" face="Verdana"><br>Kind regards,<br>Alex Saupp</font><font size="2"><br></font><br><br><font size="2">Mit freundlichen Grüßen / Kind regards</font><br><br><b>Alexander Saupp</b><br><br><font color="#0055AA">IBM Systems, Storage Platform, EMEA Storage Competence Center</font><table border="0" cellspacing="0" cellpadding="0"><tr valign="top"><td class="small_font" width="800" colspan="4" valign="middle"><hr width="100%" size="2" align="left" style="color:#696969; "></td></tr>
<tr valign="top"><td class="small_font" width="800" colspan="4" valign="middle"><img width="1" height="1" src="cid:1__=4EBB09BCDF8C10158f9e8a93df938690918c4EB@" border="0" alt=""></td></tr>
<tr valign="top"><td class="inner" width="88" valign="middle"><font size="1" color="#0055AA">Phone:</font></td><td class="inner" width="322" valign="middle"><font size="1" color="#0055AA">+49 7034-643-1512</font></td><td class="inner" width="250" valign="middle"><font size="1" color="#0055AA"> IBM Deutschland GmbH</font></td><td width="140" rowspan="4" valign="middle"><div align="right"><img src="cid:2__=4EBB09BCDF8C10158f9e8a93df938690918c4EB@" width="87" height="30" align="bottom"></div></td></tr>
<tr valign="top"><td class="inner" width="88" valign="middle"><font size="1" color="#0055AA">Mobile:</font></td><td class="inner" width="322" valign="middle"><font size="1" color="#0055AA">+49-172 7251072</font></td><td class="inner" width="250" valign="middle"><font size="1" color="#0055AA"> Am Weiher 24</font></td></tr>
<tr valign="top"><td class="inner" width="88" valign="middle"><font size="1" color="#0055AA">Email:</font></td><td class="inner" width="322" valign="middle"><font size="1" color="#0055AA">alexander.saupp@de.ibm.com</font></td><td class="inner" width="250" valign="middle"><font size="1" color="#0055AA"> 65451 Kelsterbach</font></td></tr>
<tr valign="top"><td class="inner" width="88" valign="middle"><img width="1" height="1" src="cid:1__=4EBB09BCDF8C10158f9e8a93df938690918c4EB@" border="0" alt=""></td><td class="inner" width="322" valign="middle"><img width="1" height="1" src="cid:1__=4EBB09BCDF8C10158f9e8a93df938690918c4EB@" border="0" alt=""></td><td class="inner" width="250" valign="middle"><font size="1" color="#0055AA"> Germany</font></td></tr>
<tr valign="top"><td class="small_font" width="800" colspan="4" valign="middle"><hr width="100%" size="2" align="left" style="color:#696969; "></td></tr>
<tr valign="top"><td class="foot" width="800" colspan="4" valign="middle"><font size="1" color="#999999">IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Martin Jetter<br>Geschäftsführung: Matthias Hartmann (Vorsitzender), Norbert Janzen, Stefan Lutz, Nicole Reimer, Dr. Klaus Seifert, Wolfgang Wendt<br>Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, HRB 14562 / WEEE-Reg.-Nr. DE 99369940 </font></td></tr></table><br><BR>
</body></html>