<div dir="ltr">Thanks for the reply - but that explanation does not mean no downtime without elaborating on "cut over." I can do the sync via rsync or tar today but eventually I will have to cut over to the new system. <div><br></div><div>Is this the case with AFM as well - once everything is synced over - cutting over means users will have to "cut over" by: </div><div><br><div>1. either mounting new AFM-synced system on all compute nodes with same mount as the old system (which means downtime to unmount the existing filesystem and mounting new filesystem)</div><div><br></div><div>or</div><div><br></div><div>2. end-user training i.e. starting using new filesystem, move your own files you need because eventually we will shutdown the old filesystem.</div><div><br></div><div>If, then, it's true that AFM requires some sort of cut over (either by disconnecting the old system and mounting new system as the old mount point, or by instruction to users to start using new filesystem at once) I am not sure that AFM gets me anything more than rsync or tar when it comes to taking a downtime (cutting over) for the end user. </div><div><br></div><div>Thanks,</div><div>Damir</div><div><br></div><div><br></div><div><div><br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Feb 22, 2016 at 7:39 AM Yaron Daniel <<a href="mailto:YARD@il.ibm.com">YARD@il.ibm.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font size="2" face="sans-serif">Hi</font><br><br><font size="2" face="sans-serif">AFM - </font><font size="3">Active File
Management (AFM) is an asynchronous cross cluster utility </font><br><br><font size="2" face="sans-serif">It means u create new GPFS cluster -
migrate the data without downtime , and when u r ready - u do last sync
and cut-over.</font><br><br><font size="2" face="sans-serif">Hope this help.<br></font><br><font size="1" face="Arial"> </font><p><font size="2" face="Arial">Regards</font><p><font size="1" face="Arial"> </font><p><table width="780" style="border-collapse:collapse"><tr height="8"><td width="780" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><div align="center"><hr noshade></div><p><font size="1" face="Arial"> </font><tr height="8"><td width="780" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" face="Arial"> </font><tr height="8"><td width="516" colspan="2" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="blue" face="Arial"><b>Yaron
Daniel</b></font><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial"> 94
Em Ha'Moshavot Rd</font><td width="96" rowspan="3" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><div align="right"><img align="bottom" src="cid:_1_0D92A1E80D929C54004B017AC2257F61" style="border:0px solid"></div><tr height="8"><td width="516" colspan="2" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="blue" face="Arial"><b>Server,
</b></font><a href="https://w3-03.ibm.com/services/isd/secure/client.wss/Somt?eventType=getHomePage&somtId=115" target="_blank"><font size="2" color="blue" face="Arial"><b>Storage
and Data Services</b></font></a><font size="2" color="blue" face="Arial"><b>- Team Leader</b></font><font size="1" color="blue" face="Arial"><b>
</b></font><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial"> Petach
Tiqva, 49527</font><tr height="8"><td width="516" colspan="2" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="blue" face="Arial"><b>Global
Technology Services</b></font><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial"> Israel</font><tr height="8"><td width="90" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">Phone:</font><td width="426" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">+972-3-916-5672</font><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial"> </font><td width="96" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2"> </font><tr height="8"><td width="90" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">Fax:</font><td width="426" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">+972-3-916-5672</font><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">
</font><td width="96" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2"> </font><tr height="8"><td width="90" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">Mobile:</font><td width="426" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">+972-52-8395593</font><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">
</font><td width="96" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2"> </font><tr height="8"><td width="90" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">e-mail:</font><td width="426" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial"><a href="mailto:yard@il.ibm.com" target="_blank">yard@il.ibm.com</a></font><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">
</font><td width="96" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2"> </font><tr height="8"><td width="516" colspan="2" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><a href="http://www.ibm.com/il/he/" target="_blank"><font size="2" color="blue" face="Arial"><u>IBM
Israel</u></font></a><td width="168" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2" color="#5f5f5f" face="Arial">
</font><td width="96" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="2"> </font><tr height="8"><td width="780" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#5f5f5f" face="Arial"> </font><tr height="8"><td width="780" colspan="4" style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px"><font size="1" color="#5f5f5f" face="Arial"> </font></td></tr></td></tr></td></td></td></tr></td></td></td></td></tr></td></td></td></td></tr></td></td></td></td></tr></td></td></td></td></tr></td></td></tr></td></td></tr></td></td></td></tr></td></tr></p></td></tr></table><br><p><font size="3"> </font><br><br><tt><font size="2"><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a> wrote on
02/22/2016 03:12:14 PM:<br><br>> From: Damir Krstic <<a href="mailto:damir.krstic@gmail.com" target="_blank">damir.krstic@gmail.com</a>></font></tt><br><tt><font size="2">> To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a>></font></tt><br><tt><font size="2">> Date: 02/22/2016 03:12 PM</font></tt></p></p></p></p><p><p><p><p><br><tt><font size="2">> Subject: Re: [gpfsug-discuss] migrating data
from GPFS3.5 to ESS <br>> appliance (GPFS4.1)</font></tt><br></p></p></p></p><p><p><p><p><tt><font size="2">> Sent by: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a></font></tt></p></p></p></p><p><p><p><p><br><tt><font size="2">> <br>> Sorry to revisit this question - AFM seems to be the best way to do
<br>> this. I was wondering if anyone has done AFM migration. I am looking<br>> at this wiki page for instructions: </font></tt><br><tt><font size="2">> </font></tt><a href="https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/" target="_blank"><tt><font size="2">https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/</font></tt></a><tt><font size="2"><br>> wiki/General%20Parallel%20File%20System%20(GPFS)/page/Migrating%<br>> 20Data%20Using%20AFM </font></tt><br><tt><font size="2">> and I am little confused by step 3 "cut
over users" <-- does this <br>> mean, unmount existing filesystem and point users to new filesystem? </font></tt><br><tt><font size="2">> <br>> The reason we were looking at AFM is to not have downtime - make the<br>> transition as seamless as possible to the end user. Not sure what,
<br>> then, AFM buys us if we still have to take "downtime" in
order to <br>> cut users over to the new system.</font></tt><br><tt><font size="2">> <br>> Thanks,</font></tt><br><tt><font size="2">> Damir</font></tt><br><tt><font size="2">> <br>> On Thu, Feb 4, 2016 at 3:15 PM Damir Krstic <<a href="mailto:damir.krstic@gmail.com" target="_blank">damir.krstic@gmail.com</a>>
wrote:</font></tt><br><tt><font size="2">> Thanks all for great suggestions. We will most
likely end up using <br>> either AFM or some mechanism of file copy (tar/rsync etc.). </font></tt><br><tt><font size="2">> <br>> On Mon, Feb 1, 2016 at 12:39 PM Wahl, Edward <<a href="mailto:ewahl@osc.edu" target="_blank">ewahl@osc.edu</a>>
wrote:</font></tt><br><tt><font size="2">> Along the same vein I've patched rsync to maintain
source atimes in <br>> Linux for large transitions such as this. Along with the stadnard
<br>> "patches" mod for destination atimes it is quite useful.
Works in <br>> 3.0.8 and 3.0.9. I've not yet ported it to 3.1.x<br>> </font></tt><a href="https://www.osc.edu/sites/osc.edu/files/staff_files/ewahl/onoatime.diff" target="_blank"><tt><font size="2">https://www.osc.edu/sites/osc.edu/files/staff_files/ewahl/onoatime.diff</font></tt></a><tt><font size="2"><br>> <br>> Ed Wahl<br>> OSC<br>> <br>> ________________________________________<br>> From: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a> [gpfsug-discuss-<br>> <a href="mailto:bounces@spectrumscale.org" target="_blank">bounces@spectrumscale.org</a>] on behalf of Orlando Richards [<br>> <a href="mailto:orlando.richards@ed.ac.uk" target="_blank">orlando.richards@ed.ac.uk</a>]<br>> Sent: Monday, February 01, 2016 4:25 AM<br>> To: <a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank">gpfsug-discuss@spectrumscale.org</a><br>> Subject: Re: [gpfsug-discuss] migrating data from GPFS3.5 to ESS <br>> appliance (GPFS4.1)<br>> <br>> For what it's worth - there's a patch for rsync which IBM provided
a<br>> while back that will copy NFSv4 ACLs (maybe other stuff?). I put it
up<br>> on the gpfsug github here:<br>> <br>> </font></tt><a href="https://github.com/gpfsug/gpfsug-tools/tree/master/bin/rsync" target="_blank"><tt><font size="2">https://github.com/gpfsug/gpfsug-tools/tree/master/bin/rsync</font></tt></a><tt><font size="2"><br>> <br>> <br>> <br>> On 29/01/16 22:36, Sven Oehme wrote:<br>> > Doug,<br>> ><br>> > This won't really work if you make use of ACL's or use special
GPFS<br>> > extended attributes or set quotas, filesets, etc<br>> > so unfortunate the answer is you need to use a combination of
things and<br>> > there is work going on to make some of this simpler (e.g. for
ACL's) ,<br>> > but its a longer road to get there. so until then you need
to think<br>> > about multiple aspects .<br>> ><br>> > 1. you need to get the data across and there are various ways
to do this.<br>> ><br>> > a) AFM is the simplest of all as it not just takes care of ACL's
and<br>> > extended attributes and alike as it understands the GPFS internals
it<br>> > also is operating in parallel can prefetch data, etc so its a
efficient<br>> > way to do this but as already pointed out doesn't transfer quota
or<br>> > fileset informations.<br>> ><br>> > b) you can either use rsync or any other pipe based copy program.
the<br>> > downside is that they are typical single threaded and do a file
by file<br>> > approach, means very metadata intensive on the source as well
as target<br>> > side and cause a lot of ios on both side.<br>> ><br>> > c) you can use the policy engine to create a list of files to
transfer<br>> > to at least address the single threaded scan part, then partition
the<br>> > data and run multiple instances of cp or rsync in parallel, still<br>> > doesn't fix the ACL / EA issues, but the data gets there faster.<br>> ><br>> > 2. you need to get ACL/EA informations over too. there are several<br>> > command line options to dump the data and restore it, they kind
of<br>> > suffer the same problem as data transfers , which is why using
AFM is<br>> > the best way of doing this if you rely on ACL/EA informations.<br>> ><br>> > 3. transfer quota / fileset infos. there are several ways
to do this,<br>> > but all require some level of scripting to do this.<br>> ><br>> > if you have TSM/HSM you could also transfer the data using SOBAR
it's<br>> > described in the advanced admin book.<br>> ><br>> > sven<br>> ><br>> ><br>> > On Fri, Jan 29, 2016 at 11:35 AM, Hughes, Doug<br>> > <<a href="mailto:Douglas.Hughes@deshawresearch.com" target="_blank">Douglas.Hughes@deshawresearch.com</a><br>> > <</font></tt><a href="mailto:Douglas.Hughes@deshawresearch.com" target="_blank"><tt><font size="2">mailto:Douglas.Hughes@deshawresearch.com</font></tt></a><tt><font size="2">>>
wrote:<br>> ><br>> > I have found that a tar pipe is much faster
than rsync for this sort<br>> > of thing. The fastest of these is ‘star’
(schily tar). On average it<br>> > is about 2x-5x faster than rsync for doing
this. After one pass with<br>> > this, you can use rsync for a subsequent or
last pass synch.____<br>> ><br>> > __ __<br>> ><br>> > e.g.____<br>> ><br>> > $ cd /export/gpfs1/foo____<br>> ><br>> > $ star –c H=xtar | (cd /export/gpfs2/foo;
star –xp)____<br>> ><br>> > __ __<br>> ><br>> > This also will not preserve filesets and quotas,
though. You should<br>> > be able to automate that with a little bit
of awk, perl, or whatnot.____<br>> ><br>> > __ __<br>> ><br>> > __ __<br>> ><br>> > *From:*<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a><br>> > <</font></tt><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank"><tt><font size="2">mailto:gpfsug-discuss-bounces@spectrumscale.org</font></tt></a><tt><font size="2">><br>> > [</font></tt><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank"><tt><font size="2">mailto:gpfsug-discuss-bounces@spectrumscale.org</font></tt></a><tt><font size="2"><br>> > <</font></tt><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank"><tt><font size="2">mailto:gpfsug-discuss-bounces@spectrumscale.org</font></tt></a><tt><font size="2">>]
*On Behalf Of<br>> > *Damir Krstic<br>> > *Sent:* Friday, January 29, 2016 2:32 PM<br>> > *To:* gpfsug main discussion list<br>> > *Subject:* [gpfsug-discuss] migrating data
from GPFS3.5 to ESS<br>> > appliance (GPFS4.1)____<br>> ><br>> > __ __<br>> ><br>> > We have recently purchased ESS appliance from
IBM (GL6) with 1.5PT<br>> > of storage. We are in planning stages of implementation.
We would<br>> > like to migrate date from our existing GPFS
installation (around<br>> > 300TB) to new solution. ____<br>> ><br>> > __ __<br>> ><br>> > We were planning of adding ESS to our existing
GPFS cluster and<br>> > adding its disks and then deleting our old
disks and having the data<br>> > migrated this way. However, our existing block
size on our projects<br>> > filesystem is 1M and in order to extract as
much performance out of<br>> > ESS we would like its filesystem created with
larger block size.<br>> > Besides rsync do you have any suggestions
of how to do this without<br>> > downtime and in fastest way possible? ____<br>> ><br>> > __ __<br>> ><br>> > I have looked at AFM but it does not seem
to migrate quotas and<br>> > filesets so that may not be an optimal solution.
____<br>> ><br>> ><br>> > _______________________________________________<br>> > gpfsug-discuss mailing list<br>> > gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a> <</font></tt><a href="http://spectrumscale.org/" target="_blank"><tt><font size="2">http://spectrumscale.org</font></tt></a><tt><font size="2">><br>> > </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br>> ><br>> ><br>> ><br>> ><br>> > _______________________________________________<br>> > gpfsug-discuss mailing list<br>> > gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>> > </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br>> ><br>> <br>> --<br>> --<br>> Dr Orlando Richards<br>> Research Services Manager<br>> Information Services<br>> IT Infrastructure Division<br>> Tel: 0131 650 4994<br>> skype: orlando.richards<br>> <br>> The University of Edinburgh is a charitable body, registered in<br>> Scotland, with registration number SC005336.<br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a><br>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size="2">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size="2"><br></font></tt><br>_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</p></p></p></p></blockquote></div>