[gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over NFS?

Andi Christiansen andi at christiansen.xxx
Tue Nov 17 20:40:39 GMT 2020


Hi Jonathan,

yes you are correct! but we plan to resync this once or twice every week for the next 3-4months to be sure everything is as it should be.

Right now we are focused on getting them synced up and then we will run scheduled resyncs/checks once or twice a week depending on the data growth :)

Thanks
Andi Christiansen

> On 11/17/2020 2:53 PM Jonathan Buzzard <jonathan.buzzard at strath.ac.uk> wrote:
> 
>  
> On 17/11/2020 11:51, Andi Christiansen wrote:
> > Hi all,
> > 
> > thanks for all the information, there was some interesting things
> > amount it..
> > 
> > I kept on going with rsync and ended up making a file with all top
> > level user directories and splitting them into chunks of 347 per
> > rsync session(total 42000 ish folders). yesterday we had only 14
> > sessions with 3000 folders in each and that was too much work for one
> > rsync session..
> 
> Unless you use something similar to my DB suggestion it is almost 
> inevitable that some of those rsync sessions are going to have issues 
> and you will have no way to track it or even know it has happened unless 
> you do a single final giant catchup/check rsync.
> 
> I should add that a copy of the sqlite DB is cover your backside 
> protection when a user pops up claiming that you failed to transfer one 
> of their vitally important files six months down the line and the old 
> system is turned off and scrapped.
> 
> 
> JAB.
> 
> -- 
> Jonathan A. Buzzard                         Tel: +44141-5483420
> HPC System Administrator, ARCHIE-WeSt.
> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss



More information about the gpfsug-discuss mailing list