[gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over NFS?
Andi Christiansen
andi at christiansen.xxx
Tue Nov 17 20:40:39 GMT 2020
Hi Jonathan,
yes you are correct! but we plan to resync this once or twice every week for the next 3-4months to be sure everything is as it should be.
Right now we are focused on getting them synced up and then we will run scheduled resyncs/checks once or twice a week depending on the data growth :)
Thanks
Andi Christiansen
> On 11/17/2020 2:53 PM Jonathan Buzzard <jonathan.buzzard at strath.ac.uk> wrote:
>
>
> On 17/11/2020 11:51, Andi Christiansen wrote:
> > Hi all,
> >
> > thanks for all the information, there was some interesting things
> > amount it..
> >
> > I kept on going with rsync and ended up making a file with all top
> > level user directories and splitting them into chunks of 347 per
> > rsync session(total 42000 ish folders). yesterday we had only 14
> > sessions with 3000 folders in each and that was too much work for one
> > rsync session..
>
> Unless you use something similar to my DB suggestion it is almost
> inevitable that some of those rsync sessions are going to have issues
> and you will have no way to track it or even know it has happened unless
> you do a single final giant catchup/check rsync.
>
> I should add that a copy of the sqlite DB is cover your backside
> protection when a user pops up claiming that you failed to transfer one
> of their vitally important files six months down the line and the old
> system is turned off and scrapped.
>
>
> JAB.
>
> --
> Jonathan A. Buzzard Tel: +44141-5483420
> HPC System Administrator, ARCHIE-WeSt.
> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
More information about the gpfsug-discuss
mailing list