[gpfsug-discuss] Follow-up: migrating billions of files

[Jonathan Buzzard]

On Wed, 2019-03-06 at 12:44 +0000, Oesterlin, Robert wrote:
> Some of you had questions to my original post. More information:
>  
> Source:
> - Files are straight GPFS/Posix - no extended NFSV4 ACLs
> - A solution that requires $’s to be spent on software (ie, Aspera)
> isn’t a very viable option
> - Both source and target clusters are in the same DC
> - Source is stand-alone NSD servers (bonded 10g-E) and 8gb FC SAN
> storage
> - Approx 40 file systems, a few large ones with 300M-400M files each,
> others smaller
> - no independent file sets
> - migration must pose minimal disruption to existing users
>  
> Target architecture is a small number of file systems (2-3) on ESS
> with independent filesets
> - Target (ESS) will have multiple 40gb-E links on each NSD server
> (GS4)
>  
> My current thinking is AFM with a pre-populate of the file space and
> switch the clients over to have them pull data they need (most of the
> data is older and less active) and them let AFM populate the rest in
> the background.
>  

As it's not been mentioned yet "dsmc restore" or equivalent depending
on your backup solution.

JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG