[gpfsug-discuss] GPFS cpoying huge batches of files from one stgpool to another- how do you do it?

[Orlando Richards]

We don't do this often, so when we do we tend to do a "noddy" 
parallelisation of rsync - manually divvy the folders up and spawn 
multiple threads.

We might have a requirement soon(ish) to do this on a much larger scale 
though, with virtually no interruption to service - so I'm very keen to 
see what how the AFM solution looks, since this should allow us to 
present a continual and single view into the filesystem, whilst 
migrating it all in the background to the new filesystem, with just a 
brief wobble whilst we flip between the old and new views.


On 20/03/12 00:47, Jez Tucker wrote:
> In answer to my own question …
>
> http://www.gnu.org/software/parallel
>
> http://www.gnu.org/software/parallel/man.html#example__parallelizing_rsync
>
> Or
>
> http://code.google.com/p/parallel-ssh/ (parallel versions of rsync etc).
>
> One for the bookmarks… hopefully you’ll find it useful.
>
> Jez
>
> *From:*gpfsug-discuss-bounces at gpfsug.org
> [mailto:gpfsug-discuss-bounces at gpfsug.org] *On Behalf Of *Jez Tucker
> *Sent:* 19 March 2012 23:36
> *To:* gpfsug main discussion list
> *Subject:* [gpfsug-discuss] GPFS cpoying huge batches of files from one
> stgpool to another- how do you do it?
>
> Hello
>
> Just wondering how other people go about copying loads of files in a
> many, many deep directory path from one file system to another. Assume
> filenames are full UNICODE and can contain almost any character.
>
> r to use.
>
>
> Rushes Postproduction Limited, 66 Old Compton Street, London W1D 4UH
> tel: +44 (0)20 7437 8676
> web: http://www.rushes.co.uk
> The information contained in this e-mail is confidential and may be
> subject to legal privilege. If you are not the intended recipient, you
> must not use, copy, distribute or disclose the e-mail or any part of its
> contents or take any action in reliance on it. If you have received this
> e-mail in error, please e-mail the sender by replying to this message.
> All reasonable precautions have been taken to ensure no viruses are
> present in this e-mail. Rushes Postproduction Limited cannot accept
> responsibility for loss or damage arising from the use of this e-mail or
> attachments and recommend that you subject these to your virus checking
> procedures prior to use.
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss


-- 
             --
    Dr Orlando Richards
   Information Services
IT Infrastructure Division
        Unix Section
     Tel: 0131 650 4994

The University of Edinburgh is a charitable body, registered in 
Scotland, with registration number SC005336.