[gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over NFS?

[Dugan, Michael J]

I have a cluster with two filesystems and I need to migrate a fileset from one to the other.
I would normally do this with tar and rsync but I decided to experiment with AFM following
the document below. In my test setup I'm finding that hardlinks are not preserved by the migration.
Is that expected or am I doing something wrong?

I'm using gpfs-5.0.5.4.

Thanks.

--Mike


--
Michael J. Dugan
Manager of Systems Programming and Administration
Research Computing Services | IS&T | Boston University
617-358-0030
dugan at bu.edu
http://www.bu.edu/tech


________________________________
From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Venkateswara R Puvvada <vpuvvada at in.ibm.com>
Sent: Monday, November 23, 2020 9:41 PM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Cc: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over NFS?

AFM provides near zero downtime for migration.  As of today,  AFM migration does not support ACLs or other EAs migration from non scale (GPFS) source.

https://www.ibm.com/support/knowledgecenter/STXKQY_5.1.0/com.ibm.spectrum.scale.v5r10.doc/bl1ins_uc_migrationusingafmmigrationenhancements.htm

~Venkat (vpuvvada at in.ibm.com)



From:        "Frederick Stock" <stockf at us.ibm.com>
To:        gpfsug-discuss at spectrumscale.org
Cc:        gpfsug-discuss at spectrumscale.org
Date:        11/17/2020 03:14 AM
Subject:        [EXTERNAL] Re: [gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over        NFS?
Sent by:        gpfsug-discuss-bounces at spectrumscale.org
________________________________



Have you considered using the AFM feature of Spectrum Scale?  I doubt it will provide any speed improvement but it would allow for data to be accessed as it was being migrated.

Fred
__________________________________________________
Fred Stock | IBM Pittsburgh Lab | 720-430-8821
stockf at us.ibm.com


----- Original message -----
From: Andi Christiansen <andi at christiansen.xxx>
Sent by: gpfsug-discuss-bounces at spectrumscale.org
To: "gpfsug-discuss at spectrumscale.org" <gpfsug-discuss at spectrumscale.org>
Cc:
Subject: [EXTERNAL] [gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over NFS?
Date: Mon, Nov 16, 2020 2:44 PM

Hi all,

i have got a case where a customer wants 700TB migrated from isilon to Scale and the only way for him is exporting the same directory on NFS from two different nodes...

as of now we are using multiple rsync processes on different parts of folders within the main directory. this is really slow and will take forever.. right now 14 rsync processes spread across 3 nodes fetching from 2..

does anyone know of a way to speed it up? right now we see from 1Gbit to 3Gbit if we are lucky(total bandwidth) and there is a total of 30Gbit from scale nodes and 20Gbits from isilon so we should be able to reach just under 20Gbit...


if anyone have any ideas they are welcome!


Thanks in advance
Andi Christiansen
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201204/98d8701f/attachment-0001.htm>