<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>>><span style="font-family: Calibri, Arial, Helvetica, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 13.3333px;">For the record, I have completed today my AFM migration of
a filesystem </span><span style="font-family: Calibri, Arial, Helvetica, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 13.3333px;">with 100 million files. Users are now accessing
the new filesystem.</span></p>
<p><span style="font-family: Calibri, Arial, Helvetica, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 13.3333px;"><br>
</span></p>
<p><span style="font-family: Calibri, Arial, Helvetica, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 13.3333px;">Loric-> Hi. I was wondering, what version of GPFS where you
running on the home and cache clusters? I take it you broke up the prefetch list into smaller (for example <2 million) file lists? If not, how? How much capacity did you migfrate over and how long did this process take? Thanks.</span></p>
<p><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Bill Pappas</p>
<p>901-619-0585</p>
<p>bpappas@dstonline.com</p>
<p><br>
</p>
<p><img class="EmojiInsert" alt="1466780990050_DSTlogo.png" src="cid:bf3727e8-fd7d-4440-8d16-381f3b5e9597"><br>
</p>
<p><br>
</p>
<p><img class="EmojiInsert" alt="http://www.prweb.com/releases/2016/06/prweb13504050.htm" src="cid:431e08cd-0247-4440-83f0-05428517cbc4"><br>
</p>
<p><a href="http://www.prweb.com/releases/2016/06/prweb13504050.htm" class="OWAAutoLink" id="LPNoLP">http://www.prweb.com/releases/2016/06/prweb13504050.htm</a><br>
</p>
</div>
</div>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of gpfsug-discuss-request@spectrumscale.org
<gpfsug-discuss-request@spectrumscale.org><br>
<b>Sent:</b> Monday, October 24, 2016 12:05 PM<br>
<b>To:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Subject:</b> gpfsug-discuss Digest, Vol 57, Issue 60</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Send gpfsug-discuss mailing list submissions to<br>
gpfsug-discuss@spectrumscale.org<br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
or, via email, send a message with subject or body 'help' to<br>
gpfsug-discuss-request@spectrumscale.org<br>
<br>
You can reach the person managing the list at<br>
gpfsug-discuss-owner@spectrumscale.org<br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of gpfsug-discuss digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Using AFM to migrate within the same cluster (Eric Horst)<br>
2. Re: Using AFM to migrate files. (Peter Childs) (Peter Childs)<br>
- URL encoding for pathnames (Loic Tortay)<br>
3. Re: Using AFM to migrate within the same cluster (Scott Fadden)<br>
4. Re: Using AFM to migrate files. (Peter Childs) (Peter Childs)<br>
- URL encoding for pathnames (Yaron Daniel)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Mon, 24 Oct 2016 09:16:56 -0700<br>
From: Eric Horst <erich@uw.edu><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: [gpfsug-discuss] Using AFM to migrate within the same cluster<br>
Message-ID:<br>
<CAF-To-2_BQSXKY1Qv2gs6BnxOC2RU=ywfiL9McW5CRehZvDpiw@mail.gmail.com><br>
Content-Type: text/plain; charset=UTF-8<br>
<br>
The recent conversation about AFM has been interesting. I've read the<br>
documentation several times and this is my question. Can AFM be used<br>
to migrate between two filesystems in the same cluster? There are<br>
examples of moving between clusters with NFS or native protocol but<br>
I've got a simple situation of needing to transparently move 100M<br>
files between two existing filesystems.<br>
<br>
Thanks,<br>
<br>
-Eric<br>
<br>
--<br>
Eric Horst<br>
University of Washington<br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Mon, 24 Oct 2016 18:50:51 +0200<br>
From: Loic Tortay <tortay@cc.in2p3.fr><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: Re: [gpfsug-discuss] Using AFM to migrate files. (Peter<br>
Childs) (Peter Childs) - URL encoding for pathnames<br>
Message-ID: <f6d09494-e9c9-cb08-3ab5-5d26371aacbc@cc.in2p3.fr><br>
Content-Type: text/plain; charset=utf-8<br>
<br>
On 10/24/2016 11:44 AM, Venkateswara R Puvvada wrote:<br>
> <br>
> mmafmctl prefecth expects encoded list file, and it is not documented <br>
> correctly. Issues like memory leak, file descriptor leak, and fileset <br>
> going into Unmounted state were fixed in later releases (4.2.1/4.2.2). All <br>
> your points are correct with respect to AFM migration. There is manual <br>
> intervention required. Also prefetch does not give list of files which <br>
> were failed during data read. Users need to run policy to find all <br>
> uncached files today.<br>
> <br>
Hello,<br>
For the record, I have completed today my AFM migration of a filesystem<br>
with 100 million files. Users are now accessing the new filesystem.<br>
<br>
After disabling user access and a last "prefetch", the AFM filesets were<br>
converted to independent filesets.<br>
Less than 600 files were then found to be different between the "home"<br>
and the "cache" filesystems with a metadata comparison (I just copied<br>
the files from the old filesystem to the new one).<br>
I have compared the MD5 of a few thousand randomly selected files and<br>
found no differences between the "home" and the "cache" filesystems.<br>
I expect the users to let us know if they find something different (they<br>
have been instructed to do so). We'll keep the "home" filesystem around<br>
for some time, just in case there is a problem.<br>
<br>
Maybe something else that should be mentionned in the documentation is<br>
what to do with the ".ptrash" directories after the AFM filesets have<br>
been converted. I removed them since they contained files that had<br>
clearly been deleted by the users.<br>
<br>
<br>
Lo?c.<br>
-- <br>
| Lo?c Tortay <tortay@cc.in2p3.fr> - IN2P3 Computing Centre |<br>
<br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Mon, 24 Oct 2016 09:57:33 -0700<br>
From: "Scott Fadden" <sfadden@us.ibm.com><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: Re: [gpfsug-discuss] Using AFM to migrate within the same<br>
cluster<br>
Message-ID:<br>
<OF42D09886.A30A7C7B-ON88258056.005D0734-88258056.005D2900@notes.na.collabserv.com><br>
<br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
Yes you can use AFM to move data within a cluster. If you are using the <br>
NSD protocol the target needs to be a separate file system, if you are <br>
using NFS it needs to be an NFS export. <br>
<br>
<br>
Scott Fadden<br>
Spectrum Scale - Technical Marketing <br>
Phone: (503) 880-5833 <br>
sfadden@us.ibm.com<br>
<a href="http://www.ibm.com/systems/storage/spectrum/scale">http://www.ibm.com/systems/storage/spectrum/scale</a><br>
<br>
<br>
<br>
From: Eric Horst <erich@uw.edu><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Date: 10/24/2016 09:17 AM<br>
Subject: [gpfsug-discuss] Using AFM to migrate within the same <br>
cluster<br>
Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>
<br>
<br>
<br>
The recent conversation about AFM has been interesting. I've read the<br>
documentation several times and this is my question. Can AFM be used<br>
to migrate between two filesystems in the same cluster? There are<br>
examples of moving between clusters with NFS or native protocol but<br>
I've got a simple situation of needing to transparently move 100M<br>
files between two existing filesystems.<br>
<br>
Thanks,<br>
<br>
-Eric<br>
<br>
--<br>
Eric Horst<br>
University of Washington<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
<br>
<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161024/866c55cf/attachment-0001.html">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161024/866c55cf/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Mon, 24 Oct 2016 20:05:00 +0300<br>
From: "Yaron Daniel" <YARD@il.ibm.com><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: Re: [gpfsug-discuss] Using AFM to migrate files. (Peter<br>
Childs) (Peter Childs) - URL encoding for pathnames<br>
Message-ID:<br>
<OFBC94A30E.47D6D231-ONC2258056.005DCCB6-C2258056.005DD821@notes.na.collabserv.com><br>
<br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Hi<br>
<br>
Maybe worth also to check if there are any orphan files in the NEW fs ?<br>
<br>
<br>
Regards<br>
<br>
<br>
<br>
<br>
<br>
Yaron Daniel<br>
94 Em Ha'Moshavot Rd<br>
<br>
Server, Storage and Data Services - Team Leader <br>
Petach Tiqva, 49527<br>
Global Technology Services<br>
Israel<br>
Phone:<br>
+972-3-916-5672<br>
<br>
<br>
Fax:<br>
+972-3-916-5672<br>
<br>
<br>
Mobile:<br>
+972-52-8395593<br>
<br>
<br>
e-mail:<br>
yard@il.ibm.com<br>
<br>
<br>
IBM Israel<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
From: Loic Tortay <tortay@cc.in2p3.fr><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Date: 10/24/2016 07:50 PM<br>
Subject: Re: [gpfsug-discuss] Using AFM to migrate files. (Peter <br>
Childs) (Peter Childs) - URL encoding for pathnames<br>
Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>
<br>
<br>
<br>
On 10/24/2016 11:44 AM, Venkateswara R Puvvada wrote:<br>
> <br>
> mmafmctl prefecth expects encoded list file, and it is not documented <br>
> correctly. Issues like memory leak, file descriptor leak, and fileset <br>
> going into Unmounted state were fixed in later releases (4.2.1/4.2.2). <br>
All <br>
> your points are correct with respect to AFM migration. There is manual <br>
> intervention required. Also prefetch does not give list of files which <br>
> were failed during data read. Users need to run policy to find all <br>
> uncached files today.<br>
> <br>
Hello,<br>
For the record, I have completed today my AFM migration of a filesystem<br>
with 100 million files. Users are now accessing the new filesystem.<br>
<br>
After disabling user access and a last "prefetch", the AFM filesets were<br>
converted to independent filesets.<br>
Less than 600 files were then found to be different between the "home"<br>
and the "cache" filesystems with a metadata comparison (I just copied<br>
the files from the old filesystem to the new one).<br>
I have compared the MD5 of a few thousand randomly selected files and<br>
found no differences between the "home" and the "cache" filesystems.<br>
I expect the users to let us know if they find something different (they<br>
have been instructed to do so). We'll keep the "home" filesystem around<br>
for some time, just in case there is a problem.<br>
<br>
Maybe something else that should be mentionned in the documentation is<br>
what to do with the ".ptrash" directories after the AFM filesets have<br>
been converted. I removed them since they contained files that had<br>
clearly been deleted by the users.<br>
<br>
<br>
Lo?c.<br>
-- <br>
| Lo?c Tortay <tortay@cc.in2p3.fr> - IN2P3 Computing Centre |<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161024/449a0cf1/attachment.html">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161024/449a0cf1/attachment.html</a>><br>
-------------- next part --------------<br>
A non-text attachment was scrubbed...<br>
Name: not available<br>
Type: image/gif<br>
Size: 1851 bytes<br>
Desc: not available<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161024/449a0cf1/attachment.gif">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161024/449a0cf1/attachment.gif</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
End of gpfsug-discuss Digest, Vol 57, Issue 60<br>
**********************************************<br>
</div>
</span></font></div>
</div>
</body>
</html>