<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
</head>
<body dir="ltr">
<p dir="ltr">Yes, most of the filesets are based on research groups, projects or departments, with the exception of scratch and home, hence the idea to use a different method for these filesets.</p>
<p dir="ltr">There are approximately 230 million files, the largest of the filesets has 52TB and 63 million files. 300TB in total.</p>
<p dir="ltr">Peter Childs<br>
Research Storage<br>
ITS Research and Teaching Support<br>
Queen Mary, University of London</p>
<br>
<br>
---- Bill Pappas wrote ----<br>
<br>
<div>
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>I have some ideas to suggest given some of my experiences. First, I have some questions:</p>
<p><br>
</p>
<p>How many files are you migrating? <br>
</p>
<p>Will you be creating multiple file sets on the target system based off of business or project needs? Like, file set a is for "department a" and file set b is for "large scale project a"</p>
<p><br>
</p>
<p>Thanks.<br>
</p>
<p><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Bill Pappas</p>
<p>901-619-0585</p>
<p>bpappas@dstonline.com</p>
<p><br>
</p>
<p><img class="EmojiInsert" alt="1466780990050_DSTlogo.png" src="cid:eed7e12d-f79c-4ad7-968b-5b4ee98ed00e"><br>
</p>
<p><br>
</p>
<p><img class="EmojiInsert" alt="http://www.prweb.com/releases/2016/06/prweb13504050.htm" src="cid:99dc08d8-a1e9-4273-b22b-277d0b2cbc6f"><br>
</p>
<p><a id="LPNoLP" href="http://www.prweb.com/releases/2016/06/prweb13504050.htm" class="OWAAutoLink">http://www.prweb.com/releases/2016/06/prweb13504050.htm</a><br>
</p>
</div>
</div>
<br>
<br>
<div style="color:rgb(0,0,0)">
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of gpfsug-discuss-request@spectrumscale.org
<gpfsug-discuss-request@spectrumscale.org><br>
<b>Sent:</b> Wednesday, October 19, 2016 3:12 PM<br>
<b>To:</b> gpfsug-discuss@spectrumscale.org<br>
<b>Subject:</b> gpfsug-discuss Digest, Vol 57, Issue 49</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt">
<div class="PlainText">Send gpfsug-discuss mailing list submissions to<br>
gpfsug-discuss@spectrumscale.org<br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
or, via email, send a message with subject or body 'help' to<br>
gpfsug-discuss-request@spectrumscale.org<br>
<br>
You can reach the person managing the list at<br>
gpfsug-discuss-owner@spectrumscale.org<br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of gpfsug-discuss digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Using AFM to migrate files. (Peter Childs)<br>
2. subnets (Brian Marshall)<br>
3. Re: subnets (Simon Thompson (Research Computing - IT Services))<br>
4. Re: subnets (Uwe Falke)<br>
5. Will there be any more GPFS 4.2.0-x releases?<br>
(Buterbaugh, Kevin L)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Wed, 19 Oct 2016 14:12:41 +0000<br>
From: Peter Childs <p.childs@qmul.ac.uk><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: [gpfsug-discuss] Using AFM to migrate files.<br>
Message-ID:<br>
<HE1PR0701MB2554710DD534587615543AE5A4D20@HE1PR0701MB2554.eurprd07.prod.outlook.com><br>
<br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
<br>
We are planning to use AFM to migrate our old GPFS file store to a new GPFS file store. This will give us the advantages of Spectrum Scale (GPFS) 4.2, such as larger block and inode size. I would like to attempt to gain some insight on my plans before I start.<br>
<br>
The old file store was running GPFS 3.5 with 512 byte inodes and 1MB block size. We have now upgraded it to 4.1 and are working towards 4.2 with 300TB of files. (385TB max space) this is so we can use both the old and new storage via multi-cluster.<br>
<br>
We are moving to a new GPFS cluster so we can use the new protocol nodes eventually and also put the new storage machines as cluster managers, as this should be faster and future proof<br>
<br>
The new hardware has 1PB of space running GPFS 4.2<br>
<br>
We have multiple filesets, and would like to maintain our namespace as far as possible.<br>
<br>
My plan was to.<br>
<br>
1. Create a read-only (RO) AFM cache on the new storage (ro)<br>
2a. Move old fileset and replace with SymLink to new.<br>
2b. Convert RO AFM to Local Update (LU) AFM pointing to new parking area of old files.<br>
2c. move user access to new location in cache.<br>
3. Flush everything into cache and disconnect.<br>
<br>
I've read the docs including the ones on migration but it's not clear if it's safe to move the home of a cache and update the target. It looks like it should be possible and my tests say it works.<br>
<br>
An alternative plan is to use a Independent Writer (IW) AFM Cache to move the home directories which are pointed to by LDAP. Hence we can move users one at a time and only have to drain the HPC cluster at the end to disconnect the cache. I assume that migrating
users over an Independent Writer is safe so long as the users don't use both sides of the cache at once (ie home and target)<br>
<br>
I'm also interested in any recipe people have on GPFS policies to preseed and flush the cache.<br>
<br>
We plan to do all the migration using AFM over GPFS we're not currently using NFS and have no plans to start. I believe using GPFS is the faster method to preform the migration.<br>
<br>
Any suggestions and experience of doing similar migration jobs would be helpful.<br>
<br>
Peter Childs<br>
Research Storage<br>
ITS Research and Teaching Support<br>
Queen Mary, University of London<br>
<br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Wed, 19 Oct 2016 13:46:02 -0400<br>
From: Brian Marshall <mimarsh2@vt.edu><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: [gpfsug-discuss] subnets<br>
Message-ID:<br>
<CAD0XtKRDTXe9Y5qQB5-qVRdo_RTbv9WctoJKf+CB97kNkmss0g@mail.gmail.com><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
All,<br>
<br>
We are setting up communication between 2 clusters using ethernet and<br>
IPoFabric.<br>
<br>
The Daemon interface is running on ethernet, so all admin traffic will use<br>
it.<br>
<br>
We are still getting the subnets setting correct.<br>
<br>
Question:<br>
<br>
Does GPFS have a way to query how it is connecting to a given cluster/node?<br>
i.e. once we have subnets setup how can we tell GPFS is actually using<br>
them. Currently we just do a large transfer and check tcpdump for any<br>
packets flowing on the high-speed/data/non-admin subnet.<br>
<br>
<br>
Thank you,<br>
Brian Marshall<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161019/5b59ed8e/attachment-0001.html">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161019/5b59ed8e/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Wed, 19 Oct 2016 18:10:38 +0000<br>
From: "Simon Thompson (Research Computing - IT Services)"<br>
<S.J.Thompson@bham.ac.uk><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: Re: [gpfsug-discuss] subnets<br>
Message-ID:<br>
<CF45EE16DEF2FE4B9AA7FF2B6EE26545F584168E@EX13.adf.bham.ac.uk><br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
<br>
mmdiag --network<br>
<br>
Simon<br>
________________________________________<br>
From: gpfsug-discuss-bounces@spectrumscale.org [gpfsug-discuss-bounces@spectrumscale.org] on behalf of Brian Marshall [mimarsh2@vt.edu]<br>
Sent: 19 October 2016 18:46<br>
To: gpfsug main discussion list<br>
Subject: [gpfsug-discuss] subnets<br>
<br>
All,<br>
<br>
We are setting up communication between 2 clusters using ethernet and IPoFabric.<br>
<br>
The Daemon interface is running on ethernet, so all admin traffic will use it.<br>
<br>
We are still getting the subnets setting correct.<br>
<br>
Question:<br>
<br>
Does GPFS have a way to query how it is connecting to a given cluster/node? i.e. once we have subnets setup how can we tell GPFS is actually using them. Currently we just do a large transfer and check tcpdump for any packets flowing on the high-speed/data/non-admin
subnet.<br>
<br>
<br>
Thank you,<br>
Brian Marshall<br>
<br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Wed, 19 Oct 2016 20:15:52 +0200<br>
From: "Uwe Falke" <UWEFALKE@de.ibm.com><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: Re: [gpfsug-discuss] subnets<br>
Message-ID:<br>
<OF96AC7F85.6594A994-ONC1258051.0064379D-C1258051.0064547E@notes.na.collabserv.com><br>
<br>
Content-Type: text/plain; charset="ISO-8859-1"<br>
<br>
Hi Brian, <br>
you might use<br>
<br>
mmfsadm saferdump tscomm <br>
<br>
to check on which route peer cluster members are reached.<br>
<br>
<br>
Mit freundlichen Gr??en / Kind regards<br>
<br>
<br>
Dr. Uwe Falke<br>
<br>
IT Specialist<br>
High Performance Computing Services / Integrated Technology Services / <br>
Data Center Services<br>
-------------------------------------------------------------------------------------------------------------------------------------------<br>
IBM Deutschland<br>
Rathausstr. 7<br>
09111 Chemnitz<br>
Phone: +49 371 6978 2165<br>
Mobile: +49 175 575 2877<br>
E-Mail: uwefalke@de.ibm.com<br>
-------------------------------------------------------------------------------------------------------------------------------------------<br>
IBM Deutschland Business & Technology Services GmbH / Gesch?ftsf?hrung: <br>
Frank Hammer, Thorsten Moehring<br>
Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, <br>
HRB 17122 <br>
<br>
<br>
<br>
<br>
From: Brian Marshall <mimarsh2@vt.edu><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Date: 10/19/2016 07:46 PM<br>
Subject: [gpfsug-discuss] subnets<br>
Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>
<br>
<br>
<br>
All,<br>
<br>
We are setting up communication between 2 clusters using ethernet and <br>
IPoFabric.<br>
<br>
The Daemon interface is running on ethernet, so all admin traffic will use <br>
it.<br>
<br>
We are still getting the subnets setting correct.<br>
<br>
Question:<br>
<br>
Does GPFS have a way to query how it is connecting to a given <br>
cluster/node? i.e. once we have subnets setup how can we tell GPFS is <br>
actually using them. Currently we just do a large transfer and check <br>
tcpdump for any packets flowing on the high-speed/data/non-admin subnet.<br>
<br>
<br>
Thank you,<br>
Brian Marshall_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
<br>
<br>
<br>
<br>
------------------------------<br>
<br>
Message: 5<br>
Date: Wed, 19 Oct 2016 20:11:57 +0000<br>
From: "Buterbaugh, Kevin L" <Kevin.Buterbaugh@Vanderbilt.Edu><br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Subject: [gpfsug-discuss] Will there be any more GPFS 4.2.0-x<br>
releases?<br>
Message-ID: <142FECE0-E157-42D9-BC10-4C48E78FA065@vanderbilt.edu><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi All,<br>
<br>
We?re currently running GPFS 4.2.0-4 with an efix installed and now we need a 2nd efix. I?m not a big fan of adding efix to efix and would prefer to go to a new PTF that contains both efixes.<br>
<br>
So ? is there going to be a GPFS 4.2.0-5 (it?s been a longer than normal interval since PTF 4 came out) or do we need to go to GPFS 4.2.1-x? If the latter, any major changes to watch out for? Thanks?<br>
<br>
Kevin<br>
<br>
?<br>
Kevin Buterbaugh - Senior System Administrator<br>
Vanderbilt University - Advanced Computing Center for Research and Education<br>
Kevin.Buterbaugh@vanderbilt.edu<mailto:Kevin.Buterbaugh@vanderbilt.edu> - (615)875-9633<br>
<br>
<br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161019/3a0a91e7/attachment.html">http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20161019/3a0a91e7/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
End of gpfsug-discuss Digest, Vol 57, Issue 49<br>
**********************************************<br>
</div>
</span></font></div>
</div>
</div>
</body>
</html>