[gpfsug-discuss] Online data migration tool

Luis Bolinches luis.bolinches at fi.ibm.com
Fri Dec 1 14:00:19 GMT 2017


For Hana 2.0 only SP1 and 2 are supported

--
Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations
Luis Bolinches
Consultant IT Specialist
Mobile Phone: +358503112585
https://www.youracclaim.com/user/luis-bolinches

"If you always give you will always have" --  Anonymous

> On 1 Dec 2017, at 14.53, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
> 
> Bill, could you say something about what the metadata-storage here was? ESS/NL-SAS/3way replication?
> 
> I just asked about this in the internal slack channel #scale-help today..
> 
> 
> 
> -jf
> 
>> fre. 1. des. 2017 kl. 13:44 skrev Bill Hartner <bhartner at us.ibm.com>:
>> > "It has a significant performance penalty for small files in large 
>> > block size filesystems"
>> 
>> 
>> Aaron,
>> 
>> Below are mdtest results for a test we ran for CORAL - file size was 32k.
>> 
>> We have not gone back and ran the test on a file system formatted without > 32 subblocks. We'll do that at some point...
>> 
>> -Bill
>> 
>> -- started at 10/28/2017 17:51:38 --
>> 
>> mdtest-1.9.3 was launched with 228 total task(s) on 12 node(s)
>> Command line used: /tmp/mdtest-binary-dir/mdtest -d /ibm/fs2-16m-10/mdtest-60000 -i 3 -n 294912 -w 32768 -C -F -r -p 360 -u -y
>> Path: /ibm/fs2-16m-10
>> FS: 128.1 TiB Used FS: 0.3% Inodes: 476.8 Mi Used Inodes: 0.0%
>> 
>> 228 tasks, 67239936 files
>> 
>> SUMMARY: (of 3 iterations)
>> Operation Max Min Mean Std Dev
>> --------- --- --- ---- -------
>> File creation : 51953.498 50558.517 51423.221 616.643
>> File stat : 0.000 0.000 0.000 0.000
>> File read : 0.000 0.000 0.000 0.000
>> File removal : 96746.376 92149.535 94658.774 1900.187
>> Tree creation : 1.588 0.070 0.599 0.700
>> Tree removal : 0.213 0.034 0.097 0.082
>> 
>> -- finished at 10/28/2017 19:51:54 --
>> 
>> Bill Hartner
>> IBM Systems
>> Scalable I/O Development
>> Austin, Texas
>> bhartner at us.ibm.com
>> home office 512-784-0980
>> 
>> 
>> gpfsug-discuss-bounces at spectrumscale.org wrote on 11/29/2017 04:41:48 PM:
>> 
>> > From: Aaron Knister <aaron.knister at gmail.com>
>> 
>> 
>> > To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
>> 
>> > Date: 11/29/2017 04:42 PM
>> 
>> 
>> > Subject: Re: [gpfsug-discuss] Online data migration tool
>> > Sent by: gpfsug-discuss-bounces at spectrumscale.org
>> 
>> > 
>> 
>> > Thanks, Nikhil. Most of that was consistent with my understnading, 
>> > however I was under the impression that the >32 subblocks code is 
>> > required to achieve the touted 50k file creates/second that Sven has
>> > talked about a bunch of times:
>> > 
>> > http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf
>> > http://files.gpfsug.org/presentations/2017/Ehningen/31_-_SSUG17DE_-
>> > _Sven_Oehme_-_News_from_Research.pdf
>> > http://files.gpfsug.org/presentations/2016/SC16/12_-
>> > _Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf
>> 
>> 
>> > from those presentations regarding 32 subblocks:
>> > 
>> > "It has a significant performance penalty for small files in large 
>> > block size filesystems"
>> 
>> > although I'm not clear on the specific definition of "large". Many 
>> > filesystems I encounter only have a 1M block size so it may not 
>> > matter there, although that same presentation clearly shows the 
>> > benefit of larger block sizes which is yet *another* thing for which
>> > a migration tool would be helpful.
>> 
>> > -Aaron
>> > 
>> > On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal <nikhilk at us.ibm.com> wrote:
>> 
>> > Hi,
>> > 
>> > I would like to clarify migration path to 5.0.0 from 4.X.X clusters.
>> > For all Spectrum Scale clusters that are currently at 4.X.X, it is 
>> > possible to migrate to 5.0.0 with no offline data migration and no 
>> > need to move data. Once these clusters are at 5.0.0, they will 
>> > benefit from the performance improvements, new features (such as 
>> > file audit logging), and various enhancements that are included in 5.0.0.
>> > 
>> > That being said, there is one enhancement that will not be applied 
>> > to these clusters, and that is the increased number of sub-blocks 
>> > per block for small file allocation. This means that for file 
>> > systems with a large block size and a lot of small files, the 
>> > overall space utilization will be the same it currently is in 4.X.X.
>> > Since file systems created at 4.X.X and earlier used a block size 
>> > that kept this allocation in mind, there should be very little 
>> > impact on existing file systems.
>> > 
>> > Outside of that one particular function, the remainder of the 
>> > performance improvements, metadata improvements, updated 
>> > compatibility, new functionality, and all of the other enhancements 
>> > will be immediately available to you once you complete the upgrade 
>> > to 5.0.0 -- with no need to reformat, move data, or take your data offline.
>> > 
>> > I hope that clarifies things a little and makes the upgrade path 
>> > more accessible.
>> > 
>> > Please let me know if there are any other questions or concerns.
>> > 
>> > Thank you,
>> > Nikhil Khandelwal
>> > Spectrum Scale Development
>> > Client Adoption
>> > 
>> > _______________________________________________
>> > gpfsug-discuss mailing list
>> > gpfsug-discuss at spectrumscale.org
>> > http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>> 
>> 
>> > _______________________________________________
>> > gpfsug-discuss mailing list
>> > gpfsug-discuss at spectrumscale.org
>> 
>> > https://urldefense.proofpoint.com/v2/url?
>> > u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-
>> > siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=DHoqgBeMFgcM0LpXEI0VCYvvb8ollct5aSYUDln2t68&s=iOxGm-853L_W0XkB3jGsGzCTVlSYUvANOTSewcR_Ue8&e=
>> 
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
Oy IBM Finland Ab
PL 265, 00101 Helsinki, Finland
Business ID, Y-tunnus: 0195876-3 
Registered in Finland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171201/e09df482/attachment-0002.htm>


More information about the gpfsug-discuss mailing list