<html><body><p><font size="2">ESS GL4 4u106 w/ 10 TB drives - same HW Sven reported some of the results @ user group meeting.</font><br><br><font size="2">-Bill</font><br><font size="2"><br>Bill Hartner<br>IBM Systems<br>Scalable I/O Development<br>Austin, Texas<br>bhartner@us.ibm.com<br>home office 512-784-0980<br></font><br><br><img width="16" height="16" src="cid:1__=8FBB0B7ADFC850AE8f9e8a93df938690918c8FB@" border="0" alt="Inactive hide details for Jan-Frode Myklebust ---12/01/2017 06:53:44 AM---Bill, could you say something about what the metadata"><font size="2" color="#424282">Jan-Frode Myklebust ---12/01/2017 06:53:44 AM---Bill, could you say something about what the metadata-storage here was? ESS/NL-SAS/3way replication?</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Jan-Frode Myklebust <janfrode@tanso.net></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">12/01/2017 06:53 AM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [gpfsug-discuss] Online data migration tool</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br>Bill, could you say something about what the metadata-storage here was? ESS/NL-SAS/3way replication?<br><br>I just asked about this in the internal slack channel #scale-help today..<br><br><br><br>-jf<br><br>fre. 1. des. 2017 kl. 13:44 skrev Bill Hartner <<a href="mailto:bhartner@us.ibm.com"><u><font color="#0000FF">bhartner@us.ibm.com</font></u></a>>:
<ul><tt><font size="2">> "It has a significant performance penalty for small files in large <br>> block size filesystems"</font></tt><br>
<p><font size="2">Aaron,</font><br><font size="2"><br>Below are mdtest results for a test we ran for CORAL - file size was 32k.</font><br><font size="2"><br>We have not gone back and ran the test on a file system formatted without > 32 subblocks. We'll do that at some point...</font><br><font size="2"><br>-Bill</font><br><font size="2" face="Courier New"><br>-- started at 10/28/2017 17:51:38 --</font><br><font size="2" face="Courier New"><br>mdtest-1.9.3 was launched with 228 total task(s) on 12 node(s)<br>Command line used: /tmp/mdtest-binary-dir/mdtest -d /ibm/fs2-16m-10/mdtest-60000 -i 3 -n 294912 -w 32768 -C -F -r -p 360 -u -y<br>Path: /ibm/fs2-16m-10<br>FS: 128.1 TiB Used FS: 0.3% Inodes: 476.8 Mi Used Inodes: 0.0%</font><br><font size="2" face="Courier New"><br>228 tasks, 67239936 files</font><br><font size="2" face="Courier New"><br>SUMMARY: (of 3 iterations)<br>Operation Max Min Mean Std Dev<br>--------- --- --- ---- -------<br>File creation : 51953.498 50558.517 51423.221 616.643<br>File stat : 0.000 0.000 0.000 0.000<br>File read : 0.000 0.000 0.000 0.000<br>File removal : 96746.376 92149.535 94658.774 1900.187<br>Tree creation : 1.588 0.070 0.599 0.700<br>Tree removal : 0.213 0.034 0.097 0.082</font><br><font size="2" face="Courier New"><br>-- finished at 10/28/2017 19:51:54 --</font><font size="2"><br><br>Bill Hartner<br>IBM Systems<br>Scalable I/O Development<br>Austin, Texas</font><u><font size="2" color="#0000FF"><br></font></u><a href="mailto:bhartner@us.ibm.com" target="_blank"><u><font size="2" color="#0000FF">bhartner@us.ibm.com</font></u></a><font size="2"><br>home office 512-784-0980</font><br><br><tt><u><font size="2" color="#0000FF"><br></font></u></tt><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank"><tt><u><font size="2" color="#0000FF">gpfsug-discuss-bounces@spectrumscale.org</font></u></tt></a><tt><font size="2"> wrote on 11/29/2017 04:41:48 PM:<br><br>> From: Aaron Knister <</font></tt><a href="mailto:aaron.knister@gmail.com" target="_blank"><tt><u><font size="2" color="#0000FF">aaron.knister@gmail.com</font></u></tt></a><tt><font size="2">></font></tt><p><tt><font size="2"><br>> To: gpfsug main discussion list <</font></tt><a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank"><tt><u><font size="2" color="#0000FF">gpfsug-discuss@spectrumscale.org</font></u></tt></a><tt><font size="2">></font></tt><p><tt><font size="2">> Date: 11/29/2017 04:42 PM</font></tt><p><tt><font size="2"><br>> Subject: Re: [gpfsug-discuss] Online data migration tool<br>> Sent by: </font></tt><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank"><tt><u><font size="2" color="#0000FF">gpfsug-discuss-bounces@spectrumscale.org</font></u></tt></a><p><tt><font size="2">> </font></tt><p><tt><font size="2">> Thanks, Nikhil. Most of that was consistent with my understnading, <br>> however I was under the impression that the >32 subblocks code is <br>> required to achieve the touted 50k file creates/second that Sven has<br>> talked about a bunch of times:<br>> <br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=UGLr4Z6sa2yWvKL99g7SuQdgwxnoZwhVmDuIbYsLqYY&e=" target="_blank"><tt><u><font size="2" color="#0000FF">http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf</font></u></tt></a><tt><font size="2"><br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Ehningen_31-5F-2D-5FSSUG17DE-5F-2D&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Il2rMx4AtGwjVRzX89kobZ0W25vW8TGm0KJevLd7KQ8&e=" target="_blank"><tt><u><font size="2" color="#0000FF">http://files.gpfsug.org/presentations/2017/Ehningen/31_-_SSUG17DE_-</font></u></tt></a><tt><font size="2"><br>> _Sven_Oehme_-_News_from_Research.pdf<br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2016_SC16_12-5F-2D&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=u_qcvB--uvtByHp9H471EowagobMpPLXYT_FFzMkQiw&e=" target="_blank"><tt><u><font size="2" color="#0000FF">http://files.gpfsug.org/presentations/2016/SC16/12_-</font></u></tt></a><tt><font size="2"><br>> _Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf</font></tt><p><tt><font size="2"><br>> from those presentations regarding 32 subblocks:<br>> <br>> "It has a significant performance penalty for small files in large <br>> block size filesystems"</font></tt><br><tt><font size="2"><br>> although I'm not clear on the specific definition of "large". Many <br>> filesystems I encounter only have a 1M block size so it may not <br>> matter there, although that same presentation clearly shows the <br>> benefit of larger block sizes which is yet *another* thing for which<br>> a migration tool would be helpful.</font></tt><br><tt><font size="2"><br>> -Aaron<br>> <br>> On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal <</font></tt><a href="mailto:nikhilk@us.ibm.com" target="_blank"><tt><u><font size="2" color="#0000FF">nikhilk@us.ibm.com</font></u></tt></a><tt><font size="2">> wrote:</font></tt><p><tt><font size="2">> Hi,<br>> <br>> I would like to clarify migration path to 5.0.0 from 4.X.X clusters.<br>> For all Spectrum Scale clusters that are currently at 4.X.X, it is <br>> possible to migrate to 5.0.0 with no offline data migration and no <br>> need to move data. Once these clusters are at 5.0.0, they will <br>> benefit from the performance improvements, new features (such as <br>> file audit logging), and various enhancements that are included in 5.0.0.<br>> <br>> That being said, there is one enhancement that will not be applied <br>> to these clusters, and that is the increased number of sub-blocks <br>> per block for small file allocation. This means that for file <br>> systems with a large block size and a lot of small files, the <br>> overall space utilization will be the same it currently is in 4.X.X.<br>> Since file systems created at 4.X.X and earlier used a block size <br>> that kept this allocation in mind, there should be very little <br>> impact on existing file systems.<br>> <br>> Outside of that one particular function, the remainder of the <br>> performance improvements, metadata improvements, updated <br>> compatibility, new functionality, and all of the other enhancements <br>> will be immediately available to you once you complete the upgrade <br>> to 5.0.0 -- with no need to reformat, move data, or take your data offline.<br>> <br>> I hope that clarifies things a little and makes the upgrade path <br>> more accessible.<br>> <br>> Please let me know if there are any other questions or concerns.<br>> <br>> Thank you,<br>> Nikhil Khandelwal<br>> Spectrum Scale Development<br>> Client Adoption<br>> <br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Q-P8kRqnjsWB7ePz6YtA3U0xguo7-lVWKmb_zyZPndE&e=" target="_blank"><tt><u><font size="2" color="#0000FF">spectrumscale.org</font></u></tt></a><tt><font size="2"><br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=WolSBY_TPJVJVPj5WEZ6JAbDZQK3j7oqn8u_Y5xORkE&e=" target="_blank"><tt><u><font size="2" color="#0000FF">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></u></tt></a><br>
<p><tt><font size="2">> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Q-P8kRqnjsWB7ePz6YtA3U0xguo7-lVWKmb_zyZPndE&e=" target="_blank"><tt><u><font size="2" color="#0000FF">spectrumscale.org</font></u></tt></a><p><tt><font size="2">> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?" target="_blank"><tt><u><font size="2" color="#0000FF">https://urldefense.proofpoint.com/v2/url?</font></u></tt></a><tt><font size="2"><br>> u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-<br>> siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=DHoqgBeMFgcM0LpXEI0VCYvvb8ollct5aSYUDln2t68&s=iOxGm-853L_W0XkB3jGsGzCTVlSYUvANOTSewcR_Ue8&e=</font></tt><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=Q-P8kRqnjsWB7ePz6YtA3U0xguo7-lVWKmb_zyZPndE&e=" target="_blank"><u><font color="#0000FF">spectrumscale.org</font></u></a><u><font color="#0000FF"><br></font></u><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=WolSBY_TPJVJVPj5WEZ6JAbDZQK3j7oqn8u_Y5xORkE&e=" target="_blank"><u><font color="#0000FF">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></u></a><tt><font size="2">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=WolSBY_TPJVJVPj5WEZ6JAbDZQK3j7oqn8u_Y5xORkE&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=KLv9eH4GG8WlXC5ENj_jXnzCpm60QSNAADfp6s94oa4&s=WolSBY_TPJVJVPj5WEZ6JAbDZQK3j7oqn8u_Y5xORkE&e=</a></font></tt><tt><font size="2"><br></font></tt><br></ul><BR>
</body></html>