<html><body><p><font size="2">Hi Aaron, </font><br><br><font size="2">By large block size we are primarily talking about block sizes 4 MB and greater. You are correct, in my previous message I neglected to mention the file create performance for small files on these larger block sizes due to the subblock change. In addition to the added space efficiency, small file creation (for example 32kB files) on large block size filesystems will improve.</font><br><br><font size="2">In the case of a 1 MB block size, there would be no real difference in file creates. For a 16 MB block size, however there will be a performance improvement for small file creation as a part of the subblock change for new filesystems. For users who are upgrading from 4.X.X to 5.0.0, the file creation speed will remain the same after the upgrade.</font><br><br><font size="2">I hope that helps, sorry for the confusion.</font><br><br><font size="2">Thank you,</font><br><font size="2">Nikhil Khandelwal</font><br><font size="2">Spectrum Scale Development</font><br><font size="2">Client Adoption</font><br><br><img width="16" height="16" src="cid:1__=88BB0B74DF13858D8f9e8a93df938690918c88B@" border="0" alt="Inactive hide details for Aaron Knister ---11/29/2017 03:42:20 PM---Thanks, Nikhil. Most of that was consistent with my underst"><font size="2" color="#424282">Aaron Knister ---11/29/2017 03:42:20 PM---Thanks, Nikhil. Most of that was consistent with my understnading, however I was under the impressio</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Aaron Knister <aaron.knister@gmail.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">11/29/2017 03:42 PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [gpfsug-discuss] Online data migration tool</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br>Thanks, Nikhil. Most of that was consistent with my understnading, however I was under the impression that the >32 subblocks code is required to achieve the touted 50k file creates/second that Sven has talked about a bunch of times:<br><u><font color="#0000FF"><br></font></u><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Manchester_08-5FResearch-5FTopics.pdf&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=GNrHjCLvQL1u_WHVimX2lAlYOGPzciCFrYHGlae3h_E&s=EotsJVyv0GB8dJ4sTWzsdvIaJ8AzUkepmcNgg5yzSsw&e="><u><font color="#0000FF">http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf</font></u></a><u><font color="#0000FF"><br></font></u><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2017_Ehningen_31-5F-2D-5FSSUG17DE-5F-2D-5FSven-5FOehme-5F-2D-5FNews-5Ffrom-5FResearch.pdf&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=GNrHjCLvQL1u_WHVimX2lAlYOGPzciCFrYHGlae3h_E&s=K25ejUDRbafphuPDsrlpFxCC7104c4gEDnbUDa22x_g&e="><u><font color="#0000FF">http://files.gpfsug.org/presentations/2017/Ehningen/31_-_SSUG17DE_-_Sven_Oehme_-_News_from_Research.pdf</font></u></a><u><font color="#0000FF"><br></font></u><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__files.gpfsug.org_presentations_2016_SC16_12-5F-2D-5FSven-5FOehme-5FDean-5FHildebrand-5F-2D-5FNews-5Ffrom-5FIBM-5FResearch.pdf&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=GNrHjCLvQL1u_WHVimX2lAlYOGPzciCFrYHGlae3h_E&s=Qr7EfUdIXNpWMUnLVjGiVW39oGaA_F50VAe9Fv12d_g&e="><u><font color="#0000FF">http://files.gpfsug.org/presentations/2016/SC16/12_-_Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf</font></u></a><br><br>from those presentations regarding 32 subblocks:<br><br>"It has a significant performance penalty for small files in large block size filesystems"<br><br>although I'm not clear on the specific definition of "large". Many filesystems I encounter only have a 1M block size so it may not matter there, although that same presentation clearly shows the benefit of larger block sizes which is yet *another* thing for which a migration tool would be helpful.<br><br>-Aaron<br><br><br>On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal <<a href="mailto:nikhilk@us.ibm.com" target="_blank"><u><font color="#0000FF">nikhilk@us.ibm.com</font></u></a>> wrote:
<ul><font size="2">Hi,</font><br><font size="2"><br>I would like to clarify migration path to 5.0.0 from 4.X.X clusters. For all Spectrum Scale clusters that are currently at 4.X.X, it is possible to migrate to 5.0.0 with no offline data migration and no need to move data. Once these clusters are at 5.0.0, they will benefit from the performance improvements, new features (such as file audit logging), and various enhancements that are included in 5.0.0.</font><br><font size="2"><br>That being said, there is one enhancement that will not be applied to these clusters, and that is the increased number of sub-blocks per block for small file allocation. This means that for file systems with a large block size and a lot of small files, the overall space utilization will be the same it currently is in 4.X.X. Since file systems created at 4.X.X and earlier used a block size that kept this allocation in mind, there should be very little impact on existing file systems.</font><br><font size="2"><br>Outside of that one particular function, the remainder of the performance improvements, metadata improvements, updated compatibility, new functionality, and all of the other enhancements will be immediately available to you once you complete the upgrade to 5.0.0 -- with no need to reformat, move data, or take your data offline.</font><br><font size="2"><br>I hope that clarifies things a little and makes the upgrade path more accessible.</font><br><font size="2"><br>Please let me know if there are any other questions or concerns.</font><br><font size="2"><br>Thank you,<br>Nikhil Khandelwal<br>Spectrum Scale Development<br>Client Adoption</font><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__spectrumscale.org&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=GNrHjCLvQL1u_WHVimX2lAlYOGPzciCFrYHGlae3h_E&s=sLi9sAM3s2fJ0_6G81cY28hFsxFAYGZGLtdNbMBalIY&e=" target="_blank"><u><font color="#0000FF">spectrumscale.org</font></u></a><u><font color="#0000FF"><br></font></u><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=GNrHjCLvQL1u_WHVimX2lAlYOGPzciCFrYHGlae3h_E&s=VtVgCRl7kxNRgcl5QeHdZJ0Rz6jCA-jfQXyLztbr5TY&e=" target="_blank"><u><font color="#0000FF">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></u></a><br></ul><tt><font size="2">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><tt><font size="2"><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=GNrHjCLvQL1u_WHVimX2lAlYOGPzciCFrYHGlae3h_E&s=VtVgCRl7kxNRgcl5QeHdZJ0Rz6jCA-jfQXyLztbr5TY&e=">https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=GNrHjCLvQL1u_WHVimX2lAlYOGPzciCFrYHGlae3h_E&s=VtVgCRl7kxNRgcl5QeHdZJ0Rz6jCA-jfQXyLztbr5TY&e=</a></font></tt><tt><font size="2"><br></font></tt><br><br><BR>
</body></html>