[gpfsug-discuss] Online data migration tool

[Aaron Knister]

Thanks, Nikhil. Most of that was consistent with my understnading, however
I was under the impression that the >32 subblocks code is required to
achieve the touted 50k file creates/second that Sven has talked about a
bunch of times:

http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf
http://files.gpfsug.org/presentations/2017/Ehningen/31_-_SSUG17DE_-_Sven_Oehme_-_News_from_Research.pdf
http://files.gpfsug.org/presentations/2016/SC16/12_-_Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf

from those presentations regarding 32 subblocks:

"It has a significant performance penalty for small files in large block
size filesystems"

although I'm not clear on the specific definition of "large". Many
filesystems I encounter only have a 1M block size so it may not matter
there, although that same presentation clearly shows the benefit of larger
block sizes which is yet *another* thing for which a migration tool would
be helpful.

-Aaron


On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal <nikhilk at us.ibm.com>
wrote:

> Hi,
>
> I would like to clarify migration path to 5.0.0 from 4.X.X clusters. For
> all Spectrum Scale clusters that are currently at 4.X.X, it is possible to
> migrate to 5.0.0 with no offline data migration and no need to move data.
> Once these clusters are at 5.0.0, they will benefit from the performance
> improvements, new features (such as file audit logging), and various
> enhancements that are included in 5.0.0.
>
> That being said, there is one enhancement that will not be applied to
> these clusters, and that is the increased number of sub-blocks per block
> for small file allocation. This means that for file systems with a large
> block size and a lot of small files, the overall space utilization will be
> the same it currently is in 4.X.X. Since file systems created at 4.X.X and
> earlier used a block size that kept this allocation in mind, there should
> be very little impact on existing file systems.
>
> Outside of that one particular function, the remainder of the performance
> improvements, metadata improvements, updated compatibility, new
> functionality, and all of the other enhancements will be immediately
> available to you once you complete the upgrade to 5.0.0 -- with no need to
> reformat, move data, or take your data offline.
>
> I hope that clarifies things a little and makes the upgrade path more
> accessible.
>
> Please let me know if there are any other questions or concerns.
>
> Thank you,
> Nikhil Khandelwal
> Spectrum Scale Development
> Client Adoption
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171129/1b594a6a/attachment-0002.htm>