[gpfsug-discuss] Online data migration tool

[Nikhil Khandelwal]

That is fair, there certainly are compromises that have to be made with
regards to file space/size/performance when choosing a block size,
especially with varied workloads or users who may create 200,000 files at a
time :).  With an increased the number of subblocks, the compromises and
parameters going into this choice change.

However, I just didn't want to lose sight of the fact that the remainder of
the 5.0.0 features and enhancements (and there are a lot :-) ) are
available to all systems, with no need to go through painful data movement
or recreating of filesystems.

Thanks,
Nikhil Khandelwal
Spectrum Scale Development
Client Adoption



From:	Jonathan Buzzard <jonathan.buzzard at strath.ac.uk>
To:	gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:	11/30/2017 09:13 AM
Subject:	Re: [gpfsug-discuss] Online data migration tool
Sent by:	gpfsug-discuss-bounces at spectrumscale.org



On Wed, 2017-11-29 at 12:08 -0700, Nikhil Khandelwal wrote:

[SNIP]

> Since file systems created at 4.X.X and earlier used a block size
> that kept this allocation in mind, there should be very little impact
> on existing file systems.

That is quite a presumption. I would say that file systems created at
4.X.X and earlier potentially used a block size that was the best
*compromise*, and the new options would work a lot better.

So for example supporting a larger block size for users who have sane
workflows while still not wasting a ton of space for the biomedical
folks who abuse the file system as a database.

Though I have come to the conclusion to stop them using the file system
as a database (no don't do ls in that directory there is 200,000 files
and takes minutes to come back) is to put your BOFH hat on quota them
on maximum file numbers and suggest to them that they use a database
even if it is just sticking it all in SQLite :-D

JAB.

--
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=WUJ15T9xHCCIfLm1wqC74jhfu28fXGLotYoHQvJlMCg&m=RrwCj4KWyu_ykACVG1SYu8EJiDZnH6edu-2rnoalOg4&s=p7xlojuTYL5csXYA94NyL-R5hk7OgLH0qKGTN0peGFk&e=




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171130/ecc3a471/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171130/ecc3a471/attachment-0002.gif>