[gpfsug-discuss] Follow-up: ESS File systems

Tomer Perry TOMP at il.ibm.com
Wed Apr 10 21:19:15 BST 2019


Just to clarify - its 2M block size, so 64k subblock size.

Regards,

Tomer Perry
Scalable I/O Development (Spectrum Scale)
email: tomp at il.ibm.com
1 Azrieli Center, Tel Aviv 67021, Israel
Global Tel:    +1 720 3422758
Israel Tel:      +972 3 9188625
Mobile:         +972 52 2554625




From:   "Tomer Perry" <TOMP at il.ibm.com>
To:     gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:   10/04/2019 23:11
Subject:        Re: [gpfsug-discuss] Follow-up: ESS File systems
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



Its also important to look into the actual space "wasted" by the "subblock 
mismatch".
For example, a snip from a filehist output I've found somewhere:

File%ile represents the cummulative percentage of files.
Space%ile represents the cummulative percentage of total space used.
AvlSpc%ile represents the cummulative percentage used of total available 
space.

Histogram of files <= one 2M block in size
Subblocks    Count  File%ile  Space%ile   AvlSpc%ile
--------- -------- ---------- ----------  ----------
        0  1297314      2.65%      0.00%       0.00%
        1 34014892     72.11%      0.74%       0.59%
        2  2217365     76.64%      0.84%       0.67%
        3  1967998     80.66%      0.96%       0.77%
        4   798170     82.29%      1.03%       0.83%
        5  1518258     85.39%      1.20%       0.96%
        6   581539     86.58%      1.27%       1.02%
        7   659969     87.93%      1.37%       1.10%
        8  1178798     90.33%      1.58%       1.27%
        9   189220     90.72%      1.62%       1.30%
       10   130197     90.98%      1.64%       1.32%


So, 72% of the files are smaller then 1 subblock ( 2M in the above case 
BTW). If, for example, we'll double it - we will "waste" ~76% of the 
files, and if we'll push it to 16M it will be ~90% of the files...
But, we really care about capacity, right? So, going into the 16M extreme, 
we'll "waste" 1.58% of the capacity ( worst case of course).

So, if it will give you ( highly depends on the workload of course) 4X the 
performance ( just for the sake of discussion) - will it be OK to pay the 
1.5% "premium" ?



Regards,

Tomer Perry
Scalable I/O Development (Spectrum Scale)
email: tomp at il.ibm.com
1 Azrieli Center, Tel Aviv 67021, Israel
Global Tel:    +1 720 3422758
Israel Tel:      +972 3 9188625
Mobile:         +972 52 2554625




From:        "Marc A Kaplan" <makaplan at us.ibm.com>
To:        gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:        10/04/2019 20:57
Subject:        Re: [gpfsug-discuss] Follow-up: ESS File systems
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



If you're into pondering some more tweaks:

-i InodeSize   is tunable

system pool : --metadata-block-size is tunable separately from  -B 
blocksize

On ESS you might want to use different block size and error correcting 
codes for (v)disks that hold system pool.
Generally I think you'd want to set up system pool for best performance 
for relatively short reads and updates.

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=mLPyKeOa1gNDrORvEXBgMw&m=qbhRxpvXiJPC72GAztszQ27LP3W7o1nmJYNV1rP2k2U&s=T5j2wkoj3NuxnK-RAMPlSc9vYHIViTOe8hGF68u5VsU&e=





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190410/2457c7a1/attachment-0002.htm>


More information about the gpfsug-discuss mailing list