<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div><div style="direction: inherit;">Not pendant but correct</div><div style="direction: inherit;"><br></div><div style="direction: inherit;">I flip there it is 1/32</div><br>--<div>Cheers</div></div><div><br>On 23 Sep 2016, at 22.16, Stephen Ulmer <<a href="mailto:ulmer@ulmer.org">ulmer@ulmer.org</a>> wrote:<br><br></div><blockquote type="cite"><div><meta http-equiv="Content-Type" content="text/html charset=utf-8">Not to be too pedantic, but I believe the the subblock size is 1/32 of the block size (which strengthens Luis’s arguments below).<div class=""><br class=""></div><div class="">I thought the the original question was NOT about inode size, but about metadata block size. You can specify that the system pool have a different block size from the rest of the filesystem, providing that it ONLY holds metadata (—metadata-block-size option to mmcrfs).</div><div class=""><br class=""></div><div class="">So with 4K inodes (which should be used for all new filesystems without some counter-indication), I would think that we’d want to use a metadata block size of 4K*32=128K. This is independent of the regular block size, which you can calculate based on the workload if you’re lucky.</div><div class=""><br class=""></div><div class="">There could be a great reason NOT to use 128K metadata block size, but I don’t know what it is. I’d be happy to be corrected about this if it’s out of whack.<br class=""><div class=""><br class=""><div class="">-- <br class="">Stephen<br class=""><br class=""><br class="">
</div><br class=""><div><blockquote type="cite" class=""><div class="">On Sep 22, 2016, at 3:37 PM, Luis Bolinches <<a href="mailto:luis.bolinches@fi.ibm.com" class="">luis.bolinches@fi.ibm.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr" class="">Hi</div><div dir="ltr" class=""> </div><div dir="ltr" class="">My 2 cents.</div><div dir="ltr" class=""> </div><div dir="ltr" class="">Leave at least 4K inodes, then you get massive improvement on small files (less 3.5K minus whatever you use on xattr)</div><div dir="ltr" class=""> </div><div dir="ltr" class="">About blocksize for data, unless you have actual data that suggest that you will actually benefit from smaller than 1MB block, leave there. GPFS uses sublocks where 1/16th of the BS can be allocated to different files, so the "waste" is much less than you think on 1MB and you get the throughput and less structures of much more data blocks.</div><div dir="ltr" class=""> </div><div dir="ltr" class="">No<strong class=""> warranty at all</strong> but I try to do this when the BS talk comes in: (might need some clean up it could not be last note but you get the idea)</div><div dir="ltr" class=""> </div><div dir="ltr" class=""><div class="">POSIX<br class="">find . -type f -name '*' -exec ls -l {} \; > find_ls_files.out</div><div class="">GPFS<br class="">cd /usr/lpp/mmfs/samples/ilm<br class="">gcc mmfindUtil_processOutputFile.c -o mmfindUtil_processOutputFile<br class="">./mmfind /gpfs/shared -ls -type f > find_ls_files.out</div><div class=""> CONVERT to CSV</div><div class=""><br class="">POSIX<br class="">cat find_ls_files.out | awk '{print $5","}' > find_ls_files.out.csv<br class="">GPFS<br class="">cat find_ls_files.out | awk '{print $7","}' > find_ls_files.out.csv</div><div class=""> LOAD in octave</div><div class=""><br class="">FILESIZE = int32 (dlmread ("find_ls_files.out.csv", ","));</div><div class=""> Clean the second column (OPTIONAL as the next clean up will do the same)</div><div class=""><br class="">FILESIZE(:,[2]) = [];</div><div class=""> If we are on 4K aligment we need to clean the files that go to inodes (WELL not exactly ... extended attributes! so maybe use a lower number!)</div><div class=""><br class="">FILESIZE(FILESIZE<=3584) =[];</div><div class=""> If we are not we need to clean the 0 size files</div><div class=""><br class="">FILESIZE(FILESIZE==0) =[];</div><div class=""> Median</div><div class=""><br class="">FILESIZEMEDIAN = int32 (median (FILESIZE))</div><div class=""> Mean</div><div class=""><br class="">FILESIZEMEAN = int32 (mean (FILESIZE))</div><div class=""> Variance</div><div class=""><br class="">int32 (var (FILESIZE))</div><div class=""> iqr interquartile range, i.e., the difference between the upper and lower quartile, of the input data.</div><div class=""><br class="">int32 (iqr (FILESIZE))</div><div class=""> Standard deviation</div><div class=""> </div><div class=""> </div><div class="">For some FS with lots of files you might need a rather powerful machine to run the calculations on octave, I never hit anything could not manage on a 64GB RAM Power box. Most of the times it is enough with my laptop.</div></div><div dir="ltr" class=""> </div><div dir="ltr" class=""> </div><div dir="ltr" class=""><br class="">--<br class="">Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations<br class=""><br class="">Luis Bolinches<br class="">Lab Services<br class=""><a href="http://www-03.ibm.com/systems/services/labservices/" class="">http://www-03.ibm.com/systems/services/labservices/</a><br class=""><br class="">IBM Laajalahdentie 23 (main Entrance) Helsinki, 00330 Finland<br class="">Phone: +358 503112585<br class=""><br class="">"If you continually give you will continually have." Anonymous</div><div dir="ltr" class=""> </div><div dir="ltr" class=""> </div><blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" class="">----- Original message -----<br class="">From: Stef Coene <<a href="mailto:stef.coene@docum.org" class="">stef.coene@docum.org</a>><br class="">Sent by: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a><br class="">To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" class="">gpfsug-discuss@spectrumscale.org</a>><br class="">Cc:<br class="">Subject: Re: [gpfsug-discuss] Blocksize<br class="">Date: Thu, Sep 22, 2016 10:30 PM<br class="">
<div class=""><font size="2" face="Default Monospace,Courier New,Courier,monospace" class="">On 09/22/2016 09:07 PM, J. Eric Wonderley wrote:<br class="">> It defaults to 4k:<br class="">> mmlsfs testbs8M -i<br class="">> flag value description<br class="">> ------------------- ------------------------<br class="">> -----------------------------------<br class="">> -i 4096 Inode size in bytes<br class="">><br class="">> I think you can make as small as 512b. Gpfs will store very small<br class="">> files in the inode.<br class="">><br class="">> Typically you want your average file size to be your blocksize and your<br class="">> filesystem has one blocksize and one inodesize.<br class=""><br class="">The files are not small, but around 20 MB on average.<br class="">So I calculated with IBM that a 1 MB or 2 MB block size is best.<br class=""><br class="">But I'm not sure if it's better to use a smaller block size for the<br class="">metadata.<br class=""><br class="">The file system is not that large (400 TB) and will hold backup data<br class="">from CommVault.<br class=""><br class=""><br class="">Stef<br class="">_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class=""><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font><br class=""> </div></blockquote><div dir="ltr" class=""> </div></div><br class="">Ellei edellä ole toisin mainittu: / Unless stated otherwise above:<br class="">Oy IBM Finland Ab<br class="">PL 265, 00101 Helsinki, Finland<br class="">Business ID, Y-tunnus: 0195876-3 <br class="">Registered in Finland<br class=""><br class="">
_______________________________________________<br class="">gpfsug-discuss mailing list<br class="">gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class=""><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class=""></div></blockquote></div><br class=""></div></div></div></blockquote><BR>
Ellei edellä ole toisin mainittu: / Unless stated otherwise above:<BR>
Oy IBM Finland Ab<BR>
PL 265, 00101 Helsinki, Finland<BR>
Business ID, Y-tunnus: 0195876-3 <BR>
Registered in Finland<BR>
<BR>
</body></html>