<font size=2 face="sans-serif">HI Carl, </font><br><font size=2 face="sans-serif">8k for 4 M Blocksize</font><br><font size=2 face="sans-serif">files < ~3,x KB fits into the inode
, for "larger" files (> 3,x KB) at least one
"subblock" be allocated .. </font><br><br><font size=2 face="sans-serif">in R < 5.x ... it was fixed 1/32
from blocksize so subblocksize is retrieved from the blocksize ...
</font><br><font size=2 face="sans-serif">since R >5 (so new created file systems)
.. the new default block size is 4 MB, fragment size is 8k (512 subblocks)
</font><br><font size=2 face="sans-serif">for even larger block sizes ... more
subblocks are available per block </font><br><font size=2 face="sans-serif">so e.g. </font><br><font size=2 face="sans-serif">8M .... 1024 subblocks (fragment size
is 8 k again)</font><br><br><font size=2 face="sans-serif">@Sven.. correct me, if I'm wrong ...
</font><br><br><br><div><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Carl <mutantllama@gmail.com></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">07/02/2018 08:55 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
subblock sanity check in 5.0</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><font size=3>Hi Sven,</font><br><br><font size=3>What is the resulting indirect-block size with a
4mb metadata block size? </font><br><br><font size=3>Does the new sub-block magic mean that it will take up
32k, or will it occupy 128k?</font><br><br><font size=3>Cheers,</font><br><br><font size=3>Carl.</font><br><br><br><font size=3>On Mon, 2 Jul 2018 at 15:26, Sven Oehme <</font><a href="mailto:oehmes@gmail.com"><font size=3 color=blue><u>oehmes@gmail.com</u></font></a><font size=3>>
wrote:</font><br><font size=3>Hi,</font><br><br><font size=3 color=#2f2f2f>most traditional raid controllers can't
deal well with blocksizes above 4m, which is why the new default is 4m
and i would leave it at that unless you know for sure you get better performance
with 8mb which typically requires your raid controller volume full block
size to be 8mb with maybe a 8+2p @1mb strip size (many people confuse strip
size with full track size) .</font><br><font size=3 color=#2f2f2f>if you don't have dedicated SSDs for metadata
i would recommend to just use a 4mb blocksize with mixed data and metadata
disks, if you have a reasonable number of SSD's put them in a raid 1 or
raid 10 and use them as dedicated metadata and the other disks as dataonly
, but i would not use the --metadata-block-size parameter as it prevents
the datapool to use large number of subblocks.</font><br><font size=3 color=#2f2f2f>as long as your SSDs are on raid 1 or 10
there is no read/modify/write penalty, so using them with the 4mb blocksize
has no real negative impact at least on controllers i have worked with.</font><br><br><font size=3 color=#2f2f2f>hope this helps.</font><br><br><font size=3>On Tue, Jun 26, 2018 at 5:18 PM Joseph Mendoza <</font><a href="mailto:jam@ucar.edu" target="_blank"><font size=3 color=blue><u>jam@ucar.edu</u></font></a><font size=3>>
wrote:</font><br><font size=3>Hi, it's for a traditional NSD setup.</font><p><font size=3>--Joey</font><p><br><font size=3>On 6/26/18 12:21 AM, Sven Oehme wrote:</font><br><font size=3>Joseph, </font><br><br><font size=3>the subblocksize will be derived from the smallest blocksize
in the filesytem, given you specified a metadata block size of 512k thats
what will be used to calculate the number of subblocks, even your data
pool is 4mb. </font><br><font size=3>is this setup for a traditional NSD Setup or for GNR as
the recommendations would be different. </font><br><br><font size=3>sven<br></font><br><font size=3>On Tue, Jun 26, 2018 at 2:59 AM Joseph Mendoza <</font><a href="mailto:jam@ucar.edu" target="_blank"><font size=3 color=blue><u>jam@ucar.edu</u></font></a><font size=3>>
wrote:</font><br><font size=3>Quick question, anyone know why GPFS wouldn't respect
the default for<br>the subblocks-per-full-block parameter when creating a new filesystem? <br>I'd expect it to be set to 512 for an 8MB block size but my guess is<br>that also specifying a metadata-block-size is interfering with it (by<br>being too small). This was a parameter recommended by the vendor
for a<br>4.2 installation with metadata on dedicated SSDs in the system pool, any<br>best practices for 5.0? I'm guessing I'd have to bump it up to at
least<br>4MB to get 512 subblocks for both pools.<br><br>fs1 created with:<br># mmcrfs fs1 -F fs1_ALL -A no -B 8M -i 4096 -m 2 -M 2 -r 1 -R 2 -j<br>cluster -n 9000 --metadata-block-size 512K --perfileset-quota<br>--filesetdf -S relatime -Q yes --inode-limit 20000000:10000000 -T /gpfs/fs1<br><br># mmlsfs fs1<br><snipped><br><br>flag
value
description<br>------------------- ------------------------<br>-----------------------------------<br> -f
8192
Minimum fragment (subblock)<br>size in bytes (system pool)<br>
131072
Minimum fragment (subblock)<br>size in bytes (other pools)<br> -i
4096
Inode size in bytes<br> -I
32768
Indirect block size in bytes<br><br> -B
524288
Block size (system pool)<br>
8388608
Block size (other pools)<br><br> -V
19.01 (5.0.1.0) File
system version<br><br> --subblocks-per-full-block 64
Number of subblocks per<br>full block<br> -P
system;DATA
Disk storage pools in file<br>system<br><br><br>Thanks!<br>--Joey Mendoza<br>NCAR<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font><a href="http://spectrumscale.org" target="_blank"><font size=3 color=blue><u>spectrumscale.org</u></font></a><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><font size=3 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></a><br><font size=3><br></font><br><tt><font size=3>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font></tt><a href="http://spectrumscale.org" target="_blank"><tt><font size=3 color=blue><u>spectrumscale.org</u></font></tt></a><tt><font size=3><br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><tt><font size=3 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></tt></a><tt><font size=3><br></font></tt><br><br><font size=3>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font><a href="http://spectrumscale.org" target="_blank"><font size=3 color=blue><u>spectrumscale.org</u></font></a><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><font size=3 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></a><tt><font size=2>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br><br></div><BR>