<div dir="ltr">Hi,<div><br></div><div><span style="color:rgb(33,33,33)">most traditional raid controllers can't deal well with blocksizes above 4m, which is why the new default is 4m and i would leave it at that unless you know for sure you get better performance with 8mb which typically requires your raid controller volume full block size to be 8mb with maybe a 8+2p @1mb strip size (many people confuse</span><span style="color:rgb(33,33,33)"> strip size with full track size) .</span><br></div><div><font color="#212121">if you don't have dedicated SSDs for metadata i would recommend to just use a 4mb blocksize with mixed data and metadata disks, if you have a reasonable number of SSD's put them in a raid 1 or raid 10 and use them as dedicated metadata and the other disks as dataonly , but i would not use the --metadata-block-size parameter as it prevents the datapool to use large number of subblocks.</font></div><div><font color="#212121">as long as your SSDs are on raid 1 or 10 there is no read/modify/write penalty, so using them with the 4mb blocksize has no real negative impact at least on controllers i have worked with.</font></div><div><font color="#212121"><br></font></div><div><font color="#212121">hope this helps.</font></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jun 26, 2018 at 5:18 PM Joseph Mendoza <<a href="mailto:jam@ucar.edu">jam@ucar.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Hi, it's for a traditional NSD setup.</p>
<p>--Joey<br>
</p></div><div text="#000000" bgcolor="#FFFFFF">
<br>
<div class="m_-2092002706174047791moz-cite-prefix">On 6/26/18 12:21 AM, Sven Oehme wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Joseph,
<div><br>
</div>
<div>the subblocksize will be derived from the smallest
blocksize in the filesytem, given you specified a metadata
block size of 512k thats what will be used to calculate the
number of subblocks, even your data pool is 4mb. </div>
<div>is this setup for a traditional NSD Setup or for GNR as the
recommendations would be different. </div>
<div><br>
</div>
<div>sven<br>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, Jun 26, 2018 at 2:59 AM Joseph
Mendoza <<a href="mailto:jam@ucar.edu" target="_blank">jam@ucar.edu</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Quick
question, anyone know why GPFS wouldn't respect the
default for<br>
the subblocks-per-full-block parameter when creating a new
filesystem? <br>
I'd expect it to be set to 512 for an 8MB block size but
my guess is<br>
that also specifying a metadata-block-size is interfering
with it (by<br>
being too small). This was a parameter recommended by the
vendor for a<br>
4.2 installation with metadata on dedicated SSDs in the
system pool, any<br>
best practices for 5.0? I'm guessing I'd have to bump it
up to at least<br>
4MB to get 512 subblocks for both pools.<br>
<br>
fs1 created with:<br>
# mmcrfs fs1 -F fs1_ALL -A no -B 8M -i 4096 -m 2 -M 2 -r 1
-R 2 -j<br>
cluster -n 9000 --metadata-block-size 512K
--perfileset-quota<br>
--filesetdf -S relatime -Q yes --inode-limit
20000000:10000000 -T /gpfs/fs1<br>
<br>
# mmlsfs fs1<br>
<snipped><br>
<br>
flag value description<br>
------------------- ------------------------<br>
-----------------------------------<br>
-f 8192 Minimum
fragment (subblock)<br>
size in bytes (system pool)<br>
131072 Minimum
fragment (subblock)<br>
size in bytes (other pools)<br>
-i 4096 Inode size in
bytes<br>
-I 32768 Indirect
block size in bytes<br>
<br>
-B 524288 Block size
(system pool)<br>
8388608 Block size
(other pools)<br>
<br>
-V 19.01 (5.0.1.0) File system
version<br>
<br>
--subblocks-per-full-block 64 Number of
subblocks per<br>
full block<br>
-P system;DATA Disk storage
pools in file<br>
system<br>
<br>
<br>
Thanks!<br>
--Joey Mendoza<br>
NCAR<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote>
</div>
</div>
</div>
<br>
<fieldset class="m_-2092002706174047791mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at <a href="http://spectrumscale.org" target="_blank">spectrumscale.org</a>
<a class="m_-2092002706174047791moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>
</pre>
</blockquote>
<br>
</div></blockquote></div>