<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi All,
<div class=""><br class="">
</div>
<div class="">So I was looking at the presentation referenced below and it states - on multiple slides - that there is one system storage pool per cluster. Really? Shouldn’t that be one system storage pool per filesystem?!? If not, please explain how in
my GPFS cluster with two (local) filesystems I see two different system pools with two different sets of NSDs, two different capacities, and two different percentages full???</div>
<div class=""><br class="">
</div>
<div class="">Thanks…</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class=""><br class="">
<div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class="">—</div>
<div class="">Kevin Buterbaugh - Senior System Administrator</div>
<div class="">Vanderbilt University - Advanced Computing Center for Research and Education</div>
<div class=""><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" class="">Kevin.Buterbaugh@vanderbilt.edu</a> - (615)875-9633</div>
</div>
</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Mar 26, 2019, at 11:27 AM, Dorigo Alvise (PSI) <<a href="mailto:alvise.dorigo@psi.ch" class="">alvise.dorigo@psi.ch</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; direction: ltr; font-family: Tahoma; font-size: 10pt;" class="">
<div class="">Hi Marc,<br class="">
</div>
<div class="">"Indirect block size" is well explained in this presentation:<span class="Apple-converted-space"> </span><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Ffiles.gpfsug.org%2Fpresentations%2F2016%2Fsouth-bank%2FD2_P2_A_spectrum_scale_metadata_dark_V2a.pdf&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C5b28a9a0d39a47fd3f0608d6b208186a%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636892145179816620&sdata=SDAw4EXR1jNDYZRdIPtNVuhR1ZfBiUFcF7%2FCrprNwag%3D&reserved=0" originalsrc="http://files.gpfsug.org/presentations/2016/south-bank/D2_P2_A_spectrum_scale_metadata_dark_V2a.pdf" shash="cTgclma2BYLWO8+U2OSNl3mbJ5gHwSxskH4XL3Lqw8okCCRbJRsIrI5w53SGaJlxdkanrAi4R1qxnF9aBK4wdjpGYHlyNuxUVXQomC6uQFY0wjEbeX/WlaShhvNApZMyNeiCd9IVdFvOdm7PuYcQfnQNLksrS1XVpXwlAIAZl1U=" target="_blank" class="">http://files.gpfsug.org/presentations/2016/south-bank/D2_P2_A_spectrum_scale_metadata_dark_V2a.pdf</a></div>
<div class=""><br class="">
</div>
<div class="">pages 37-41<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Cheers,</div>
<div class=""><br class="">
</div>
<div class=""> Alvise<br class="">
</div>
<div class=""><br class="">
</div>
<div style="font-family: "Times New Roman"; font-size: 16px;" class="">
<hr tabindex="-1" class="">
<div id="divRpF933280" style="direction: ltr;" class=""><font size="2" face="Tahoma" class=""><b class="">From:</b><span class="Apple-converted-space"> </span><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a><span class="Apple-converted-space"> </span>[<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a>]
on behalf of Caubet Serrabou Marc (PSI) [<a href="mailto:marc.caubet@psi.ch" class="">marc.caubet@psi.ch</a>]<br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>Tuesday, March 26, 2019 4:39 PM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>gpfsug main discussion list<br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>[gpfsug-discuss] GPFS v5: Blocksizes and subblocks<br class="">
</font><br class="">
</div>
<div class=""></div>
<div class="">
<div style="direction: ltr; font-family: Tahoma; font-size: 10pt;" class="">
<div class="">Hi all,</div>
<div class=""><br class="">
</div>
<div class="">according to several GPFS presentations as well as according to the man pages:</div>
<div class=""><br class="">
</div>
<div class=""><font face="Courier New" class=""> Table 1. Block sizes and subblock sizes<br class="">
<br class="">
+-------------------------------+-------------------------------+<br class="">
| Block size | Subblock size |<br class="">
+-------------------------------+-------------------------------+<br class="">
| 64 KiB | 2 KiB |<br class="">
+-------------------------------+-------------------------------+<br class="">
| 128 KiB | 4 KiB |<br class="">
+-------------------------------+-------------------------------+<br class="">
| 256 KiB, 512 KiB, 1 MiB, 2 | 8 KiB |<br class="">
| MiB, 4 MiB | |<br class="">
+-------------------------------+-------------------------------+<br class="">
| 8 MiB, 16 MiB | 16 KiB |<br class="">
+-------------------------------+-------------------------------+</font><br class="">
<br class="">
</div>
<div class="">A block size of 8MiB or 16MiB should contain subblocks of 16KiB.</div>
<div class=""><br class="">
</div>
<div class="">However, when creating a new filesystem with 16MiB blocksize, looks like is using 128KiB subblocks:</div>
<div class=""><br class="">
</div>
<div class=""><font face="Courier New" class="">[root@merlindssio01 ~]# mmlsfs merlin<br class="">
flag value description<br class="">
------------------- ------------------------ -----------------------------------<br class="">
-f 8192 Minimum fragment (subblock) size in bytes (system pool)<br class="">
131072 Minimum fragment (subblock) size in bytes (other pools)<br class="">
-i 4096 Inode size in bytes<br class="">
-I 32768 Indirect block size in bytes<br class="">
.</font></div>
<div class=""><font face="Courier New" class="">.</font></div>
<div class=""><font face="Courier New" class="">.<br class="">
</font></div>
<div class=""><font face="Courier New" class=""> -n 128 Estimated number of nodes that will mount file system<br class="">
-B 1048576 Block size (system pool)<br class="">
16777216 Block size (other pools)<br class="">
.</font></div>
<div class=""><font face="Courier New" class="">.</font></div>
<div class=""><font face="Courier New" class="">.</font><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">What am I missing? According to documentation, I expect this to be a fixed value, or it isn't at all?<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">On the other hand, I don't really understand the concept 'Indirect block size in bytes', can somebody clarify or provide some details about this setting?</div>
<div class=""><br class="">
</div>
<div class="">Thanks a lot and best regards,</div>
<div class="">Marc <span class="Apple-converted-space"> </span><br class="">
<div style="font-family: Tahoma; font-size: 13px;" class="">
<div style="font-family: Tahoma; font-size: 13px;" class="">
<div class="BodyFragment"><font size="2" class=""><span style="font-size: 10pt;" class="">
<div class="PlainText">_________________________________________<br class="">
Paul Scherrer Institut<span class="Apple-converted-space"> </span><br class="">
High Performance Computing<br class="">
Marc Caubet Serrabou<br class="">
Building/Room: WHGA/019A</div>
<div class="PlainText"><font size="2" class=""><span style="font-size: 10pt;" class="">Forschungsstrasse, 111</span></font></div>
<div class="PlainText">5232 Villigen PSI<br class="">
Switzerland<br class="">
<br class="">
Telephone: +41 56 310 46 67<br class="">
E-Mail:<span class="Apple-converted-space"> </span><a href="mailto:marc.caubet@psi.ch" class="">marc.caubet@psi.ch</a></div>
</span></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">gpfsug-discuss
mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">gpfsug-discuss
at<span class="Apple-converted-space"> </span></span><a href="http://spectrumscale.org/" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">spectrumscale.org</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<a href="https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C5b28a9a0d39a47fd3f0608d6b208186a%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636892145179836634&sdata=23F22sUiyCYEg0H3AdbkBAnhPpLVBVTh39zRr%2FLYCmc%3D&reserved=0" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C5b28a9a0d39a47fd3f0608d6b208186a%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636892145179836634&sdata=23F22sUiyCYEg0H3AdbkBAnhPpLVBVTh39zRr%2FLYCmc%3D&reserved=0</a></div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>