<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi All,
<div class=""><br class="">
</div>
<div class="">I do have metadata replication set to two, so Alex, does that make more sense?</div>
<div class=""><br class="">
</div>
<div class="">And I had forgotten about indirect blocks for large files, which actually makes sense with the user in question … my apologies for that … due to a very gravely ill pet and a recovering at home from pneumonia family member I’m way more sleep deprived
right now than I’d like. :-(</div>
<div class=""><br class="">
</div>
<div class="">Fred - I think you’ve already answered this … but mmchfs can only create / allocate more inodes … it cannot be used to shrink the number of inodes? That would make sense, and if that’s the case then I can allocate more NSDs to the system pool.</div>
<div class=""><br class="">
</div>
<div class="">Thanks…</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Jan 23, 2018, at 11:27 AM, Alex Chekholko <<a href="mailto:alex@calicolabs.com" class="">alex@calicolabs.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">2.8TB seems quite high for only 350M inodes. Are you sure you only have metadata in there?</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On Tue, Jan 23, 2018 at 9:25 AM, Frederick Stock <span dir="ltr" class="">
<<a href="mailto:stockf@us.ibm.com" target="_blank" class="">stockf@us.ibm.com</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<font size="2" face="sans-serif" class="">One possibility is the creation/expansion of directories or allocation of indirect blocks for large files.</font><br class="">
<br class="">
<font size="2" face="sans-serif" class="">Not sure if this is the issue here but at one time inode allocation was considered slow and so folks may have pre-allocated inodes to avoid that overhead during file creation. To my understanding inode creation time
is not so slow that users need to pre-allocate inodes. Yes, there are likely some applications where pre-allocating may be necessary but I expect they would be the exception. I mention this because you have a lot of free inodes and of course once they are
allocated they cannot be de-allocated. </font><br class="">
<br class="">
<font size="3" face="sans-serif" class="">Fred<br class="">
______________________________<wbr class="">____________________<br class="">
Fred Stock | IBM Pittsburgh Lab | <a href="tel:(720)%20430-8821" value="+17204308821" target="_blank" class="">
720-430-8821</a><br class="">
<a href="mailto:stockf@us.ibm.com" target="_blank" class="">stockf@us.ibm.com</a></font><br class="">
<br class="">
<br class="">
<br class="">
<font size="1" color="#5f5f5f" face="sans-serif" class="">From: </font><font size="1" face="sans-serif" class="">"Buterbaugh, Kevin L" <Kevin.Buterbaugh@Vanderbilt.<wbr class="">Edu></font><span class=""><br class="">
<font size="1" color="#5f5f5f" face="sans-serif" class="">To: </font><font size="1" face="sans-serif" class="">gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" class="">gpfsug-discuss@spectrumscale.<wbr class="">org</a>></font><br class="">
</span><font size="1" color="#5f5f5f" face="sans-serif" class="">Date: </font><font size="1" face="sans-serif" class="">01/23/2018 12:17 PM</font><span class=""><br class="">
<font size="1" color="#5f5f5f" face="sans-serif" class="">Subject: </font><font size="1" face="sans-serif" class="">[gpfsug-discuss] Metadata only system pool</font><br class="">
</span><font size="1" color="#5f5f5f" face="sans-serif" class="">Sent by: </font><font size="1" face="sans-serif" class=""><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank" class="">gpfsug-discuss-bounces@<wbr class="">spectrumscale.org</a></font><br class="">
<hr noshade="" class="">
<div class="">
<div class="h5"><br class="">
<br class="">
<br class="">
<font size="3" class="">Hi All, </font><br class="">
<br class="">
<font size="3" class="">I was under the (possibly false) impression that if you have a filesystem where the system pool contains metadata only then the only thing that would cause the amount of free space in that pool to change is the creation of more inodes
… is that correct? In other words, given that I have a filesystem with 130 million free (but allocated) inodes:</font><br class="">
<br class="">
<font size="3" class="">Inode Information</font><br class="">
<font size="3" class="">-----------------</font><br class="">
<font size="3" class="">Number of used inodes: 218635454</font><br class="">
<font size="3" class="">Number of free inodes: 131364674</font><br class="">
<font size="3" class="">Number of allocated inodes: 350000128</font><br class="">
<font size="3" class="">Maximum number of inodes: 350000128</font><br class="">
<font size="3" class=""><br class="">
I would not expect that a user creating a few hundred or thousands of files could cause a “no space left on device” error (which I’ve got one user getting). There’s plenty of free data space, BTW.</font><br class="">
<br class="">
<font size="3" class="">Now my system pool is almost “full”:</font><br class="">
<br class="">
<font size="3" class="">(pool total) 2.878T 34M ( 0%) 140.9M ( 0%)</font><br class="">
<br class="">
<font size="3" class="">But again, what - outside of me creating more inodes - would cause that to change??</font><br class="">
<br class="">
<font size="3" class="">Thanks…</font><br class="">
<br class="">
<font size="3" class="">Kevin</font><br class="">
<br class="">
<font size="3" class="">—</font><br class="">
<font size="3" class="">Kevin Buterbaugh - Senior System Administrator</font><br class="">
<font size="3" class="">Vanderbilt University - Advanced Computing Center for Research and Education</font><br class="">
<a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" target="_blank" class=""><font size="3" color="blue" class=""><u class="">Kevin.Buterbaugh@vanderbilt.<wbr class="">edu</u></font></a><font size="3" class="">-
<a href="tel:(615)%20875-9633" value="+16158759633" target="_blank" class="">(615)875-9633</a></font><br class="">
<br class="">
<br class="">
</div>
</div>
<tt class=""><font size="2" class="">______________________________<wbr class="">_________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspectrumscale.org&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C1607a3fe872e4241587b08d56286a746%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636523252830007825&sdata=h%2BCEzaXFYl%2By89m3IVPo960AeN2CL7idGpgCLZgOip8%3D&reserved=0" target="_blank" class="">
spectrumscale.org</a><br class="">
</font></tt><a href="https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss%26d%3DDwICAg%26c%3Djf_iaSHvJObTbx-siA1ZOg%26r%3Dp_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw%26m%3Dgou0xYZwz8M-5i8mT6Tthafi8JW2aMrzQGMK1hUEUls%26s%3DjcHOB_vmJjE8PnrpfHqzMkm1nk6QWwkn2npTEP6kcKs%26e%3D&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C1607a3fe872e4241587b08d56286a746%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636523252830007825&sdata=%2B87gWCFaIiPJUDwTWW5KxsR11rJJB6o6EfrIgIOGPKE%3D&reserved=0" target="_blank" class=""><tt class=""><font size="2" class="">https://urldefense.proofpoint.<wbr class="">com/v2/url?u=http-3A__gpfsug.<wbr class="">org_mailman_listinfo_gpfsug-<wbr class="">2Ddiscuss&d=DwICAg&c=jf_<wbr class="">iaSHvJObTbx-siA1ZOg&r=p_<wbr class="">1XEUyoJ7-VJxF_w8h9gJh8_<wbr class="">Wj0Pey73LCLLoxodpw&m=<wbr class="">gou0xYZwz8M-<wbr class="">5i8mT6Tthafi8JW2aMrzQGMK1hUEUl<wbr class="">s&s=jcHOB_<wbr class="">vmJjE8PnrpfHqzMkm1nk6QWwkn2npT<wbr class="">EP6kcKs&e=</font></tt></a><tt class=""><font size="2" class=""><br class="">
</font></tt><br class="">
<br class="">
<br class="">
<br class="">
______________________________<wbr class="">_________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspectrumscale.org&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C1607a3fe872e4241587b08d56286a746%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636523252830007825&sdata=h%2BCEzaXFYl%2By89m3IVPo960AeN2CL7idGpgCLZgOip8%3D&reserved=0" rel="noreferrer" target="_blank" class="">
spectrumscale.org</a><br class="">
<a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C1607a3fe872e4241587b08d56286a746%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636523252830007825&sdata=rIFx3lzbAIH5SZtFxJsVqWMMSo%2F0LssNc4K4tZH3uQc%3D&reserved=0" rel="noreferrer" target="_blank" class="">http://gpfsug.org/mailman/<wbr class="">listinfo/gpfsug-discuss</a><br class="">
<br class="">
</blockquote>
</div>
<br class="">
</div>
_______________________________________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class="">
<a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C1607a3fe872e4241587b08d56286a746%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636523252830007825&sdata=rIFx3lzbAIH5SZtFxJsVqWMMSo%2F0LssNc4K4tZH3uQc%3D&reserved=0" class="">https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C1607a3fe872e4241587b08d56286a746%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636523252830007825&sdata=rIFx3lzbAIH5SZtFxJsVqWMMSo%2F0LssNc4K4tZH3uQc%3D&reserved=0</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>