<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">Dear Marc,</div>
<div class=""><br class="">
</div>
<div class="">OK, so let’s give it a try:</div>
<div class=""><br class="">
</div>
<div class="">[root@datdsst100 pr74qo]# mmlsfileset dsstestfs01</div>
<div class="">Filesets in file system 'dsstestfs01':</div>
<div class="">Name Status Path </div>
<div class="">root Linked /dss/dsstestfs01 </div>
<div class="">...</div>
<div class="">quota_test_independent Linked /dss/dsstestfs01/quota_test_independent </div>
<div class="">quota_test_dependent Linked /dss/dsstestfs01/quota_test_independent/quota_test_dependent</div>
<div class=""><br class="">
</div>
<div class="">[root@datdsst100 pr74qo]# mmsetquota dsstestfs01:quota_test_independent --user a2822bp --block 1G:1G --files 10:10</div>
<div class="">[root@datdsst100 pr74qo]# mmsetquota dsstestfs01:quota_test_dependent --user a2822bp --block 10G:10G --files 100:100</div>
<div class=""><br class="">
</div>
<div class="">[root@datdsst100 pr74qo]# mmrepquota -u -v dsstestfs01:quota_test_independent </div>
<div class="">*** Report for USR quotas on dsstestfs01</div>
<div class=""> Block Limits | File Limits</div>
<div class="">Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType</div>
<div class="">a2822bp quota_test_independent USR 0 1048576 1048576 0 none | 0 10 10 0 none e </div>
<div class="">root quota_test_independent USR 0 0 0 0 none | 1 0 0 0 none i </div>
<div class=""><br class="">
</div>
<div class="">[root@datdsst100 pr74qo]# mmrepquota -u -v dsstestfs01:quota_test_dependent</div>
<div class="">*** Report for USR quotas on dsstestfs01</div>
<div class=""> Block Limits | File Limits</div>
<div class="">Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType</div>
<div class="">a2822bp quota_test_dependent USR 0 10485760 10485760 0 none | 0 100 100 0 none e </div>
<div class="">root quota_test_dependent USR 0 0 0 0 none | 1 0 0 0 none i </div>
<div class=""><br class="">
</div>
<div class="">Looks good …</div>
<div class=""><br class="">
</div>
<div class="">[root@datdsst100 pr74qo]# cd /dss/dsstestfs01/quota_test_independent/quota_test_dependent/</div>
<div class="">[root@datdsst100 quota_test_dependent]# for foo in `seq 1 99`; do touch file${foo}; chown a2822bp:pr28fa file${foo}; done</div>
<div class=""><br class="">
</div>
<div class="">[root@datdsst100 quota_test_dependent]# mmrepquota -u -v dsstestfs01:quota_test_dependent</div>
<div class="">*** Report for USR quotas on dsstestfs01</div>
<div class=""> Block Limits | File Limits</div>
<div class="">Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType</div>
<div class="">a2822bp quota_test_dependent USR 0 10485760 10485760 0 none | 99 100 100 0 none e </div>
<div class="">root quota_test_dependent USR 0 0 0 0 none | 1 0 0 0 none i </div>
<div class=""> </div>
<div class="">[root@datdsst100 quota_test_dependent]# mmrepquota -u -v dsstestfs01:quota_test_independent </div>
<div class="">*** Report for USR quotas on dsstestfs01</div>
<div class=""> Block Limits | File Limits</div>
<div class="">Name fileset type KB quota limit in_doubt grace | files quota limit in_doubt grace entryType</div>
<div class="">a2822bp quota_test_independent USR 0 1048576 1048576 0 none | 0 10 10 0 none e </div>
<div class="">root quota_test_independent USR 0 0 0 0 none | 1 0 0 0 none i </div>
<div class=""><br class="">
</div>
<div class="">So it seems that per fileset per user quota is really not depending on independence. But what is the documentation then meaning with:</div>
<div class="">>>> User group and user quotas can be tracked at the file system level or per independent fileset.</div>
<div class="">???</div>
<div class=""><br class="">
</div>
<div class="">However, there still remains the problem with mmbackup and mmapplypolicy …</div>
<div class="">And if you look at some of the RFEs, like the one from DESY, they want even more than 10k independent filesets …</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Best Regards,</div>
<div class="">Stephan Peinkofer</div>
<div class="">
<div class="">
<blockquote type="cite" class="">
<div class="" style="margin-top: 0px; margin-bottom: 0px;"></div>
</blockquote>
</div>
<div class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;">
-- <br class="">
Stephan Peinkofer<br class="">
Dipl. Inf. (FH), M. Sc. (TUM)<br class="">
<br class="">
Leibniz Supercomputing Centre<br class="">
Data and Storage Division<br class="">
Boltzmannstraße 1, 85748 Garching b. München<br class="">
Tel: +49(0)89 35831-8715 Fax: +49(0)89 35831-9700<br class="">
URL: <a href="http://www.lrz.de" class="">http://www.lrz.de</a></div>
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 12. Aug 2018, at 15:05, Marc A Kaplan <<a href="mailto:makaplan@us.ibm.com" class="">makaplan@us.ibm.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class=""><span style=" font-size:10pt;font-family:sans-serif" class="">That's interesting, I confess I never read that piece of documentation.</span><br class="">
<span style=" font-size:10pt;font-family:sans-serif" class="">What's also interesting, is that if you look at this doc for quotas:</span><br class="">
<br class="">
<a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_change_quota_anynum_users_onproject_basis_acrs_protocols.htm" class=""><span style=" font-size:10pt;color:blue;font-family:sans-serif" class="">https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_change_quota_anynum_users_onproject_basis_acrs_protocols.htm</span></a><br class="">
<br class="">
<span style=" font-size:10pt;font-family:sans-serif" class="">The word independent appears only once in a "Note":
</span><span style=" font-size:10pt;color:#2f2f2f;font-family:Arial" class=""><i class="">It is recommended to create an independent fileset for the project.</i></span><span style=" font-size:12pt" class=""><i class=""></i></span><span style=" font-size:10pt;font-family:sans-serif" class=""><br class="">
</span><br class="">
<span style=" font-size:10pt;font-family:sans-serif" class="">AND if you look at the mmchfs or mmchcr command you see:</span><br class="">
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:10pt;color:#2f2f2f;font-family:DejaVu Sans Mono" class="">--perfileset-quota
</span></div>
<br class="">
<span style=" font-size:6pt;color:#2f2f2f;font-family:Arial" class=""> </span><span style=" font-size:10pt;color:#2f2f2f;font-family:Arial" class="">Sets the scope of user and group quota limit checks to the individual fileset level, rather than to the entire
file system.</span><span style=" font-size:6pt;color:#2f2f2f;font-family:Arial" class=""> </span><br class="">
<br class="">
<span style=" font-size:10pt;font-family:sans-serif" class="">With no mention of "dependent" nor "independent"...</span><br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">From: </span><span style=" font-size:9pt;font-family:sans-serif" class="">"Peinkofer, Stephan" <<a href="mailto:Stephan.Peinkofer@lrz.de" class="">Stephan.Peinkofer@lrz.de</a>></span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">To: </span><span style=" font-size:9pt;font-family:sans-serif" class="">gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" class="">gpfsug-discuss@spectrumscale.org</a>></span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">Date: </span><span style=" font-size:9pt;font-family:sans-serif" class="">08/11/2018 03:03 AM</span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">Subject: </span><span style=" font-size:9pt;font-family:sans-serif" class="">Re: [gpfsug-discuss] GPFS Independent Fileset Limit</span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">Sent by: </span><span style=" font-size:9pt;font-family:sans-serif" class=""><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a></span><br class="">
<hr noshade="" class="">
<br class="">
<br class="">
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">Dear Marc,</span></div>
<p style="margin-top:0px;margin-Bottom:0px" class=""></p>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">so at least your documentation says:</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/bl1hlp_filesfilesets.htm" class=""><span style=" font-size:12pt;color:blue;font-family:Calibri" class=""><u class="">https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/bl1hlp_filesfilesets.htm</u></span></a></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;color:#2f2f2f;font-family:Arial" class="">>>> User group and user quotas can be tracked at the file system level or per independent fileset.</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">But obviously as a customer I don't know if that "Really" depends on independence.</span></div>
<p style="margin-top:0px;margin-Bottom:0px" class=""></p>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">Currently about 70% of our filesets in the Data Science Storage systems get backed up to ISP. But that number may change over time as it depends
on the requirements of our projects. For them it is just selecting "Protect this DSS Container by ISP" in a Web form an our portal then automatically does all the provisioning of the ISP Node to one of our ISP servers, rolling out the new dsm config files
to the backup workers and so on.</span></div>
<br class="">
<span style=" font-size:12pt;font-family:Calibri" class=""><br class="">
Best Regards, </span><br class="">
<span style=" font-size:12pt;font-family:Calibri" class="">Stephan Peinkofer</span><br class="">
<hr class="">
<br class="">
<span style=" font-size:11pt;font-family:Calibri" class=""><b class="">From:</b> <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">
gpfsug-discuss-bounces@spectrumscale.org</a> <<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a>> on behalf of Marc A Kaplan <<a href="mailto:makaplan@us.ibm.com" class="">makaplan@us.ibm.com</a>><b class=""><br class="">
Sent:</b> Friday, August 10, 2018 7:15 PM<b class=""><br class="">
To:</b> gpfsug main discussion list<b class=""><br class="">
Subject:</b> Re: [gpfsug-discuss] GPFS Independent Fileset Limit</span><span style=" font-size:12pt;font-family:Calibri" class=""></span><br class="">
<span style=" font-size:12pt;font-family:Calibri" class=""> </span><br class="">
<span style=" font-size:10pt;font-family:sans-serif" class="">I know quota stuff was cooked into GPFS before we even had "independent filesets"...<br class="">
So which particular quota features or commands or options now depend on "independence"?! Really?</span><span style=" font-size:12pt;font-family:Calibri" class=""><br class="">
</span><span style=" font-size:10pt;font-family:sans-serif" class=""><br class="">
Yes, independent fileset performance for mmapplypolicy and mmbackup scales with the inodespace sizes. But I'm curious to know how many of those indy filesets are mmback-ed-up.</span><span style=" font-size:12pt;font-family:Calibri" class=""><br class="">
</span><span style=" font-size:10pt;font-family:sans-serif" class=""><br class="">
Appreciate your elaborations, 'cause even though I've worked on some of this code, I don't know how/when/if customers push which limits.</span><span style=" font-size:12pt;font-family:Calibri" class=""><br class="">
</span><span style=" font-size:10pt;font-family:sans-serif" class=""><br class="">
---------------------</span><span style=" font-size:12pt;font-family:Calibri" class=""></span>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">Dear Marc,</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">well the primary reasons for us are:</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">- Per fileset quota (this seems to work also for dependent filesets as far as I know)
</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">- Per user per fileset quota (this seems only to work for independent filesets)</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">- The dedicated inode space to speedup mmpolicy runs which only have to be applied to a specific subpart of the file system</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">- Scaling mmbackup by backing up different filesets to different TSM Servers economically</span></div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><span style=" font-size:12pt;font-family:Calibri" class="">We have currently more than 1000 projects on our HPC machines and several different existing and planned file systems (use cases):</span></div>
<br class="">
<span style=" font-size:12pt;font-family:Calibri" class=""><br class="">
</span><tt class=""><span style=" font-size:10pt" class="">_______________________________________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class="">
</span></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class=""><tt class=""><span style=" font-size:10pt" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt class=""><span style=" font-size:10pt" class=""><br class="">
</span></tt><br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class="">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>