<span style=" font-size:10pt;font-family:sans-serif">If you "must"
exceed 1000 filesets because you are assigning each project to its own
fileset, my suggestion is this:</span><br><br><span style=" font-size:10pt;font-family:sans-serif">Yes, there are
scaling/performance/manageability benefits to using mmbackup over independent
filesets.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">But maybe you
don't need 10,000 independent filesets -- </span><br><span style=" font-size:10pt;font-family:sans-serif">maybe you can
hash or otherwise randomly assign projects that each have their own (dependent)
fileset name to a lesser number of independent filesets that will serve
as management groups for (mm)backup...<br></span><br><span style=" font-size:10pt;font-family:sans-serif">Like many things
in life, sometimes compromises are necessary!</span><br><br><br><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:
</span><span style=" font-size:9pt;font-family:sans-serif">"Peinkofer,
Stephan" <Stephan.Peinkofer@lrz.de></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:
</span><span style=" font-size:9pt;font-family:sans-serif">gpfsug
main discussion list <gpfsug-discuss@spectrumscale.org></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:
</span><span style=" font-size:9pt;font-family:sans-serif">08/13/2018
03:26 AM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:
</span><span style=" font-size:9pt;font-family:sans-serif">Re:
[gpfsug-discuss] GPFS Independent Fileset Limit vs Quotas?</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Sent
by: </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><br><hr noshade><br><br><br><span style=" font-size:12pt">Dear Marc,</span><br><br><span style=" font-size:12pt">OK, so let’s give it a try:</span><br><br><span style=" font-size:12pt">[root@datdsst100 pr74qo]# mmlsfileset
dsstestfs01</span><br><span style=" font-size:12pt">Filesets in file system 'dsstestfs01':</span><br><span style=" font-size:12pt">Name
Status Path
</span><br><span style=" font-size:12pt">root
Linked /dss/dsstestfs01
</span><br><span style=" font-size:12pt">...</span><br><span style=" font-size:12pt">quota_test_independent Linked
/dss/dsstestfs01/quota_test_independent </span><br><span style=" font-size:12pt">quota_test_dependent Linked
/dss/dsstestfs01/quota_test_independent/quota_test_dependent</span><br><br><span style=" font-size:12pt">[root@datdsst100 pr74qo]# mmsetquota
dsstestfs01:quota_test_independent --user a2822bp --block 1G:1G --files
10:10</span><br><span style=" font-size:12pt">[root@datdsst100 pr74qo]# mmsetquota
dsstestfs01:quota_test_dependent --user a2822bp --block 10G:10G --files
100:100</span><br><br><span style=" font-size:12pt">[root@datdsst100 pr74qo]# mmrepquota
-u -v dsstestfs01:quota_test_independent </span><br><span style=" font-size:12pt">*** Report for USR quotas on dsstestfs01</span><br><span style=" font-size:12pt">
Block Limits
|
File Limits</span><br><span style=" font-size:12pt">Name fileset
type KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</span><br><span style=" font-size:12pt">a2822bp quota_test_independent
USR 0 1048576
1048576 0
none | 0 10
10 0 none e
</span><br><span style=" font-size:12pt">root quota_test_independent
USR 0
0 0
0 none | 1
0 0 0
none i </span><br><br><span style=" font-size:12pt">[root@datdsst100 pr74qo]# mmrepquota
-u -v dsstestfs01:quota_test_dependent</span><br><span style=" font-size:12pt">*** Report for USR quotas on dsstestfs01</span><br><span style=" font-size:12pt">
Block Limits
|
File Limits</span><br><span style=" font-size:12pt">Name fileset
type KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</span><br><span style=" font-size:12pt">a2822bp quota_test_dependent
USR 0 10485760
10485760 0 none
| 0 100 100
0 none e
</span><br><span style=" font-size:12pt">root quota_test_dependent
USR 0
0 0
0 none | 1
0 0 0
none i </span><br><br><span style=" font-size:12pt">Looks good …</span><br><br><span style=" font-size:12pt">[root@datdsst100 pr74qo]# cd /dss/dsstestfs01/quota_test_independent/quota_test_dependent/</span><br><span style=" font-size:12pt">[root@datdsst100 quota_test_dependent]#
for foo in `seq 1 99`; do touch file${foo}; chown a2822bp:pr28fa file${foo};
done</span><br><br><span style=" font-size:12pt">[root@datdsst100 quota_test_dependent]#
mmrepquota -u -v dsstestfs01:quota_test_dependent</span><br><span style=" font-size:12pt">*** Report for USR quotas on dsstestfs01</span><br><span style=" font-size:12pt">
Block Limits
|
File Limits</span><br><span style=" font-size:12pt">Name fileset
type KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</span><br><span style=" font-size:12pt">a2822bp quota_test_dependent
USR 0 10485760
10485760 0 none
| 99 100 100
0 none e
</span><br><span style=" font-size:12pt">root quota_test_dependent
USR 0
0 0
0 none | 1
0 0 0
none i </span><br><span style=" font-size:12pt"> </span><br><span style=" font-size:12pt">[root@datdsst100 quota_test_dependent]#
mmrepquota -u -v dsstestfs01:quota_test_independent </span><br><span style=" font-size:12pt">*** Report for USR quotas on dsstestfs01</span><br><span style=" font-size:12pt">
Block Limits
|
File Limits</span><br><span style=" font-size:12pt">Name fileset
type KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</span><br><span style=" font-size:12pt">a2822bp quota_test_independent
USR 0 1048576
1048576 0
none | 0 10
10 0 none e
</span><br><span style=" font-size:12pt">root quota_test_independent
USR 0
0 0
0 none | 1
0 0 0
none i </span><br><br><span style=" font-size:12pt">So it seems that per fileset per user
quota is really not depending on independence. But what is the documentation
then meaning with:</span><br><span style=" font-size:12pt">>>> User group and user quotas
can be tracked at the file system level or per independent fileset.</span><br><span style=" font-size:12pt">???</span><br><br><span style=" font-size:12pt">However, there still remains the problem
with mmbackup and mmapplypolicy …</span><br><span style=" font-size:12pt">And if you look at some of the RFEs,
like the one from DESY, they want even more than 10k independent filesets
…</span><br><br><br><span style=" font-size:12pt">Best Regards,</span><br><span style=" font-size:12pt">Stephan Peinkofer</span><br><span style=" font-size:7pt;font-family:Helvetica">-- <br>Stephan Peinkofer<br>Dipl. Inf. (FH), M. Sc. (TUM)<br> <br>Leibniz Supercomputing Centre<br>Data and Storage Division<br>Boltzmannstraße 1, 85748 Garching b. München<br>Tel: +49(0)89 35831-8715 Fax: +49(0)89 35831-9700<br>URL: </span><a href="http://www.lrz.de"><span style=" font-size:7pt;color:blue;font-family:Helvetica"><u>http://www.lrz.de</u></span></a><br><br><span style=" font-size:12pt">On 12. Aug 2018, at 15:05, Marc A Kaplan
<</span><a href="mailto:makaplan@us.ibm.com"><span style=" font-size:12pt;color:blue"><u>makaplan@us.ibm.com</u></span></a><span style=" font-size:12pt">>
wrote:</span><br><br><span style=" font-size:10pt;font-family:sans-serif">That's interesting,
I confess I never read that piece of documentation.<br>What's also interesting, is that if you look at this doc for quotas:</span><span style=" font-size:12pt"><br></span><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_change_quota_anynum_users_onproject_basis_acrs_protocols.htm"><span style=" font-size:10pt;color:blue;font-family:sans-serif"><u>https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_change_quota_anynum_users_onproject_basis_acrs_protocols.htm</u></span></a><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>The word independent appears only once in a "Note": </span><span style=" font-size:10pt;color:#2f2f2f;font-family:Arial"><i>It
is recommended to create an independent fileset for the project.</i></span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>AND if you look at the mmchfs or mmchcr command you see:</span><br><span style=" font-size:10pt;color:#2f2f2f">--perfileset-quota </span><br><span style=" font-size:6pt;color:#2f2f2f;font-family:Arial"><br> </span><span style=" font-size:10pt;color:#2f2f2f;font-family:Arial">Sets
the scope of user and group quota limit checks to the individual fileset
level, rather than to the entire file system.</span><span style=" font-size:6pt;color:#2f2f2f;font-family:Arial"></span><span style=" font-size:12pt"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>With no mention of "dependent" nor "independent"...</span><span style=" font-size:12pt"><br><br><br><br></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>From: </span><span style=" font-size:9pt;font-family:sans-serif">"Peinkofer,
Stephan" <</span><a href="mailto:Stephan.Peinkofer@lrz.de"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>Stephan.Peinkofer@lrz.de</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>To: </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug
main discussion list <</span><a href="mailto:gpfsug-discuss@spectrumscale.org"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>gpfsug-discuss@spectrumscale.org</u></span></a><span style=" font-size:9pt;font-family:sans-serif">></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Date: </span><span style=" font-size:9pt;font-family:sans-serif">08/11/2018
03:03 AM</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Subject: </span><span style=" font-size:9pt;font-family:sans-serif">Re:
[gpfsug-discuss] GPFS Independent Fileset Limit</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Sent by: </span><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><span style=" font-size:9pt;color:blue;font-family:sans-serif"><u>gpfsug-discuss-bounces@spectrumscale.org</u></span></a><span style=" font-size:12pt"><br></span><hr noshade><span style=" font-size:12pt"><br></span><br><span style=" font-size:12pt;font-family:Calibri">Dear Marc,</span><br><span style=" font-size:12pt;font-family:Calibri">so at least your
documentation says:</span><br><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/bl1hlp_filesfilesets.htm"><span style=" font-size:12pt;color:blue;font-family:Calibri"><u>https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/bl1hlp_filesfilesets.htm</u></span></a><br><span style=" font-size:12pt;color:#2f2f2f;font-family:Arial">>>>
User group and user quotas can be tracked at the file system level or per
independent fileset.</span><br><span style=" font-size:12pt;font-family:Calibri">But obviously as
a customer I don't know if that "Really" depends on independence.</span><br><span style=" font-size:12pt;font-family:Calibri">Currently about 70%
of our filesets in the Data Science Storage systems get backed up to ISP.
But that number may change over time as it depends on the requirements
of our projects. For them it is just selecting "Protect this DSS Container
by ISP" in a Web form an our portal then automatically does all the
provisioning of the ISP Node to one of our ISP servers, rolling out the
new dsm config files to the backup workers and so on.</span><br><span style=" font-size:12pt;font-family:Calibri"><br><br>Best Regards, <br>Stephan Peinkofer</span><span style=" font-size:12pt"><br></span><hr><span style=" font-size:11pt;font-family:Calibri"><b><br>From:</b> </span><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><span style=" font-size:11pt;color:blue;font-family:Calibri"><u>gpfsug-discuss-bounces@spectrumscale.org</u></span></a><span style=" font-size:11pt;font-family:Calibri"><</span><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><span style=" font-size:11pt;color:blue;font-family:Calibri"><u>gpfsug-discuss-bounces@spectrumscale.org</u></span></a><span style=" font-size:11pt;font-family:Calibri">>
on behalf of Marc A Kaplan <</span><a href="mailto:makaplan@us.ibm.com"><span style=" font-size:11pt;color:blue;font-family:Calibri"><u>makaplan@us.ibm.com</u></span></a><span style=" font-size:11pt;font-family:Calibri">><b><br>Sent:</b> Friday, August 10, 2018 7:15 PM<b><br>To:</b> gpfsug main discussion list<b><br>Subject:</b> Re: [gpfsug-discuss] GPFS Independent Fileset Limit</span><span style=" font-size:12pt;font-family:Calibri"><br> </span><span style=" font-size:10pt;font-family:sans-serif"><br>I know quota stuff was cooked into GPFS before we even had "independent
filesets"...<br>So which particular quota features or commands or options now depend on
"independence"?! Really?<br><br>Yes, independent fileset performance for mmapplypolicy and mmbackup scales
with the inodespace sizes. But I'm curious to know how many of those indy
filesets are mmback-ed-up.<br><br>Appreciate your elaborations, 'cause even though I've worked on some of
this code, I don't know how/when/if customers push which limits.<br><br>---------------------</span><span style=" font-size:12pt"> </span><br><span style=" font-size:12pt;font-family:Calibri">Dear Marc,</span><br><span style=" font-size:12pt;font-family:Calibri">well the primary
reasons for us are:</span><br><span style=" font-size:12pt;font-family:Calibri">- Per fileset quota
(this seems to work also for dependent filesets as far as I know) </span><br><span style=" font-size:12pt;font-family:Calibri">- Per user per fileset
quota (this seems only to work for independent filesets)</span><br><span style=" font-size:12pt;font-family:Calibri">- The dedicated inode
space to speedup mmpolicy runs which only have to be applied to a specific
subpart of the file system</span><br><span style=" font-size:12pt;font-family:Calibri">- Scaling mmbackup
by backing up different filesets to different TSM Servers economically</span><br><span style=" font-size:12pt;font-family:Calibri">We have currently
more than 1000 projects on our HPC machines and several different existing
and planned file systems (use cases):</span><br><span style=" font-size:12pt;font-family:Calibri"><br></span><tt><span style=" font-size:10pt"><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </span></tt><a href="http://spectrumscale.org"><tt><span style=" font-size:10pt;color:blue"><u>spectrumscale.org</u></span></tt></a><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style=" font-size:10pt;color:blue"><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></span></tt></a><span style=" font-size:12pt"><br><br><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </span><a href="http://spectrumscale.org"><span style=" font-size:12pt;color:blue"><u>spectrumscale.org</u></span></a><span style=" font-size:12pt;color:blue"><u><br></u></span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><span style=" font-size:12pt;color:blue"><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></span></a><br><tt><span style=" font-size:10pt">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></span></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style=" font-size:10pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt><span style=" font-size:10pt"><br></span></tt><br><br><BR>