<font size=2 face="sans-serif">as Dominic said.. .. your are absolutely
right .. for mmbackup you need dedicated inode spaces .. so "independent"
filesets .. (in case I you wanna be able to mmbackup on a fileset
level or multiple mmbackup's in parallel .. )</font><br><br><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">"Peinkofer, Stephan"
<Stephan.Peinkofer@lrz.de></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">08/13/2018 09:26 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
GPFS Independent Fileset Limit vs Quotas?</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><font size=3>Dear Marc,</font><br><br><font size=3>OK, so let’s give it a try:</font><br><br><font size=3>[root@datdsst100 pr74qo]# mmlsfileset dsstestfs01</font><br><font size=3>Filesets in file system 'dsstestfs01':</font><br><font size=3>Name
Status Path
</font><br><font size=3>root
Linked /dss/dsstestfs01
</font><br><font size=3>...</font><br><font size=3>quota_test_independent Linked /dss/dsstestfs01/quota_test_independent
</font><br><font size=3>quota_test_dependent Linked /dss/dsstestfs01/quota_test_independent/quota_test_dependent</font><br><br><font size=3>[root@datdsst100 pr74qo]# mmsetquota dsstestfs01:quota_test_independent
--user a2822bp --block 1G:1G --files 10:10</font><br><font size=3>[root@datdsst100 pr74qo]# mmsetquota dsstestfs01:quota_test_dependent
--user a2822bp --block 10G:10G --files 100:100</font><br><br><font size=3>[root@datdsst100 pr74qo]# mmrepquota -u -v dsstestfs01:quota_test_independent
</font><br><font size=3>*** Report for USR quotas on dsstestfs01</font><br><font size=3>
Block Limits
|
File Limits</font><br><font size=3>Name fileset type
KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</font><br><font size=3>a2822bp quota_test_independent USR
0 1048576
1048576 0 none |
0 10
10 0 none e
</font><br><font size=3>root quota_test_independent USR
0
0 0
0 none | 1
0 0 0
none i </font><br><br><font size=3>[root@datdsst100 pr74qo]# mmrepquota -u -v dsstestfs01:quota_test_dependent</font><br><font size=3>*** Report for USR quotas on dsstestfs01</font><br><font size=3>
Block Limits
|
File Limits</font><br><font size=3>Name fileset type
KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</font><br><font size=3>a2822bp quota_test_dependent USR
0 10485760 10485760
0 none |
0 100 100
0 none e </font><br><font size=3>root quota_test_dependent USR
0
0 0
0 none | 1
0 0 0
none i </font><br><br><font size=3>Looks good …</font><br><br><font size=3>[root@datdsst100 pr74qo]# cd /dss/dsstestfs01/quota_test_independent/quota_test_dependent/</font><br><font size=3>[root@datdsst100 quota_test_dependent]# for foo in `seq
1 99`; do touch file${foo}; chown a2822bp:pr28fa file${foo}; done</font><br><br><font size=3>[root@datdsst100 quota_test_dependent]# mmrepquota
-u -v dsstestfs01:quota_test_dependent</font><br><font size=3>*** Report for USR quotas on dsstestfs01</font><br><font size=3>
Block Limits
|
File Limits</font><br><font size=3>Name fileset type
KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</font><br><font size=3>a2822bp quota_test_dependent USR
0 10485760 10485760
0 none |
99 100 100 0
none e </font><br><font size=3>root quota_test_dependent USR
0
0 0
0 none | 1
0 0 0
none i </font><br><font size=3> </font><br><font size=3>[root@datdsst100 quota_test_dependent]# mmrepquota
-u -v dsstestfs01:quota_test_independent </font><br><font size=3>*** Report for USR quotas on dsstestfs01</font><br><font size=3>
Block Limits
|
File Limits</font><br><font size=3>Name fileset type
KB quota
limit in_doubt grace | files
quota limit in_doubt grace entryType</font><br><font size=3>a2822bp quota_test_independent USR
0 1048576
1048576 0 none |
0 10
10 0 none e
</font><br><font size=3>root quota_test_independent USR
0
0 0
0 none | 1
0 0 0
none i </font><br><br><font size=3>So it seems that per fileset per user quota is really
not depending on independence. But what is the documentation then meaning
with:</font><br><font size=3>>>> User group and user quotas can be tracked
at the file system level or per independent fileset.</font><br><font size=3>???</font><br><br><font size=3>However, there still remains the problem with mmbackup
and mmapplypolicy …</font><br><font size=3>And if you look at some of the RFEs, like the one from
DESY, they want even more than 10k independent filesets …</font><br><br><br><font size=3>Best Regards,</font><br><font size=3>Stephan Peinkofer</font><br><font size=1 face="Helvetica">-- <br>Stephan Peinkofer<br>Dipl. Inf. (FH), M. Sc. (TUM)<br> <br>Leibniz Supercomputing Centre<br>Data and Storage Division<br>Boltzmannstraße 1, 85748 Garching b. München<br>Tel: +49(0)89 35831-8715 Fax: +49(0)89 35831-9700<br>URL: </font><a href="http://www.lrz.de"><font size=1 color=blue face="Helvetica"><u>http://www.lrz.de</u></font></a><br><br><font size=3>On 12. Aug 2018, at 15:05, Marc A Kaplan <</font><a href="mailto:makaplan@us.ibm.com"><font size=3 color=blue><u>makaplan@us.ibm.com</u></font></a><font size=3>>
wrote:</font><br><br><font size=2>That's interesting, I confess I never read that piece
of documentation.<br>What's also interesting, is that if you look at this doc for quotas:</font><font size=3><br></font><font size=3 color=blue><u><br></u></font><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_change_quota_anynum_users_onproject_basis_acrs_protocols.htm"><font size=2 color=blue><u>https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_change_quota_anynum_users_onproject_basis_acrs_protocols.htm</u></font></a><font size=3><br></font><font size=2><br>The word independent appears only once in a "Note": </font><font size=2 color=#2f2f2f face="Arial"><i>It
is recommended to create an independent fileset for the project.</i></font><font size=3><br></font><font size=2><br>AND if you look at the mmchfs or mmchcr command you see:</font><br><font size=2 color=#2f2f2f>--perfileset-quota </font><br><font size=1 color=#2f2f2f face="Arial"><br> </font><font size=2 color=#2f2f2f face="Arial">Sets the scope of user
and group quota limit checks to the individual fileset level, rather than
to the entire file system.</font><font size=1 color=#2f2f2f face="Arial"></font><font size=3><br></font><font size=2><br>With no mention of "dependent" nor "independent"...</font><font size=3><br><br><br><br></font><font size=1 color=#5f5f5f><br>From: </font><font size=1>"Peinkofer, Stephan"
<</font><a href="mailto:Stephan.Peinkofer@lrz.de"><font size=1 color=blue><u>Stephan.Peinkofer@lrz.de</u></font></a><font size=1>></font><font size=1 color=#5f5f5f><br>To: </font><font size=1>gpfsug main discussion
list <</font><a href="mailto:gpfsug-discuss@spectrumscale.org"><font size=1 color=blue><u>gpfsug-discuss@spectrumscale.org</u></font></a><font size=1>></font><font size=1 color=#5f5f5f><br>Date: </font><font size=1>08/11/2018 03:03 AM</font><font size=1 color=#5f5f5f><br>Subject: </font><font size=1>Re: [gpfsug-discuss]
GPFS Independent Fileset Limit</font><font size=1 color=#5f5f5f><br>Sent by: </font><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><font size=1 color=blue><u>gpfsug-discuss-bounces@spectrumscale.org</u></font></a><font size=3><br></font><hr noshade><font size=3><br></font><br><font size=3 face="Calibri">Dear Marc,</font><br><font size=3 face="Calibri">so at least your documentation says:</font><br><a href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/bl1hlp_filesfilesets.htm"><font size=3 color=blue face="Calibri"><u>https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.1/com.ibm.spectrum.scale.v5r01.doc/bl1hlp_filesfilesets.htm</u></font></a><br><font size=3 color=#2f2f2f face="Arial">>>> User group and
user quotas can be tracked at the file system level or per independent
fileset.</font><br><font size=3 face="Calibri">But obviously as a customer I don't know
if that "Really" depends on independence.</font><br><font size=3 face="Calibri">Currently about 70% of our filesets in
the Data Science Storage systems get backed up to ISP. But that number
may change over time as it depends on the requirements of our projects.
For them it is just selecting "Protect this DSS Container by ISP"
in a Web form an our portal then automatically does all the provisioning
of the ISP Node to one of our ISP servers, rolling out the new dsm config
files to the backup workers and so on.</font><br><font size=3 face="Calibri"><br><br>Best Regards, <br>Stephan Peinkofer</font><font size=3><br></font><hr><font size=2 face="Calibri"><b><br>From:</b> </font><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><font size=2 color=blue face="Calibri"><u>gpfsug-discuss-bounces@spectrumscale.org</u></font></a><font size=2 face="Calibri"><</font><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><font size=2 color=blue face="Calibri"><u>gpfsug-discuss-bounces@spectrumscale.org</u></font></a><font size=2 face="Calibri">>
on behalf of Marc A Kaplan <</font><a href="mailto:makaplan@us.ibm.com"><font size=2 color=blue face="Calibri"><u>makaplan@us.ibm.com</u></font></a><font size=2 face="Calibri">><b><br>Sent:</b> Friday, August 10, 2018 7:15 PM<b><br>To:</b> gpfsug main discussion list<b><br>Subject:</b> Re: [gpfsug-discuss] GPFS Independent Fileset Limit</font><font size=3 face="Calibri"><br> </font><font size=2><br>I know quota stuff was cooked into GPFS before we even had "independent
filesets"...<br>So which particular quota features or commands or options now depend on
"independence"?! Really?<br><br>Yes, independent fileset performance for mmapplypolicy and mmbackup scales
with the inodespace sizes. But I'm curious to know how many of those indy
filesets are mmback-ed-up.<br><br>Appreciate your elaborations, 'cause even though I've worked on some of
this code, I don't know how/when/if customers push which limits.<br><br>---------------------</font><font size=3> </font><br><font size=3 face="Calibri">Dear Marc,</font><br><font size=3 face="Calibri">well the primary reasons for us are:</font><br><font size=3 face="Calibri">- Per fileset quota (this seems to work
also for dependent filesets as far as I know) </font><br><font size=3 face="Calibri">- Per user per fileset quota (this seems
only to work for independent filesets)</font><br><font size=3 face="Calibri">- The dedicated inode space to speedup
mmpolicy runs which only have to be applied to a specific subpart of the
file system</font><br><font size=3 face="Calibri">- Scaling mmbackup by backing up different
filesets to different TSM Servers economically</font><br><font size=3 face="Calibri">We have currently more than 1000 projects
on our HPC machines and several different existing and planned file systems
(use cases):</font><br><font size=3 face="Calibri"><br></font><tt><font size=2><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font></tt><a href="http://spectrumscale.org"><tt><font size=2 color=blue><u>spectrumscale.org</u></font></tt></a><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></tt></a><font size=3><br><br><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font><a href="http://spectrumscale.org"><font size=3 color=blue><u>spectrumscale.org</u></font></a><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><font size=3 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></a><br><tt><font size=2>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br><br><BR>