<span style=" font-size:10pt;font-family:sans-serif">My idea, not completely
thought out, is that before you hit the 1000 limit, you start putting new
customers or projects into dependent filesets and define those new dependent
filesets within either a lesser number of independent filesets expressly
created to receive the new customers OR perhaps even lump them with already
existing independent filesets that have matching backup requirements.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">I would NOT try
to create backups for each dependent fileset. But stick with the
supported facilities to manage backup per independent...</span><br><br><span style=" font-size:10pt;font-family:sans-serif">Having said that,
if you'd still like to do backup per dependent fileset -- then have at
it -- but test, test and retest.... And measure performance...</span><br><span style=" font-size:10pt;font-family:sans-serif">My GUESS is that
IF you can hack mmbackup or similar to use mmapplypolicy /path-to-dependent-fileset
--scope fileset .... </span><br><span style=" font-size:10pt;font-family:sans-serif">instead of mmapplypolicy
/path-to-independent-fileset --scope inodespace ....</span><br><br><span style=" font-size:10pt;font-family:sans-serif">You'll be okay
because the inodescan where you end up reading some extra inodes is probably
a tiny fraction of all the other IO you'll be doing! </span><br><br><span style=" font-size:10pt;font-family:sans-serif">BUT I don't think
IBM is in a position to encourage you to hack mmbackup -- it's already
very complicated!<br></span><br><br><br><br><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:
</span><span style=" font-size:9pt;font-family:sans-serif">"Peinkofer,
Stephan" <Stephan.Peinkofer@lrz.de></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:
</span><span style=" font-size:9pt;font-family:sans-serif">gpfsug
main discussion list <gpfsug-discuss@spectrumscale.org></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:
</span><span style=" font-size:9pt;font-family:sans-serif">08/17/2018
07:40 AM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:
</span><span style=" font-size:9pt;font-family:sans-serif">Re:
[gpfsug-discuss] GPFS Independent Fileset Limit vs Quotas?</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Sent
by: </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><br><hr noshade><br><br><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt;font-family:Calibri">Dear
Marc,</span></p><p style="margin-top:0px;margin-Bottom:0px"></p><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt;font-family:Calibri">well
as I think I cannot simply "move" dependent filesets between
independent ones and our customers must have the opportunity to change
data protection policy for their Containers at any given time, I cannot
map them to a "backed up" or "not backed up" independent
fileset.</span></p><p style="margin-top:0px;margin-Bottom:0px"></p><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt;font-family:Calibri">So
how much performance impact is lets say 1-10 exclude.dir directives per
independent fileset?</span></p><p style="margin-top:0px;margin-Bottom:0px"></p><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt;font-family:Calibri">Many
thanks in advance.</span></p><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt;font-family:Calibri">Best
Regards,</span></p><p style="margin-top:0px;margin-Bottom:0px"><span style=" font-size:12pt;font-family:Calibri">Stephan
Peinkofer</span></p><br><span style=" font-size:12pt;font-family:Calibri"><br></span><br><hr><br><span style=" font-size:11pt;font-family:Calibri"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org
<gpfsug-discuss-bounces@spectrumscale.org> on behalf of Marc A Kaplan
<makaplan@us.ibm.com><b><br>Sent:</b> Tuesday, August 14, 2018 5:31 PM<b><br>To:</b> gpfsug main discussion list<b><br>Subject:</b> Re: [gpfsug-discuss] GPFS Independent Fileset Limit vs Quotas?</span><span style=" font-size:12pt;font-family:Calibri"></span><br><span style=" font-size:12pt;font-family:Calibri"> </span><br><span style=" font-size:10pt;font-family:sans-serif">True, mmbackup
is designed to work best backing up either a single independent fileset
or the entire file system. So if you know some filesets do not need
to be backed up, map them to one or more indepedent filesets that will
not be backed up. </span><span style=" font-size:12pt;font-family:Calibri"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>mmapplypolicy is happy to scan a single dependent fileset, use option --scope
fileset and make the primary argument the path to the root of the fileset
you wish to scan. The overhead is not simply described. The
directory scan phase will explore or walk the (sub)tree in parallel with
multiple threads on multiple nodes, reading just the directory blocks that
need to be read.</span><span style=" font-size:12pt;font-family:Calibri"><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>The inodescan phase will read blocks of inodes from the given inodespace
... since the inodes of dependent filesets may be "mixed"
into the same blocks as other dependend filesets that are in the same independent
fileset, mmapplypolicy will incur what you might consider "extra"
overhead.</span><span style=" font-size:12pt;font-family:Calibri"><br><br><br><br></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>From: </span><span style=" font-size:9pt;font-family:sans-serif">"Peinkofer,
Stephan" <Stephan.Peinkofer@lrz.de></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>To: </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug
main discussion list <gpfsug-discuss@spectrumscale.org></span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Date: </span><span style=" font-size:9pt;font-family:sans-serif">08/14/2018
12:50 AM</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Subject: </span><span style=" font-size:9pt;font-family:sans-serif">Re:
[gpfsug-discuss] GPFS Independent Fileset Limit vs Quotas?</span><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif"><br>Sent by: </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><span style=" font-size:12pt;font-family:Calibri"><br></span><hr noshade><span style=" font-size:12pt;font-family:Calibri"><br><br><br>Dear Marc,<br><br></span><span style=" font-size:10pt;font-family:sans-serif"><br>If you "must" exceed 1000 filesets because you are assigning
each project to its own fileset, my suggestion is this:<br><br>Yes, there are scaling/performance/manageability benefits to using mmbackup
over independent filesets.<br><br>But maybe you don't need 10,000 independent filesets -- <br>maybe you can hash or otherwise randomly assign projects that each have
their own (dependent) fileset name to a lesser number of independent filesets
that will serve as management groups for (mm)backup...</span><span style=" font-size:12pt;font-family:Calibri"><br><br>OK, if that might be doable, whats then the performance impact of having
to specify Include/Exclude lists for each independent fileset in order
to specify which dependent fileset should be backed up and which one not?<br>I don’t remember exactly, but I think I’ve heard at some time, that Include/Exclude
and mmbackup have to be used with caution. And the same question holds
true for running mmapplypolicy for a “job” on a single dependent fileset?
Is the scan runtime linear to the size of the underlying independent fileset
or are there some optimisations when I just want to scan a subfolder/dependent
fileset of an independent one?<br></span><span style=" font-size:10pt;font-family:sans-serif"><br>Like many things in life, sometimes compromises are necessary!</span><span style=" font-size:12pt;font-family:Calibri"><br><br>Hmm, can I reference this next time, when we negotiate Scale License pricing
with the ISS sales people? ;)<br><br>Best Regards,<br>Stephan Peinkofer</span><span style=" font-size:10pt;font-family:Calibri"><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org</span><span style=" font-size:12pt;color:blue;font-family:Calibri"><u><br></u></span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><span style=" font-size:10pt;color:blue;font-family:Calibri"><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></span></a><span style=" font-size:12pt;font-family:Calibri"><br><br><br></span><tt><span style=" font-size:10pt">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></span></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style=" font-size:10pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt><span style=" font-size:10pt"><br></span></tt><br><br><BR>