<font size=2 face="sans-serif">HI Jan, </font><br><font size=2 face="sans-serif">yes.. but we should highlight, that
this means.. an extra / additionally copy on writes/ changes to a block
... so it adds a bit latency , when running in this mode </font><br><br><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Jan-Frode Myklebust
<janfrode@tanso.net></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">01/29/2019 08:19 PM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
Querying size of snapshots</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><font size=3>You could put snapshot data in a separate storage pool.
Then it should be visible how much space it occupies, but it’s a bit hard
to see how this will be usable/manageable..<br><br><br>-jf</font><br><font size=3>tir. 29. jan. 2019 kl. 20:08 skrev Christopher Black <</font><a href="mailto:cblack@nygenome.org"><font size=3 color=blue><u>cblack@nygenome.org</u></font></a><font size=3>>:</font><br><font size=3>Thanks for the quick and detailed reply! I had read the
manual and was aware of the warnings about -d (mentioned in my PS).</font><p><font size=3>On systems with high churn (lots of temporary files, lots
of big and small deletes along with many new files), I’ve previously used
estimates of snapshot size as a useful signal on whether we can expect
to see an increase in available space over the next few days as snapshots
expire. I’ve used this technique on a few different more mainstream storage
systems, but never on gpfs.</font><p><font size=3>I’d find it useful to have a similar way to monitor “space
to be freed pending snapshot deletes” on gpfs. It sounds like there is
not an existing solution for this so it would be a request for enhancement.</font><p><font size=3>I’m not sure how much overhead there would be keeping
a running counter for blocks changed since snapshot creation or if that
would completely fall apart on large systems or systems with many snapshots.
If that is a consideration even having only an estimate for the oldest
snapshot would be useful, but I realize that can depend on all the other
later snapshots as well. Perhaps an overall “size of all snapshots” would
be easier to manage and would still be useful to us.</font><p><font size=3>I don’t need this number to be 100% accurate, but a low
or floor estimate would be very useful.</font><p><font size=3> </font><p><font size=3>Is anyone else interested in this? Do other people have
other ways to estimate how much space they will get back as snapshots expire?
Is there a more efficient way of making such an estimate available to admins
other than running an mmlssnapshot -d every night and recording the output?</font><p><font size=3> </font><p><font size=3>Thanks all!</font><p><font size=3>Chris</font><p><font size=3> </font><p><font size=3><b>From: </b><</font><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank"><font size=3 color=blue><u>gpfsug-discuss-bounces@spectrumscale.org</u></font></a><font size=3>>
on behalf of Marc A Kaplan <</font><a href="mailto:makaplan@us.ibm.com" target="_blank"><font size=3 color=blue><u>makaplan@us.ibm.com</u></font></a><font size=3>><b><br>Reply-To: </b>gpfsug main discussion list <</font><a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank"><font size=3 color=blue><u>gpfsug-discuss@spectrumscale.org</u></font></a><font size=3>><b><br>Date: </b>Tuesday, January 29, 2019 at 1:24 PM<b><br>To: </b>gpfsug main discussion list <</font><a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank"><font size=3 color=blue><u>gpfsug-discuss@spectrumscale.org</u></font></a><font size=3>><b><br>Subject: </b>Re: [gpfsug-discuss] Querying size of snapshots</font><p><font size=3> </font><p><font size=2 face="Arial">1. First off, let's RTFM ...</font><font size=3><br><b><br>-d </b>Displays the amount of storage that is used by the snapshot.<br>This operation requires an amount of time that is proportional to the size
of the file system; therefore,<br>it can take several minutes or even hours on a large and heavily-loaded
file system.<br>This optional parameter can impact overall system performance. Avoid running
the <b>mmlssnapshot</b><br>command with this parameter frequently or during periods of high file system
activity.<br></font><font size=2 face="Arial"><br>SOOOO.. there's that. </font><font size=3><br></font><font size=2 face="Arial"><br>2. Next you may ask, HOW is that?</font><font size=3><br></font><font size=2 face="Arial"><br>Snapshots are maintained with a "COW" strategy -- They are created
quickly, essentially just making a record that the snapshot was created
and at such and such time -- when the snapshot is the same as the "live"
filesystem...</font><font size=3><br></font><font size=2 face="Arial"><br>Then over time, each change to a block of data in live system requires
that a copy is made of the old data block and that is associated with the
most recently created snapshot.... SO, as more and more changes
are made to different blocks over time the snapshot becomes bigger and
bigger. How big? Well it seems the current implementation does not
keep a "simple counter" of the number of blocks -- but rather,
a list of the blocks that were COW'ed.... So when you come and ask "How
big"... GPFS has to go traverse the file sytem metadata and count
those COW'ed blocks....</font><font size=3><br></font><font size=2 face="Arial"><br>3. So why not keep a counter? Well, it's likely not so simple. For
starters GPFS is typically running concurrently on several or many nodes...
And probably was not deemed worth the effort ..... IF a convincing case
could be made, I'd bet there is a way... to at least keep approximate numbers,
log records, exact updates periodically, etc, etc -- similar to the way
space allocation and accounting is done for the live file system...</font><font size=3><br><br></font><p><hr><br><font size=1 face="Arial">This message is for the recipient’s use
only, and may contain confidential, privileged or protected information.
Any unauthorized use or dissemination of this communication is prohibited.
If you received this message in error, please immediately notify the sender
and destroy all copies of this message. The recipient should check this
email and any attachments for the presence of viruses, as we accept no
liability for any damage caused by any virus transmitted by this email.</font><br><font size=3>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at </font><a href="http://spectrumscale.org" target="_blank"><font size=3 color=blue><u>spectrumscale.org</u></font></a><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank"><font size=3 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></a><tt><font size=2>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br><br><BR>