<div><div>Hi</div></div><div><br></div><div> Whilst it does use snapshots, I'd argue that snapshot creation is pretty lightweight - and always consistent. </div><div><br></div><div>Your alternative via the mmbackup 'tracking' route is to parse out the mmbackup shadow file. AFAIK to do this /properly in a timely fashion/ you'd need to do this as an inline post process after the scan phase of mmbackup has run, else you're instead looking at the outdated view of the shadow file post previous mmbackup run. </div><div><br></div><div>mmbackup does not 'track' file changes, it performs a comparison pass between the filesystem contents and what TSM _believes_ is the known state of the file system during each run. If a change is made oob of TSM then you need to re-generate the show file to regain total consistency.</div><div><br></div><div>Sensibly you should be running any mmbackup process from a snapshot to perform consistent backups without dsmc errors. </div><div><br></div><div>So all things being equal, using snapshots for exact consistency and not having to regenerate (very heavyweight) or parse out a shadow file periodically is a lighter weight, smoother and reliably consistent workflow. </div><div><br></div><div>YMMV with either approach depending on your management of TSM and your interpretation of 'consistent view' vs 'good enough'. </div><div><br></div><div>Jez</div><div><br></div><div><div class="gmail_quote"><div>On Mon, 27 Feb 2017 at 12:39, Simon Thompson (Research Computing - IT Services) <<a href="mailto:S.J.Thompson@bham.ac.uk">S.J.Thompson@bham.ac.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif" class="gmail_msg">
<div class="gmail_msg">Yeah but that uses snapshots, which is pretty heavy-weight for what I want to do, particularly given mmbackup seems to have a way of tracking deletes...</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">Simon</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<span id="m_3801951628406490869OLK_SRC_BODY_SECTION" class="gmail_msg">
<div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt" class="gmail_msg">
<span style="font-weight:bold" class="gmail_msg">From: </span><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="gmail_msg" target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a>> on behalf of Jez Tucker <<a href="mailto:jtucker@pixitmedia.com" class="gmail_msg" target="_blank">jtucker@pixitmedia.com</a>><br class="gmail_msg">
<span style="font-weight:bold" class="gmail_msg">Reply-To: </span>"<a href="mailto:gpfsug-discuss@spectrumscale.org" class="gmail_msg" target="_blank">gpfsug-discuss@spectrumscale.org</a>" <<a href="mailto:gpfsug-discuss@spectrumscale.org" class="gmail_msg" target="_blank">gpfsug-discuss@spectrumscale.org</a>><br class="gmail_msg">
<span style="font-weight:bold" class="gmail_msg">Date: </span>Monday, 27 February 2017 at 11:59<br class="gmail_msg">
<span style="font-weight:bold" class="gmail_msg">To: </span>"<a href="mailto:gpfsug-discuss@spectrumscale.org" class="gmail_msg" target="_blank">gpfsug-discuss@spectrumscale.org</a>" <<a href="mailto:gpfsug-discuss@spectrumscale.org" class="gmail_msg" target="_blank">gpfsug-discuss@spectrumscale.org</a>><br class="gmail_msg">
<span style="font-weight:bold" class="gmail_msg">Subject: </span>Re: [gpfsug-discuss] Tracking deleted files<br class="gmail_msg">
</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">
<div class="gmail_msg"></div></div></span></div><div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif" class="gmail_msg"><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" class="gmail_msg"><div class="gmail_msg"><div class="gmail_msg">
<div class="gmail_msg">Hi Simon</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg"> I presented exactly this (albeit briefly) at the 2016 UG. </div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">See the snapdiff section of the presentation at:</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg"><a href="http://files.gpfsug.org/presentations/2016/south-bank/ArcaPix_GPFS_Spectrum_Scale_Python_API_final_17052016.pdf" class="gmail_msg" target="_blank">http://files.gpfsug.org/presentations/2016/south-bank/ArcaPix_GPFS_Spectrum_Scale_Python_API_final_17052016.pdf</a></div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">We can track creations, modifications, deletions and moves (from, to) for files and directories between one point in time and another. </div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">The selections can be returned via a manner of your choice. </div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">If anyone wants to know more, hit me up directly. </div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">Incidentally - I will be at BVE this week (<a href="http://www.bvexpo.com/" class="gmail_msg" target="_blank">http://www.bvexpo.com/</a>) showing new things driven by the Python API and GPFS - so if anyone is in the area and wants to chat about technicals in person rather than on mail,
drop me a line and we can sort that out. </div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">Best,</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg">Jez</div>
<div class="gmail_msg"><br class="gmail_msg">
</div>
<div class="gmail_msg"><br class="gmail_msg">
<div class="gmail_quote gmail_msg">
<div class="gmail_msg">On Mon, 27 Feb 2017 at 11:30, Simon Thompson (Research Computing - IT Services) <<a href="mailto:S.J.Thompson@bham.ac.uk" class="gmail_msg" target="_blank">S.J.Thompson@bham.ac.uk</a>> wrote:<br class="gmail_msg">
</div>
<blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br class="gmail_msg">
<br class="gmail_msg">
Is there a way to track files which have been deleted easily? I'm assuming<br class="gmail_msg">
that we can't easily use a policy scan as they files are no longer in the<br class="gmail_msg">
file-system unless we do some sort of diff?<br class="gmail_msg">
<br class="gmail_msg">
I'm assuming there must be a way of doing this as mmbackup must track<br class="gmail_msg">
deleted files to notify TSM of expired objects.<br class="gmail_msg">
<br class="gmail_msg">
Basically I want a list of new files, changed files and deleted files<br class="gmail_msg">
since a certain time. I'm assuming the first two will be relatively simple<br class="gmail_msg">
with a policyscan, but the latter I'm not sure about.<br class="gmail_msg">
<br class="gmail_msg">
Thanks<br class="gmail_msg">
<br class="gmail_msg">
Simon<br class="gmail_msg">
<br class="gmail_msg">
_______________________________________________<br class="gmail_msg">
gpfsug-discuss mailing list<br class="gmail_msg">
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" class="gmail_msg" target="_blank">
spectrumscale.org</a><br class="gmail_msg">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" class="gmail_msg" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="gmail_msg">
</blockquote>
</div>
</div>
<div class="gmail_msg">-- <br class="gmail_msg">
</div>
<div data-smartmail="gmail_signature" class="gmail_msg">
<div class="gmail_msg"><img src="http://www.pixitmedia.com/sig/pxone_pt1.png" class="gmail_msg"><img src="http://www.pixitmedia.com/sig/pxone_pt2.png" class="gmail_msg"><img src="http://www.pixitmedia.com/sig/pxone_pt3.png" style="font-size:12.8px" class="gmail_msg"><img src="http://www.pixitmedia.com/sig/pxone_pt4.png" class="gmail_msg"></div>
</div>
<br class="gmail_msg">
</div></div></span></div><div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif" class="gmail_msg"><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" class="gmail_msg"><div class="gmail_msg"><div class="gmail_msg"><div class="gmail_msg"><a href="http://pixitmedia.com" style="font-size:1.3em;font-family:Arial,Helvetica,sans-serif" class="gmail_msg" target="_blank"><img src="http://pixitmedia.com/sig/BVE-Banner4.png" class="gmail_msg"></a><br class="gmail_msg">
<font face="Arial,Helvetica,sans-serif" size="1" class="gmail_msg">This email is confidential in that it is intended for the exclusive attention of the addressee(s) indicated. If you are not the intended recipient, this email should not be read or disclosed to any other person.
Please notify the sender immediately and delete this email from your computer system. Any opinions expressed are not necessarily those of the company from which this email was sent and, whilst to the best of our knowledge no viruses or defects exist, no responsibility
can be accepted for any loss or damage arising from its receipt or subsequent use of this email.</font></div>
</div>
</div>
</span>
</div>
_______________________________________________<br class="gmail_msg">
gpfsug-discuss mailing list<br class="gmail_msg">
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" class="gmail_msg" target="_blank">spectrumscale.org</a><br class="gmail_msg">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" class="gmail_msg" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="gmail_msg">
</blockquote></div></div><div dir="ltr">-- <br></div><div data-smartmail="gmail_signature"><div dir="ltr"><img src="http://www.pixitmedia.com/sig/pxone_pt1.png"><img src="http://www.pixitmedia.com/sig/pxone_pt2.png"><img src="http://www.pixitmedia.com/sig/pxone_pt3.png" style="font-size:12.8px"><img src="http://www.pixitmedia.com/sig/pxone_pt4.png"></div></div>
<br>
<div><a href="http://pixitmedia.com" style="font-size:1.3em;font-family:Arial,Helvetica,sans-serif" target="_blank"><img src="http://pixitmedia.com/sig/BVE-Banner4.png"></a><br><font face="Arial, Helvetica, sans-serif" size="1">This email is confidential in that it is intended for the exclusive attention of the addressee(s) indicated. If you are not the intended recipient, this email should not be read or disclosed to any other person. Please notify the sender immediately and delete this email from your computer system. Any opinions expressed are not necessarily those of the company from which this email was sent and, whilst to the best of our knowledge no viruses or defects exist, no responsibility can be accepted for any loss or damage arising from its receipt or subsequent use of this email.</font></div>