<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" >I would say the snapshot is not a light process. on heavy I/O and hundreds of nodes is pretty hard actually.</div>
<div dir="ltr" > </div>
<div dir="ltr" >the quiescence needed to do the actual snapshot is not to be taken lightly on such conditions.</div>
<div dir="ltr" > </div>
<div dir="ltr" >Furthermore the deletion of snapshots can be really fun too when it comes to how heavy it is.</div>
<div dir="ltr" > </div>
<div dir="ltr" >For the rest of the points I wont argue, and is some cases this product might be the best way to go, just wanted to jump in about the snapshot part.</div>
<div dir="ltr" ><br>--<br>Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations<br><br>Luis Bolinches<br>Lab Services<br>http://www-03.ibm.com/systems/services/labservices/<br><br>IBM Laajalahdentie 23 (main Entrance) Helsinki, 00330 Finland<br>Phone: +358 503112585<br><br>"If you continually give you will continually have." Anonymous</div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: Jez Tucker <jtucker@pixitmedia.com><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: Re: [gpfsug-discuss] Tracking deleted files<br>Date: Mon, Feb 27, 2017 3:12 PM<br>
<div><div>Hi</div></div>
<div> </div>
<div> Whilst it does use snapshots, I'd argue that snapshot creation is pretty lightweight - and always consistent. </div>
<div> </div>
<div>Your alternative via the mmbackup 'tracking' route is to parse out the mmbackup shadow file. AFAIK to do this /properly in a timely fashion/ you'd need to do this as an inline post process after the scan phase of mmbackup has run, else you're instead looking at the outdated view of the shadow file post previous mmbackup run. </div>
<div> </div>
<div>mmbackup does not 'track' file changes, it performs a comparison pass between the filesystem contents and what TSM _believes_ is the known state of the file system during each run. If a change is made oob of TSM then you need to re-generate the show file to regain total consistency.</div>
<div> </div>
<div>Sensibly you should be running any mmbackup process from a snapshot to perform consistent backups without dsmc errors. </div>
<div> </div>
<div>So all things being equal, using snapshots for exact consistency and not having to regenerate (very heavyweight) or parse out a shadow file periodically is a lighter weight, smoother and reliably consistent workflow. </div>
<div> </div>
<div>YMMV with either approach depending on your management of TSM and your interpretation of 'consistent view' vs 'good enough'. </div>
<div> </div>
<div>Jez</div>
<div> </div>
<div><div><div>On Mon, 27 Feb 2017 at 12:39, Simon Thompson (Research Computing - IT Services) <<a href="mailto:S.J.Thompson@bham.ac.uk" target="_blank" >S.J.Thompson@bham.ac.uk</a>> wrote:</div>
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" ><div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif" ><div>Yeah but that uses snapshots, which is pretty heavy-weight for what I want to do, particularly given mmbackup seems to have a way of tracking deletes...</div>
<div> </div>
<div>Simon</div>
<div> </div>
<div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt" ><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" ><span style="font-weight:bold" >From: </span><<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank" >gpfsug-discuss-bounces@spectrumscale.org</a>> on behalf of Jez Tucker <<a href="mailto:jtucker@pixitmedia.com" target="_blank" >jtucker@pixitmedia.com</a>><br><span style="font-weight:bold" >Reply-To: </span>"<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" >gpfsug-discuss@spectrumscale.org</a>" <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" >gpfsug-discuss@spectrumscale.org</a>><br><span style="font-weight:bold" >Date: </span>Monday, 27 February 2017 at 11:59<br><span style="font-weight:bold" >To: </span>"<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" >gpfsug-discuss@spectrumscale.org</a>" <<a href="mailto:gpfsug-discuss@spectrumscale.org" target="_blank" >gpfsug-discuss@spectrumscale.org</a>><br><span style="font-weight:bold" >Subject: </span>Re: [gpfsug-discuss] Tracking deleted files</span></div>
<div> </div>
<div><div> </div></div></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif" ><div><div><div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >Hi Simon</span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" > I presented exactly this (albeit briefly) at the 2016 UG. </span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >See the snapdiff section of the presentation at:</span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" ><a href="http://files.gpfsug.org/presentations/2016/south-bank/ArcaPix_GPFS_Spectrum_Scale_Python_API_final_17052016.pdf" target="_blank" >http://files.gpfsug.org/presentations/2016/south-bank/ArcaPix_GPFS_Spectrum_Scale_Python_API_final_17052016.pdf</a></span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >We can track creations, modifications, deletions and moves (from, to) for files and directories between one point in time and another. </span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >The selections can be returned via a manner of your choice. </span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >If anyone wants to know more, hit me up directly. </span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >Incidentally - I will be at BVE this week (<a href="http://www.bvexpo.com" target="_blank" >http://www.bvexpo.com</a>/) showing new things driven by the Python API and GPFS - so if anyone is in the area and wants to chat about technicals in person rather than on mail, drop me a line and we can sort that out. </span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >Best,</span></div>
<div> </div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >Jez</span></div>
<div> </div>
<div>
<div><div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >On Mon, 27 Feb 2017 at 11:30, Simon Thompson (Research Computing - IT Services) <<a href="mailto:S.J.Thompson@bham.ac.uk" target="_blank" >S.J.Thompson@bham.ac.uk</a>> wrote:</span></div>
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" ><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >Hi,<br><br>Is there a way to track files which have been deleted easily? I'm assuming<br>that we can't easily use a policy scan as they files are no longer in the<br>file-system unless we do some sort of diff?<br><br>I'm assuming there must be a way of doing this as mmbackup must track<br>deleted files to notify TSM of expired objects.<br><br>Basically I want a list of new files, changed files and deleted files<br>since a certain time. I'm assuming the first two will be relatively simple<br>with a policyscan, but the latter I'm not sure about.<br><br>Thanks<br><br>Simon<br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank" > spectrumscale.org</a><br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank" >http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></span></blockquote></div></div>
<div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" >-- </span></div>
<div data-smartmail="gmail_signature" ><div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt1.png" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt2.png" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt3.png" style="font-size:12.8px" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt4.png" ></span></div></div></div></div></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif" ><div><div><div><span id="m_3801951628406490869OLK_SRC_BODY_SECTION" ><a href="http://pixitmedia.com" style="font-size:1.3em;font-family:Arial,Helvetica,sans-serif" tabindex="-1" target="_blank" ><img s_org_src="http://pixitmedia.com/sig/BVE-Banner4.png" ></a><br><font size="1" face="Arial,Helvetica,sans-serif" >This email is confidential in that it is intended for the exclusive attention of the addressee(s) indicated. If you are not the intended recipient, this email should not be read or disclosed to any other person. Please notify the sender immediately and delete this email from your computer system. Any opinions expressed are not necessarily those of the company from which this email was sent and, whilst to the best of our knowledge no viruses or defects exist, no responsibility can be accepted for any loss or damage arising from its receipt or subsequent use of this email.</font></span></div></div></div></div>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank" >spectrumscale.org</a><br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank" >http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></blockquote></div></div>
<div dir="ltr" >--</div>
<div data-smartmail="gmail_signature" ><div dir="ltr" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt1.png" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt2.png" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt3.png" style="font-size:12.8px" ><img s_org_src="http://www.pixitmedia.com/sig/pxone_pt4.png" ></div></div>
<div><a href="http://pixitmedia.com" style="font-size:1.3em;font-family:Arial,Helvetica,sans-serif" tabindex="-1" target="_blank" ><img s_org_src="http://pixitmedia.com/sig/BVE-Banner4.png" ></a><br><font size="1" face="Arial, Helvetica, sans-serif" >This email is confidential in that it is intended for the exclusive attention of the addressee(s) indicated. If you are not the intended recipient, this email should not be read or disclosed to any other person. Please notify the sender immediately and delete this email from your computer system. Any opinions expressed are not necessarily those of the company from which this email was sent and, whilst to the best of our knowledge no viruses or defects exist, no responsibility can be accepted for any loss or damage arising from its receipt or subsequent use of this email.</font></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank" >http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font></div></blockquote>
<div dir="ltr" > </div></div><BR>
Ellei edellä ole toisin mainittu: / Unless stated otherwise above:<BR>
Oy IBM Finland Ab<BR>
PL 265, 00101 Helsinki, Finland<BR>
Business ID, Y-tunnus: 0195876-3 <BR>
Registered in Finland<BR>
<BR>