[gpfsug-discuss] Tracking deleted files

[Jez Tucker]

Hi

  Whilst it does use snapshots, I'd argue that snapshot creation is pretty
lightweight - and always consistent.

Your alternative via the mmbackup 'tracking' route is to parse out the
mmbackup shadow file.  AFAIK to do this /properly in a timely fashion/
you'd need to do this as an inline post process after the scan phase of
mmbackup has run, else you're instead looking at the outdated view of the
shadow file post previous mmbackup run.

mmbackup does not 'track' file changes, it performs a comparison pass
between the filesystem contents and what TSM _believes_ is the known state
of the file system during each run.  If a change is made oob of TSM then
you need to re-generate the show file to regain total consistency.

Sensibly you should be running any mmbackup process from a snapshot to
perform consistent backups without dsmc errors.

So all things being equal, using snapshots for exact consistency and not
having to regenerate (very heavyweight) or parse out a shadow file
periodically is a lighter weight, smoother and reliably consistent
workflow.

YMMV with either approach depending on your management of TSM and your
interpretation of 'consistent view' vs 'good enough'.

Jez

On Mon, 27 Feb 2017 at 12:39, Simon Thompson (Research Computing - IT
Services) <S.J.Thompson at bham.ac.uk> wrote:

> Yeah but that uses snapshots, which is pretty heavy-weight for what I want
> to do, particularly given mmbackup seems to have a way of tracking
> deletes...
>
> Simon
>
> From: <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Jez Tucker <
> jtucker at pixitmedia.com>
> Reply-To: "gpfsug-discuss at spectrumscale.org" <
> gpfsug-discuss at spectrumscale.org>
> Date: Monday, 27 February 2017 at 11:59
> To: "gpfsug-discuss at spectrumscale.org" <gpfsug-discuss at spectrumscale.org>
> Subject: Re: [gpfsug-discuss] Tracking deleted files
>
> Hi Simon
>
>   I presented exactly this (albeit briefly) at the 2016 UG.
>
> See the snapdiff section of the presentation at:
>
>
> http://files.gpfsug.org/presentations/2016/south-bank/ArcaPix_GPFS_Spectrum_Scale_Python_API_final_17052016.pdf
>
> We can track creations, modifications, deletions and moves (from, to) for
> files and directories between one point in time and another.
>
> The selections can be returned via a manner of your choice.
>
> If anyone wants to know more, hit me up directly.
>
> Incidentally - I will be at BVE this week (http://www.bvexpo.com/)
> showing new things driven by the Python API and GPFS - so if anyone is in
> the area and wants to chat about technicals in person rather than on mail,
> drop me a line and we can sort that out.
>
> Best,
>
> Jez
>
>
> On Mon, 27 Feb 2017 at 11:30, Simon Thompson (Research Computing - IT
> Services) <S.J.Thompson at bham.ac.uk> wrote:
>
> Hi,
>
> Is there a way to track files which have been deleted easily? I'm assuming
> that we can't easily use a policy scan as they files are no longer in the
> file-system unless we do some sort of diff?
>
> I'm assuming there must be a way of doing this as mmbackup must track
> deleted files to notify TSM of expired objects.
>
> Basically I want a list of new files, changed files and deleted files
> since a certain time. I'm assuming the first two will be relatively simple
> with a policyscan, but the latter I'm not sure about.
>
> Thanks
>
> Simon
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
> --
>
> <http://pixitmedia.com>
> This email is confidential in that it is intended for the exclusive
> attention of the addressee(s) indicated. If you are not the intended
> recipient, this email should not be read or disclosed to any other person.
> Please notify the sender immediately and delete this email from your
> computer system. Any opinions expressed are not necessarily those of the
> company from which this email was sent and, whilst to the best of our
> knowledge no viruses or defects exist, no responsibility can be accepted
> for any loss or damage arising from its receipt or subsequent use of this
> email.
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
--

-- 
 <http://pixitmedia.com>
This email is confidential in that it is intended for the exclusive 
attention of the addressee(s) indicated. If you are not the intended 
recipient, this email should not be read or disclosed to any other person. 
Please notify the sender immediately and delete this email from your 
computer system. Any opinions expressed are not necessarily those of the 
company from which this email was sent and, whilst to the best of our 
knowledge no viruses or defects exist, no responsibility can be accepted 
for any loss or damage arising from its receipt or subsequent use of this 
email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20170227/c057d557/attachment-0002.htm>