[gpfsug-discuss] List of deleted files via policies

[Alec]

I do something where I make sure all my files are in my backup server...

So I will pull the full index from the backup server...

And I will use a generated list of files on the file system...  Using a
GPFS policy...

And pulling indexes and comparing about 1.3 million files on both sides of
the diff takes about 3 minutes.  The sorts takes a bit of time but the
whole thing runs in 6 minutes as I run both sorts and pulls in parallel.

So I would recommend just generating the file listing using a mmfind or an
apply policy at time of snapshot and doing it again at a different time and
just running a diff command to find missing entries.

Depending on the amount of data you could probably combine the date and
inode into a single field and only sort those and compare then join the
filename using the inode..  depends on how big your filesystem index would
be.

Good luck on this one.  But it is very nice to know every morning which 16
files out of millions aren't in the backup because their filenames are too
crazy.. lol

Alec

On Tue, May 30, 2023, 9:57 AM Talamo Ivano Giuseppe <ivano.talamo at psi.ch>
wrote:

> Hello everyone,
>
> I am currently looking for the most efficient way to get a list of all
> deleted files since time T.
> So far the best I could come to is to create a snapshot at time T,
> generate two lists via policies, one for the files inside the .snapshots
> dir, one for the current filesystem, and compare the two with standard diff
> command.
> But I was wondering if is there any smarter way with policies only.
>
> Thanks,
> Ivano
>
> __________________________________________
> Paul Scherrer Institut
> Ivano Talamo
> WHGA/038
> Forschungsstrasse 111
> 5232 Villigen PSI
> Schweiz
>
> Phone: +41 56 310 47 11
> E-Mail: ivano.talamo at psi.ch
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230530/ad8a45f3/attachment.htm>