[gpfsug-discuss] Quick delete of huge tree

Alec anacreo at gmail.com
Tue Apr 20 17:44:53 BST 2021


I would start with the mv to hide it, and then allow the delete to progress
in the background.  I would seperate out the delete of files from
directories... And i would try using mmxargs with a rm command to get
parallelism plus reduce the number of execs in one policy, followed by a
simple rm -r of the directory tree.

Maybe it's cheaper to make a new filesystem and just retain the data you
want though...

Alec

On Tue, Apr 20, 2021, 6:51 AM Jonathan Buzzard <
jonathan.buzzard at strath.ac.uk> wrote:

> On 20/04/2021 13:09, Ulrich Sibiller wrote:
>
> >>
> >> Consider using mv to move it out the way or hide it while the delete is
> >> in progress. If you do that think carefully about backups, you don't
> >> want to back it all up again while it is being deleted :-)
> >
> > ;-) Yeah, that's why I did not the do the mv in the first place ;-)
> >
>
> I would estimate (based on my experience) is that you should be able to
> delete that amount of data/files in under 24 hours anyway with a simple
> rm -rf. Which is why I question trying to find faster methods. You have
> already wasted a significant amount of that time :-)
>
> If your using TSM for the backup then just exclude it from the backup in
> your dsm.opts file
>
> exclude.dir  <path to directory>
>
> We have a NOBACK option that allows users to select if they don't want
> something backing up. Helpful if your job generates lots of temporary
> files or data that are junked as soon as the job finishes. Anything
> under a directory called NOBACK does not get backed up.
>
> exclude.dir  /.../NOBACK/
>
>
> JAB.
>
> --
> Jonathan A. Buzzard                         Tel: +44141-5483420
> HPC System Administrator, ARCHIE-WeSt.
> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210420/86cec6ae/attachment-0002.htm>


More information about the gpfsug-discuss mailing list