[gpfsug-discuss] Quick delete of huge tree
u.sibiller at science-computing.de
Tue Apr 20 12:18:15 BST 2021
I have to delete a subtree of about ~50 million files in thousands of subdirs, ~14TB of data.
Running a recursive rm is very slow so I setup a simple policy file:
RULE 'delstuff' DELETE
WHERE PATH_NAME LIKE '/mypath/%'
This kinda works but is not really fast, either. It even requires a second run because files and
directories within the tree will be processed in arbitrary order so it will happen quite frequently
that a directory is going to be deleted before its content has been removed completely. For those
dirs I see an error message and have to delete afterwards.
I am wondering if there's a quicker way. Given the fact that this is a whole tree I think there's
should be a quick way to unlink the complete inode hierachy.
Unfortunately we are not using a fileset for that tree...
So are there any ideas how to solve that more efficiently?
Science + Computing AG
Vorstandsvorsitzender/Chairman of the board of management:
Dr. Martin Matzke
Vorstand/Board of Management:
Matthias Schempp, Sabine Hohenstein
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Martin Wibbe, Ursula Morgenstern
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196
More information about the gpfsug-discuss