<html><body><p><font size="2">Deleting a file is a two stage process. The original user thread unlinks the file from the directory and reduces the link count. If the count is zero and the file is not open, then it gets queued for the background deletion thread. The background thread then deletes the blocks and frees the space. If there is a snapshot, the data blocks may be captured and not actually freed. After a crash, the recovery code looks for files that were being deleted and restarts the deletion if necessary.</font><br><br><font size="2">-Wayne</font><br><br><br><tt><font size="2">gpfsug-discuss-bounces@spectrumscale.org wrote on 02/18/2020 06:05:41 AM:<br><br>> From: TURNER Aaron <aaron.turner@ed.ac.uk></font></tt><br><tt><font size="2">> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></font></tt><br><tt><font size="2">> Date: 02/18/2020 06:05 AM</font></tt><br><tt><font size="2">> Subject: [EXTERNAL] Re: [gpfsug-discuss] Odd behaviour with regards <br>> to reported free space</font></tt><br><tt><font size="2">> Sent by: gpfsug-discuss-bounces@spectrumscale.org</font></tt><br><tt><font size="2">> <br>> Dear Jonathan,<br>> <br>> This is what I had assumed was the case. Since the system ended up <br>> with an enforced reboot before we had time for further investigation<br>> I wasn't able to confirm this.<br>> <br>> > I can be very confusing for end users, especially when what is <br>> holding onto the file is some random zombie process on another node <br>> that died last month.<br>> <br>> Yes, that's very likely to have been the case.<br>> <br>> Regards<br>> <br>> Aaron Turner<br>> <br>> -----Original Message-----<br>> From: gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-<br>> bounces@spectrumscale.org> On Behalf Of Jonathan Buzzard<br>> Sent: 18 February 2020 10:50<br>> To: gpfsug-discuss@spectrumscale.org<br>> Subject: Re: [gpfsug-discuss] Odd behaviour with regards to reportedfree space<br>> <br>> On Tue, 2020-02-18 at 09:28 +0000, TURNER Aaron wrote:<br>> > Dear All,<br>> ><br>> > This has happened more than once with both 4.2.3 and 5.0. The<br>> > instances may not be related.<br>> ><br>> > In the first instance, usage was high (over 90%) and so users were<br>> > encouraged to delete files. One user deleted a considerable number of<br>> > files equal to around 10% of the total storage. Reported usage did not<br>> > fall. There were not obviously any waiters. Has anyone seen anything<br>> > similar?<br>> ><br>> <br>> I have seen similar behaviour a number of times.<br>> <br>> I my experience it is because a process somewhere has an open file <br>> handle on one or more files/directories. So you can delete the file <br>> and it goes from a directory listing; it's no long visible when you do ls.<br>> <br>> However the file has not actually gone, and will continue to count <br>> towards total file system usage, user/group/fileset quota's etc.<br>> <br>> Once the errant process is found and killed magically the space becomes free.<br>> <br>> I can be very confusing for end users, especially when what is <br>> holding onto the file is some random zombie process on another node <br>> that died last month.<br>> <br>> <br>> JAB.<br>> <br>> --<br>> Jonathan A. Buzzard Tel: +44141-5483420<br>> HPC System Administrator, ARCHIE-WeSt.<br>> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG <br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> <a href="INVALID URI REMOVED">INVALID URI REMOVED</a><br>> u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-<br>> siA1ZOg&r=GtPIT10cORUM6qwFnTVtIiDUFmESkxW3I0wu8GDxmgc&m=QkF9KAzl1dxqONkEkh7ZLNsDYktsFHJCkI2oGi6qyHk&s=_Z-<br>> E_VtMDAiXmR8oSZym4G9OIzxRhcs5rJxMEjxK1RI&e= <br>> The University of Edinburgh is a charitable body, registered in <br>> Scotland, with registration number SC005336.<br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> <a href="INVALID URI REMOVED">INVALID URI REMOVED</a><br>> u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-<br>> siA1ZOg&r=GtPIT10cORUM6qwFnTVtIiDUFmESkxW3I0wu8GDxmgc&m=QkF9KAzl1dxqONkEkh7ZLNsDYktsFHJCkI2oGi6qyHk&s=_Z-<br>> E_VtMDAiXmR8oSZym4G9OIzxRhcs5rJxMEjxK1RI&e= <br>> <br></font></tt><BR>
</body></html>