[gpfsug-discuss] Question about Policies

[Jonathan Buzzard]

On 30/12/2019 16:20, Marc A Kaplan wrote:
> Now apart from the mechanics of handling and manipulating pathnames ...
> 
> the idea to manage storage by "mv"ing instead of MIGRATEing (GPFS-wise) 
> may be ill-advised.
> 
> I suspect this is a hold-over or leftover from the old days -- when a 
> filesystem was comprised of just a few storage devices (disk drives) and 
> the only way available to manage space was to mv files to another 
> filesystem or archive to tape or whatnot..
> 

I suspect based on the OP is from (a cancer research institute which is 
basically life sciences) that this is an incorrect assumption.

I would guess this is about "archiving" results coming off experimental 
equipment. I use the term "archiving" in the same way that various email 
programs try and "archive" my old emails. That is to prevent the output 
directory of the equipment filling up with many thousands of files 
and/or directories I want to automate the placement in a directory 
hierarchy of old results.

Imagine a piece of equipment that does 50 different analysis's a day 
every working day. That's a 1000 a month or ~50,000 a year. It's about 
logically moving stuff to keep ones working directory manageable but 
making finding an old analysis easy to find.

I would also note that some experimental equipment would do many more 
than 50 different analysis's a day.

It's a common requirement in any sort of research facility, especially 
when they have central facilities for doing analysis on equipment that 
would be too expensive for an individual group or where it makes sense 
to "outsource" repetitive basics analysis to lower paid staff.


JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG