[gpfsug-discuss] Question about Policies

[Jonathan Buzzard]

On 27/12/2019 14:20, david_johnson at brown.edu wrote:
> You would want to look for examples of external scripts that work on the 
> result of running the policy engine in listing mode.  The one issue that 
> might need some attention is the way that gpfs quotes unprintable 
> characters in the pathname. So the policy engine generates the list and 
> your external script does the moving.
> 

In my experience a good starting point would be to scan the list of 
files from the policy engine and separate the files out into "normal"; 
that is files using basic ASCII and no special characters and the rest 
also known as the "wacky pile".

Given that you are UK based it is not unreasonable to expect all path 
and file names to be in English. There might (and if not probably 
should) be an institutional policy mandating it. Not much use if a 
researcher saves everything in Greek then gets knocked over by a bus and 
person picking up the work is Spanish for example.

Hopefully the "wacky pile" is small, however expect to find all sorts of 
bizarre file and path names in it. We are talking wildcards, back ticks, 
even newline characters to name but a few.

Depending on the amount of data in the "wacky" pile you might just want 
to forget about moving them, as they are orders of magnitude more 
difficult to deal with than files with "sane" path and file names and 
can rapidly soak up large chunks of time trying to deal with them in 
scripts.

JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG