<div dir="ltr">In our environment managing the GPFS policies (or anything GPFS) are a bit of an Art form and really only a few people out of a sea of engineers can/will do it. So I like to keep the GPFS policies simple and durable.<div><br></div><div>If you really wanted to you could generate a list of every US (or whatever country) holiday you wanted and use Python, Awk, whatever to produce an exact number of business days versus not;</div><div><br></div><div>You can also use directory structure, GPFS attributes in parent directory, or a config file to maintain the requirements, which may be more manageable by other team members.</div><div><br></div><div>Anyhow my point being don't just look at the technical of how to implement, but what is the most maintainable, .documentable, and recoverable solution.</div><div><br></div><div>Some things I would consider in my design?</div><div>* Supportability - What is the skill level of the engineer required to maintain the system.</div><div>* Serviceability - What component would change most frequently, what's the complexity and impact of a change.</div><div>* Fail Safe - If a component fails, what will the negative impact be.</div><div>* Trackability - Are all the activities logged.</div><div>* Sanity Checking - Can you define safety mechanisms to prevent a failure. For instance if your purge script detects that there are 10k+ files to purge, can you interrupt the process?</div><div><br></div><div>For GPFS we try to keep the service components simple, and the logic in shell scripting where there is a plethora of knowledgeable engineers. So my attack would be like this:</div><div> * Policy runs, purges any file with purgeafter beyond the current date (maybe validate that the purge.date and modify/access date agree still). <-- is simple</div><div> * I would have it run the logic twice, and bomb if a safety threshhold is exceeded. Should log the results.</div><div> * Config in /gpfs/configs/purge_configuration</div><div> /some/path 10bd</div><div> /some/scratchspace 30d</div><div> /some/reallybigfiles 1d</div><div> * mmfind $(awk '{print $1}' < gpfs/configs/purge_configuration) -type f --exec /gpfs/scripts/setpurgedates</div><div> * Could also be a regular mmapplypolicy command as well...</div><div> * /gpfs/scripts/setpurgedates can rescan /gpfs/configs/purge_configuration and read in the date, check against the path passed, and then perform the necessary date logic, and exec the mmchattr, preserve the file's modify/access time to a purge.date time..</div><div> * Could log its decisions to a log file.</div><div> </div><div>Happy Hunting, I've already learned quite a few things from monitoring this list in a very short period of time.</div><div><br></div><div>* If a candidate ever tells me they know everything about Unix... I pass. We're all very experienced noobies in this ever progressing field/platform. *</div><div><br></div><div>Alec</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 1, 2021 at 1:58 PM Jonathan Buzzard <<a href="mailto:jonathan.buzzard@strath.ac.uk">jonathan.buzzard@strath.ac.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 01/02/2021 21:09, Owen Morgan wrote:<br>
> CAUTION: This email originated outside the University. Check before <br>
> clicking links or attachments.<br>
> Jonathan,<br>
> <br>
> If I have a single policy file with all the related department rules and <br>
> each time they want to add additional rules with different working day <br>
> thresholds maybe using this -M method is easier. Its clear that the <br>
> 'maths' and date/timestamp manipulation is easier in shell (my preferred <br>
> is bash) than in the SQL of the policy (your example is succinct but <br>
> needs to be repeated everytime a new rule is added with a different <br>
> working day threshold, which is what I'm trying (if possiblr) to avoid.<br>
> <br>
<br>
I actually think there is a much better way to achieve it than the case <br>
statement, but that would have required me to do some more thinking and <br>
testing to make sure it worked ;-)<br>
<br>
Anyway how many different "working" days old do you need?<br>
<br>
<br>
JAB.<br>
<br>
-- <br>
Jonathan A. Buzzard Tel: +44141-5483420<br>
HPC System Administrator, ARCHIE-WeSt.<br>
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</blockquote></div>