[gpfsug-discuss] gpfsug-discuss Digest, Vol 108, Issue 18

[Jez Tucker]

Hi Owen,

  This is a great thread and raises as is usual with GPFS that there is
more often than not many tools in the bag and more than one way to
achieve a requirement.

If you stick with policy (indeed worthwhile aspect of GPFS to learn
well) you can also build up libraries of centralised macros and
reference them into any policy file alike so at the top of the file:

include(/mmfs1/policies/macros/general_macros.m4)
include(/mmfs1/policies/macros/pixit_excludes.m4)

Check your PixStor system under /mmfs1/policies/..

The .m4 extension is not required, but a nod to
https://www.gnu.org/savannah-checkouts/gnu/m4/manual/m4-1.4.18/m4.html
which the policy engine also encapsulates.  Marc Kaplan once told me
"You can do just about anything with m4 and cleverness and patience...",
though he failed to mention the required amounts of coffee and pizza
;-)  Search the list digests for his valuable insights over the years.

Alternatively you can achieve all this and more using the Python API
https://www.arcapix.com/pixstorapi/index.html (especially your date
processing).

If you'd like direct support with any this, ping us over an email via
support@ - alternatively there are indeed many great minds a wealth of
experience and views on this list (and it's nice to meet the community too).

SSUG also has a Slack channel too.. ssug-poweraiug.slack.com

Most of all - have fun learning.


Kind regards,

Jez

p.s. yes the threading has all gone wonky. ah well :-)


On 01/02/2021 21:09, Owen Morgan wrote:
> Jonathan,
>
> If I have a single policy file with all the related department rules
> and each time they want to add additional rules with different working
> day thresholds maybe using this -M method is easier. Its clear that
> the 'maths' and date/timestamp manipulation is easier in shell (my
> preferred is bash) than in the SQL of the policy (your example is
> succinct but needs to be repeated everytime a new rule is added with a
> different working day threshold, which is what I'm trying (if
> possiblr) to avoid.
>
> It seems to me the IBM SQL engine is perhaps missing more 'SQL' in
> built date/time functions like DateAdd and DateDiff etc..  as this
> would be a moot point. Its a shame I can't make one function that
> given a working day input as an argument spits out how many 'real'
> days exist between them for the file age comparison all in the SQL. It
> can be done for 1 specific input argument, but needs the whole
> function repeated manually for a different input argument, and further
> repeated for a different argument etc..
>
> Maybe I'm also compounding the issue by trying to make the policy file
> as concise as possible (for sake of clarity as to what the rules are
> trying to achieve, and easy expandability), and demanding too much of
> the SQL-like syntax that IBM have created.
>
> I have options for mmfind or even (as suggested) -M inoput to
> mmapplypolicy where I us bash to create a small function that does
> what I need, spits out 'real days' given a working day input, and
> using arrays and for-loop create a dynamic calling of the
> mmapplypolicy command (which I'm kinda half doing anyways for other
> reasons in my launcher script.
>
>
> As always, I'm seriously amazed at people with soo much experience and
> knowledge taking time out to help, guide, and offer input like
> everyone has been doing!! I'm relatively early in my career, so being
> able to interact and learn from experienced persons is giving me such
> a wider insight!
>
> Thanks!
>
> Owen.
> Sent from Front
>   	 
> 	
> Owen Morgan​
> Data Wrangler
> Motion Picture Solutions Ltd
> T: ** <tel:>
>
> E: *owen.morgan at motionpicturesolutions.com*
> <mailto:owen.morgan at motionpicturesolutions.com> 	 | 
> W: *motionpicturesolutions.com* <https://www.motionpicturesolutions.com/>
>
> A:  	Mission Hall, 9‑11 North End Road 	,  	London 	,  	W14 8ST
>
> Motion Picture Solutions Ltd is a company registered in England and
> Wales under number 5388229, VAT number 201330482
>
>> On 1 February 2021, 20:17 GMT jonathan.buzzard at strath.ac.uk
>> <mailto:jonathan.buzzard at strath.ac.uk> wrote:
>>
>> On 01/02/2021 18:11, Jan-Frode Myklebust wrote:
>>>
>>> > CAUTION: This email originated outside the University. Check before
>>> > clicking links or attachments.
>>> > Agree.. Write a policy that takes a "mmapplypolicy -M var=val"
>>> argument,
>>> > and figure out the workdays outside of the policy. Something like:
>>> >
>>> > # cat test.poilcy
>>> > define( access_age,     (DAYS(CURRENT_TIMESTAMP) - DAYS(ACCESS_TIME)))
>>> > /* list migrated files */
>>> > RULE EXTERNAL LIST 'oldFiles' EXEC ''
>>> > RULE 'oldFiles' LIST 'oldFiles'
>>> >     WHERE (access_age > MINAGE)
>>> >
>>> > # mmapplypolicy gpfs01  -P test.policy -I defer -f ./filelist -M
>>> MINAGE=5
>>> >
>>>
>>> Why bother when you can do it all in the policy?
>>>
>>> JAB.
>>>
>>> -- 
>>> Jonathan A. Buzzard Tel: +44141-5483420
>>> HPC System Administrator, ARCHIE-WeSt.
>>> University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
>>>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

-- 
*Jez Tucker*
VP Research and Development | Pixit Media
m: +44 (0) 776 419 3820
e: jtucker at pixitmedia.com <mailto:jtucker at pixitmedia.com>
Visit www.pixitmedia.com <https://www.pixitmedia.com>

-- 


This email is confidential in that it is intended for the exclusive 
attention of the addressee(s) indicated. If you are not the intended 
recipient, this email should not be read or disclosed to any other person. 
Please notify the sender immediately and delete this email from your 
computer system. Any opinions expressed are not necessarily those of the 
company from which this email was sent and, whilst to the best of our 
knowledge no viruses or defects exist, no responsibility can be accepted 
for any loss or damage arising from its receipt or subsequent use of this 
email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20210201/e69f7855/attachment-0002.htm>