[gpfsug-discuss] Dry-Run functionality ?

[Marc A Kaplan]

What I always recommend is to make a directory of test cases and run

mmapplypolicy /gpfs-path/test-cases -I test (or -I yes if you want to try 
operating on the test files) -L 2 (or -L 6 or something in between)  -P 
policy-rules [other-options]

Because as I think you're saying

mmapplypolicy gpfs-fsname -I test ... 

Can consume a lot of compute and IO resource, just scanning the metadata.





From:   Jez Tucker <jtucker at pixitmedia.com>
To:     gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:   06/27/2016 12:22 PM
Subject:        Re: [gpfsug-discuss] Dry-Run functionality ?
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



Hi Marc,

    Quite.  We have had coverage of (1) and (2) in the API for some time.

My overriding thought is that (QoS aside) if one executes an 
mmapplypolicy, for whatever reason, then a certain amount of resource is 
consumed.
This may not be preferred.

So let me rephrase to the users:    in your real-world working 
environment, would you prefer:

a) Return what would be called.  E.G. mmapplypolicy -P mypol.pol --flag1 
--flag2 --etc  which is a sanity check, but not a policy run, hence no 
load
b) mmapplypolicy /gpfs-path -P policy-rules-file   -I test -L 2  [other 
options]`  will do a dry run and show what actions would be performed on 
each file.

To be clear; you can currently achieve either launch with the API as it 
stands.
However we'd like to know what the general concensus would prefer to be 
the norm in such an implementation.

Regarding dry run functionality this can be achieved globally as follows:

setDryRun(True)
<API commands to delete filesets>
<API commands to delete snapshots>
<API commands to migrate data>

or as a more granular decorator:

@dryrun
... def delete_filesets(): 
    <API commands to delete filesets>

Jez


On 20/06/16 16:03, Marc A Kaplan wrote:
Jez,

  Regarding your recent post.  Do the mmchpolicy and mmapplypolicy 
commands have sufficient functionality for your purposes?  
Are you suggesting some improvements?  If so, what?  Please provide 
examples and/or specific suggestions.

WRT your numbered items:

(1)  `mmchpolicy fsname  -I  policy-rules-file test`   does a complete 
syntax check on policy-rules-file and some other sanity checking.

(2) `mmapplypolicy /gpfs-path/empty-directory -P policy-rules-file -I test 
-L 0` is another way to validate policy-rules-file.  Rules like MIGRATE 
and LIST that are interpreted by mmapplypolicy will   be scrutinized. For 
example check that each named pool is defined either within the file 
system or by an EXTERNAL POOL rules.

(3) `mmapplypolicy /gpfs-path -P policy-rules-file   -I test -L 2  [other 
options]`  will do a dry run and show what actions would be performed on 
each file.


--marc

-----------

API calls for Policies have the ability to run in 'test' or 'run' mode. 
 (ref: man mmapplypolicy)

How would you expect to use dry-run functionality on a policy?
We have our own opinion, but we'd like to hear yours.

1) Validate the policy and print the policy content to stdout/other.  No 
mmapplypolicy is performed.
2) Validate the policy and enforce mmapplypolicy with -I test
3) Return what would be called.  E.G. mmapplypolicy -P mypol.pol --flag1 
--flag2 --etc
4) Other

Best regards,

Jez
-- 
Jez Tucker
Head of Research & Development



_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss



-- 
Jez Tucker
Head of Research & Development
Pixit Media
www.pixitmedia.com


This email is confidential in that it is intended for the exclusive 
attention of the addressee(s) indicated. If you are not the intended 
recipient, this email should not be read or disclosed to any other person. 
Please notify the sender immediately and delete this email from your 
computer system. Any opinions expressed are not necessarily those of the 
company from which this email was sent and, whilst to the best of our 
knowledge no viruses or defects exist, no responsibility can be accepted 
for any loss or damage arising from its receipt or subsequent use of this 
email._______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160627/c3d818d3/attachment-0002.htm>