[gpfsug-discuss] Question about Policies
Marc A Kaplan
makaplan at us.ibm.com
Fri Dec 27 14:19:43 GMT 2019
The MIGRATE rule is for moving files from one pool to another, without
changing the pathname or any attributes, except the storage devices holding
the data blocks of the file. Also can be use for "external" pools to
migrate to an HSM system.
"moving" from one folder to another is a different concept.
The mmapplypolicy LIST and EXTERNAL LIST rules can be used to find files
older than 30 days and then do any operations you like on them, but you
have to write a script to do those operations.
See also -- the "Information Lifecycle Management" (ILM) chapter of the SS
Admin Guide
AND/OR for an easy to use parallel function equivalent to the classic Unix
pipline `find ... | xargs ... `
Try the `mmfind ... -xargs ... ` from the samples/ilm directory.
[root@~/.../samples/ilm]$./mmfind
Usage: ./mmfind [mmfind args] { <find invocation> | -inputFileList f
-policyFile f }
mmfind args: [-polFlags 'flag 1 flag 2 ...'] [-logLvl {0|1|2}] [-logFile
f] [-saveTmpFiles]
[-fs fsName] [-mmapplypolicyOutputFile f]
find invocation -- logic: ! ( ) -a -o
/path1 [/path2 ...] [expression]
-atime N -ctime N -mtime N -true -false -perm mode
-iname PATTERN -name PATTERN -path PATTERN -ipath PATTERN
-uid N -user NAME -gid N -group NAME -nouser -nogroup
-newer FILE -older FILE -mindepth LEVEL -maxdepth LEVEL
-links N -size N -empty -type [bcdpflsD] -inum N
-exec COMMAND -execdir COMMAND
-ea NAME -eaWithValue NAME===VALUE -setEA NAME[===VALUE] -deleteEA NAME
-gpfsImmut -gpfsAppOnly -gpfsEnc -gpfsPool POOL_NAME
-gpfsMigrate poolFrom,poolTo -gpfsSetPool poolTo
-gpfsCompress -gpfsUncompress -gpfsSetRep m,r
-gpfsWeight NumericExpr
-ls -fls -print -fprint -print0 -fprint0 -exclude PATH
-xargs [-L maxlines] [-I rplstr] COMMAND
Give -h for a more verbose usage message
From: Kevin Doyle <kevin.doyle at manchester.ac.uk>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: 12/27/2019 08:57 AM
Subject: [EXTERNAL] Re: [gpfsug-discuss] Question about Policies
Sent by: gpfsug-discuss-bounces at spectrumscale.org
Hi
Thanks for your reply, once I have a set of files that are older than 30
days would I then use the migration rule to move them ?
Looking at the migration Rule syntax implies a “From” pool and a “To” pool,
I only have a single pool so would I use the same pool name for From and
To ? if it is the same pool
How do I specify the folder to move it to which needs to be different from
the current location.
Thanks
Kevin
RULE ['RuleName'] [WHEN TimeBooleanExpression]
MIGRATE [COMPRESS ({'yes' | 'no' | 'lz4' | 'z'})]
[FROM POOL 'FromPoolName']
[THRESHOLD (HighPercentage[,LowPercentage[,PremigratePercentage]])]
[WEIGHT (WeightExpression)]
TO POOL 'ToPoolName'
[LIMIT (OccupancyPercentage)]
[REPLICATE (DataReplication)]
[FOR FILESET ('FilesetName'[,'FilesetName']...)]
[SHOW (['String'] SqlExpression)]
[SIZE (numeric-sql-expression)]
[ACTION (SqlExpression)]
[WHERE SqlExpression]
Kevin Doyle | Linux Administrator, Scientific Computing
Cancer Research UK, Manchester Institute
The University of Manchester
Room 13G40, Alderley Park, Macclesfield SK10 4TG
Mobile: 07554 223480
Email: Kevin.Doyle at manchester.ac.uk
/Users/kdoyle/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_1131538866
From: <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Yaron Daniel
<YARD at il.ibm.com>
Reply-To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: Friday, 27 December 2019 at 12:55
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Question about Policies
Hi
U can create list of diretories in output file which were not modify in the
last 30 days, and than second script will move this directories to the new
location that u want.
Regards
Yaron Daniel 94 Em Ha'Moshavot Rd cid:_1_10392F3C103929880046F589C22584DD
Storage Architect – IL Lab Petach Tiqva, 49527
Services (Storage)
IBM Global Markets, Systems HW Israel
Sales
Phone: +972-3-916-5672
Fax: +972-3-916-5672
Mobile: +972-52-8395593
e-mail: yard at il.ibm.com
Webex:
https://ibm.webex.com/meet/yard
IBM Israel
cid:_2_103C9B0C103C96FC0046F589C22584DD
cid:_2_103C9D14103C96FC0046F589C22584DD
cid:_2_103C9F1C103C96FC0046F589C22584DD
cid:_2_103CA124103C96FC0046F589C22584DD
cid:_2_103CA32C103C96FC0046F589C22584DD
cid:_2_103CA534103C96FC0046F589C22584DD
cid:_2_103CA73C103C96FC0046F589C22584DD
cid:_2_103CA944103C96FC0046F589C22584DD
From: Kevin Doyle <kevin.doyle at manchester.ac.uk>
To: "gpfsug-discuss at spectrumscale.org"
<gpfsug-discuss at spectrumscale.org>
Date: 27/12/2019 13:45
Subject: [EXTERNAL] [gpfsug-discuss] Question about Policies
Sent by: gpfsug-discuss-bounces at spectrumscale.org
Hi
I work for Cancer Research UK MI in Manchester UK. I am new to GPFS and
have been tasked with creating a policy that will
Move files older that 30 days to a new folder within the same pool. There
are a lot of files so using a policy based move will be faster.
I have read about the migration Rule but it states a source and destination
pool, we only have one pool. Will it work if I define the same source and
destination pool ?
Many thanks
Kevin
Kevin Doyle | Linux Administrator, Scientific Computing
Cancer Research UK, Manchester Institute
The University of Manchester
Room 13G40, Alderley Park, Macclesfield SK10 4TG
Mobile: 07554 223480
Email: Kevin.Doyle at manchester.ac.uk
/Users/kdoyle/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_1799188038
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=w3zKI5uOkIxqfgnHm53Al4Q3apC0htUiiuFcMnh2U9s&s=rkD5iWzjhbTA_9kEHL9Laggb4NGjiYS4qoM8yXbAoyM&e=
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0010.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16547711.gif
Type: image/gif
Size: 16051 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0011.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16942257.gif
Type: image/gif
Size: 1115 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0012.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ecblank.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0013.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16264175.jpg
Type: image/jpeg
Size: 3848 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0016.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16010102.jpg
Type: image/jpeg
Size: 4267 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0017.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16098719.jpg
Type: image/jpeg
Size: 3748 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0018.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16043707.jpg
Type: image/jpeg
Size: 3794 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0019.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16546771.jpg
Type: image/jpeg
Size: 4302 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0020.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16875824.jpg
Type: image/jpeg
Size: 3740 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0021.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16069185.jpg
Type: image/jpeg
Size: 3856 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0022.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16639470.jpg
Type: image/jpeg
Size: 4339 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0023.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16809363.gif
Type: image/gif
Size: 16052 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20191227/39672f4b/attachment-0014.gif>
More information about the gpfsug-discuss
mailing list