[gpfsug-discuss] wanted...gpfs policy that places larger files onto a pool based on size

[J. Eric Wonderley]

Placement policy only applies to writes and I had thought that gpfs did
enough writing to memory "pagepool" to figure out the size before
committing the write to pool.

I also admit I don't know all of the innards of gpfs.  Pehaps being a copy
on write type filesystem prevents this for occurring.

On Mon, Oct 31, 2016 at 1:29 PM, Chris Scott <chrisjscott at gmail.com> wrote:

> Hi Brian
>
> This is exactly what I do with a SSD tier on top of 10K and 7.2K tiers.
>
> HAWC is another recent option that might address Eric's requirement but
> needs further consideration of the read requirements you want from the
> small files.
>
> Cheers
> Chris
>
> On 31 October 2016 at 17:23, Brian Marshall <mimarsh2 at vt.edu> wrote:
>
>> When creating a "fast tier" storage pool in a filesystem is the normal
>> case to create a placement policy that places all files in the fast tier
>> and migrates out old and large files?
>>
>>
>> Brian Marshall
>>
>> On Mon, Oct 31, 2016 at 1:20 PM, Jez Tucker <jez.tucker at gpfsug.org>
>> wrote:
>>
>>> Hey Bryan
>>>
>>>   There was a previous RFE for path placement from the UG, but Yuri told
>>> me this was not techically possible as an inode has no knowledge about the
>>> parent dentry.  (IIRC).    You can see this in effect in the C API.  It is
>>> possible to work this out at kernel level, but it's so costly that it
>>> becomes non-viable at scale / performance.
>>>
>>> IBMers please chip in and expand if you will.
>>>
>>> Jez
>>>
>>>
>>> On 31/10/16 17:09, Bryan Banister wrote:
>>>
>>> The File Placement Policy that you are trying to set cannot use the size
>>> of the file to determine the placement of the file in a GPFS Storage Pool.
>>> This is because GPFS has no idea what the file size will be when the file
>>> is open()’d for writing.
>>>
>>>
>>>
>>> Hope that helps!
>>>
>>> -Bryan
>>>
>>>
>>>
>>> PS. I really wish that we could use a path for specifying data placement
>>> in a GPFS Pool, and not just the file name, owner, etc.  I’ll submit a RFE
>>> for this.
>>>
>>>
>>>
>>> *From:* gpfsug-discuss-bounces at spectrumscale.org [
>>> mailto:gpfsug-discuss-bounces at spectrumscale.org
>>> <gpfsug-discuss-bounces at spectrumscale.org>] *On Behalf Of *J. Eric
>>> Wonderley
>>> *Sent:* Monday, October 31, 2016 11:53 AM
>>> *To:* gpfsug main discussion list
>>> *Subject:* [gpfsug-discuss] wanted...gpfs policy that places larger
>>> files onto a pool based on size
>>>
>>>
>>>
>>> I wanted to do something like this...
>>>
>>>
>>> [root at cl001 ~]# cat /opt/gpfs/home.ply
>>> /*Failsafe migration of old small files back to spinning media
>>> pool(fc_8T) */
>>> RULE 'theshold' MIGRATE FROM POOL 'system' THRESHOLD(90,70)
>>> WEIGHT(ACCESS_TIME) TO POOL 'fc_8T'
>>> /*Write files larger than 16MB to pool called "fc_8T" */
>>> RULE 'bigfiles' SET POOL 'fc_8T' WHERE FILE_SIZE>16777216
>>> /*Move anything else to system pool */
>>> RULE 'default' SET POOL 'system'
>>>
>>> Apparently there is no happiness using FILE_SIZE in a placement policy:
>>> [root at cl001 ~]# mmchpolicy home /opt/gpfs/home.ply
>>> Error while validating policy `home.ply': rc=22:
>>> PCSQLERR: 'FILE_SIZE' is an unsupported or unknown attribute or variable
>>> name in this context.
>>> PCSQLCTX: at line 4 of 6: RULE 'bigfiles' SET POOL 'fc_8T' WHERE
>>> {{{FILE_SIZE}}}>16777216
>>> runRemoteCommand_v2: cl002.cl.arc.internal: tschpolicy /dev/home
>>> /var/mmfs/tmp/tspolicyFile.mmchpolicy.113372 -t home.ply   failed.
>>> mmchpolicy: Command failed. Examine previous error messages to determine
>>> cause.
>>>
>>> Can anyone suggest a way to accomplish this using policy?
>>>
>>> ------------------------------
>>>
>>> Note: This email is for the confidential use of the named addressee(s)
>>> only and may contain proprietary, confidential or privileged information.
>>> If you are not the intended recipient, you are hereby notified that any
>>> review, dissemination or copying of this email is strictly prohibited, and
>>> to please notify the sender immediately and destroy this email and any
>>> attachments. Email transmission cannot be guaranteed to be secure or
>>> error-free. The Company, therefore, does not make any guarantees as to the
>>> completeness or accuracy of this email or any attachments. This email is
>>> for informational purposes only and does not constitute a recommendation,
>>> offer, request or solicitation of any kind to buy, sell, subscribe, redeem
>>> or perform any type of transaction of a financial product.
>>>
>>>
>>> _______________________________________________
>>> gpfsug-discuss mailing list
>>> gpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>>
>>>
>>> _______________________________________________
>>> gpfsug-discuss mailing list
>>> gpfsug-discuss at spectrumscale.org
>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>>
>>>
>>
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20161031/b1db3076/attachment-0002.htm>