[gpfsug-discuss] Introduction/Question

[Edward Wahl]

On Mon, 6 Nov 2017 09:20:11 +0000
"Chase, Peter" <peter.chase at metoffice.gov.uk> wrote:

>  how can I automate sending files to a cloud object store as they arrive in
> GPFS and keep a copy of the file in GPFS?

Sounds like you already have an idea how to do this by using ILM policies.
Either quota based as you mention or 'placement' policies should work, though I
cannot speak to placement in an S3 environment, the policy engine has a way to
call external commands for that if necessary.  Though if you create an external
pool, a placement policy may be much simpler and possibly faster as well as
data would be sent to S3 on write, rather than on a quota trigger.    


If an external storage pool works properly for S3, I'd probably use a placement
policy myself. This also would depend on how/when I needed the data on S3 and
your mention of timeliness tells me placement rather than quota may be best.
Weighing the solutions for this may be better tested(and timed!) than anything.  

EVERYONE wants a timely weather forecast.   ^_-

Ed


-- 

Ed Wahl
Ohio Supercomputer Center
614-292-9302