[gpfsug-discuss] GPFS and Flash/SSD Storage tiered storage

[valleru at cbio.mskcc.org]

Hi All,

I am trying to figure out a GPFS tiering architecture with flash storage in front end and near line storage as backend, for Supercomputing

The Backend storage will be a GPFS storage on near line of about 8-10PB. The backend storage will/can be tuned to give out large streaming bandwidth and enough metadata disks to make the stat of all these files fast enough.

I was thinking if it would be possible to use a GPFS flash cluster or GPFS SSD cluster in front end that uses AFM and acts as a cache cluster with the backend GPFS cluster.

At the end of this .. the workflow that i am targeting is where:


“
If the compute nodes read headers of thousands of large files ranging from 100MB to 1GB, the AFM cluster should be able to bring up enough threads to bring up all of the files from the backend to the faster SSD/Flash GPFS cluster.
The working set might be about 100T, at a time which i want to be on a faster/low latency tier, and the rest of the files to be in slower tier until they are read by the compute nodes.
“


I do not want to use GPFS policies to achieve the above, is because i am not sure - if policies could be written in a way, that files are moved from the slower tier to faster tier depending on how the jobs interact with the files.
I know that the policies could be written depending on the heat, and size/format but i don’t think thes policies work in a similar way as above.

I did try the above architecture, where an SSD GPFS cluster acts as an AFM cache cluster before the near line storage. However the AFM cluster was really really slow, It took it about few hours to copy the files from near line storage to AFM cache cluster.
I am not sure if AFM is not designed to work this way, or if AFM is not tuned to work as fast as it should.

I have tried LROC too, but it does not behave the same way as i guess AFM works.

Has anyone tried or know if GPFS supports an architecture - where the fast tier can bring up thousands of threads and copy the files almost instantly/asynchronously from the slow tier, whenever the jobs from compute nodes reads few blocks from these files?
I understand that with respect to hardware - the AFM cluster should be really fast, as well as the network between the AFM cluster and the backend cluster.

Please do also let me know, if the above workflow can be done using GPFS policies and be as fast as it is needed to be.

Regards,
Lohit


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180222/23a08988/attachment-0001.htm>