[gpfsug-discuss] Proposal for dynamic heat assisted file tiering

[Andreas Landhäußer]

On Tue, 27 Sep 2016, Eric Horst <erich at uw.edu> wrote:

Thanks Eric for the hint,

shouldn't we as the users define a requirement for such a dynamic heat 
assisted file tiering option (DHAFTO).

Keeping track which files have increased heat and triggering a transparent 
move to a faster tier.

Since I haven't tested it on a GPFS FS, I would like to know about the 
performance penalties being observed, when frequently running the 
policies, just a rough estimate.

Of course its depending on the speed of the Metadata disks (yes, we use 
different devices for Metadata) we are also running GPFS on various GSS 
Systems.

IBM might also want bundling this option together with GSS/ESS hardware 
for better performance.

Just my 2¢

Andreas

>>>
>>> if a file gets hot again, there is no rule for putting the file back
>>> into a faster storage device?
>>
>>
>> The file will get moved when you run the policy again.  You can run the
>> policy as often as you like.
>
> I think its worth stating clearly that if a file is in the Thrifty
> slow pool and a user opens and reads/writes the file there is nothing
> that moves this file to a different tier. A policy run is the only
> action that relocates files. So if you apply the policy daily and over
> the course of the day users access many cold files, the performance
> accessing those cold files may not be ideal until the next day when
> they are repacked by heat. A file is not automatically moved to the
> fast tier on access read or write. I mention this because this aspect
> of tiering was not immediately clear from the docs when I was a
> neophyte GPFS admin and I had to learn by observation. It is easy for
> one to make an assumption that it is a more dynamic tiering system
> than it is.

-- 
Andreas Landhäußer				+49 151 12133027 (mobile)
alandhae at gmx.de