[gpfsug-discuss] File_heat for GPFS File Systems

[Andreas Landhäußer]

On Mon, 26 Sep 2016, Marc A Kaplan <makaplan at us.ibm.com> wrote:

Marc,

thanks for your explanation,

> fileHeatLossPercent=10, fileHeatPeriodMinutes=1440
>
> means any file that has not been accessed for 1440 minutes (24 hours = 1
> day) will lose 10% of its Heat.
>
> So if it's heat was X at noon today, tomorrow  0.90 X, the next day 0.81X,
> on the k'th day   (.90)**k * X.
> After 63 fileHeatPeriods, we always round down and compute file heat as
> 0.0.
>
> The computation (in floating point with some approximations) is done "on
> demand" based on a heat value stored in the Inode the last time the unix
> access "atime"  and the current time.  So the cost of maintaining
> FILE_HEAT for a file is some bit twiddling, but only when the file is
> accessed and the atime would be updated in the inode anyway.
>
> File heat increases by approximately 1.0 each time the entire file is read
> from disk.   This is done proportionately so if you read in half of the
> blocks the increase is 0.5.
> If you read all the blocks twice FROM DISK the file heat is increased by
> 2. And so on.  But only IOPs are charged.  If you repeatedly do posix
> read()s but the data is in cache, no heat is added.

with the above definition file heat >= 0.0 e.g. any positive floating 
point value is valid.

I need to categorize the files into categories hot, warm, lukewarm and 
cold.

How do I achieve this, since the maximum heat is varying and need to be 
defined every time when requesting the report.

We are wishing to migrate data according to the heat onto different 
storage categories (expensive --> cheap devices)

> The easiest way to observe FILE_HEAT is with the mmapplypolicy directory
> -I test -L 2 -P fileheatrule.policy
>
> RULE 'fileheatrule' LIST 'hot' SHOW('Heat=' || varchar(FILE_HEAT))  /* in
> file fileheatfule.policy */
>
> Because policy reads metadata from inodes as stored on disk, when
> experimenting/testing you may need to
>
> mmfsctl fs suspend-write;  mmfsctl fs resume

Doing this on a production file system, a valid change request need to be 
filed, and description of the risks for customers data and so on have to 
be defined (ITIL) ...

Any help and ideas will be appreciated

 	Andreas

-- 
Andreas Landhäußer				+49 151 12133027 (mobile)
alandhae at gmx.de