[gpfsug-discuss] Fileheat - does work! Complete test/example provided here.

Marc A Kaplan makaplan at us.ibm.com
Tue Aug 13 14:32:53 BST 2019 

Yes, you are correct.  It should only be necessary to set
fileHeatPeriodMinutes, since the loss percent does have a default value.
But IIRC (I implemented part of this!) you must restart the daemon to get
those fileheat parameter(s) "loaded"and initialized into the daemon
processes.

Not fully trusting my memory... I will now "prove" this works today as
follows:

To test, create and re-read a large file with dd...

[root@/main/gpfs-git]$mmchconfig fileHeatPeriodMinutes=60
mmchconfig: Command successfully completed
  ...
[root@/main/gpfs-git]$mmlsconfig | grep -i heat
fileHeatPeriodMinutes 60

[root@/main/gpfs-git]$mmshutdown
 ...
[root@/main/gpfs-git]$mmstartup
...
[root@/main/gpfs-git]$mmmount c23
...
[root@/main/gpfs-git]$ls -l /c23/10g
-rw-r--r--. 1 root root 10737418240 May 16 15:09 /c23/10g

[root@/main/gpfs-git]$mmlsattr -d -X /c23/10g
file name:            /c23/10g
security.selinux

(NO fileheat attribute yet...)

[root@/main/gpfs-git]$dd if=/c23/10g bs=1M of=/dev/null
...
After the command finishes, you may need to wait a while for the metadata
to flush to the inode on disk ... or you can force that with an unmount or
a mmfsctl...

Then the fileheat attribute will appear (I just waited by answering another
email... No need to do any explicit operations on the file system..)

[root@/main/gpfs-git]$mmlsattr -d -X /c23/10g
file name:            /c23/10g
security.selinux
gpfs.FileHeat

To see its hex string value:

[root@/main/gpfs-git]$mmlsattr -d -X -L /c23/10g
file name:            /c23/10g
    ...
security.selinux:
0x756E636F6E66696E65645F753A6F626A6563745F723A756E6C6162656C65645F743A733000
gpfs.FileHeat:        0x000000EE42A40400

Which will be interpreted by mmapplypolicy...

YES, the interpretation is relative to last access time and current time,
and done by a policy/sql function "computeFileHeat"
(You could find this using m4 directives in your policy file...)

define([FILE_HEAT],[computeFileHeat(CURRENT_TIMESTAMP-ACCESS_TIME,xattr
('gpfs.FileHeat'),KB_ALLOCATED)])

Well gone that far, might as well try mmapplypolicy too....

[root@/main/gpfs-git]$cat /gh/policies/fileheat.policy
define(DISPLAY_NULL,[CASE WHEN ($1) IS NULL THEN '_NULL_' ELSE varchar($1)
END])

rule fh1 external list 'fh' exec ''
rule fh2 list 'fh' weight(FILE_HEAT) show(DISPLAY_NULL(xattr_integer
('gpfs.FileHeat',1,4,'B')) || ' ' ||
DISPLAY_NULL(xattr_integer('gpfs.FileHeat',5,2,'B')) || ' ' ||
DISPLAY_NULL(xattr_integer('gpfs.FileHeat',7,2,'B')) || ' ' ||
DISPLAY_NULL(FILE_HEAT) || ' ' ||
DISPLAY_NULL(hex(xattr('gpfs.FileHeat'))) || ' ' ||
getmmconfig('fileHeatPeriodMinutes') || ' ' ||
getmmconfig('fileHeatLossPercent') || ' ' ||
getmmconfig('clusterName') )


[root@/main/gpfs-git]$mmapplypolicy /c23 --maxdepth 1
-P /gh/policies/fileheat.policy -I test -L 3
   ...
<1> /c23/10g     RULE 'fh2' LIST 'fh' WEIGHT(0.022363) SHOW( 238 17060 1024
+2.23632812500000E-002 000000EE42A40400 60 10 makaplan.sl.cloud9.ibm.com)
 ...
WEIGHT(0.022363) LIST 'fh' /c23/10g SHOW(238 17060 1024
+2.23632812500000E-002 000000EE42A40400 60 10 makaplan.sl.cloud9.ibm.com)






From:	Jan-Frode Myklebust <janfrode at tanso.net>
To:	gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date:	08/13/2019 06:22 AM
Subject:	[EXTERNAL] Re: [gpfsug-discuss] Fileheat
Sent by:	gpfsug-discuss-bounces at spectrumscale.org




What about filesystem atime updates. We recently changed the default to
«relatime». Could that maybe influence heat tracking?



  -jf


tir. 13. aug. 2019 kl. 11:29 skrev Ulrich Sibiller <
u.sibiller at science-computing.de>:
  On 12.08.19 15:38, Marc A Kaplan wrote:
  > My Admin guide says:
  >
  > The loss percentage and period are set via the configuration
  > variables *fileHeatLossPercent *and *fileHeatPeriodMinutes*. By
  default, the file access temperature
  > is not
  > tracked. To use access temperature in policy, the tracking must first
  be enabled. To do this, set
  > the two
  > configuration variables as follows:*

  Yes, I am aware of that.

  > fileHeatLossPercent*
  > The percentage (between 0 and 100) of file access temperature
  dissipated over the*
  > fileHeatPeriodMinutes *time. The default value is 10.
  > Chapter 25. Information lifecycle management for IBM Spectrum Scale
  *361**
  > fileHeatPeriodMinutes*
  > The number of minutes defined for the recalculation of file access
  temperature. To turn on
  > tracking, *fileHeatPeriodMinutes *must be set to a nonzero value. The
  default value is 0
  >
  >
  > SO Try setting both!

  Well, I have not because the documentation explicitly mentions a default.
  What's the point of a
  default if I have to explicitly configure it?

  > ALSO to take effect you may have to mmshutdown and mmstartup, at least
  on the (client gpfs) nodes
  > that are accessing the files of interest.

  I have now configured both parameters and restarted GPFS. Ran a tar over
  a directory - still no
  change. I will wait for 720minutes and retry (tomorrow).

  Thanks

  Uli

  --
  Science + Computing AG
  Vorstandsvorsitzender/Chairman of the board of management:
  Dr. Martin Matzke
  Vorstand/Board of Management:
  Matthias Schempp, Sabine Hohenstein
  Vorsitzender des Aufsichtsrats/
  Chairman of the Supervisory Board:
  Philippe Miltin
  Aufsichtsrat/Supervisory Board:
  Martin Wibbe, Ursula Morgenstern
  Sitz/Registered Office: Tuebingen
  Registergericht/Registration Court: Stuttgart
  Registernummer/Commercial Register No.: HRB 382196
  _______________________________________________
  gpfsug-discuss mailing list
  gpfsug-discuss at spectrumscale.org
  http://gpfsug.org/mailman/listinfo/gpfsug-discuss
  _______________________________________________
  gpfsug-discuss mailing list
  gpfsug-discuss at spectrumscale.org
  https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=q-xjYq0Bimv3bYK1rhVMZ7jLvoEssmvfyMF0kcf5slc&s=buMugfNwUbsXJ2Gi04A3ehIQ0v-ORRc-Mb7sxaGgLrA&e=



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190813/2f62ab3b/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190813/2f62ab3b/attachment-0002.gif>

More information about the gpfsug-discuss mailing list