[gpfsug-discuss] mmapplypolicy - sql math on crack? - Please try a workaround

[Marc A Kaplan]

This may be a bug and/or a peculiarity of the SQL type system.  A proper 
investigation and full explanation will take more time than I have right 
now.

In the meanwhile please try forcing the computation/arithmetic to use 
floating point by changing your "30" to "30.0" 
and let us know if that helps you move along in your task at hand. 



From:   valdis.kletnieks at vt.edu
To:     gpfsug-discuss at spectrumscale.org
Date:   09/28/2018 06:36 PM
Subject:        [gpfsug-discuss] mmapplypolicy - sql math on crack?
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



OK, so we're running ltfs/ee for archiving to tape, and currently we 
migrate based
sheerly on "largest file first".  I'm trying to cook up something that 
does "largest LRU
first" (basically, large unaccessed files get moved earlier than large 
used files).

So I need to do some testing for what function works best.  First cut was 
"file
size times number of months idle".  And the policy looks like (Yes, I know
FILE_SIZE instead of KB_ALLOCATED when you have terabyte files is silly.. 
bear
with me...)

--
define(user_exclude_list,(PATH_NAME LIKE '/gpfs/archive/.ltfsee/%' OR 
PATH_NAME LIKE '/gpfs/archive/.SpaceMan/%' OR PATH_NAME LIKE 
'/gpfs/archive/ces/%' OR PATH_NAME LIKE '/gpfs/archive/config/%'))
define(is_premigrated,(MISC_ATTRIBUTES LIKE '%M%' AND MISC_ATTRIBUTES NOT 
LIKE '%V%'))
define(is_migrated,(MISC_ATTRIBUTES LIKE '%V%'))
define(is_resident,(NOT MISC_ATTRIBUTES LIKE '%M%'))

define(months_old,((DAYS(CURRENT_TIMESTAMP)-DAYS(ACCESS_TIME))/30))
define(STALE,(months_old * FILE_SIZE))

define(attr,varchar($1) || ' ')
define(all_attrs,attr(USER_ID) || attr(GROUP_ID) || attr(FILE_SIZE) || 
attr(STALE) || attr(months_old) || 
attr(DAYS(CURRENT_TIMESTAMP)-DAYS(ACCESS_TIME)) || attr(ACCESS_TIME) )

RULE 'SYSTEM_POOL_PLACEMENT_RULE' SET POOL 'system'

RULE EXTERNAL LIST 'testdrive_files'
EXEC ''

RULE 'FILES_PRUNE' LIST 'testdrive_files'
THRESHOLD(85,83)
WEIGHT(FILE_SIZE)
SHOW('candidate ' || all_attrs)
WHERE is_premigrated
AND (LENGTH(PATH_NAME) < 200)
AND NOT user_exclude_list
--

And something odd happens (tossing out files where 'months_old' is 0 and 
thus STALE is
as well):

 grep -v ' 0 0 ' ../list.testdrive_files | head -10| cut -f1-4 -d/ | sed 
's/$/(...)/'
528469735 1844029648 0  candidate 21008 675 132918210560 -897871872 23 714 
2016-10-14 07:00:52.000000  -- /gpfs/archive/vbi(...)
528469499 452828309 0  candidate 21008 675 128930785280 1880627200 23 714 
2016-10-14 05:51:37.000000  -- /gpfs/archive/vbi(...)
528994319 1729658954 0  candidate 21008 675 122826721280 -1073891328 23 
714 2016-10-14 08:03:26.000000  -- /gpfs/archive/vbi(...)
521263267 1704365147 0  candidate 1691616 1691616 111187014003 
111187014003 1 44 2018-08-15 19:15:18.534360  -- /gpfs/archive/arc(...)
529340511 975566740 0  candidate 21008 675 92028549120 -762247168 23 715 
2016-10-13 20:02:50.000000  -- /gpfs/archive/vbi(...)
528289152 1660557219 0  candidate 21008 675 91083468800 -1024258048 23 714 
2016-10-14 05:32:01.000000  -- /gpfs/archive/vbi(...)
-- this pair particularly interesting
513739717 1569434383 0  candidate 1691616 1691616 83291991689 -919741166 2 
63 2018-07-27 20:34:49.532723  -- /gpfs/archive/arc(...)
513739667 229891659 0  candidate 1691616 1691616 82634076418 2059395588 2 
63 2018-07-27 20:27:23.532723  -- /gpfs/archive/arc(...)
--
8007095 9763066 0  candidate 501 502 79531799574 -1823771078 119 3595 
2008-11-24 17:00:25.000000  -- /gpfs/archive/edisc(...)
527263956 829892493 0  candidate 1921090 1921090 73720064000 73720064000 1 
47 2018-08-12 02:18:07.534040  -- /gpfs/archive/arc(...)

It *looks* like "if you multiply by 0, you get zero, multiply by 1 (as in 
the
last line), you get full precision out  to at least 2**38, but multiply by 
anything else you get the
value mod 2**32 and treated as a signed integer"....

Is that the intended/documented behavior of "(value * value)" in a policy 
statement?
[attachment "attwost2.dat" deleted by Marc A Kaplan/Watson/IBM] 
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180929/36474f01/attachment-0002.htm>