[gpfsug-discuss] Recharging where HSM is used

[Jonathan Buzzard]

On Thu, 2018-05-03 at 15:02 +0000, Sobey, Richard A wrote:
> Stephen, Bryan,
>  
> Thanks for the input, it’s greatly appreciated.
>  
> For us we’re trying – as many people are – to drive down the usage of
> under-the-desk NAS appliances and USB HDDs. We offer space on disk,
> but you can’t charge for 3TB of storage the same as you would down PC
> World and many customers don’t understand the difference between what
> we do, and what a USB disk offers.
>  
> So, offering tape as a medium to store cold data, but not archive
> data, is one offering we’re just getting round to discussing. The
> solution is in place. To answer the specific question: for our
> customers that adopt HSM, how much less should/could/can we charge
> them per TB. We know how much a tape costs, but we don’t necessarily
> have the means (or knowledge?) to say that for a given fileset, 80%
> of the data is on tape. Then you get into 80% of 1TB is not the same
> as 80% of 10TB.
>  

The test that I have used in the past for if a file is migrated with a
high degree of accuracy is 

    if the space allocated on the file system is less than the
    file size, and equal to the stub size then presume the file
    is migrated.

There is a small chance it could be sparse instead. However this is
really rather remote as sparse files are not common in the first place
and even less like that the amount of allocated data in the sparse file
exactly matches the stub size.

It is an easy step to write a policy to list all the UID and FILE_SIZE
where KB_ALLOCATED<FILE_SIZE and KB_ALLOCATED=XXX where XXX is the sub
size. You can then post process that to get a per user report of the
amount of data migrated, and this then becomes the basis for your
charging. Depending on how much effort you wish to expend you can use
things like Excel::Writer::XLSX or RTF::Writer to produce fancy reports
if Perl is your poison. I imagined similar modules exist for Python.


JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG