[gpfsug-discuss] RAID config for SSD's used for data

[Jonathan Buzzard]

On Wed, 2017-04-19 at 14:23 -0700, Alex Chekholko wrote:
> On 04/19/2017 12:53 PM, Buterbaugh, Kevin L wrote:
> >
> > So you’re considering the purchase of a dual-controller FC storage array
> > with 12 or so 1.8 TB SSD’s in it, with the idea being that that storage
> > would be in its’ own storage pool and that pool would be the default
> > location for I/O for your main filesystem … at least for smaller files.
> >  You intend to use mmapplypolicy nightly to move data to / from this
> > pool and the spinning disk pools.
> 
> We did this and failed in interesting (but in retrospect obvious) ways. 
> You will want to ensure that your users cannot fill your write target 
> pool within a day.  The faster the storage, the more likely that is to 
> happen.  Or else your users will get ENOSPC.

Eh? Seriously you should have a fail over rule so that when your "fast"
pool is filled up it starts allocating in the "slow" pool (nice good
names that are descriptive and less than 8 characters including
termination character). Now there are issues when you get close to very
full so you need to set the fail over to as sizeable bit less than the
full size, 95% is a good starting point.

The pool names size is important because if the fast pool is less than
eight characters and the slow is more because you called in
"nearline" (which is 9 including termination character) once the files
get moved they get backed up again by TSM, yeah!!!

The 95% bit comes about from this. Imagine you had 12KB left in the fast
pool and you go to write a file. You open the file with 0B in size and
then start writing. At 12KB you run out of space in the fast pool and as
the file can only be in one pool you get a ENOSPC, and the file gets
canned. This then starts repeating on a regular basis.

So if you start allocating at significantly less than 100%, say 95%
where that 5% is larger than the largest file you expect that file
works, but all subsequent files get allocated in the slow pool, till you
flush the fast pool.

Something like this as the last two rules in your policy should do the
trick.

/* by default new files to the fast disk unless full, then to slow */
RULE 'new' SET POOL 'fast' LIMIT(95)
RULE 'spillover' SET POOL 'slow'

However in general your fast pool needs to have sufficient capacity to
take your daily churn and then some.

JAB.

-- 
Jonathan A. Buzzard                 Email: jonathan (at) buzzard.me.uk
Fife, United Kingdom.