[gpfsug-discuss] Spectrum Scale licensing

[Jonathan Buzzard]

On 17/04/2020 14:15, Aaron Knister wrote:
> Yeah, I had similar experiences in the past (over a decade ago) with
> Lustre and was heavily heavily anti-Lustre. That said, I just
> finished several weeks of what I’d call grueling testing of DDN
> Lustre and GPFS on the same hardware and I’m reasonably convinced
> much of that is behind us now (things like stability, metadata
> performance, random I/O performance just don’t appear to be issues
> anymore and in some cases these operations are now faster in Lustre).

Several weeks testing frankly does not cut the mustard to demonstrate 
stability. Our Lustre would run for months on end then boom, metadata 
server kernel panics. Sometimes but not always this would introduce the 
incorrectable file system corruption. You are going to need to have 
several years behind it to claim it is now stable.

At this point I would note that basically a fsck on Lustre is not 
possible. Sure there is a somewhat complicated procedure for it, but 
firstly it is highly likely to take weeks to run, and even then it might 
not be able to actually fix the problem.

> Full disclosure, I work for DDN, but the source of my paycheck has
> relatively little bearing on my technical opinions. All I’m saying is
> for me to honestly believe Lustre is worth another shot after the
> experiences I had years ago is significant. I do think it’s key to
> have a vendor behind you, vs rolling your own. I have seen that make
> a difference. I’m happy to take any further conversation/questions
> offline, I’m in no way trying to turn this into a marketing
> campaign.

Lustre is as of two years ago still behind GPFS 3.0 in terms of features 
and stability in my view. The idea it has caught up to GPFS 5.x in the 
last two years is in my view errant nonsense, software development just 
does not work like that.

Let me put it another way, in our experience the loss of compute 
capacity from the downtime of Lustre exceeded the cost of GPFS licenses. 
That excludes the wage costs of researches twiddling their thumbs whilst 
the system was restored to working order.

If I am being cynical if you can afford DDN storage in the first place 
stop winging about GPFS license costs.


JAB.

-- 
Jonathan A. Buzzard                         Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG