[gpfsug-discuss] A GPFS newbie

Matthew Wallis mattw at vpac.org
Tue Aug 7 12:43:00 BST 2012


Hi Robert,

On 07/08/2012, at 9:09 PM, Robert Esnouf <robert at strubi.ox.ac.uk> wrote:

> 
> Dear GPFS users,
> 
> My question is how should we go about configuring the number 
> and specifications of the NSDs? Are there any good rules of 
> thumb? And are there any folk out there using GPFS for high 
> I/O rates like this in a similar setup who would be happy to 
> have their brains/experiences picked?

From IBM, a x3650 M3 should be able to provide around 2.4GB/sec 
over QDR IB. That's with 12GB of RAM and dual quad core X5667s
 
They believe with the M4 you should be able to sustain somewhere
near double that,  but we'll say 4GB/sec for safety.

So with 4 of those you should be pushing somewhere north of 
16GB/sec. With FDR IB and PCIe 3.0, I can certainly believe it's 
possible, I think they've doubled the minimum RAM in the recent 
proposal we had from them.

In our benchmarks we certainly found the M3's capable of it, for daily 
use, our workloads are too mixed, we don't have anyone doing 
sustained reads or writes on those types of files.

Might have to be a bit more expansive on your node configuration
though, I can get 2000 cores in 32 nodes these days, so that spec 
would give you 512MB/sec per node if everyone is reading and 
writing at once. If you're only doing 16 cores per node, then that's
125 nodes, and only 131MB/sec per node. 

Matt.




More information about the gpfsug-discuss mailing list