[gpfsug-discuss] A GPFS newbie
Matthew Wallis
mattw at vpac.org
Tue Aug 7 12:43:00 BST 2012
Hi Robert,
On 07/08/2012, at 9:09 PM, Robert Esnouf <robert at strubi.ox.ac.uk> wrote:
>
> Dear GPFS users,
>
> My question is how should we go about configuring the number
> and specifications of the NSDs? Are there any good rules of
> thumb? And are there any folk out there using GPFS for high
> I/O rates like this in a similar setup who would be happy to
> have their brains/experiences picked?
From IBM, a x3650 M3 should be able to provide around 2.4GB/sec
over QDR IB. That's with 12GB of RAM and dual quad core X5667s
They believe with the M4 you should be able to sustain somewhere
near double that, but we'll say 4GB/sec for safety.
So with 4 of those you should be pushing somewhere north of
16GB/sec. With FDR IB and PCIe 3.0, I can certainly believe it's
possible, I think they've doubled the minimum RAM in the recent
proposal we had from them.
In our benchmarks we certainly found the M3's capable of it, for daily
use, our workloads are too mixed, we don't have anyone doing
sustained reads or writes on those types of files.
Might have to be a bit more expansive on your node configuration
though, I can get 2000 cores in 32 nodes these days, so that spec
would give you 512MB/sec per node if everyone is reading and
writing at once. If you're only doing 16 cores per node, then that's
125 nodes, and only 131MB/sec per node.
Matt.
More information about the gpfsug-discuss
mailing list