[gpfsug-discuss] Online data migration tool
Sven Oehme
oehmes at gmail.com
Fri Dec 22 00:02:43 GMT 2017
thats not how GPFS aehm Scale works :-)
each client has pre-allocated inodes in memory and creating files is a
matter of spooling records. yes, eventually you need to destage this to the
disk, but that happens only every few seconds and given this i/os are
usually very colocated so good storage cache technology can reduce i/os to
physical media significant.
to proof the point look at this numbers :
-- started at 10/17/2017 14:29:13 --
mdtest-1.9.3 was launched with 110 total task(s) on 11 node(s)
Command line used: /ghome/oehmes/mpi/bin/mdtest-pcmpi9131-existingdir -d
/ibm/fs2-16m-09/shared/mdtest-ec -i 1 -n 10000 -F -w 0 -Z -p 8 -N 11 -u
Path: /ibm/fs2-16m-09/shared
FS: 128.1 TiB Used FS: 0.2% Inodes: 476.8 Mi Used Inodes: 0.0%
110 tasks, 1100000 files
SUMMARY: (of 1 iterations)
Operation Max Min Mean
Std Dev
--------- --- --- ----
-------
File creation : 444221.343 444221.343 444221.343
0.000
File stat : 6704498.841 6704498.841 6704498.841
0.000
File read : 3859105.596 3859105.596 3859105.596
0.000
File removal : 409336.606 409336.606 409336.606
0.000
Tree creation : 5.344 5.344 5.344
0.000
Tree removal : 1.145 1.145 1.145
0.000
-- finished at 10/17/2017 14:29:27 --
this is a run against a 16mb blocksize filesystem with only spinning disks
(just one GL6 ESS) , not a single SSD and as you can see , this system on
11 nodes produces 444k creates / second something far above and beyond of
what drives can do.
and yes i know this stuff is all very complicated and not easy to explain
:-)
sven
On Thu, Dec 21, 2017 at 8:35 PM <valdis.kletnieks at vt.edu> wrote:
> On Thu, 21 Dec 2017 16:38:27 +0000, Sven Oehme said:
>
> > size. so if you now go to a 16MB blocksize and you have just 50 iops @
> 2MB
> > each you can write ~800 MB/sec with the exact same setup and same size
> > small writes, that's a factor of 8 .
>
> That's assuming your metadata storage is able to handle
> open/read/write/close
> on enough small files per second to push 800MB/sec. If you're talking
> 128K subblocks,
> you're going to need some 6,400 small files per second to fill that pipe...
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171222/a66aca84/attachment-0002.htm>
More information about the gpfsug-discuss
mailing list