<font size=2 face="sans-serif">Hi Scott,</font><br><br><font size=2 color=red face="sans-serif">>>- Should the number
of threads equal the number of NSDs for the file system? or equal to the
number of nodes? <br>>>- If I execute a large multi-threaded run of this tool from a single
node in the cluster, will that give me an accurate result of the performance
of the file system? </font><font size=3 color=red><br></font><br><font size=2 face="sans-serif">To add to Valdis's note, the answer
to above also depends on the node, network used for GPFS communication
between client and server, as well as storage performance capabilities
constituting the GPFS cluster/network/storage stack. </font><br><br><font size=2 face="sans-serif">As an example, if the storage subsystem
(including controller + disks) hosting the file-system can deliver ~20
GB/s and the networking between NSD client and server is FDR 56Gb/s Infiniband
(with verbsRdma = ~6GB/s). Assuming, one FDR-IB link (verbsPorts) is configured
per NSD server as well as client, then you could need minimum of 4 x NSD
servers (4 x 6GB/s ==> 24 GB/s) to saturate the backend storage. So,
you would need to run gpfsperf (or anyother parallel I/O benchmark) across
minimum of 4 x GPFS NSD clients to saturate the backend storage. You
can scale the gpfsperf thread counts (-th parameter) depending on access
pattern (buffered/dio etc) but this would only be able to drive load from
single NSD client node. If you would like to drive I/O load from multiple
NSD client nodes + synchronize the parallel runs across multiple nodes
for accuracy, then gpfsperf-mpi would be strongly recommended. You would
need to use MPI to launch gpfsperf-mpi across multiple NSD client nodes
and scale the MPI processes (across NSD clients with 1 or more MPI process
per NSD client) accordingly to drive the I/O load for good performance.
</font><br><br><font size=2 color=red face="sans-serif">>>The cluster that I
will be running this tool on will not have MPI installed and will have
multiple file systems in the cluster. </font><br><br><font size=2 face="sans-serif">Without MPI, alternative would be to
use ssh or pdsh to launch gpfsperf across multiple nodes however if there
are slow NSD clients then the performance may not be accurate (slow clients
taking longer and after faster clients finished it will get all the network/storage
resources skewing the performance analysis. You may also consider using
parallel Iozone as it can be run across multiple node using rsh/ssh with
combination of "-+m" and "-t" option. </font><br><br><a href=http://iozone.org/docs/IOzone_msword_98.pdf><font size=2 color=blue face="sans-serif">http://iozone.org/docs/IOzone_msword_98.pdf</font></a><br><br><font size=2 face="sans-serif">##</font><br><font size=2 face="sans-serif">-+m filename </font><br><font size=2 face="sans-serif">Use this file to obtain the configuration
informati</font><br><font size=2 face="sans-serif">on of the clients for cluster testing.
The file </font><br><font size=2 face="sans-serif">contains one line for each client. Each
line has th</font><br><font size=2 face="sans-serif">ree fields. The fields are space delimited.
A # </font><br><font size=2 face="sans-serif">sign in column zero is a comment line.
The first fi</font><br><font size=2 face="sans-serif">eld is the name of the client. The second
field is </font><br><font size=2 face="sans-serif">the path, on the client, for the working
directory </font><br><font size=2 face="sans-serif">where Iozone will execute. The third
field is the </font><br><font size=2 face="sans-serif">path, on the client, for the executable
Iozone. </font><br><font size=2 face="sans-serif">To use this option one must be able
to execute comm</font><br><font size=2 face="sans-serif">ands on the clients without being challenged
</font><br><font size=2 face="sans-serif">for a password. Iozone will start remote
execution </font><br><font size=2 face="sans-serif">by using “rsh"</font><br><br><font size=2 face="sans-serif">To use ssh, export RSH=/usr/bin/ssh
</font><br><br><font size=2 face="sans-serif">-t #</font><br><font size=2 face="sans-serif">Run Iozone in a throughput mode. This
option allows</font><br><font size=2 face="sans-serif"> the user to specify how </font><br><font size=2 face="sans-serif">many threads or processes to have active
during th</font><br><font size=2 face="sans-serif">e measurement.</font><br><font size=2 face="sans-serif">##</font><br><br><font size=2 face="sans-serif">Hope this helps,</font><br><font size=2 face="sans-serif">-Kums</font><br><br><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">valdis.kletnieks@vt.edu</font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">07/25/2017 07:59 PM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
Baseline testing GPFS with gpfsperf</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><tt><font size=2>On Tue, 25 Jul 2017 15:46:45 -0500, "Scott C
Batchelder" said:<br><br>> - Should the number of threads equal the number of NSDs for the file<br>> system? or equal to the number of nodes?<br><br>Depends on what definition of "throughput" you are interested
in. If your<br>configuration has 50 clients banging on 5 NSD servers, your numbers for
5<br>threads and 50 threads are going to tell you subtly different things...<br><br>(Basically, one thread per NSD is going to tell you the maximum that<br>one client can expect to get with little to no contention, while one<br>per client will tell you about the maximum *aggregate* that all 50<br>can get together - which is probably still giving each individual client<br>less throughput than one-to-one....)<br><br>We usually test with "exactly one thread total", "one thread
per server",<br>and "keep piling the clients on till the total number doesn't get
any bigger".<br><br>Also be aware that it only gives you insight to your workload performance
if<br>your workload is comprised of large file access - if your users are actually<br>doing a lot of medium or small files, that changes the results dramatically<br>as you end up possibly pounding on metadata more than the actual data....<br>[attachment "att0twxd.dat" deleted by Kumaran Rajaram/Arlington/IBM]
_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br><BR>