[gpfsug-discuss] Announcing IO-500 and soliciting submissions

[John Bent]

Hello GPFS community,

After BoFs at last year's SC and the last two ISC's, the IO-500 is
formalized and is now accepting submissions in preparation for our first
IO-500 list at this year's SC BoF:
http://sc17.supercomputing.org/presentation/?id=bof108&sess=sess319

The goal of the IO-500 is simple: to improve parallel file systems by
ensuring that sites publish results of both "hero" and "anti-hero" runs and
by sharing the tuning and configuration they applied to achieve those
results.

After receiving feedback from a few trial users, the framework is
significantly improved:
> git clone https://github.com/VI4IO/io-500-dev
> cd io-500-dev
> ./utilities/prepare.sh
> ./io500.sh
> # tune and rerun
> # email results to submit at io500.org

This, perhaps with a bit of tweaking and please consult our 'doc' directory
for troubleshooting, should get a very small toy problem up and running
quickly.  It then does become a bit challenging to tune the problem size as
well as the underlying file system configuration (e.g. striping parameters)
to get a valid, and impressive, result.

The basic format of the benchmark is to run both a "hero" and "antihero"
IOR test as well as a "hero" and "antihero" mdtest.  The write/create phase
of these tests must last for at least five minutes to ensure that the test
is not measuring cache speeds.

One of the more challenging aspects is that there is a requirement to
search through the metadata of the files that this benchmark creates.
Currently we provide a simple serial version of this test (i.e. the GNU
find command) as well as a simple python MPI parallel tree walking
program.  Even with the MPI program, the find can take an extremely long
amount of time to finish.  You are encouraged to replace these provided
tools with anything of your own devise that satisfies the required
functionality.  This is one area where we particularly hope to foster
innovation as we have heard from many file system admins that metadata
search in current parallel file systems can be painfully slow.

Now is your chance to show the community just how awesome we all know GPFS
to be.  We are excited to introduce this benchmark and foster this
community.  We hope you give the benchmark a try and join our community if
you haven't already.  Please let us know right away in any of our various
communications channels (as described in our documentation) if you
encounter any problems with the benchmark or have questions about tuning or
have suggestions for others.

We hope to see your results in email and to see you in person at the SC BoF.

Thanks,

IO 500 Committee
John Bent, Julian Kunkle, Jay Lofstead
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171027/b36e544a/attachment-0001.htm>