[gpfsug-discuss] mmap I/O on GPFS

[Matthew Wallis]

Hi Stuart,

On 16/05/2013, at 2:16 AM, Stuart Barkley <stuartb at 4gh.net> wrote:

> Has anyone done mmap() I/O with GPFS?

We've used it with GPFS under Windows, and I believe it's generally a Bad Idea™.

<Not A Programmer>
From what I understand of mmap() I/O, you're basically opening up a file handle _on disk_
that maps to a location in memory. The problem is that those file handles are 4K in size.
</Not A Programmer>

No striped filesystem likes 4K files. The application we were supporting was dealing with
5000 or so 110MB files. Initially GPFS performance was worse than a system with a single 
disk. After we convinced them to ditch mmap() I/O, our system became "the fastest we've
ever run this on." So fast in fact, that we found all the other limitations on scaling that they'd
built into their system.

So for us mmap() IO became something of a curse word. Your mileage may vary.

Matt.


--
Matthew Wallis, Senior HPC Systems Administrator
Victorian Partnership for Advanced Computing.
Ph: +61 3 9925 4452  Fax: +61 3 9925 4647