[gpfsug-discuss] Anybody running GPFS over iSCSI?

Buterbaugh, Kevin L Kevin.Buterbaugh at Vanderbilt.Edu
Thu Dec 13 20:54:39 GMT 2018

Hi All,

Googling “GPFS and iSCSI” doesn’t produce a ton of hits!  But we are interested to know if anyone is actually using GPFS over iSCSI?

The reason why I’m asking is that we currently use an 8 Gb FC SAN … QLogic SANbox 5800’s, QLogic HBA’s in our NSD servers … but we’re seeing signs that, especially when we start using beefier storage arrays with more disks behind the controllers, the 8 Gb FC could be a bottleneck.

As many / most of you are already aware, I’m sure, while 16 Gb FC exists, there’s basically only one vendor in that game.  And guess what happens to prices when there’s only one vendor???  We bought our 8 Gb FC switches for approximately $5K apiece.  List price on a <vendor redacted mainly because I don’t even want to honor them by mentioning their name> 16 Gb FC switch - $40K.  Ouch.

So the idea of being able to use commodity 10 or 40 Gb Ethernet switches and HBA’s is very appealing … both from a cost and a performance perspective (last I checked 40 Gb was more than twice 16 Gb!).  Anybody doing this already?

As those of you who’ve been on this list for a while and don’t filter out e-mails from me (<grin>) already know, we have a much beefier Infortrend storage array we’ve purchased that I’m currently using to test various metadata configurations (and I will report back results on that when done, I promise).  That array also supports iSCSI, so I actually have our test cluster GPFS filesystem up and running over iSCSI.  It was surprisingly easy to set up.  But any tips, suggestions, warnings, etc. about running GPFS over iSCSI are appreciated!

Two things that I am already aware of are:  1) use jumbo frames, and 2) run iSCSI over it’s own private network.  Other things I should be aware of?!?

Thanks all…

Kevin Buterbaugh - Senior System Administrator
Vanderbilt University - Advanced Computing Center for Research and Education
Kevin.Buterbaugh at vanderbilt.edu<mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20181213/97ef5e4f/attachment-0001.htm>

More information about the gpfsug-discuss mailing list