<html><body><p>The issue of "GNR as software" is a pretty convoluted mixture of technical, business, and resource constraints issues. While some of the technical issues can be discussed here, obviously the other considerations cannot be discussed in a public forum. So you won't be able to get a complete understanding of the situation by discussing it here.<br><br>> <tt>I understand the support concerns, but I naively thought that assuming <br>> the hardware meets a basic set of requirements (e.g. redundant sas <br>> paths, x type of drives) it would be fairly supportable with GNR. The <br>> DS3700 shelves are re-branded NetApp E-series shelves and pretty vanilla <br>> I thought.<br></tt><br><tt>Setting business issues aside, this is more complicated on the technical level than one may think. At present, GNR requires a set of twin-tailed external disk enclosures. This is not a particularly exotic kind of hardware, but it turns out that this corner of the storage world is quite insular. GNR has a very close relationship with physical disk devices, much more so than regular GPFS. In an ideal world, SCSI and SES standards are supposed to provide a framework which would allow software like GNR to operate on an arbitrary disk enclosure. In the real world, the actual SES implementations on various enclosures that we've been dealing with are, well, peculiar. Apparently SES is one of those standards where vendors feel a lot of freedom in "re-interpreting" the standard, and since typically enclosures talk to a small set of RAID controllers, there aren't bad enough consequences to force vendors to be religious about SES standard compliance. Furthermore, the SAS fabric topology in configurations with an external disk enclosures is surprisingly complex, and that complexity predictably leads to complex failures which don't exist in simpler configurations. Thus far, every single one of the five enclosures we've had a chance to run GNR on required some adjustments, workarounds, hacks, etc. And the consequences of a misbehaving SAS fabric can be quite dire. There are various approaches to dealing with those complications, from running a massive 3rd party hardware qualification program to basically declaring any complications from an unknown enclosure to be someone else's problem (how would ZFS deal with a SCSI reset storm due to a bad SAS expander?), but there's much debate on what is the right path to take. Customer input/feedback is obviously very valuable in tilting such discussions in the right direction.</tt><br><br><tt>yuri</tt><br><br><img width="16" height="16" src="cid:1__=07BB0AADDF8EE6C78f9e8a93df938690918c07B@" border="0" alt="Inactive hide details for Aaron Knister ---09/28/2016 06:44:23 PM---Thanks Everyone for your replies! (Quick disclaimer, these "><font color="#424282">Aaron Knister ---09/28/2016 06:44:23 PM---Thanks Everyone for your replies! (Quick disclaimer, these opinions are my own, and not those of my</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Aaron Knister <aaron.knister@gmail.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">gpfsug-discuss@spectrumscale.org, </font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">09/28/2016 06:44 PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [gpfsug-discuss] gpfs native raid</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><tt>Thanks Everyone for your replies! (Quick disclaimer, these opinions are <br>my own, and not those of my employer or NASA).<br><br>Not knowing what's coming at the NDA session, it seems to boil down to <br>"it ain't gonna happen" because of:<br><br>- Perceived difficulty in supporting whatever creative hardware <br>solutions customers may throw at it.<br><br>I understand the support concerns, but I naively thought that assuming <br>the hardware meets a basic set of requirements (e.g. redundant sas <br>paths, x type of drives) it would be fairly supportable with GNR. The <br>DS3700 shelves are re-branded NetApp E-series shelves and pretty vanilla <br>I thought.<br><br>- IBM would like to monetize the product and compete with the likes of <br>DDN/Seagate<br><br>This is admittedly a little disappointing. GPFS as long as I've known it <br>has been largely hardware vendor agnostic. To see even a slight shift <br>towards hardware vendor lockin and certain features only being supported <br>and available on IBM hardware is concerning. It's not like the software <br>itself is free. Perhaps GNR could be a paid add-on license for non-IBM <br>hardware? Just thinking out-loud.<br><br>The big things I was looking to GNR for are:<br><br>- end-to-end checksums<br>- implementing a software RAID layer on (in my case enterprise class) JBODs<br><br>I can find a way to do the second thing, but the former I cannot. <br>Requiring IBM hardware to get end-to-end checksums is a huge red flag <br>for me. That's something Lustre will do today with ZFS on any hardware <br>ZFS will run on (and for free, I might add). I would think GNR being <br>openly available to customers would be important for GPFS to compete <br>with Lustre. Furthermore, I had opened an RFE (#84523) a while back to <br>implement checksumming of data for non-GNR environments. The RFE was <br>declined because essentially it would be too hard and it already exists <br>for GNR. Well, considering I don't have a GNR environment, and hardware <br>vendor lock in is something many sites are not interested in, that's <br>somewhat of a problem.<br><br>I really hope IBM reconsiders their stance on opening up GNR. The <br>current direction, while somewhat understandable, leaves a really bad <br>taste in my mouth and is one of the (very few, in my opinion) features <br>Lustre has over GPFS.<br><br>-Aaron<br><br><br>On 9/1/16 9:59 AM, Marc A Kaplan wrote:<br>> I've been told that it is a big leap to go from supporting GSS and ESS<br>> to allowing and supporting native raid for customers who may throw<br>> together "any" combination of hardware they might choose.<br>><br>> In particular the GNR "disk hospital" functions...<br>> </tt><tt><a href="https://www.ibm.com/support/knowledgecenter/SSFKCN_3.5.0/com.ibm.cluster.gpfs.v3r5.gpfs200.doc/bl1adv_introdiskhospital.htm">https://www.ibm.com/support/knowledgecenter/SSFKCN_3.5.0/com.ibm.cluster.gpfs.v3r5.gpfs200.doc/bl1adv_introdiskhospital.htm</a></tt><tt><br>> will be tricky to support on umpteen different vendor boxes -- and keep<br>> in mind, those will be from IBM competitors!<br>><br>> That said, ESS and GSS show that IBM has some good tech in this area and<br>> IBM has shown with the Spectrum Scale product (sans GNR) it can support<br>> just about any semi-reasonable hardware configuration and a good slew of<br>> OS versions and architectures... Heck I have a demo/test version of GPFS<br>> running on a 5 year old Thinkpad laptop.... And we have some GSSs in the<br>> lab... Not to mention Power hardware and mainframe System Z (think 360,<br>> 370, 290, Z)<br>><br>><br>><br>><br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </tt><tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></tt><tt><br>><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></tt><tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></tt><tt><br><br></tt><br><BR>
</body></html>