[gpfsug-discuss] Spectrum Scale next generation hadoop connector is available for public download

Frank Kraemer kraemerf at de.ibm.com
Wed Dec 2 18:14:49 GMT 2015



Spectrum Scale next generation hadoop connector is available for public
download

from YONG ZHENG,

Spectrum Scale HDFS transparency (or HDFS protocol), the next generation
Hadoop connector, is available for public download from IBM developerWorks:

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20%28GPFS%29/page/Hadoop%20Connector%20Download%20%26%20Info

The 1st release 2.7.0-0 supports FPO storage model (internal disk model).
For shared storage model (e.g. ESS, SAN-based storage), it's in progress
and will be available around Jan, 2016.

The key advantage for HDFS transparency includes:
1.GPFS client free(it doesn't need every Hadoop node to install GPFS
client)
2.Full kerberos support for enterprise security requirement
3.make some hdfs hard-coded components working(e.g. Impala which will call
HDFS client directly instead calling Hadoop FileSystem interface)
4.make some popular features working: discp, webhdfs, multiple hadoop
cluster over the same Spectrum Scale file systems etc

refer the above link for more details.

-frank-

Frank Kraemer
IBM Consulting IT Specialist  / Client Technical Architect
Hechtsheimer Str. 2, 55131 Mainz
mailto:kraemerf at de.ibm.com
voice: +49171-3043699
IBM Germany
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20151202/9148490c/attachment-0001.htm>


More information about the gpfsug-discuss mailing list