[gpfsug-discuss] Alternative to Scale S3 API.

Nishaan Docrat NISHAAN at za.ibm.com
Wed Oct 28 07:45:45 GMT 2020


Hi Luis

Thanks for your reply.. It should address Andi's issue as the underlying
Swift version is what is important and the functionality he needs is in the
latest releases (I was told 5.1 includes Swift Train which is the latest
version).

Am sure there is a beta program for Spectrum Scale.. Perhaps Andi should
speak to his software sales rep and ask to be included on it to get access
so that he can test.

Kind Regards

Nishaan Docrat
Client Technical Specialist - Storage Systems
IBM Systems Hardware

Work: +27 (0)11 302 5001
Mobile: +27 (0)81 040 3793
Email: nishaan at za.ibm.com









From:	"Luis Bolinches" <luis.bolinches at fi.ibm.com>
To:	gpfsug-discuss at spectrumscale.org
Cc:	gpfsug-discuss at spectrumscale.org
Date:	2020/10/28 09:29
Subject:	[EXTERNAL] Re: [gpfsug-discuss] Alternative to Scale S3 API.
Sent by:	gpfsug-discuss-bounces at spectrumscale.org



Hi

5.1.x is going GA very soon (TM). Would it address the issues Andi sees on
his environment or not I cannot say.

I can take it with Andi for more details on the GA date


--
Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations /
Salutacions
Luis Bolinches
Consultant IT Specialist
IBM Spectrum Scale development
Mobile Phone: +358503112585

https://www.youracclaim.com/user/luis-bolinches

Ab IBM Finland Oy
Laajalahdentie 23
00330 Helsinki
Uusimaa - Finland

"If you always give you will always have" --  Anonymous



 ----- Original message -----
 From: "Nishaan Docrat" <NISHAAN at za.ibm.com>
 Sent by: gpfsug-discuss-bounces at spectrumscale.org
 To: Andi Christiansen <andi at christiansen.xxx>
 Cc: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
 Subject: [EXTERNAL] Re: [gpfsug-discuss] Alternative to Scale S3 API.
 Date: Wed, Oct 28, 2020 08:47



 Hi Andi

 The s3api middleware does support multipart uploads..

 https://docs.openstack.org/swift/latest/s3_compat.html

 The current version of Swift (PIKE) that is bundled with Spectrum Scale
 5.0.X doesn't..
 https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.5/com.ibm.spectrum.scale.v5r05.doc/bl1adm_ManagingOpenStackACLsviaAmazonS3API.htm


 According to the Spectrum Scale Roadmap, 5.1 is due out 2H20.. Not sure if
 someone from development can confirm the GA date.

 Does Veeam have a test utility? You could always test it using the current
 Swift AIO or if you can provide me with a test utility I can test that for
 you.

 Kind Regards

 Nishaan Docrat
 Client Technical Specialist - Storage Systems
 IBM Systems Hardware

 Work: +27 (0)11 302 5001
 Mobile: +27 (0)81 040 3793
 Email: nishaan at za.ibm.com




 Inactive hide details for Andi Christiansen ---2020/10/28 08:24:55---Hi
 Nishaan, Thanks for you reply.Andi Christiansen ---2020/10/28
 08:24:55---Hi Nishaan, Thanks for you reply.

 From: Andi Christiansen <andi at christiansen.xxx>
 To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>,
 Nishaan Docrat <NISHAAN at za.ibm.com>
 Date: 2020/10/28 08:24
 Subject: [EXTERNAL] Re: [gpfsug-discuss] Alternative to Scale S3 API.





 Hi Nishaan, Thanks for you reply. When you say 5.1? is that 5.1.x.x or
 5.0.5.1? Some...
                                                                            
                                                                            
                                                                            
                                                                            
                                                                            
                                                                            
                                                                            
               This Message Is From an External Sender                      
               This message came from outside your organization.            
                                                                            
                                                                            
                                                                            
                                                                            

 Hi Nishaan,

 Thanks for you reply.

 When you say 5.1? is that 5.1.x.x or 5.0.5.1?

 Some of the limitations we have encountered is the multipart upload not
 being supported and some md5sum that the s3 api does that veeam actually
 dont like. also interms of the management on the Scale GUI, that has to be
 on one of the S3 CES nodes in order to be able to show project, container
 etc... but when you have a HAProxy for enabling SSL then a GUI is not
 available as they both use port 443?


 i know min.io is not the full stack of S3 API commands but as far as i can
 read it comes with more features out of the box than Scale S3 does,
 multipart for an example...

 I looked through your documentation and its very close to what we have set
 up today and found to not work...

 If multipart uploads would be supported today on scale S3 i would think
 about still using scale for the s3 part but as i expect that you talk
 about 5.1.x.x i dont see that being released any time soon? and dont know
 if that is actually going to be supported in that release then i cant wait
 for that to happen..

 Best Regards
 Andi Christiansen
             On 10/27/2020 2:38 PM Nishaan Docrat <nishaan at za.ibm.com>
             wrote:



             Hi Andi

             The current S3 compatibility in Spectrum Scale is delivered
             via the Swift3 middleware. This middleware has since been
             replaced by s3api in later versions of Swift. Spectrum Scale
             5.1 will make use of Swift Train release which will include
             the new s3api middleware.

             I've tested the S3 compatibility with a few applications
             including Spectrum Scale itself (i.e. Cloud Data Sharing to
             another Scale Object store using S3 API) and Spectrum Protect
             etc. and haven't had any issue. I've also ran a few
             application tools to test for an S3 compliant object stores
             and again had no issues.

             You can use s3compat to test the current compatibility.. Or
             you can check here for the current compatibility..
             https://docs.openstack.org/swift/latest/s3_compat.html

             Not sure if there is any other way to talk HTTPS without using
             HAProxy.

             In any case, I've documented the process to setup an S3
             compliant object store including supporting virtual-hosted
             style bucket addressing which you can find here..

             https://www.linkedin.com/feed/update/urn:li:activity:6720227398756909056/


             Most storage vendors including minio would not support the
             full S3 API stack as alot of the calls are specific to AWS
             (like billing stuff etc.).

             Anyway, good luck with your testing.

             Kind Regards

             Nishaan Docrat
             Client Technical Specialist - Storage Systems
             IBM Systems Hardware

             Work: +27 (0)11 302 5001
             Mobile: +27 (0)81 040 3793
             Email: nishaan at za.ibm.com




             Inactive hide details for Andi Christiansen ---2020/10/27
             13:59:30---Hi all, We have over a longer period used the S3
             API withiAndi Christiansen ---2020/10/27 13:59:30---Hi all, We
             have over a longer period used the S3 API within spectrum
             Scale.. And that has shown that

             From: Andi Christiansen <andi at christiansen.xxx>
             To: "gpfsug-discuss at spectrumscale.org"
             <gpfsug-discuss at spectrumscale.org>
             Date: 2020/10/27 13:59
             Subject: [EXTERNAL] [gpfsug-discuss] Alternative to Scale S3
             API.
             Sent by: gpfsug-discuss-bounces at spectrumscale.org








             Hi all,



             We have over a longer period used the S3 API within spectrum
             Scale.. And that has shown that it does not support very many
             applications because of limitations of the API..



             Has anyone got any experience with any other product we can
             deploy on-top of Spectrum Scale that will give us a true S3
             API with full functionalities and able to answer on port 443?
             As of now we use HAProxy to forware ssl request back and forth
             from Scale S3 API.



             We have looked at MinIO which seems to be fairly simple and
             maybe might solve a lot of incompatibilities with clients
             software. But the product seems to be very badly documented at
             least for me.

             The idea is basically that a client uses their backup
             application(rubrik, veeam etc.) to connect to a domain(for
             example backup.mycompany.com) with their access and secret key
             and have access to their bucket only. and it must be over
             https/ssl.



             If someone has any knowledge to minio or any other product
             that might solve our problem I will be glad to hear from you!
             😊

             Thank you in advance!
             _______________________________________________
             gpfsug-discuss mailing list
             gpfsug-discuss at spectrumscale.org
             http://gpfsug.org/mailman/listinfo/gpfsug-discuss



             _______________________________________________
             gpfsug-discuss mailing list
             gpfsug-discuss at spectrumscale.org
             http://gpfsug.org/mailman/listinfo/gpfsug-discuss









 _______________________________________________
 gpfsug-discuss mailing list
 gpfsug-discuss at spectrumscale.org
 http://gpfsug.org/mailman/listinfo/gpfsug-discuss


Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
Oy IBM Finland Ab
PL 265, 00101 Helsinki, Finland
Business ID, Y-tunnus: 0195876-3
Registered in Finland
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201028/3cef7908/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 16781831.jpg
Type: image/jpeg
Size: 35643 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201028/3cef7908/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20201028/3cef7908/attachment-0002.gif>


More information about the gpfsug-discuss mailing list