[gpfsug-discuss] EDR and omnipath

[Aaron Knister]

Ah, that makes complete sense. Thanks!

I had been doing some reading about OmniPath and for some reason was 
under the impression the OmniPath adapter could load itself as a driver 
under the verbs stack of OFED. Even so, that raises support concerns as 
you say.

I wonder what folks are doing who have IB-based block storage fabrics 
but wanting to connect to OmniPath-based fabrics?

I'm also curious how GNR customers would be able to serve both IB-based 
and an OmniPath-based fabrics over RDMA where performance is best? This 
is is along the lines of my GPFS protocol router question from the other 
day.

-Aaron

On 9/19/16 4:43 PM, Sven Oehme wrote:
> Because they both require a different distribution of OFED, which are
> mutual exclusive to install.
> in theory if you deploy plain OFED it might work, but that will be hard
> to find somebody to support.
>
>
> Sent from IBM Verse
>
> Aaron Knister --- Re: [gpfsug-discuss] EDR and omnipath ---
>
> From:	"Aaron Knister" <aaron.s.knister at nasa.gov>
> To:	gpfsug-discuss at spectrumscale.org
> Date:	Mon, Sep 19, 2016 1:35 PM
> Subject:	Re: [gpfsug-discuss] EDR and omnipath
>
> ------------------------------------------------------------------------
>
> I must admit, I'm curious as to why one cannot use GPFS with IB and OPA
> both in RDMA mode. Granted, I know very little about OPA but if it just
> presents as another verbs device I wonder why it wouldn't "Just work" as
> long as GPFS is configured correctly.
>
> -Aaron
>
> On 9/19/16 3:11 AM, Vic Cornell wrote:
>> Bump
>>
>> I can see no reason why that wouldn't work. But it would be nice to a
>> have an official answer or evidence that it works.
>>
>> Vic
>>
>>
>>> On 15 Sep 2016, at 5:49 pm, Brian Marshall <mimarsh2 at vt.edu
>>> <mailto:mimarsh2 at vt.edu>> wrote:
>>>
>>> All,
>>>
>>> I see in the GPFS FAQ A6.3 the statement below.  Is it possible to
>>> have GPFS do RDMA over EDR infiniband and non-RDMA communication over
>>> omnipath (IP over fabric) when each NSD server has an EDR card and a
>>> OPA card installed?
>>>
>>>
>>>
>>> RDMA is not supported on a node when both Mellanox HCAs and Intel
>>> Omni-Path HFIs are enabled for RDMA.
>>>
>>> _______________________________________________
>>> gpfsug-discuss mailing list
>>> gpfsug-discuss at spectrumscale.org <http://spectrumscale.org>
>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>>
>>
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>
> --
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>

-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776