<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>Out of curiosity could you send us the output of "ibv_devinfo -v"?</div><div id="AppleMailSignature"><br></div><div id="AppleMailSignature">-Aaron<br><br>Sent from my iPhone</div><div><br>On Jun 10, 2017, at 06:55, Frank Tower <<a href="mailto:frank.tower@outlook.com">frank.tower@outlook.com</a>> wrote:<br><br></div><blockquote type="cite"><div>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;" dir="ltr">
<p>Hi everybody, </p>
<p><br>
</p>
<p>I don't get why one of our compute node cannot start GPFS over IB.</p>
<p><br>
</p>
<p>I have the following error: </p>
<p><br>
</p>
<p>[I] VERBS RDMA starting with verbsRdmaCm=no verbsRdmaSend=no verbsRdmaUseMultiCqThreads=yes verbsRdmaUseCompVectors=yes<br>
</p>
<p>[I] VERBS RDMA library libibverbs.so (version >= 1.1) loaded and initialized.<br>
</p>
<p>[I] VERBS RDMA verbsRdmasPerNode reduced from 1000 to 514 to match (nsdMaxWorkerThreads 512 + (nspdThreadsPerQueue 2 * nspdQueues 1)).</p>
<p>[I] VERBS RDMA parse verbsPorts mlx4_0/1</p>
<p>[W] VERBS RDMA parse error verbsPort mlx4_0/1 ignored due to device mlx4_0 not found</p>
<p>[I] VERBS RDMA library libibverbs.so unloaded.</p>
<p>[E] VERBS RDMA failed to start, no valid verbsPorts defined.</p>
<p><br>
</p>
<p><br>
</p>
<p>I'm using Centos 7.3, Kernel 3.10.0-514.21.1.el7.x86_64.</p>
<p><br>
</p>
<p>I have 2 infinibands card, both have an IP and working well.</p>
<p><br>
</p>
<p>[root@rdx110 ~]# ibstat -l</p>
<p>mlx4_0</p>
<p>mlx4_1</p>
<p>[root@rdx110 ~]#<br>
</p>
<p><br>
</p>
<p>I tried configuration with both card, and no one work with GPFS.</p>
<p><span style="font-size: 12pt;"><br>
</span></p>
<p><span style="font-size: 12pt;">I also tried with </span><span style="font-size: 12pt;">mlx4_0/1, but same problem.</span><br>
</p>
<p><span style="font-size: 12pt;"></span></p>
<p><span style="font-size: 12pt;"><br>
</span></p>
<p>Someone already have the issue ?</p>
<p><br>
</p>
<p>Kind Regards,</p>
<p>Frank</p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<p></p>
</div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>gpfsug-discuss mailing list</span><br><span>gpfsug-discuss at <a href="http://spectrumscale.org">spectrumscale.org</a></span><br><span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></span><br></div></blockquote></body></html>