[gpfsug-discuss] RDMA data from Zimon

[Kristy Kallback-Rose]

Thanks Eric. No one who is a ZIMon developer has jumped up to contradict this, so I’ll go with it :-)

Many thanks. This is helpful to understand where the data is coming from and would be a welcome addition to the documentation.

Cheers,
Kristy

> On Feb 15, 2018, at 9:08 AM, Eric Agar <agar at us.ibm.com> wrote:
> 
> Kristy,
> 
> I experimented a bit with this some months ago and looked at the ZIMon source code. I came to the conclusion that ZIMon is reporting values obtained from the IB counters (actually, delta values adjusted for time) and that yes, for port_xmit_data and port_rcv_data, one would need to multiply the values by 4 to make sense of them.
> 
> To obtain a port_xmit_data value, the ZIMon sensor first looks for /sys/class/infiniband/<ibdev>/ports/<port>/counters_ext/port_xmit_data_64, and if that is not found then looks for /sys/class/infiniband/<ibdev>/ports/<port>/counters/port_xmit_data. Similarly for other counters/metrics.
> 
> Full disclosure: I am not an IB expert nor a ZIMon developer.
> 
> I hope this helps.
> 
> 
> Eric M. Agar
> agar at us.ibm.com
> 
> 
> <graycol.gif>Kristy Kallback-Rose ---02/14/2018 08:47:59 PM---Hi, Can one of the IBMers tell me if port_xmit_data and port_rcv_data from Zimon can be interpreted
> 
> From: Kristy Kallback-Rose <kkr at lbl.gov>
> To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
> Date: 02/14/2018 08:47 PM
> Subject: [gpfsug-discuss] RDMA data from Zimon
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
> 
> 
> 
> 
> Hi,
> 
> Can one of the IBMers tell me if port_xmit_data and port_rcv_data from Zimon can be interpreted as RDMA Bytes/sec? Ideally, also how this data is being collected? I’m looking here: https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm <https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm>
> 
> But then I also look here: https://community.mellanox.com/docs/DOC-2751 <https://urldefense.proofpoint.com/v2/url?u=https-3A__community.mellanox.com_docs_DOC-2D2751&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=ataLzjZMTIJfHxa5qXIwQdCTq09CQveIxYNnxJQ5pgs&e=>
> 
> and see "Total number of data octets, divided by 4 (lanes), received on all VLs. This is 64 bit counter.” So I wasn’t sure if some multiplication by 4 was in order.
> 
> Please advise.
> 
> Cheers,
> Kristy_______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8&e= <https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8&e=>
> 
> 
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180305/c264e5b7/attachment-0001.htm>