[gpfsug-discuss] mmsysmon exception with pmcollector socket being absent

[Ragho Mahalingam]

Hi Frederick,

In our case the issue started appearing after upgrading from 5.0.4 to
5.1.1.  If you've recently upgraded, then the following may be useful.

Turns out that mmsysmon (gpfs-base package) requires the new
gpfs.gss.pmcollector (from zimon packages) to function correctly (the
AF_INET -> AF_UNIX switch seems to have happened between 5.0 and 5.1).  In
our case, we'd upgraded all the mandatory packages but had not upgraded the
optional ones; the mmsysmonc python libs appears to be updated by the
pmcollector package from my study.

If you're running >5.1, I'd suggest checking the versions of gpfs.gss.*
packages installed.  If gpfs.gss.pmcollector isn't installed, you'd
definitely need that to make this runaway logging stop.

Hope that helps!

Ragu

On Wed, Nov 10, 2021 at 5:40 AM Frederik Ferner <
frederik.ferner at diamond.ac.uk> wrote:

> Hi Ragu,
>
> have you ever received any reply to this or managed to solve it? We are
> seeing exactly the same error and it's filling up our logs. It seems all
> the monitoring data is still extracted, so I'm not sure when it
> started so not sure if this is related to any upgrade on our side, but
> it may have been going on for a while. We only noticed because the log
> file now is filling up the local log partition.
>
> Kind regards,
> Frederik
>
> On 26/08/2021 11:49, Ragho Mahalingam wrote:
> > We've been working on setting up mmperfmon; after creating a new
> > configuration with the new collector on the same manager node, mmsysmon
> > keeps throwing exceptions.
> >
> >   File "/usr/lpp/mmfs/lib/mmsysmon/container/PerfmonController.py", line
> > 123, in _getDataFromZimonSocket
> >     sock.connect(SOCKET_PATH)
> > FileNotFoundError: [Errno 2] No such file or directory
> >
> > Tracing this a bit, it appears that SOCKET_PATH is
> >  /var/run/perfmon/pmcollector.socket and this unix domain socket is
> absent,
> > even though pmcollector has started and is running successfully.
> >
> > Under what scenarios is pmcollector supposed to create this socket?  I
> > don't see any configuration for this in
> /opt/IBM/zimon/ZIMonCollector.cfg,
> > so I'm assuming the socket is automatically created when pmcollector
> starts.
> >
> > Any thoughts on how to debug and resolve this?
> >
> > Thanks, Ragu
>
> --
> Frederik Ferner (he/him)
> Senior Computer Systems Administrator (storage) phone: +44 1235 77 8624
> Diamond Light Source Ltd.                       mob:   +44 7917 08 5110
>
> SciComp Help Desk can be reached on x8596
>
>
> (Apologies in advance for the lines below. Some bits are a legal
> requirement and I have no control over them.)
>
> --
> This e-mail and any attachments may contain confidential, copyright and or
> privileged material, and are for the use of the intended addressee only. If
> you are not the intended addressee or an authorised recipient of the
> addressee please notify us of receipt by returning the e-mail and do not
> use, copy, retain, distribute or disclose the information in or attached to
> the e-mail.
> Any opinions expressed within this e-mail are those of the individual and
> not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
> attachments are free from viruses and we cannot accept liability for any
> damage which you may sustain as a result of software viruses which may be
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England
> and Wales with its registered office at Diamond House, Harwell Science and
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>

-- 
*Disclaimer: This email and any corresponding attachments may contain 
confidential information. If you're not the intended recipient, any 
copying, distribution, disclosure, or use of any information contained in 
the email or its attachments is strictly prohibited. If you believe to have 
received this email in error, please email security at pathai.com 
<mailto:security at pathai.com> immediately, then destroy the email and any 
attachments without reading or saving.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20211110/016a38b0/attachment-0002.htm>