<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-2022-jp">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:PMingLiU;
panose-1:2 1 6 1 0 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@PMingLiU";
panose-1:2 1 6 1 0 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:ZH-TW;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I’ve been experiencing this “start request repeated too quickly” issue, but IIRC for the pmsensors service instead, for instance when the GUI was set up against Spectrum Scale nodes on which the
gpfs.gss.pmsensors RPM was not properly installed. That is, something was misconfigured at the cluster level, and not necessarily on the node for which the service is failing. Your issue might point at something similar but on the other end of the spectrum
(sic).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">In this case the issue is usually resolved by deleting/recreating the performance monitoring configuration for the whole cluster:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">mmchnode --noperfmon -N all # required before deleting the perfmon config<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">mmperfmon config delete --all<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">mmperfmon config generate --collectors <GUINODES> # start the pmcollector service on the GUI nodes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">mmchnode --perfmon -N all # start the pmsensors service on all nodes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">It might work when targeting individual nodes instead, though again the problem might be caused by cluster inconsistencies.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">HTH<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">-- <o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Nicolas Calimet, PhD | HPC System Architect | Lenovo ISG | Meitnerstrasse 9, D-70563 Stuttgart, Germany | +49 71165690146 |
<a href="https://www.lenovo.com/dssg">https://www.lenovo.com/dssg</a><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org>
<b>On Behalf Of </b>Oesterlin, Robert<br>
<b>Sent:</b> Monday, November 15, 2021 19:44<br>
<b>To:</b> gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Subject:</b> [External] [gpfsug-discuss] Pmcollector fails to start<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any idea why pmcollector fails to start via service? If I start it manually, it runs just fine. Scale 5.1.1.4<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This worksfrom the command line: /opt/IBM/zimon/sbin/pmcollector -C /opt/IBM/zimon/ZIMonCollector.cfg -R /var/run/perfmon<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">“service pmcollector start” – fails:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Redirecting to /bin/systemctl status pmcollector.service<o:p></o:p></p>
<p class="MsoNormal">● pmcollector.service - zimon collector daemon<o:p></o:p></p>
<p class="MsoNormal"> Loaded: loaded (/usr/lib/systemd/system/pmcollector.service; enabled; vendor preset: disabled)<o:p></o:p></p>
<p class="MsoNormal"> Active: failed (Result: start-limit) since Mon 2021-11-15 13:22:34 EST; 10min ago<o:p></o:p></p>
<p class="MsoNormal"> Process: 2055 ExecStart=/opt/IBM/zimon/sbin/pmcollector -C /opt/IBM/zimon/ZIMonCollector.cfg -R /var/run/perfmon (code=exited, status=203/EXEC)<o:p></o:p></p>
<p class="MsoNormal">Main PID: 2055 (code=exited, status=203/EXEC)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Nov 15 13:22:33 nrg1-zimon1 systemd[1]: Unit pmcollector.service entered failed state.<o:p></o:p></p>
<p class="MsoNormal">Nov 15 13:22:33 nrg1-zimon1 systemd[1]: pmcollector.service failed.<o:p></o:p></p>
<p class="MsoNormal">Nov 15 13:22:34 nrg1-zimon1 systemd[1]: pmcollector.service holdoff time over, scheduling restart.<o:p></o:p></p>
<p class="MsoNormal">Nov 15 13:22:34 nrg1-zimon1 systemd[1]: Stopped zimon collector daemon.<o:p></o:p></p>
<p class="MsoNormal">Nov 15 13:22:34 nrg1-zimon1 systemd[1]: start request repeated too quickly for pmcollector.service<o:p></o:p></p>
<p class="MsoNormal">Nov 15 13:22:34 nrg1-zimon1 systemd[1]: Failed to start zimon collector daemon.<o:p></o:p></p>
<p class="MsoNormal">Nov 15 13:22:34 nrg1-zimon1 systemd[1]: Unit pmcollector.service entered failed state.<o:p></o:p></p>
<p class="MsoNormal">Nov 15 13:22:34 nrg1-zimon1 systemd[1]: pmcollector.service failed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">Bob Oesterlin </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:black">Sr Principal Storage Engineer<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:black">Nuance Communications</span><o:p></o:p></p>
</div>
</body>
</html>