<html><body><p><font size="2">Hello, </font><br><br><font size="2">The GUI tasks which are not daily tasks will start periodically at a random time.</font><br><br><font size="2">The exception are daily tasks which are defined at fixed start times.</font><br><font size="2">It seems this is the issue you are experiencing, as the HW_INVENTORY task only runs once a day adn starts at identical times on both GUI nodes.</font><br><font size="2">Tweaking the cache database is unfortunately not a workaround as the hard coded and fixed starting times will be reset for every GUI restart.</font><br><br><font size="2">I have created a task to address this issue in a future release. </font><br><font size="2">We could for example add a random delay to the daily tasks, or a fixed delay based on the number of GUI nodes that are active.</font><br><br><font size="2" face="Arial">Mit freundlichen Grüßen / Kind regards</font><br><br><b><font face="Arial">Dr. Markus Rohwedder</font></b><br><br><font size="1" face="Arial">Spectrum Scale GUI Development</font><table border="0" cellspacing="0" cellpadding="0"><tr valign="top"><td class="small_font" width="600" colspan="4" valign="middle"><hr width="100%" size="2" align="left" style="color:#0055AA; "></td></tr>
<tr valign="top"><td class="small_font" width="600" colspan="4" valign="middle"><img width="1" height="1" src="cid:1__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" border="0" alt=""></td></tr>
<tr valign="top"><td class="inner" width="39" valign="middle"><font size="1" color="#0055AA" face="Arial">Phone:</font></td><td class="inner" width="157" valign="middle"><font size="1" color="#0055AA" face="Arial">+49 162 4159920</font></td><td class="inner" width="246" valign="middle"><font size="1" color="#0055AA" face="Arial"> IBM Deutschland Research & Development</font></td><td width="158" rowspan="4" valign="middle"><div align="right"><img src="cid:2__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" width="158" height="40" align="bottom"></div></td></tr>
<tr valign="top"><td class="inner" width="39" valign="middle"><font size="1" color="#0055AA" face="Arial">E-Mail:</font></td><td class="inner" width="157" valign="middle"><font size="1" color="#0055AA" face="Arial">rohwedder@de.ibm.com</font></td><td class="inner" width="246" valign="middle"><font size="1" color="#0055AA" face="Arial"> Am Weiher 24</font></td></tr>
<tr valign="top"><td class="inner" width="39" valign="middle"><img width="1" height="1" src="cid:1__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" border="0" alt=""></td><td class="inner" width="157" valign="middle"><img width="1" height="1" src="cid:1__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" border="0" alt=""></td><td class="inner" width="246" valign="middle"><font size="1" color="#0055AA" face="Arial"> 65451 Kelsterbach</font></td></tr>
<tr valign="top"><td class="inner" width="39" valign="middle"><img width="1" height="1" src="cid:1__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" border="0" alt=""></td><td class="inner" width="157" valign="middle"><img width="1" height="1" src="cid:1__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" border="0" alt=""></td><td class="inner" width="246" valign="middle"><font size="1" color="#0055AA" face="Arial"> Germany</font></td></tr>
<tr valign="top"><td class="small_font" width="600" colspan="4" valign="middle"><hr width="100%" size="2" align="left" style="color:#0055AA; "></td></tr>
<tr valign="top"><td class="small_font" width="600" colspan="4" valign="middle"><img width="1" height="1" src="cid:1__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" border="0" alt=""></td></tr></table><br><img width="16" height="16" src="cid:3__=8FBB0E6DDFD8CC818f9e8a93df938690918c8FB@" border="0" alt="Inactive hide details for "Billich Heinrich Rainer (ID SD)" ---29.01.2020 14:41:31---Hello, Can I change the times at which th"><font size="2" color="#424282">"Billich Heinrich Rainer (ID SD)" ---29.01.2020 14:41:31---Hello, Can I change the times at which the GUI runs HW_INVENTORY and related tasks?</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">"Billich Heinrich Rainer (ID SD)" <heinrich.billich@id.ethz.ch></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">29.01.2020 14:41</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">[EXTERNAL] [gpfsug-discuss] gui_refresh_task_failed for HW_INVENTORY with two active GUI nodes</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><font face="Calibri">Hello,</font><br><font face="Calibri"> </font><br><font face="Calibri">Can I change the times at which the GUI runs HW_INVENTORY and related tasks?</font><br><font face="Calibri"> </font><br><font face="Calibri">we frequently get messages like</font><br><font face="Calibri"> </font><br><font face="Calibri"> gui_refresh_task_failed GUI WARNING 12 hours ago The following GUI refresh task(s) failed: HW_INVENTORY</font><br><font face="Calibri"> </font><br><font face="Calibri">The tasks fail due to timeouts. Running the task manually most times succeeds. We do run two gui nodes per cluster and I noted that both servers seem run the HW_INVENTORY at the exact same time which may lead to locking or congestion issues, actually the logs show messages like</font><br><font face="Calibri"> </font><br><font face="Calibri">EFSSA0194I Waiting for concurrent operation to complete.</font><br><font face="Calibri"> </font><br><font face="Calibri">The gui calls ‘rinv’ on the xCat servers. Rinv for a single little-endian server takes a long time – about 2-3 minutes , while it finishes in about 15s for big-endian server.</font><br><font face="Calibri"> </font><br><font face="Calibri">Hence the long runtime of rinv on little-endian systems may be an issue, too</font><br><font face="Calibri"> </font><br><font face="Calibri">We run 5.0.4-1 efix9 on the gui and ESS 5.3.4.1 on the GNR systems (5.0.3.2 efix4). We run a mix of ppc64 and ppc64le systems, which a separate xCat/ems server for each type. The GUI nodes are ppc64le.</font><br><font face="Calibri"> </font><br><font face="Calibri">We did see this issue with several gpfs version on the gui and with at least two ESS/xCat versions.</font><br><font face="Calibri"> </font><br><font face="Calibri">Just to be sure I did purge the Posgresql tables. </font><br><font face="Calibri"> </font><br><font face="Calibri">I did try</font><br><font face="Calibri"> </font><br><font face="Calibri">/usr/lpp/mmfs/gui/cli/lstasklog HW_INVENTORY</font><br><font face="Calibri">/usr/lpp/mmfs/gui/cli/runtask HW_INVENTORY –debug</font><br><font face="Calibri"> </font><br><font face="Calibri">And also tried to read the logs in /var/log/cnlog/mgtsrv/ - but they are difficult.</font><br><font face="Calibri"> </font><br><font face="Calibri">Thank you,</font><br><font face="Calibri"> </font><br><font face="Calibri">Heiner</font><br><font face="Calibri"> </font><br><font face="Calibri"> </font><br><font face="Calibri">-- </font><br><font color="#18376A" face="Arial">=======================</font><br><font color="#18376A" face="Arial">Heinrich Billich</font><br><font color="#18376A" face="Arial">ETH Zürich</font><br><font color="#18376A" face="Arial">Informatikdienste</font><br><font color="#18376A" face="Arial">Tel.: +41 44 632 72 56</font><br><font color="#18376A" face="Arial">heinrich.billich@id.ethz.ch</font><br><font color="#18376A" face="Arial">========================</font><br><font face="Calibri"> </font><br><font face="Calibri"> </font><br><font face="Calibri"><br><br><br><br></font><tt><font size="2">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><tt><font size="2"><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></font></tt><tt><font size="2"> <br></font></tt><br><br><BR>
</body></html>