<html><body><div style="font-family: arial, helvetica, sans-serif; font-size: 12pt; color: #000000"><div>Dear, <br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div> We have reinstall recently from gpfs 3.5 to SpectrumScale 4.2.3-6 version redhat 7.<br data-mce-bogus="1"></div><div> <br data-mce-bogus="1"></div><div> We are running two nsd servers and a a gui, there is no firewall on gpfs network, and selinux is disable, I have checked changing the manager and cluster manager node between server with the same result, server 01 always increase the CLOSE_WAIT :<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div> Node Daemon node name IP address Admin node name Designation<br>--------------------------------------------------------------------------------<br> 1 gpfs01.ifca.es 10.10.0.111 gpfs01.ifca.es quorum-manager-perfmon<br> 2 gpfs02.ifca.es 10.10.0.112 gpfs02.ifca.es quorum-manager-perfmon<br> 3 gpfsgui.ifca.es 10.10.0.60 gpfsgui.ifca.es quorum-perfmon<br>.......<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>Installation and configuration works fine, but now we see that one of the servers do not close the mmfsd connections and this growing for ever while the othe nsd servers is always in the same range:<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>[root@gpfs01 ~]# netstat -putana | grep 1191 | wc -l<br>19701<br></div><div>[root@gpfs01 ~]# netstat -putana | grep 1191 | grep CLOSE_WAIT| wc -l<br>19528<br></div><div>....<br data-mce-bogus="1"></div><div>[root@gpfs02 ~]# netstat -putana | grep 1191 | wc -l<br>215<br>[root@gpfs02 ~]# netstat -putana | grep 1191 | grep CLOSE_WAIT| wc -l<br>0<br></div><div><br data-mce-bogus="1"></div><div>this is causing that gpfs01 do not answer to cluster commands<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>NSD are balance between server (same size):<br data-mce-bogus="1"></div><div>[root@gpfs02 ~]# mmlsnsd<br><br> File system Disk name NSD servers <br>---------------------------------------------------------------------------<br> gpfs nsd1 gpfs01,gpfs02 <br> gpfs nsd2 gpfs01,gpfs02 <br> gpfs nsd3 gpfs02,gpfs01 <br> gpfs nsd4 gpfs02,gpfs01<br>.....<br></div><div><br data-mce-bogus="1"></div><div>proccess seems to be similar in both servers, only mmccr is running on server 1 and not in 2<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>gpfs01<br data-mce-bogus="1"></div><div>#######<br data-mce-bogus="1"></div><div>root 9169 1 0 feb07 ? 22:27:54 python /usr/lpp/mmfs/bin/mmsysmon.py<br>root 11533 6154 0 13:41 ? 00:00:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmsdrquery sdrq_fs_info all<br>root 11713 1 0 13:41 ? 00:00:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmccrmonitor 15<br>root 12367 11533 0 13:43 ? 00:00:00 /usr/lpp/mmfs/bin/mmccr vget mmRunningCommand<br>root 12641 6162 0 13:44 ? 00:00:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmsdrquery sdrq_nsd_info sdrq_nsd_name:sdrq_fs_name:sdrq_storage_pool<br>root 12668 12641 0 13:44 ? 00:00:00 /usr/lpp/mmfs/bin/mmccr fget -c 835 mmsdrfs /var/mmfs/gen/mmsdrfs.12641<br>root 12950 11713 0 13:44 ? 00:00:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmccrmonitor 15<br>root 12959 9169 13 13:44 ? 00:00:00 /usr/lpp/mmfs/bin/mmccr check -Y -e<br>root 12968 3150 0 13:45 pts/3 00:00:00 grep --color=auto mm<br>root 19620 26468 38 jun14 ? 11:28:36 /usr/lpp/mmfs/bin/mmfsd<br>root 19701 2 0 jun14 ? 00:00:00 [mmkproc]<br>root 19702 2 0 jun14 ? 00:00:00 [mmkproc]<br>root 19703 2 0 jun14 ? 00:00:00 [mmkproc]<br>root 26468 1 0 jun05 ? 00:00:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/runmmfs</div><div><br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>[root@gpfs02 ~]# ps -feA | grep mm<br>root 5074 1 0 feb07 ? 01:00:34 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmccrmonitor 15<br>root 5128 31456 28 jun14 ? 06:18:07 /usr/lpp/mmfs/bin/mmfsd<br>root 5255 2 0 jun14 ? 00:00:00 [mmkproc]<br>root 5256 2 0 jun14 ? 00:00:00 [mmkproc]<br>root 5257 2 0 jun14 ? 00:00:00 [mmkproc]<br>root 15196 5074 0 13:47 ? 00:00:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/mmccrmonitor 15<br>root 15265 13117 0 13:47 pts/0 00:00:00 grep --color=auto mm<br>root 31456 1 0 jun05 ? 00:00:00 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/runmmfs<br></div><div><br data-mce-bogus="1"></div><div>Any idea will be appreciated.<br data-mce-bogus="1"></div><div>Regards, I<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div></div></body></html>