<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Tejas,
<div class=""><br class="">
</div>
<div class="">One thing that I have had occasional success with is a lazy unmount - i.e. “unmount -l /gpfs01”. Doesn’t help more of the time than it does help in my experience, but it’s one thing I try before bouncing a node. HTHAL…</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Jun 29, 2016, at 10:43 AM, Tejas Rao <<a href="mailto:raot@bnl.gov" class="">raot@bnl.gov</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Hi all,<br class="">
<br class="">
I cannot get the filesystem mounted again on few nodes.<br class="">
<br class="">
[root@ ~]# /usr/lpp/mmfs/bin/mmmount gpfs01<br class="">
Wed Jun 29 11:29:50 EDT 2016: mmmount: Mounting file systems ...<br class="">
mount: /dev/gpfs01 already mounted or /gpfs01 busy<br class="">
mmmount: Command failed. Examine previous error messages to determine cause.<br class="">
<br class="">
<br class="">
<br class="">
Things I have tried and checked.<br class="">
<br class="">
<br class="">
<br class="">
1. Checked /proc/mounts and /etc/mtab and lsof /dev/gpfs01 and both /dev/gpfs01 and /gpfs01 are not seen anywhere.<br class="">
<br class="">
2. Tried mmshutdown/mmstartup and it fails to unload mmfs modules. See below.<br class="">
<br class="">
3. mmumount gpfs01 -f<br class="">
<br class="">
<br class="">
A reboot of the client would probably fix it but as the client is used in production, this is not easy to do.<br class="">
<br class="">
<br class="">
Are there any tricks/hacks to remount the filesystem again without rebooting the client.<br class="">
<br class="">
<br class="">
[root@~]# /usr/lpp/mmfs/bin/mmshutdown<br class="">
Wed Jun 29 11:10:52 EDT 2016: mmshutdown: Starting force unmount of GPFS file systems<br class="">
Wed Jun 29 11:10:57 EDT 2016: mmshutdown: Shutting down GPFS daemons<br class="">
Shutting down!<br class="">
'shutdown' command about to kill process 11431<br class="">
Unloading modules from /lib/modules/2.6.32-504.8.1.el6.x86_64/extra<br class="">
mmfsenv: Module mmfslinux is still in use.<br class="">
Unmount all GPFS file systems and issue the command:<br class="">
mmfsadm cleanup<br class="">
Unloading module mmfs26<br class="">
ERROR: Module mmfs26 is in use<br class="">
mmfsenv: Error unloading module mmfs26.<br class="">
Unloading module mmfslinux<br class="">
ERROR: Module mmfslinux is in use by mmfs26<br class="">
mmfsenv: Error unloading module mmfslinux.<br class="">
Wed Jun 29 11:11:03 EDT 2016: mmshutdown: Finished<br class="">
[root@~]#<br class="">
<br class="">
Thanks.<br class="">
_______________________________________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class="">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div class="">—</div>
<div class="">Kevin Buterbaugh - Senior System Administrator</div>
<div class="">Vanderbilt University - Advanced Computing Center for Research and Education</div>
<div class=""><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" class="">Kevin.Buterbaugh@vanderbilt.edu</a> - (615)875-9633</div>
<div class=""><br class="">
</div>
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</body>
</html>