[gpfsug-discuss] File system manager - won't change to new node

[Bryan Banister]

Hi Bob,

Also tracing waiters on the cluster can help you understand if there is something that is blocking this kind of operation.

Beyond the command output, which is usually too terse to understand what is actually happening, do the logs on the nodes in the cluster give you any further details about the operation?

Cheers,
-Bryan

From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> On Behalf Of Jordan Robertson
Sent: Friday, June 22, 2018 9:10 AM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] File system manager - won't change to new node

Note: External Email
________________________________
Two thoughts:

1) Has your config data update fully propagated after the mmchnode?  We've (rarely) seen some weird stuff happen when that process isn't complete yet, or if a node in question simply didn't get the update (try md5sum'ing the mmsdrfs file on nrg1-gpfs13 and compare to the cluster manager's md5sum, make sure the push process isn't still running, etc.).  If you see discrepancies, you could try an mmsdrrestore to get that node back into spec.

2) If everything looks fine; what are the chances you could simply try restarting GPFS on nrg1-gpfs13?  Might be particularly interesting to see what the cluster tries to do with the filesystem once that node is down.

-Jordan

On Fri, Jun 22, 2018 at 9:28 AM, Oesterlin, Robert <Robert.Oesterlin at nuance.com<mailto:Robert.Oesterlin at nuance.com>> wrote:
Yep. And nrg1-gpfs13 isn’t even a manager node anymore!

[root at nrg1-gpfs01 ~]# mmchmgr dataeng nrg1-gpfs05
Sending migrate request to current manager node 10.30.43.136 (nrg1-gpfs13).
Node 10.30.43.136 (nrg1-gpfs13) resigned as manager for dataeng.
Node 10.30.43.136 (nrg1-gpfs13) appointed as manager for dataeng.

2018-06-22_09:26:08.305-0400: [I] Command: mmchmgr /dev/dataeng nrg1-gpfs05.nrg1.us.grid.nuance.com<http://nrg1-gpfs05.nrg1.us.grid.nuance.com>
2018-06-22_09:26:09.178-0400: [N] Node 10.30.43.136 (nrg1-gpfs13) resigned as manager for dataeng.
2018-06-22_09:26:09.179-0400: [N] Node 10.30.43.136 (nrg1-gpfs13) appointed as manager for dataeng.
2018-06-22_09:26:09.179-0400: [I] Command: successful mmchmgr /dev/dataeng nrg1-gpfs05.nrg1.us.grid.nuance.com<http://nrg1-gpfs05.nrg1.us.grid.nuance.com>
2018-06-22_09:26:10.116-0400: [I] Node 10.30.43.136 (nrg1-gpfs13) completed take over for dataeng.

Bob Oesterlin
Sr Principal Storage Engineer, Nuance


From: <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> on behalf of "Buterbaugh, Kevin L" <Kevin.Buterbaugh at Vanderbilt.Edu<mailto:Kevin.Buterbaugh at Vanderbilt.Edu>>
Reply-To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Date: Friday, June 22, 2018 at 8:21 AM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Subject: [EXTERNAL] Re: [gpfsug-discuss] File system manager - won't change to new node

Hi Bob,

Have you tried explicitly moving it to a specific manager node?  That’s what I always do … I personally never let GPFS pick when I’m moving the management functions for some reason.  Thanks…

Kevin



_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


________________________________

Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential, or privileged information and/or personal data. If you are not the intended recipient, you are hereby notified that any review, dissemination, or copying of this email is strictly prohibited, and requested to notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request, or solicitation of any kind to buy, sell, subscribe, redeem, or perform any type of transaction of a financial product. Personal data, as defined by applicable data privacy laws, contained in this email may be processed by the Company, and any of its affiliated or related companies, for potential ongoing compliance and/or business-related purposes. You may have rights regarding your personal data; for information on exercising these rights or the Company’s treatment of personal data, please email datarequests at jumptrading.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180622/f7c9ec02/attachment-0002.htm>