[gpfsug-discuss] control which hosts become token manager

Bryan Banister bbanister at jumptrading.com
Fri Jul 27 18:09:56 BST 2018 

Yes, the token managers will reside on the NSD Server Cluster which has the NSD Servers that provide access to the underlying data and metadata storage.  I believe that all nodes that have the “manager” designation will participate in the token management operations as needed.  Though there is not a way to specify which node will be assigned the primary file system manager or overall cluster manager, which are two different roles but may reside on the same node.

Tokens themselves, however, are distributed and managed by clients directly.  When a file is first opened then the node that opened the file will be the “metanode” for the file, and all metadata updates on the file will be handled by this metanode until it closes the file handle, in which case another node will become the “metanode”.  For byte range locking, the file system manager will handle revoking tokens from nodes that have a byte range lock when another node requests access to the same byte range region.  This ensures that nodes cannot hold byte range locks that prevent other nodes from accessing byte range regions of a file.

Hope that helps,
-Bryan

From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> On Behalf Of Billich Heinrich Rainer (PSI)
Sent: Friday, July 27, 2018 11:50 AM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] control which hosts become token manager

Note: External Email
________________________________
Hello,

So probably I was wrong from the beginning –  please can somebody clarify: In a multicluster environment with all storage and filesystem hosted by a single cluster all token managers will reside in this central cluster?

Or are there also token managers in the storage-less clusters which just mount? This managers wouldn’t be accessible by all nodes which access the file system, hence I doubt this exists.

Still it would be nice to know how to influence the token manager placement and how to exclude certain machines. And the output of ‘mmdiag –tokenmgr’ indicates that there _are_ token manager in the remote-mounting cluster – confusing.

I would greatly appreciate if somebody could sort this out. A point to the relevant documentation would also be welcome.

Thank you & Kind regards,

Heiner

--
Paul Scherrer Institut
Science IT
Heiner Billich
WHGA 106
CH 5232  Villigen PSI
056 310 36 02
https://www.psi.ch


From: <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> on behalf of "Billich Heinrich Rainer (PSI)" <heiner.billich at psi.ch<mailto:heiner.billich at psi.ch>>
Reply-To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Date: Friday 27 July 2018 at 17:33
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Subject: Re: [gpfsug-discuss] control which hosts become token manager

Thank you,

The cluster was freshly set up and the VM node never was denoted as manager, it was created as quorum-client. What I didn’t mention but probably should have: This is a multicluster mount, the cluster has no own storage. Hence the filesystem manager are on the home cluster, according to mmlsmgr. Hm, probably more complicated as I initially thought. Still I would expect that for file-access that is restricted to this cluster all token management is handled inside the cluster, too? And I don’t want the weakest node to participate.

Kind regards,

Heiner

--
Paul Scherrer Institut
Science IT
Heiner Billich
WHGA 106
CH 5232  Villigen PSI
056 310 36 02
https://www.psi.ch


From: <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> on behalf of Bryan Banister <bbanister at jumptrading.com<mailto:bbanister at jumptrading.com>>
Reply-To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Date: Tuesday 24 July 2018 at 23:12
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Subject: Re: [gpfsug-discuss] control which hosts become token manager

Agree with Peter here.  And if the file system and workload are of significant size then isolating the token manager to a dedicated node is definitely best practice.

Unfortunately there isn’t a way to specify a preferred manager per FS… (Bryan starts typing up a new RFE…).

Cheers,
-Bryan

From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> On Behalf Of Peter Childs
Sent: Tuesday, July 24, 2018 2:29 PM
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Subject: Re: [gpfsug-discuss] control which hosts become token manager

Note: External Email
________________________________

What does mmlsmgr show?

Your config looks fine.

I suspect you need to do a

mmchmgr perf node-1.psi.ch<http://node-1.psi.ch>
mmchmgr tiered node-2.psi.ch<http://node-2.psi.ch>

It looks like the node was set up as a manager and was demoted to just quorum but since its still currently the manager it needs to be told to stop.

From experience it's also worth having different file system managers on different nodes, if at all possible.

But that's just a guess without seeing the output of mmlsmgr.


Peter Childs
Research Storage
ITS Research and Teaching Support
Queen Mary, University of London


---- Billich Heinrich Rainer (PSI) wrote ----
Hello,

I want to control which nodes can become token manager. In detail I run a virtual machine as quorum node. I don’t want this machine to become a token manager - it has no access to Infiniband and only very limited memory.

What I see is that ‘mmdiag –tokenmgr’ lists the machine as active token manager. The machine has role ‘quorum-client’. This doesn’t seem sufficient to exclude it.

Is there any way to tell spectrum scale to exclude this single machine with role quorum-client?

I run 5.0.1-1.

Sorry if this is a faq, I did search  quite a bit before I wrote to the list.

Thank you,

Heiner Billich


[root at node-2 ~]# mmlscluster

GPFS cluster information
========================
  GPFS cluster name:         node.psi.ch
  GPFS cluster id:           5389874024582403895
  GPFS UID domain:           node.psi.ch
  Remote shell command:      /usr/bin/ssh
  Remote file copy command:  /usr/bin/scp
  Repository type:           CCR

Node  Daemon node name       IP address     Admin node name        Designation
--------------------------------------------------------------------------------
   1   node-1.psi.ch       a.b.95.31  node-1.psi.ch       quorum-manager
   2   node-2.psi.ch       a.b.95.32  node-2.psi.ch       quorum-manager
   3   node-quorum.psi.ch  a.b.95.30  node-quorum.psi.ch  quorum                       <<<< VIRTUAL MACHINE >>>>>>>>>

[root at node-2 ~]# mmdiag --tokenmgr

=== mmdiag: tokenmgr ===
  Token Domain perf
    There are 3 active token servers in this domain.
    Server list:
      a.b.95.120
      a.b.95.121
      a.b.95.122    <<<< VIRTUAL MACHINE >>>>>>>>>
  Token Domain tiered
    There are 3 active token servers in this domain.
    Server list:
      a.b.95.120
      a.b.95.121
      a.b.95.122   <<<< VIRTUAL MACHINE >>>>>>>>>

--
Paul Scherrer Institut
Science IT
Heiner Billich
WHGA 106
CH 5232  Villigen PSI
056 310 36 02
https://www.psi.ch


________________________________

Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential, or privileged information and/or personal data. If you are not the intended recipient, you are hereby notified that any review, dissemination, or copying of this email is strictly prohibited, and requested to notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request, or solicitation of any kind to buy, sell, subscribe, redeem, or perform any type of transaction of a financial product. Personal data, as defined by applicable data privacy laws, contained in this email may be processed by the Company, and any of its affiliated or related companies, for potential ongoing compliance and/or business-related purposes. You may have rights regarding your personal data; for information on exercising these rights or the Company’s treatment of personal data, please email datarequests at jumptrading.com<mailto:datarequests at jumptrading.com>.


________________________________

Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential, or privileged information and/or personal data. If you are not the intended recipient, you are hereby notified that any review, dissemination, or copying of this email is strictly prohibited, and requested to notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request, or solicitation of any kind to buy, sell, subscribe, redeem, or perform any type of transaction of a financial product. Personal data, as defined by applicable data privacy laws, contained in this email may be processed by the Company, and any of its affiliated or related companies, for potential ongoing compliance and/or business-related purposes. You may have rights regarding your personal data; for information on exercising these rights or the Company’s treatment of personal data, please email datarequests at jumptrading.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20180727/c0579944/attachment-0002.htm>

More information about the gpfsug-discuss mailing list