[gpfsug-discuss] Mismatch between local and Scale directories

Tue Jun 30 14:22:09 BST 2020

Hi Renar,

Yes, not Hadoop directories but Yarn and mapreduce. They were configured as follows:

mmdsh -N $host "ln -s %GPFS_DIR/$host $LOCAL_DIR"

Yarn Namenode Directories are storing into $LOCAL_DIR. And yes I’m using transparency on the gpfs-nodes.

Thank you.

Best regards,

Pedro Real Baptista
Consulting | Analytics & Cognitive
Deloitte Consultores, S.A.
Av. Eng. Duarte Pacheco, 7, 1070-100 Lisboa, Portugal
M: +351 962369236
pebaptista at deloitte.pt<mailto:%20pebaptista at deloitte.pt> | www.deloitte.pt<http://www.deloitte.pt/>

[cid:image002.png at 01D64EE9.D259DF20]<http://www.deloitte.pt/>

From: gpfsug-discuss-bounces at spectrumscale.org <gpfsug-discuss-bounces at spectrumscale.org> On Behalf Of Grunenberg, Renar
Sent: 30 de junho de 2020 13:59
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: [EXT] Re: [gpfsug-discuss] Mismatch between local and Scale directories

Hallo Pedro,
what do you mean you had linked the local hadoop-Directory with the gpfs fs. Can you clarify? Do you use transparency on the gpfs-nodes?
You should use on the local-hadoop site hdfs dfs –ls <path> cmd’s here. No os cmds like ls only!

Renar Grunenberg
Abteilung Informatik - Betrieb

HUK-COBURG
Bahnhofsplatz
96444 Coburg
Telefon:
09561 96-44110
Telefax:
09561 96-44104
E-Mail:
Renar.Grunenberg at huk-coburg.de<mailto:Renar.Grunenberg at huk-coburg.de>
Internet:
www.huk.de<http://www.huk.de>
________________________________
HUK-COBURG Haftpflicht-Unterstützungs-Kasse kraftfahrender Beamter Deutschlands a. G. in Coburg
Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021
Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg
Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin.
Vorstand: Klaus-Jürgen Heitmann (Sprecher), Stefan Gronbach, Dr. Hans Olav Herøy, Dr. Jörg Rheinländer, Sarah Rössler, Daniel Thomas.
________________________________
Diese Nachricht enthält vertrauliche und/oder rechtlich geschützte Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese Nachricht irrtümlich erhalten haben,
informieren Sie bitte sofort den Absender und vernichten Sie diese Nachricht.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Nachricht ist nicht gestattet.

This information may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this information in error) please notify the
sender immediately and destroy this information.
Any unauthorized copying, disclosure or distribution of the material in this information is strictly forbidden.
________________________________
Von: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> Im Auftrag von Baptista, Pedro Real
Gesendet: Dienstag, 30. Juni 2020 14:29
An: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Betreff: Re: [gpfsug-discuss] Mismatch between local and Scale directories

Hi Yaron,

Thank you for your reply.

No, /var is not full and all other disks have free space also.

Best regards

Pedro Real Baptista
Consulting | Analytics & Cognitive
Deloitte Consultores, S.A.
Av. Eng. Duarte Pacheco, 7, 1070-100 Lisboa, Portugal
M: +351 962369236
pebaptista at deloitte.pt<mailto:%20pebaptista at deloitte.pt> | www.deloitte.pt<http://secure-web.cisco.com/1Qoaf_rxTf_RFpeXPJnbVzy5zgta4xW8L9Ai1aN4eWYPLU8N-OritaykbXDNmTUXbJq4lnoS0A9WUTGH-Ckm5RJWc-8uvoowHnPF96thPazDpO1u0VgB6kCAxBrk17BIlx1nJWKFhTsvgiRKeIZ7Y_vv7UlVS2PIs7uw7Tr1mJUja0z3TrM5MQaY-NLfL0Cvm5KMTNpJinPeJ1kYuNK4_YsWrTjw0YCpkRgj_Dfoq0oCfGneMPYV2l4dxxNQvo10A8kORorKsXNhu5Kp98n-VlripJQFp-HmHT15Rv5Ye7Ft1C_zhkSBDwRxy4NA3XnjOHBrAGWsix_wrkGoPSxvc6w/http%3A%2F%2Fwww.deloitte.pt%2F>

[cid:image020.png at 01D64EE8.4B08B790]<http://secure-web.cisco.com/1Qoaf_rxTf_RFpeXPJnbVzy5zgta4xW8L9Ai1aN4eWYPLU8N-OritaykbXDNmTUXbJq4lnoS0A9WUTGH-Ckm5RJWc-8uvoowHnPF96thPazDpO1u0VgB6kCAxBrk17BIlx1nJWKFhTsvgiRKeIZ7Y_vv7UlVS2PIs7uw7Tr1mJUja0z3TrM5MQaY-NLfL0Cvm5KMTNpJinPeJ1kYuNK4_YsWrTjw0YCpkRgj_Dfoq0oCfGneMPYV2l4dxxNQvo10A8kORorKsXNhu5Kp98n-VlripJQFp-HmHT15Rv5Ye7Ft1C_zhkSBDwRxy4NA3XnjOHBrAGWsix_wrkGoPSxvc6w/http%3A%2F%2Fwww.deloitte.pt%2F>

From: gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org> <gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>> On Behalf Of Yaron Daniel
Sent: 30 de junho de 2020 13:06
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Subject: [EXT] Re: [gpfsug-discuss] Mismatch between local and Scale directories

HI

what is the output of :
#df -h

Look like /, /var is full ?

Regards

________________________________

Yaron Daniel
 94 Em Ha'Moshavot Rd
[cid:image021.gif at 01D64EE8.4B08B790]
Storage Architect – IL Lab Services (Storage)
 Petach Tiqva, 49527
IBM Global Markets, Systems HW Sales
 Israel

Phone:
+972-3-916-5672

Fax:
+972-3-916-5672

Mobile:
+972-52-8395593

e-mail:
yard at il.ibm.com<mailto:yard at il.ibm.com>

Webex:            https://ibm.webex.com/meet/yard<Webex:%20%20%20%20%20%20%20%20%20%20%20%20%20https://ibm.webex.com/meet/yard%0dIBM%20Israel>
IBM Israel<Webex:%20%20%20%20%20%20%20%20%20%20%20%20%20https://ibm.webex.com/meet/yard%0dIBM%20Israel>

  [IBM Storage for Cyber Resiliency & Modern Data Protection V1] [cid:image023.jpg at 01D64EE8.4B08B790] [cid:image024.jpg at 01D64EE8.4B08B790] [cid:image025.jpg at 01D64EE8.4B08B790] [cid:image026.jpg at 01D64EE8.4B08B790]
[cid:image027.jpg at 01D64EE8.4B08B790][cid:image028.jpg at 01D64EE8.4B08B790][cid:image029.jpg at 01D64EE8.4B08B790] [IBM Storage and Cloud Essentials] [cid:image031.jpg at 01D64EE8.4B08B790]

From:        "Baptista, Pedro Real" <pebaptista at deloitte.pt<mailto:pebaptista at deloitte.pt>>
To:        "gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>" <gpfsug-discuss at spectrumscale.org<mailto:gpfsug-discuss at spectrumscale.org>>
Date:        30/06/2020 14:54
Subject:        [EXTERNAL] [gpfsug-discuss] Mismatch between local and Scale directories
Sent by:        gpfsug-discuss-bounces at spectrumscale.org<mailto:gpfsug-discuss-bounces at spectrumscale.org>
________________________________

Hi all,

I’m finding diferences between my local directories (Hadoop cluster) and GPFS filesystem. I’ve linked both yarn and mapreduce directories to Scale.

For example, in one specific worker node:

[cid:image032.png at 01D64EE8.4B08B790]

If I list the usercache folder, I see differences.

Local:

[cid:image033.png at 01D64EE8.4B08B790]

GPFS

[cid:image034.png at 01D64EE8.4B08B790]

I see that GPFS is working ok in the node

[cid:image035.png at 01D64EE8.4B08B790]

However, if I check the node health:

[cid:image036.png at 01D64EE8.4B08B790]

I’m new to Spectrum Scale and I don’t know what’s csm_resync_needed and local_fs_filled.

Can anyone give a hand with this?

Best regards,

Pedro Real Baptista
Consulting | Analytics & Cognitive
Deloitte Consultores, S.A.
Av. Eng. Duarte Pacheco, 7, 1070-100 Lisboa, Portugal
M: +351 962369236
pebaptista at deloitte.pt<mailto:%20pebaptista at deloitte.pt>| www.deloitte.pt<http://secure-web.cisco.com/1rJhl7ESWYrTmt1WoVdb6kWjihCNE8HMLybWnRnzY5Y1r-z_Inptagcam9z0AVh1nWVm8zad5VeQwzrz1QszH5nUOlyTGb2v4yzGu92aCricPjOoikjmy31svec85RmOYKkoKUgAj6wC-hoOhF7HX9rYgaKGfZS1mDeO_uq7izfjGCZu8ANk_v33Jrsg8WtVjARiBQVrCgTl0ghg-eH170fllK4e_8AsSRQ-agoa15wA76wzW1k83MCwTt9zJcIFR93_tOcl0i4VbjFJylSRZL5c0Ia7sO1wazIRbVG61y7Q_54jY_RhtWcMgYQP97GAB/http%3A%2F%2Fwww.deloitte.pt%2F>

[cid:image020.png at 01D64EE8.4B08B790]<http://secure-web.cisco.com/1rJhl7ESWYrTmt1WoVdb6kWjihCNE8HMLybWnRnzY5Y1r-z_Inptagcam9z0AVh1nWVm8zad5VeQwzrz1QszH5nUOlyTGb2v4yzGu92aCricPjOoikjmy31svec85RmOYKkoKUgAj6wC-hoOhF7HX9rYgaKGfZS1mDeO_uq7izfjGCZu8ANk_v33Jrsg8WtVjARiBQVrCgTl0ghg-eH170fllK4e_8AsSRQ-agoa15wA76wzW1k83MCwTt9zJcIFR93_tOcl0i4VbjFJylSRZL5c0Ia7sO1wazIRbVG61y7Q_54jY_RhtWcMgYQP97GAB/http%3A%2F%2Fwww.deloitte.pt%2F>

*Disclaimer:*

Deloitte refers to a Deloitte member firm, one of its related entities, or Deloitte Touche Tohmatsu Limited (“DTTL”).
Each Deloitte member firm is a separate legal entity and a member of DTTL. DTTL does not provide services to clients.
Please see www.deloitte.com/about<http://www.deloitte.com/about>to learn more.

Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer do not consent to Internet email for messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of my firm shall be understood as neither given nor endorsed by it._______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss<http://secure-web.cisco.com/1HMlNwUFuvF1Cxz4WEr5vNOzTCiigxD1EkNgqCxytj5F_2uhft0nUnqEsuaNiccKA5pGjtyNHgYytzvHetLNWkwCSOwN9vEGg3SiCRNiqnEFTU3sQWCUdRHvOTmxFaR8R4OH_UNsSiJyi_DXPVf-VeKsXEqw8UPLQeAatNNdzfwPtA10hBmG5-ZMvJELI5N6c8S_z8J4zy7sxgdEkLO-CzygqSZm_FRz13HXbplzA29PByMkU-iiBFjCtpQHfv5eIlcyfi7pMmWv38kzIpYmGKtN83ZtJD1nq0_llxQc8r5hBqx7ob2P4LslqIAFetkxx/http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss>

*Disclaimer:*

Deloitte refers to a Deloitte member firm, one of its related entities, or Deloitte Touche Tohmatsu Limited (“DTTL”).
Each Deloitte member firm is a separate legal entity and a member of DTTL. DTTL does not provide services to clients.
Please see www.deloitte.com/about<http://www.deloitte.com/about> to learn more.

Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer do not consent to Internet email for messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of my firm shall be understood as neither given nor endorsed by it.
*Disclaimer:*

Deloitte refers to a Deloitte member firm, one of its related entities, or Deloitte Touche Tohmatsu Limited (“DTTL”).
Each Deloitte member firm is a separate legal entity and a member of DTTL. DTTL does not provide services to clients.
Please see www.deloitte.com/about <http://www.deloitte.com/about> to learn more.

Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email. Please advise immediately if you or your employer do not consent to Internet email for messages of this kind. Opinions, conclusions and other information in this message that do not relate to the official business of my firm shall be understood as neither given nor endorsed by it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image020.png
Type: image/png
Size: 2030 bytes
Desc: image020.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0014.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image021.gif
Type: image/gif
Size: 1114 bytes
Desc: image021.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0006.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image022.gif
Type: image/gif
Size: 4105 bytes
Desc: image022.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0007.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image023.jpg
Type: image/jpeg
Size: 3847 bytes
Desc: image023.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0016.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image024.jpg
Type: image/jpeg
Size: 4266 bytes
Desc: image024.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0017.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image025.jpg
Type: image/jpeg
Size: 3747 bytes
Desc: image025.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0018.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image026.jpg
Type: image/jpeg
Size: 3793 bytes
Desc: image026.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0019.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image027.jpg
Type: image/jpeg
Size: 4301 bytes
Desc: image027.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0020.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image028.jpg
Type: image/jpeg
Size: 3739 bytes
Desc: image028.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0021.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image029.jpg
Type: image/jpeg
Size: 3855 bytes
Desc: image029.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0022.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image030.gif
Type: image/gif
Size: 4084 bytes
Desc: image030.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0008.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image031.jpg
Type: image/jpeg
Size: 3776 bytes
Desc: image031.jpg
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0023.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image032.png
Type: image/png
Size: 15587 bytes
Desc: image032.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0015.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image033.png
Type: image/png
Size: 11114 bytes
Desc: image033.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0016.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image034.png
Type: image/png
Size: 17038 bytes
Desc: image034.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0017.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image035.png
Type: image/png
Size: 5902 bytes
Desc: image035.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0018.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image036.png
Type: image/png
Size: 23157 bytes
Desc: image036.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0019.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 2030 bytes
Desc: image002.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20200630/6c5704fe/attachment-0020.png>