<font size=2 face="sans-serif">there's a config parameter </font><font size=2 color=blue face="sans-serif">mnodeRemoteAccessThreshold</font><font size=2 face="sans-serif">,
which defines how many remote accesses to a file (i think remote means
"not from the mnode itself") happen, before it is asked to resign
from metanode ownership. i believe the node accessing the file next will
be taking over ownership, and be the next metanode. </font><br><font size=1 face="Arial"><br>Mit freundlichen Grüßen / Kind regards</font><p><font size=2 face="Arial"><b>Achim Rehor</b></font><p><table width=600 style="border-collapse:collapse;"><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=center><hr noshade></div><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 face="Arial"> </font><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 face="Arial">Software
Technical Support Specialist AIX/ Emea HPC Support</font><td width=103 rowspan=10 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=right><img align=bottom src=cid:_1_DC17B9ACDC17B42C0044AF9BC12582C7 style="border:0px solid;"></div><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 face="Arial">IBM
Certified Advanced Technical Expert - Power Systems with AIX</font><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 face="Arial">TSCC
Software Service, Dept. 7922</font><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 face="Arial">Global
Technology Services </font><tr height=8><td width=496 colspan=3 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=center><hr noshade></div><tr height=8><td width=44 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial">Phone:</font><td width=197 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial">+49-7034-274-7862</font><td width=255 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> IBM
Deutschland</font><tr height=8><td width=44 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial">E-Mail:</font><td width=197 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial">Achim.Rehor@de.ibm.com</font><td width=255 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> Am
Weiher 24</font><tr height=8><td width=44 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> </font><td width=197 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> </font><td width=255 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> 65451
Kelsterbach</font><tr height=8><td width=44 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> </font><td width=197 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> </font><td width=255 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> Germany</font><tr height=8><td width=44 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> </font><td width=197 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> </font><td width=255 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#0060a0 face="Arial"> </font><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><div align=center><hr noshade></div><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 face="Arial"> </font><tr height=8><td width=600 colspan=4 style="border-style:none none none none;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;"><font size=1 color=#a2a2a2 face="Arial">IBM
Deutschland GmbH / Vorsitzender des Aufsichtsrats: Martin Jetter <br>Geschäftsführung: Martin Hartmann (Vorsitzender), Norbert Janzen, Stefan
Lutz, Nicole Reimer, Dr. Klaus Seifert, Wolfgang Wendt <br>Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
HRB 14562 WEEE-Reg.-Nr. DE 99369940 </font></table><br><p><font size=3> </font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Peter Childs <p.childs@qmul.ac.uk></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">10/07/2018 23:06</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
Same file opened by many nodes / processes</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><font size=3>The reason I think the metanode is moving around is I'd
done a limited amount of trying to track it down using "mmfsadm saferdump
file" and it moved before I'd tracked down the correct metanode. But
I might have been chasing ghosts, so it may be operating normally and nothing
to worry about.<br><br>The user reading the file only has read access to it from the file permissions,
<br><br>Mmbackup has only slowed down while this job has been running. As I say
the scan for what to backup usally takes </font><a href="tel:40-60"><font size=3 color=blue><u>40-60</u></font></a><font size=3>minutes, but is currently taking 3-4 hours with these jobs running. I've
seen it take 3 days when our storage went bad (slow and failing disks)
but that is usally a sign of a bad disk and pulling the disk and rebuilding
the RAID "fixed" that straight away. I cant see anything like
that currently however.<br><br>It might be that its network congestion were suffering from and nothing
to do with token management but as the mmpmon bytes read data is running
very high with this job and the load is spread over 50+ nodes it's difficult
to see one culprit. It's a mixed speed ethernet network mainly 10GB connected
although the nodes in question are legacy with only 1GB connections (and
40GB to the back of the storage.<br><br>We're currently running </font><a href="tel:4.2.3-8"><font size=3 color=blue><u>4.2.3-8</u></font></a><font size=3><br><br>Peter Childs<br>Research Storage<br>ITS Research and Teaching Support<br>Queen Mary, University of London<br><br>---- IBM Spectrum Scale wrote ----<br></font><br><font size=3 face="Arial">What is in the dump that indicates the metanode
is moving around? Could you please provide an example of what you
are seeing?</font><font size=3><br></font><font size=3 face="Arial"><br>You noted that the access is all read only, is the file opened for read
only or for read and write?</font><font size=3><br></font><font size=3 face="Arial"><br>What makes you state that this particular file is interfering with the
scan done by mmbackup? Reading a file, no matter how large should
significantly impact a policy scan.</font><font size=3><br></font><font size=3 face="Arial"><br>What version of Spectrum Scale are you running and how large is your cluster?</font><font size=3><br></font><font size=2 face="sans-serif"><br>Regards, The Spectrum Scale (GPFS) team<br><br>------------------------------------------------------------------------------------------------------------------<br>If you feel that your question can benefit other users of Spectrum
Scale (GPFS), then please post it to the public IBM developerWroks Forum
at </font><a href="https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479"><font size=2 color=blue face="sans-serif"><u>https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479</u></font></a><font size=2 face="sans-serif">.
<br><br>If your query concerns a potential software error in Spectrum Scale (GPFS)
and you have an IBM software maintenance contract please contact 1-800-237-5511
in the United States or your local IBM Service Center in other countries.
<br><br>The forum is informally monitored as time permits and should not be used
for priority messages to the Spectrum Scale (GPFS) team.</font><font size=3><br><br><br></font><font size=1 color=#5f5f5f face="sans-serif"><br>From: </font><font size=1 face="sans-serif">Peter
Childs <p.childs@qmul.ac.uk></font><font size=1 color=#5f5f5f face="sans-serif"><br>To: </font><font size=1 face="sans-serif">"gpfsug-discuss@spectrumscale.org"
<gpfsug-discuss@spectrumscale.org></font><font size=1 color=#5f5f5f face="sans-serif"><br>Date: </font><font size=1 face="sans-serif">07/10/2018
10:51 AM</font><font size=1 color=#5f5f5f face="sans-serif"><br>Subject: </font><font size=1 face="sans-serif">[gpfsug-discuss]
Same file opened by many nodes / processes</font><font size=1 color=#5f5f5f face="sans-serif"><br>Sent by: </font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><font size=3><br></font><hr noshade><font size=3><br><br></font><tt><font size=2><br>We have an situation where the same file is being read by around 5000<br>"jobs" this is an array job in uge with a tc set, so the file
in<br>question is being opened by about 100 processes/jobs at the same time.<br><br>Its a ~200GB file so copying the file locally first is not an easy<br>answer, and these jobs are causing issues with mmbackup scanning the<br>file system, in that the scan is taking 3 hours instead of the normal<br>40-60 minutes.<br><br>This is read only access to the file, I don't know the specifics about<br>the job.<br><br>It looks like the metanode is moving around a fair amount (given what I<br>can see from mmfsadm saferdump file)<br><br>I'm wondering if we there is anything we can do to improve things or<br>that can be tuned within GPFS, I'm don't think we have an issue with<br>token management, but would increasing maxFileToCache on our token<br>manager node help say?<br><br>Is there anything else I should look at, to try and attempt to allow<br>GPFS to share this file better.<br><br>Thanks in advance<br><br>Peter Childs<br><br>-- <br>Peter Childs<br>ITS Research Storage<br>Queen Mary, University of London<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org</font></tt><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></tt></a><tt><font size=2><br></font></tt><font size=3><br><br><br></font><tt><font size=2>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br><br><BR>