<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<div><br>
Can you remember what version you were running? Don't worry if you can't remember.
<br>
<br>
It looks like ibm may have withdrawn <a href="tel:4.2.1">4.2.1</a> from fix central and wish to forget its existences. Never a good sign,
<a href="tel:4.2.0">4.2.0</a>, <a href="tel:4.2.2">4.2.2</a>, <a href="tel:4.2.3">
4.2.3</a> and even 3.5, so maybe upgrading is worth a try.<br>
<br>
I've looked at all the standard trouble shouting guides and got nowhere hence why I asked. But another set of slides always helps.
<br>
<br>
Thank-you for the help, still head scratching.... Which only makes the issue more random.<br>
<br>
Peter Childs<br>
Research Storage<br>
ITS Research and Teaching Support<br>
Queen Mary, University of London<br>
<br>
<br>
---- Simon Thompson (IT Research Support) wrote ----<br>
<br>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">We actually saw this for a while on one of our clusters which was new. But<br>
by the time I'd got round to looking deeper, it had gone, maybe we were<br>
using the NSDs more heavily, or possibly we'd upgraded. We are at 4.2.2-2,<br>
so might be worth trying to bump the version and see if it goes away.<br>
<br>
We saw it on the NSD servers directly as well, so not some client trying<br>
to talk to it, so maybe there was some buggy code?<br>
<br>
Simon<br>
<br>
On 11/04/2017, 16:51, "gpfsug-discuss-bounces@spectrumscale.org on behalf<br>
of Bryan Banister" <gpfsug-discuss-bounces@spectrumscale.org on behalf of<br>
bbanister@jumptrading.com> wrote:<br>
<br>
>There are so many things to look at and many tools for doing so (iostat,<br>
>htop, nsdperf, mmdiag, mmhealth, mmlsconfig, mmlsfs, etc). I would<br>
>recommend a review of the presentation that Yuri gave at the most recent<br>
>GPFS User Group:<br>
><a href="https://drive.google.com/drive/folders/0B124dhp9jJC-UjFlVjJTa2ZaVWs">https://drive.google.com/drive/folders/0B124dhp9jJC-UjFlVjJTa2ZaVWs</a><br>
><br>
>Cheers,<br>
>-Bryan<br>
><br>
>-----Original Message-----<br>
>From: gpfsug-discuss-bounces@spectrumscale.org<br>
>[<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>] On Behalf Of Peter<br>
>Childs<br>
>Sent: Tuesday, April 11, 2017 3:58 AM<br>
>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
>Subject: [gpfsug-discuss] Spectrum Scale Slow to create directories<br>
><br>
>This is a curious issue which I'm trying to get to the bottom of.<br>
><br>
>We currently have two Spectrum Scale file systems, both are running GPFS<br>
>4.2.1-1 some of the servers have been upgraded to 4.2.1-2.<br>
><br>
>The older one which was upgraded from GPFS 3.5 works find create a<br>
>directory is always fast and no issue.<br>
><br>
>The new one, which has nice new SSD for metadata and hence should be<br>
>faster. can take up to 30 seconds to create a directory but usually takes<br>
>less than a second, The longer directory creates usually happen on busy<br>
>nodes that have not used the new storage in a while. (Its new so we've<br>
>not moved much of the data over yet) But it can also happen randomly<br>
>anywhere, including from the NSD servers them selves. (times of 3-4<br>
>seconds from the NSD servers have been seen, on a single directory create)<br>
><br>
>We've been pointed at the network and suggested we check all network<br>
>settings, and its been suggested to build an admin network, but I'm not<br>
>sure I entirely understand why and how this would help. Its a mixed<br>
>1G/10G network with the NSD servers connected at 40G with an MTU of 9000.<br>
><br>
>However as I say, the older filesystem is fine, and it does not matter if<br>
>the nodes are connected to the old GPFS cluster or the new one, (although<br>
>the delay is worst on the old gpfs cluster), So I'm really playing spot<br>
>the difference. and the network is not really an obvious difference.<br>
><br>
>Its been suggested to look at a trace when it occurs but as its difficult<br>
>to recreate collecting one is difficult.<br>
><br>
>Any ideas would be most helpful.<br>
><br>
>Thanks<br>
><br>
><br>
><br>
>Peter Childs<br>
>ITS Research Infrastructure<br>
>Queen Mary, University of London<br>
>_______________________________________________<br>
>gpfsug-discuss mailing list<br>
>gpfsug-discuss at spectrumscale.org<br>
><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
><br>
>________________________________<br>
><br>
>Note: This email is for the confidential use of the named addressee(s)<br>
>only and may contain proprietary, confidential or privileged information.<br>
>If you are not the intended recipient, you are hereby notified that any<br>
>review, dissemination or copying of this email is strictly prohibited,<br>
>and to please notify the sender immediately and destroy this email and<br>
>any attachments. Email transmission cannot be guaranteed to be secure or<br>
>error-free. The Company, therefore, does not make any guarantees as to<br>
>the completeness or accuracy of this email or any attachments. This email<br>
>is for informational purposes only and does not constitute a<br>
>recommendation, offer, request or solicitation of any kind to buy, sell,<br>
>subscribe, redeem or perform any type of transaction of a financial<br>
>product.<br>
>_______________________________________________<br>
>gpfsug-discuss mailing list<br>
>gpfsug-discuss at spectrumscale.org<br>
><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</div>
</span></font>
</body>
</html>