<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hello Andrew,</p>
<p><br>
</p>
<p><span style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;">Thanks for your questions.</span></p>
<p><br>
</p>
<p><span style="font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols; font-size: 16px;"></span>We're not experiencing any other issue/slowness during
normal activity.</p>
<p>The storage is a Lenovo DSS appliance with a<span style="font-size: 12pt;"> dedicated SSD enclosure/pool for metadata only.</span></p>
<p><span style="font-size: 12pt;"><br>
</span></p>
<p>The two NSD servers have 750GB of RAM and 618 are configured as pagepool.</p>
<p><br>
</p>
<p>The issue we see is happening on both the two filesystems we have:</p>
<p><br>
</p>
<p>- perf filesystem:</p>
<p> - 1.8 PB size (71% in use)</p>
<p> - 570 milions of inodes (24% in use)</p>
<p><br>
</p>
<p>- tiered filesystem:</p>
<p> - 400 TB size (34% in use)</p>
<p> - 230 Milions of files (60% in use)</p>
<p><br>
</p>
<p>Cheers,</p>
<p>Ivano</p>
<p></p>
<p><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p></p>
<div>__________________________________________</div>
<div>Paul Scherrer Institut</div>
<div>Ivano Talamo</div>
<div>WHGA/038</div>
<div>Forschungsstrasse 111</div>
<div>5232 Villigen PSI</div>
<div>Schweiz</div>
<div><br>
</div>
<div>Telefon: +41 56 310 47 11</div>
<div>E-Mail: ivano.talamo@psi.ch</div>
<br>
<p></p>
</div>
</div>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr style="display:inline-block; width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Andrew Beattie <abeattie@au1.ibm.com><br>
<b>Sent:</b> Wednesday, February 2, 2022 10:33 AM<br>
<b>To:</b> gpfsug main discussion list<br>
<b>Subject:</b> Re: [gpfsug-discuss] snapshots causing filesystem quiesce</font>
<div> </div>
</div>
<div>Ivano,
<div><br>
</div>
<div>How big is the filesystem in terms of number of files?</div>
<div>How big is the filesystem in terms of capacity? </div>
<div>Is the Metadata on Flash or Spinning disk? </div>
<div>Do you see issues when users do an LS of the filesystem or only when you are doing snapshots.</div>
<div><br>
</div>
<div>How much memory do the NSD servers have?</div>
<div>How much is allocated to the OS / Spectrum</div>
<div> Scale Pagepool<br>
<br>
<div dir="ltr">Regards</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">Andrew Beattie</div>
<div dir="ltr">Technical Specialist - Storage for Big Data & AI</div>
<div dir="ltr">IBM Technology Group</div>
<div dir="ltr">IBM Australia & New Zealand</div>
<div dir="ltr">P. +61 421 337 927</div>
<div dir="ltr">E. abeattie@au1.IBM.com</div>
<div dir="ltr"><br>
</div>
<div dir="ltr"><br>
</div>
<div dir="ltr"><br>
<blockquote type="cite">On 2 Feb 2022, at 19:14, Talamo Ivano Giuseppe (PSI) <Ivano.Talamo@psi.ch> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p></p>
<div>Dear all,</div>
<div><br>
</div>
<div>Since a while we are experiencing an issue when dealing with snapshots.</div>
<div>Basically what happens is that when deleting a fileset snapshot (and maybe also when creating new ones) the filesystem becomes inaccessible on the clients for the duration of the operation (can take a few minutes).</div>
<div><br>
</div>
<div>The clients and the storage are on two different clusters, using remote cluster mount for the access.</div>
<div><br>
</div>
<div>On the log files many lines like the following appear (on both clusters):</div>
<div>Snapshot whole quiesce of SG perf from xbldssio1 on this node lasted 60166 msec</div>
<div><br>
</div>
<div>By looking around I see we're not the first one. I am wondering if that's considered an unavoidable part of the snapshotting and if there's any tunable that can improve the situation. Since when this occurs all the clients are stuck and users are very
quick to complain.</div>
<div><br>
</div>
<div>If it can help, the clients are running GPFS 5.1.2-1 while the storage cluster is on 5.1.1-0.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Ivano</div>
<p></p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br>
</div>
</div>
</div>
</body>
</html>