<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>pre,code,address {
margin: 0px;
}
h1,h2,h3,h4,h5,h6 {
margin-top: 0.2em;
margin-bottom: 0.2em;
}
ol,ul {
margin-top: 0em;
margin-bottom: 0em;
}
blockquote {
margin-top: 0em;
margin-bottom: 0em;
}
</style>
</head>
<body>
<div>Hello,</div>
<div><br>
</div>
<div>For the ease of testing and reproduction we use fio and iozone benchmarking tools. The issue you desrcibe looks pretty similar. However it occurs using GPFS mount only. Here are some results from out testing environment:</div>
<div><br>
</div>
<div>Linux NSD server:</div>
<div>Writes: 3472 MiB/s</div>
<div>Re-writes: 3476 MiB/s</div>
<div><br class="Apple-interchange-newline">
Windows GPFS:</div>
<div>Writes: 2487 MiB/s</div>
<div>Re-writes: 250 MiB/s</div>
<div><br>
</div>
<div>Windows Samba:</div>
<div>Writes: 997 MiB/s</div>
<div>Re-writes: 1269 MiB/s</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><span>
<pre>-- <br></pre>
<div style="font-size: 9pt; color: rgb(89, 89, 89); font-family: verdana;">S přáním pěkného dne / Best regards</div>
<br style="caret-color: rgb(46, 52, 54); color: rgb(46, 52, 54);">
<div style="font-size: 11pt; color: rgb(220, 38, 30); font-family: verdana;"><b>Mgr. Peter Hruška</b><br>
</div>
<div style="font-size: 9pt; font-family: verdana;"><font color="#595959">IT specialista</font><br>
<br>
<div style="font-size: 9pt;"><span style="color: rgb(89, 89, 89); font-size: 9pt; font-family: Verdana, sans-serif;"><b>M Computers s.r.o.</b><br>
</span><span style="color: rgb(220, 38, 30); font-size: 9pt; font-family: Verdana, sans-serif;">Úlehlova 3100/10, 628 00 Brno-Líšeň (<a href="https://mapy.cz/s/gafufehufe" title="Click to open https://mapy.cz/s/gafufehufe" style="color: rgb(149, 79, 114);"><span style="color: rgb(5, 99, 193);">mapa</span></a>)</span><span style="font-size: 9pt; font-family: Verdana, sans-serif;"><br>
<font color="#595959">T:+420 515 538 136</font><br>
<font color="#595959">E: </font></span><a href="mailto:peter.hruska@mcomputers.cz" title="Click to mail peter.hruska@mcomputers.cz" style="font-size: 9pt; color: rgb(149, 79, 114); font-family: Verdana, sans-serif;"><span style="color: rgb(5, 99, 193);">peter.hruska@mcomputers.cz</span></a><span style="font-size: 9pt; font-family: Verdana, sans-serif;"><font color="#595959"><br>
</font><font color="#0563c1"><span style="caret-color: rgb(5, 99, 193);"></span></font></span><span style="font-size: 9pt; font-family: Verdana, sans-serif;"><br>
<a href="http://www.mcomputers.cz/" title="Click to open http://www.mcomputers.cz/" style="color: rgb(149, 79, 114);"><span style="color: rgb(5, 99, 193);">www.mcomputers.cz</span></a><br>
</span></div>
</div>
<div><a href="http://www.lenovoshop.cz/" title="Click to open http://www.lenovoshop.cz/" style="font-family: Verdana, sans-serif; font-size: 9pt;">www.lenovoshop.cz</a></div>
<div><img src="cid:a57a2d9ed2ea460d8bb2abefe678c4fab3cdaeeb.camel@mcomputers.cz-0" width="457px" height="106px"></div>
<div style="font-size : 9pt; color:#595959;
font-family : verdana"><br>
</div>
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(0, 0, 0, 0.4); -webkit-text-stroke-width: 0px; text-decoration: none; font-size: 9pt; color: rgb(89, 89, 89); font-family: verdana;">
<br>
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Cantarell; font-size: 14.666666984558105px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-tap-highlight-color: rgba(0, 0, 0, 0.4); -webkit-text-stroke-width: 0px; text-decoration: none;">
<br>
</div>
</span></div>
<div><br>
</div>
<div>On Mon, 2024-03-11 at 17:35 +0000, Jonathan Buzzard wrote:</div>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div>EXTERNÍ ODESÍLATEL<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>On 11/03/2024 16:43, Peter Hruška wrote:<br>
</div>
<div><br>
</div>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div>Hello,<br>
</div>
<div><br>
</div>
<div>We've encountered yet another performance flaw. We have a GPFS<br>
</div>
<div>filesystem mounted using GPFS binaries on Windows. When we try to<br>
</div>
<div>rewrite a file on the GPFS filesystem rewriting speed is much slower<br>
</div>
<div>than writing to a new file. The difference ratio we measured is about<br>
</div>
<div>3.5 times. From the task manager it is visible that there is excessive<br>
</div>
<div>amount of reading from the network when rewriting. This is even visible<br>
</div>
<div>on the NDS server as io activity. However when rewriting on a linux<br>
</div>
<div>client there are no reads while rewriting. Has anyone encountered such<br>
</div>
<div>problems?<br>
</div>
<div>To replicate the issue is is possidle to run fio twice with the same<br>
</div>
<div>configuration to achieve rewriting or to run iozone. Both tools return<br>
</div>
<div>similar outputs.<br>
</div>
</blockquote>
<div><br>
</div>
<div>Kind of yes.<br>
</div>
<div><br>
</div>
<div>What are you using to "rewrite" the file?<br>
</div>
<div><br>
</div>
<div>What we saw initially over Samba was certain Microsoft applications when<br>
</div>
<div>rewriting a file had truly abysmal performance. The same application<br>
</div>
<div>when saving the same document to a new file and the performance was as<br>
</div>
<div>expected.<br>
</div>
<div><br>
</div>
<div>After much digging into it the cause (it was weeks of person effort) it<br>
</div>
<div>was determined to be down to the application writing the file *one* byte<br>
</div>
<div>at a time. Basically some idiot C++ developer at Microsoft decided to<br>
</div>
<div>ignore the C++ library because it has "bugs" and write their own<br>
</div>
<div>formatted output routines.<br>
</div>
<div><br>
</div>
<div>It was not noticeable saving to a local disk, but the instant you tried<br>
</div>
<div>saving to a network drive the performance was truly awful. Basically the<br>
</div>
<div>increased latency of single character IO was the issue.<br>
</div>
<div><br>
</div>
<div>Note that the issue was not confined to Samba and GPFS, as we verified<br>
</div>
<div>the same abysmal performance with a Windows 2008 R2 server running on<br>
</div>
<div>File and Print Sharing on NTFS on bare metal hardware. It was also not<br>
</div>
<div>confined to just Windows the same awful performance happened on Macs<br>
</div>
<div>too. In fact that is where it first came to light.<br>
</div>
<div><br>
</div>
<div>Might be the cause of your problem, might not.<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>JAB.<br>
</div>
<div><br>
</div>
<div>--<br>
</div>
<div>Jonathan A. Buzzard Tel: +44141-5483420<br>
</div>
<div>HPC System Administrator, ARCHIE-WeSt.<br>
</div>
<div>University of Strathclyde, John Anderson Building, Glasgow. G4 0NG<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>_______________________________________________<br>
</div>
<div>gpfsug-discuss mailing list<br>
</div>
<div>gpfsug-discuss at gpfsug.org<br>
</div>
<div><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a><br>
</div>
</blockquote>
</body>
</html>