<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi, given you see read latencies of 1 ms, you do not get the data
from disk but from some cache (on whatever level). From spinning
disks, you can never expect such read latencies (mind that GPFS
block reading, even if sequential from the application's PoV,
typically translates to random I/O at the physical disk level). <br>
</p>
<p>So, I do not know what the latencies on your other measurements
(bypassing GPFS) were but the numbers below do not represent
sustained high-scale throughputs, apparently. <br>
</p>
<p><br>
</p>
<p>It is nevertheless strange that your write rates are much below
reads (and write latencies are that high) -- from my experience
with different systems, when hammering GPFS with usual storage
backends with both read and write requests, the writes tend to
prevail. <br>
</p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<p>Your waiters indicate that the problem is above GPFS: <br>
</p>
<p>GPFS is able to serve all I/O threads within a few ms, and there
is not a long list of pending IOs. <br>
</p>
<p><br>
</p>
<p>Sorry, the iohistory options is <br>
</p>
<p>mmdiag --iohist</p>
<p><br>
</p>
<p>But to me it looks like not GPFS is the culprit here. <br>
</p>
<p><br>
</p>
<p>Uwe <br>
</p>
<div class="moz-cite-prefix">On 04.09.24 11:08, Henrik Cednert
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:GVYP280MB05744D1C9F4E4807F1F291B2EC9C2@GVYP280MB0574.SWEP280.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Uwe</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks.</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Worth noting is that we have win10 ltsc, win 2019 and have had a
single CPU win 11 22h2 (as a test) clients that all perform as
expected. Those machines is older though and connected with
10-40GbE, those client max out their NIC in read and write.</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Let me know if i missed something important here. Thanks again.</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Setups is</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Client:</div>
<ul
style="margin-top: 0px; margin-bottom: 0px; list-style-type: disc;"
data-editing-info="{"orderedStyleType":1,"unorderedStyleType":1}">
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Supermicro Workstation<br>
</li>
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Intel(R) Xeon(R) Gold 6418H 2.10 GHz (2 processors)<br>
</li>
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Mellanox ConnectX-6 Dx connected with 100GbE over dedicated
vlan via mellanox sn2100.<br>
</li>
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Windows 11 Pro for Workstations, 22H2<br>
</li>
</ul>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Storage setup</div>
<ul
style="margin-top: 0px; margin-bottom: 0px; list-style-type: disc;"
data-editing-info="{"orderedStyleType":1,"unorderedStyleType":1}">
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div>3 x 84 bay seagate chassis with spinning disks.</div>
</li>
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div class="elementToProof">Storage connected with redundant
12Gb SAS to 2 x Storage node servers</div>
</li>
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div class="elementToProof">2 x mellanox sn2100</div>
</li>
<li
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div class="elementToProof">The 2 storage node servers are for
this vlan connected with 100GbE to each switch, so in total
4 x 100GbE. And the switches are connected with 2 * 100GbE</div>
</li>
</ul>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I tested the commands you suggested. They are both new to me so
not sure what the output is supposed to be, looks like
-iohistory isn't available in windows. I ran --waiters a few
times, as seen below. Not sure what the expected output is from
that.</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
mmdiag --waiters</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
=== mmdiag: waiters ===</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0000 sec since 2024-09-04_10:05:14, monitored, thread
18616 MsgHandler@getData: for In function sendMessage</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0000 sec since 2024-09-04_10:05:14, monitored, thread
25084 WritebehindWorkerThread: on ThCond 0x31A7C360
(MsgRecordCondvar), reason 'RPC wait' for NSD I/O completion on
node 192.168.45.213 <c0n0></div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
C:\Users\m5-tkd01>mmdiag --waiters</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
=== mmdiag: waiters ===</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0009 sec since 2024-09-04_10:05:17, monitored, thread
16780 FsyncHandlerThread: on ThCond 0x37FFDAB0
(MsgRecordCondvar), reason 'RPC wait' for NSD I/O completion on
node 192.168.45.214 <c0n1></div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0009 sec since 2024-09-04_10:05:17, monitored, thread
30308 MsgHandler@getData: for In function sendMessage</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
C:\Users\m5-tkd01>mmdiag --waiters</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
=== mmdiag: waiters ===</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0055 sec since 2024-09-04_10:05:21, monitored, thread
16780 FileBlockReadFetchHandlerThread: on ThCond 0x37A25FF0
(MsgRecordCondvar), reason 'RPC wait' for NSD I/O completion on
node 192.168.45.213 <c0n0></div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
C:\Users\m5-tkd01>mmdiag --waiters</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
=== mmdiag: waiters ===</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0029 sec since 2024-09-04_10:05:23, monitored, thread
16780 FileBlockReadFetchHandlerThread: on ThCond 0x38281DE0
(MsgRecordCondvar), reason 'RPC wait' for NSD I/O completion on
node 192.168.45.213 <c0n0></div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
C:\Users\m5-tkd01>mmdiag --waiters</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
=== mmdiag: waiters ===</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0019 sec since 2024-09-04_10:05:25, monitored, thread
11832 PrefetchWorkerThread: on ThCond 0x38278D20
(MsgRecordCondvar), reason 'RPC wait' for NSD I/O completion on
node 192.168.45.214 <c0n1></div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0009 sec since 2024-09-04_10:05:25, monitored, thread
16780 AcquireBRTHandlerThread: on ThCond 0x37A324E0
(MsgRecordCondvar), reason 'RPC wait' for tmMsgBRRevoke on node
192.168.45.161 <c0n11></div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Waiting 0.0009 sec since 2024-09-04_10:05:25, monitored, thread
2576 RangeRevokeWorkerThread: on ThCond 0x5419DAA0
(BrlObjCondvar), reason 'waiting because of local byte range
lock conflict'</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
C:\Users\m5-tkd01></div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
C:\Users\m5-tkd01>mmdiag --iohistory</div>
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Unrecognized option: --iohistory.</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Run mmdiag --help for the option list</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature" style="color: inherit;">
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<p style="margin: 0px; font-family: Helvetica; font-size: 12px;">-- </p>
<p style="margin: 0px; font-family: Helvetica; font-size: 12px;">Henrik
Cednert <b>
/ </b> + 46 704 71 89 54 <b>/</b> CTO <b>/ OnePost </b>(<span
style="background-color: rgb(255, 255, 255);">formerly</span> Filmlance
Post)</p>
<p style="margin: 0px;"><span
style="font-family: Helvetica; font-size: 12px; background-color: rgb(255, 255, 255);">☝️
<b>OnePost</b>, formerly Filmlance's post-production, is now
an independent part of the Banijay Group.<br>
New name, same team – business as usual at OnePost.</span></p>
<p
style="margin: 0px; font-family: Helvetica; font-size: 9px; color: rgb(14, 14, 14);">
<br>
</p>
<p
style="margin: 0px; font-family: Helvetica; font-size: 9px; color: rgb(14, 14, 14);">
<br>
</p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>From:</b>
gpfsug-discuss <a class="moz-txt-link-rfc2396E" href="mailto:gpfsug-discuss-bounces@gpfsug.org"><gpfsug-discuss-bounces@gpfsug.org></a> on
behalf of Uwe Falke <a class="moz-txt-link-rfc2396E" href="mailto:uwe.falke@kit.edu"><uwe.falke@kit.edu></a><br>
<b>Sent:</b> Tuesday, 3 September 2024 17:35<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:gpfsug-discuss@gpfsug.org">gpfsug-discuss@gpfsug.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:gpfsug-discuss@gpfsug.org"><gpfsug-discuss@gpfsug.org></a><br>
<b>Subject:</b> Re: [gpfsug-discuss] GPFS 5.1.9.4 on Windows
11 Pro. Performance issues, write.</font>
<div> </div>
</div>
<div>
<p>Hi, Henrik, </p>
<p><br>
</p>
<p>while I am not using Windows I'd start investigating the
usual things (see below).</p>
<p><br>
</p>
<p>But first you should describe your set-up better. <br>
</p>
<p>Where are the NSDs : locally attached to the Windows box? In
some NSD servers?</p>
<p>If the latter -- what is the link to the NSD servers? via
your GbE link? FC? IB? separate Ethernet?<br>
</p>
<p>What type of storage? Spinning Disks? Flash?<br>
</p>
<p><br>
</p>
<p>How long are your I/Os waiting on the client (compare that to
the waiting times on the NSD server if applicable)?
<br>
</p>
<p>not sure whether that is available on Windows, but <br>
</p>
<p>mmdiag --waiters</p>
<p>mmdiag --iohistory</p>
<p>might be of use.<br>
</p>
<p><br>
</p>
Somewhere in the chain from your application to the storage
backend there is a delay and you should first find out where
that occurs I think.
<br>
<p><br>
</p>
<p>Bye</p>
<p>Uwe<br>
</p>
<p><br>
</p>
<p><br>
</p>
<div class="x_moz-cite-prefix">On 03.09.24 14:10, Henrik Cednert
wrote:<br>
</div>
<blockquote type="cite">
<style type="text/css" style="display:none">p
{margin-top:0;
margin-bottom:0}</style>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Still no solution here regarding this.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Have tested other cables.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Have tested to change tcp window size, no change</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Played with numa in the bios, no change</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Played with hyperthreading in bios, no change</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Have anyone managed to get some speed out of windows 11 and
gpfs?</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="x_Signature" style="color:inherit">
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<p style="margin:0px; font-family:Helvetica; font-size:12px">-- </p>
<p style="margin:0px; font-family:Helvetica; font-size:12px">Henrik
Cednert <b>/
</b> + 46 704 71 89 54 <b>/</b> CTO <b>/ OnePost </b>(<span
style="background-color:rgb(255,255,255)">formerly</span> Filmlance
Post)</p>
<p style="margin:0px"><span
style="font-family:Helvetica; font-size:12px; background-color:rgb(255,255,255)">☝️
<b>OnePost</b>, formerly Filmlance's post-production, is
now an independent part of the Banijay Group.<br>
New name, same team – business as usual at OnePost.</span></p>
<p
style="margin:0px; font-family:Helvetica; font-size:9px; color:rgb(14,14,14)">
<br>
</p>
<p
style="margin:0px; font-family:Helvetica; font-size:9px; color:rgb(14,14,14)">
<br>
</p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font
style="font-size:11pt" face="Calibri, sans-serif"
color="#000000"><b>From:</b> gpfsug-discuss
<a class="x_moz-txt-link-rfc2396E"
href="mailto:gpfsug-discuss-bounces@gpfsug.org"
moz-do-not-send="true">
<gpfsug-discuss-bounces@gpfsug.org></a> on behalf
of Henrik Cednert <a class="x_moz-txt-link-rfc2396E"
href="mailto:henrik.cednert@onepost.se"
moz-do-not-send="true">
<henrik.cednert@onepost.se></a><br>
<b>Sent:</b> Friday, 9 August 2024 17:25<br>
<b>To:</b> <a
class="x_moz-txt-link-abbreviated moz-txt-link-freetext"
href="mailto:gpfsug-discuss@gpfsug.org"
moz-do-not-send="true">
gpfsug-discuss@gpfsug.org</a> <a
class="x_moz-txt-link-rfc2396E"
href="mailto:gpfsug-discuss@gpfsug.org"
moz-do-not-send="true">
<gpfsug-discuss@gpfsug.org></a><br>
<b>Subject:</b> [gpfsug-discuss] GPFS 5.1.9.4 on Windows
11 Pro. Performance issues, write.</font>
<div> </div>
</div>
<style type="text/css" style="display:none">p
{margin-top:0;
margin-bottom:0}</style>
<div dir="ltr">
<div style="border:solid #9C6500 1.0pt">
<p class="x_x_MsoNormal"
style="background:#FFFFFF; margin:0px; padding-bottom:3px; padding-left:5px">
<b><span style="font-size:11.0pt; color:#FFFFFF"></span></b><span
style="font-size:11.0pt; color:black"><b>VARNING:
DETTA ÄR ETT EXTERNT MAIL. Klicka inte på några
länkar oavsett hur legitima de verkar utan att
verifiera.</b></span></p>
</div>
<br>
<div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hello</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I have some issues with write performance on a windows
11 pro system and I'm out of ideas here. Hopefully
someone here have some bright ideas and/or experience of
GPFS on Windows 11?</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The system is a:</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Windows 11 Pro 22H2</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
2 x Intel(R) Xeon(R) Gold 6418H 2.10 GHz</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
512 GB RAM</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
GPFS 5.1.9.4</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Mellanox ConnectX 6 Dx</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
100GbE connected to Mellanox Switch with 5m Mellanox
DAC.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Before deploying this workstation we had a single socket
system as a test bench where we got 60 GbE in both
directons with iPerf and around 6GB/sec write and
3GB/sec read from the system over GPFS (fio tests, same
tests as furhter down here). </div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
With that system I had loads of issues before getting to
that point though. MS Defender had to be forcefully
disabled via regedit some other tweaks. All those tweaks
have been performed in this new system as well, but I
can't get the proper speed out of it.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
On this new system and with iPerf to the storage servers
I get around 50-60GbE in both directions and send and
receive.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
If I mount the storage over SMB and 100GbE via the
storage gateway servers I get around 3GB/sec read and
write with Blackmagics Disk speed test. I have not
tweaked the system for samba performande, just a test to
see what it would give and part of the troubleshooting.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
If I run Blackmagics diskspeed test to the GPFS mount I
instead get around 700MB/sec write and 400MB/sec read.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Starting to think that the Blackmagic test might not run
properly on this machine with these CPUs though. Or it's
related to the mmfsd process maybe, how that threads or
not threads...?</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
But if we instead look at fio. I have a bat script that
loops through a bunch of FIO-tests. A test that I have
been using over the years so that we easily can
benchmark all deployed systems with the exakt same
tests. The tests are named like:</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
seqrw-<filesize>gb-<blocksize>mb-t<threads></div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The result when I run this is like the below list.
Number in parenthesis is the by fio reported latency.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-40gb-1mb-t1</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 162 MB/s (6 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 1940 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-20gb-1mb-t2</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 286 MB/s (7 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 3952 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-10gb-1mb-t4</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 549 MB/s (7 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 6987 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-05gb-1mb-t8</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 989 MB/s (8 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 7721 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-40gb-2mb-t1</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 161 MB/s (12 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 2261 MB/s (0 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-20gb-2mb-t2</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 348 MB/s (11 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 4266 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-10gb-2mb-t4</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 626 MB/s (13 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 4949 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-05gb-2mb-t8</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 1154 MB/s (14 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 7007 MB/s (2 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-40gb-4mb-t1</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 161 MB/s (25 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 2083 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-20gb-4mb-t2</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 352 MB/s (23 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 4317 MB/s (2 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-10gb-4mb-t4</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 696 MB/s (23 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 7358 MB/s (2 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-05gb-4mb-t8</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 1251 MB/s (25 ms)</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 6707 MB/s (5 ms)</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
So with fio I get a very nice read speed, but the write
is horrendous and I cannot find what causes it. I have
looked at affinity settings for the mmfsd process but
not sure I fully understand it. But no matter what I set
it to, I see no difference.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I have "played" with the bios and tried with/without
hyperthreading, numa and so on. And nothing affects
atleast the blackmagic disk speed test.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
the current settings for this host is like below. I
write "current" because I have tested a few different
settings here but nothing affects the write speed.
maxTcpConnsPerNodeConn for sure bumped the read speed
though.</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
nsdMaxWorkerThreads 16</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
prefetchPct 60<br>
maxTcpConnsPerNodeConn 8</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
maxMBpS 14000</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Does anyone have any suggestions or ideas on how to
troubleshoot this?</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thanks</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="x_x_Signature" style="color:inherit">
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<p
style="margin:0px; font-family:Helvetica; font-size:12px">-- </p>
<p
style="margin:0px; font-family:Helvetica; font-size:12px">Henrik
Cednert <b>/
</b> + 46 704 71 89 54 <b>/</b> CTO <b>/ OnePost
</b>(<span style="background-color:rgb(255,255,255)">formerly</span> Filmlance
Post)</p>
<p style="margin:0px"><span
style="font-family:Helvetica; font-size:12px; background-color:rgb(255,255,255)">☝️
<b>OnePost</b>, formerly Filmlance's
post-production, is now an independent part of the
Banijay Group.<br>
New name, same team – business as usual at OnePost.</span></p>
<p
style="margin:0px; font-family:Helvetica; font-size:9px; color:rgb(14,14,14)">
<br>
</p>
<p
style="margin:0px; font-family:Helvetica; font-size:9px; color:rgb(14,14,14)">
<br>
</p>
</div>
</div>
</div>
<br>
<fieldset class="x_moz-mime-attachment-header"></fieldset>
<pre class="x_moz-quote-pre">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
<a class="x_moz-txt-link-freetext moz-txt-link-freetext"
href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org"
moz-do-not-send="true">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
</pre>
</blockquote>
<pre class="x_moz-signature" cols="72">--
Karlsruhe Institute of Technology (KIT)
Scientific Computing Centre (SCC)
Scientific Data Management (SDM)
Uwe Falke
Hermann-von-Helmholtz-Platz 1, Building 442, Room 187
D-76344 Eggenstein-Leopoldshafen
Tel: +49 721 608 28024
Email: <a class="x_moz-txt-link-abbreviated moz-txt-link-freetext"
href="mailto:uwe.falke@kit.edu" moz-do-not-send="true">uwe.falke@kit.edu</a>
<a class="x_moz-txt-link-abbreviated" href="http://www.scc.kit.edu"
moz-do-not-send="true">www.scc.kit.edu</a>
Registered office:
Kaiserstraße 12, 76131 Karlsruhe, Germany
KIT – The Research University in the Helmholtz Association
</pre>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
<a class="moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Karlsruhe Institute of Technology (KIT)
Scientific Computing Centre (SCC)
Scientific Data Management (SDM)
Uwe Falke
Hermann-von-Helmholtz-Platz 1, Building 442, Room 187
D-76344 Eggenstein-Leopoldshafen
Tel: +49 721 608 28024
Email: <a class="moz-txt-link-abbreviated" href="mailto:uwe.falke@kit.edu">uwe.falke@kit.edu</a>
<a class="moz-txt-link-abbreviated" href="http://www.scc.kit.edu">www.scc.kit.edu</a>
Registered office:
Kaiserstraße 12, 76131 Karlsruhe, Germany
KIT – The Research University in the Helmholtz Association
</pre>
</body>
</html>