<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi, Henrik, </p>
<p><br>
</p>
<p>while I am not using Windows I'd start investigating the usual
things (see below).</p>
<p><br>
</p>
<p>But first you should describe your set-up better. <br>
</p>
<p>Where are the NSDs : locally attached to the Windows box? In some
NSD servers?</p>
<p>If the latter -- what is the link to the NSD servers? via your
GbE link? FC? IB? separate Ethernet?<br>
</p>
<p>What type of storage? Spinning Disks? Flash?<br>
</p>
<p><br>
</p>
<p>How long are your I/Os waiting on the client (compare that to the
waiting times on the NSD server if applicable)? <br>
</p>
<p>not sure whether that is available on Windows, but <br>
</p>
<p>mmdiag --waiters</p>
<p>mmdiag --iohistory</p>
<p>might be of use.<br>
</p>
<p><br>
</p>
Somewhere in the chain from your application to the storage backend
there is a delay and you should first find out where that occurs I
think. <br>
<p><br>
</p>
<p>Bye</p>
<p>Uwe<br>
</p>
<p><br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 03.09.24 14:10, Henrik Cednert
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:GVYP280MB05746F9D1034FB26D2AB6415EC932@GVYP280MB0574.SWEP280.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Still no solution here regarding this.</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Have tested other cables.</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Have tested to change tcp window size, no change</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Played with numa in the bios, no change</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Played with hyperthreading in bios, no change</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Have anyone managed to get some speed out of windows 11 and
gpfs?</div>
<div class="elementToProof"
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature" style="color: inherit;">
<div
style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<p style="margin: 0px; font-family: Helvetica; font-size: 12px;">-- </p>
<p style="margin: 0px; font-family: Helvetica; font-size: 12px;">Henrik
Cednert <b>
/ </b> + 46 704 71 89 54 <b>/</b> CTO <b>/ OnePost </b>(<span
style="background-color: rgb(255, 255, 255);">formerly</span> Filmlance
Post)</p>
<p style="margin: 0px;"><span
style="font-family: Helvetica; font-size: 12px; background-color: rgb(255, 255, 255);">☝️
<b>OnePost</b>, formerly Filmlance's post-production, is now
an independent part of the Banijay Group.<br>
New name, same team – business as usual at OnePost.</span></p>
<p
style="margin: 0px; font-family: Helvetica; font-size: 9px; color: rgb(14, 14, 14);">
<br>
</p>
<p
style="margin: 0px; font-family: Helvetica; font-size: 9px; color: rgb(14, 14, 14);">
<br>
</p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>From:</b>
gpfsug-discuss <a class="moz-txt-link-rfc2396E" href="mailto:gpfsug-discuss-bounces@gpfsug.org"><gpfsug-discuss-bounces@gpfsug.org></a> on
behalf of Henrik Cednert <a class="moz-txt-link-rfc2396E" href="mailto:henrik.cednert@onepost.se"><henrik.cednert@onepost.se></a><br>
<b>Sent:</b> Friday, 9 August 2024 17:25<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:gpfsug-discuss@gpfsug.org">gpfsug-discuss@gpfsug.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:gpfsug-discuss@gpfsug.org"><gpfsug-discuss@gpfsug.org></a><br>
<b>Subject:</b> [gpfsug-discuss] GPFS 5.1.9.4 on Windows 11
Pro. Performance issues, write.</font>
<div> </div>
</div>
<style type="text/css" style="display:none">p
{margin-top:0;
margin-bottom:0}</style>
<div dir="ltr">
<div style="border:solid #9C6500 1.0pt">
<p class="x_MsoNormal"
style="background:#FFFFFF; margin:0px; padding-bottom:3px; padding-left:5px">
<b><span style="font-size:11.0pt; color:#FFFFFF"></span></b><span
style="font-size:11.0pt; color:black"><b>VARNING: DETTA ÄR
ETT EXTERNT MAIL. Klicka inte på några länkar oavsett
hur legitima de verkar utan att verifiera.</b></span></p>
</div>
<br>
<div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hello</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I have some issues with write performance on a windows 11
pro system and I'm out of ideas here. Hopefully someone here
have some bright ideas and/or experience of GPFS on Windows
11?</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The system is a:</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Windows 11 Pro 22H2</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
2 x Intel(R) Xeon(R) Gold 6418H 2.10 GHz</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
512 GB RAM</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
GPFS 5.1.9.4</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Mellanox ConnectX 6 Dx</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
100GbE connected to Mellanox Switch with 5m Mellanox DAC.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Before deploying this workstation we had a single socket
system as a test bench where we got 60 GbE in both directons
with iPerf and around 6GB/sec write and 3GB/sec read from
the system over GPFS (fio tests, same tests as furhter down
here). </div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
With that system I had loads of issues before getting to
that point though. MS Defender had to be forcefully disabled
via regedit some other tweaks. All those tweaks have been
performed in this new system as well, but I can't get the
proper speed out of it.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
On this new system and with iPerf to the storage servers I
get around 50-60GbE in both directions and send and receive.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
If I mount the storage over SMB and 100GbE via the storage
gateway servers I get around 3GB/sec read and write with
Blackmagics Disk speed test. I have not tweaked the system
for samba performande, just a test to see what it would give
and part of the troubleshooting.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
If I run Blackmagics diskspeed test to the GPFS mount I
instead get around 700MB/sec write and 400MB/sec read.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Starting to think that the Blackmagic test might not run
properly on this machine with these CPUs though. Or it's
related to the mmfsd process maybe, how that threads or not
threads...?</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
But if we instead look at fio. I have a bat script that
loops through a bunch of FIO-tests. A test that I have been
using over the years so that we easily can benchmark all
deployed systems with the exakt same tests. The tests are
named like:</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
seqrw-<filesize>gb-<blocksize>mb-t<threads></div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The result when I run this is like the below list. Number in
parenthesis is the by fio reported latency.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-40gb-1mb-t1</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 162 MB/s (6 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 1940 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-20gb-1mb-t2</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 286 MB/s (7 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 3952 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-10gb-1mb-t4</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 549 MB/s (7 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 6987 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-05gb-1mb-t8</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 989 MB/s (8 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 7721 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-40gb-2mb-t1</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 161 MB/s (12 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 2261 MB/s (0 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-20gb-2mb-t2</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 348 MB/s (11 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 4266 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-10gb-2mb-t4</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 626 MB/s (13 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 4949 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-05gb-2mb-t8</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 1154 MB/s (14 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 7007 MB/s (2 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-40gb-4mb-t1</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 161 MB/s (25 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 2083 MB/s (1 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-20gb-4mb-t2</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 352 MB/s (23 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 4317 MB/s (2 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-10gb-4mb-t4</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 696 MB/s (23 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 7358 MB/s (2 ms)</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Job: seqrw-05gb-4mb-t8</div>
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Write: 1251 MB/s (25 ms)</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
• Read: 6707 MB/s (5 ms)</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
So with fio I get a very nice read speed, but the write is
horrendous and I cannot find what causes it. I have looked
at affinity settings for the mmfsd process but not sure I
fully understand it. But no matter what I set it to, I see
no difference.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I have "played" with the bios and tried with/without
hyperthreading, numa and so on. And nothing affects atleast
the blackmagic disk speed test.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
the current settings for this host is like below. I write
"current" because I have tested a few different settings
here but nothing affects the write speed.
maxTcpConnsPerNodeConn for sure bumped the read speed
though.</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
nsdMaxWorkerThreads 16</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
prefetchPct 60<br>
maxTcpConnsPerNodeConn 8</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
maxMBpS 14000</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Does anyone have any suggestions or ideas on how to
troubleshoot this?</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thanks</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div class="x_elementToProof"
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="x_Signature" style="color:inherit">
<div
style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<p style="margin:0px; font-family:Helvetica; font-size:12px">-- </p>
<p style="margin:0px; font-family:Helvetica; font-size:12px">Henrik
Cednert <b>/
</b> + 46 704 71 89 54 <b>/</b> CTO <b>/ OnePost </b>(<span
style="background-color:rgb(255,255,255)">formerly</span> Filmlance
Post)</p>
<p style="margin:0px"><span
style="font-family:Helvetica; font-size:12px; background-color:rgb(255,255,255)">☝️
<b>OnePost</b>, formerly Filmlance's post-production, is
now an independent part of the Banijay Group.<br>
New name, same team – business as usual at OnePost.</span></p>
<p
style="margin:0px; font-family:Helvetica; font-size:9px; color:rgb(14,14,14)">
<br>
</p>
<p
style="margin:0px; font-family:Helvetica; font-size:9px; color:rgb(14,14,14)">
<br>
</p>
</div>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
<a class="moz-txt-link-freetext" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Karlsruhe Institute of Technology (KIT)
Scientific Computing Centre (SCC)
Scientific Data Management (SDM)
Uwe Falke
Hermann-von-Helmholtz-Platz 1, Building 442, Room 187
D-76344 Eggenstein-Leopoldshafen
Tel: +49 721 608 28024
Email: <a class="moz-txt-link-abbreviated" href="mailto:uwe.falke@kit.edu">uwe.falke@kit.edu</a>
<a class="moz-txt-link-abbreviated" href="http://www.scc.kit.edu">www.scc.kit.edu</a>
Registered office:
Kaiserstraße 12, 76131 Karlsruhe, Germany
KIT – The Research University in the Helmholtz Association
</pre>
</body>
</html>