<!doctype html>
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
<div class="default-style">
<div>
Hi all,
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
I am in the process of replacing a beegfs cluster with a spectrum scale cluster and some of our initial tests have returned poor performance when writing from multiple client nodes to the same file.
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
If we setup a client to write into a file it takes less than 8 seconds to complete the write on Flash and about the same on NLSAS storage. But if we get 3 more nodes to do the exact same write the cluster seems to slow down and completes all writes in about 1.5 minute.
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
We are running 5.0.4-0 on 4 Lenovo SR630 servers with a V7000 control enclosure with flash drives and a 92F drawer with NLSAS drives behind the nodes attach through SAN.
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Is there something I am missing since the writes are so slow compared to beegfs?
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Beegfs completes the write from all clients within 9 second when run from 4 clients doing the same write to the same file
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
GPFS takes 1.5 min to do the same.
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Tests run:
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
time (for i in `seq 5000`;do echo "This is $(hostname) writing line number" $i >> "/gpfs_T0/test/benchmark1.txt";done) ########## this is run from 4 gpfs client nodes at the same time.
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Result for scale:
</div>
<div>
<br>
</div>
<div>
real 1m43.995s
</div>
<div>
<br>
</div>
<div>
user 0m0.821s
</div>
<div>
<br>
</div>
<div>
sys 0m3.545s
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Result for beegfs:
</div>
<div>
<br>
</div>
<div>
real 0m6.507s
</div>
<div>
<br>
</div>
<div>
user 0m0.651s
</div>
<div>
<br>
</div>
<div>
sys 0m2.534s
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
if we run writes from each client node to separate files, performance is way better with GPFS than beegfs but not when the writes are done parallel.
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
If anyone have an idea I would be glad to hear it 😊
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Best Regards
</div>
<div>
<br>
</div>
<div>
Andi Christiansen
</div>
<div>
<br>
</div>
</div>
</body>
</html>