<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body>
<div dir="ltr">Hi Lukas,
<div dir="ltr"><br>
</div>
<div dir="ltr">Check out FPO mode. That mimics Hadoop’s data placement features. You can have up to 3 replicas both data and metadata but still the downside, though, as you say is the wrong node failures will take your cluster down. </div>
<div dir="ltr"><br>
</div>
<div dir="ltr">You might want to check out something like Excelero’s NVMesh (note: not an endorsement since I can’t give such things) which can create logical volumes across all your NVMe drives. The product has erasure coding on their roadmap. I’m not sure
if they’ve released that feature yet but in theory it will give better fault tolerance *and* you’ll get more efficient usage of your SSDs. </div>
<div dir="ltr"><br>
</div>
<div dir="ltr">I’m sure there are other ways to skin this cat too. </div>
<div dir="ltr"><br>
</div>
<div dir="ltr">-Aaron</div>
</div>
<span id="draft-break"></span><br>
<br>
<span id="draft-break"></span><br>
<br>
<div>
<div class="null" dir="auto">On March 12, 2018 at 10:59:35 EDT, Lukas Hejtmanek <xhejtman@ics.muni.cz> wrote:<br class="null">
</div>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;" class="null">
<div class="null" dir="auto">
<div class="null">
<div class="null">
<div class="null">Hello, <br class="null">
<br class="null">
I'm thinking about the following setup: <br class="null">
~ 60 nodes, each with two enterprise NVMe SSDs, FDR IB interconnected <br class="null">
<br class="null">
I would like to setup shared scratch area using GPFS and those NVMe SSDs. Each <br class="null">
SSDs as on NSD. <br class="null">
<br class="null">
I don't think like 5 or more data/metadata replicas are practical here. On the <br class="null">
other hand, multiple node failures is something really expected. <br class="null">
<br class="null">
Is there a way to instrument that local NSD is strongly preferred to store <br class="null">
data? I.e. node failure most probably does not result in unavailable data for <br class="null">
the other nodes? <br class="null">
<br class="null">
Or is there any other recommendation/solution to build shared scratch with <br class="null">
GPFS in such setup? (Do not do it including.) <br class="null">
<br class="null">
-- <br class="null">
Lukáš Hejtmánek <br class="null">
_______________________________________________ <br class="null">
gpfsug-discuss mailing list <br class="null">
gpfsug-discuss at spectrumscale.org <br class="null">
http://gpfsug.org/mailman/listinfo/gpfsug-discuss <br class="null">
</div>
</div>
</div>
</div>
</blockquote>
</div>
</body>
</html>