<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi All,
<div class=""><br class="">
</div>
<div class="">Just wanted to follow up with the results of my survey … I received a grand total of two responses (Thanks Alex and John). In their case, they’re using SSDs with a 10 DWPD rating.</div>
<div class=""><br class="">
</div>
<div class="">The motivation behind my asking this question was … money! ;-). Seriously, 10 DWPD drives are still very expensive, while 3 DWPD drives are significantly less expensive and 1 DWPD drives are even cheaper still. While we would NOT feel comfortable
using anything less than 10 DWPD drives for metadata, we’re wondering about using less expensive drives for data.</div>
<div class=""><br class="">
</div>
<div class="">For example, let’s just say that you’re getting ready to set up a brand new GPFS 5 formatted filesystem of 1-2 PB in size. You’re considering having 3 pools:</div>
<div class=""><br class="">
</div>
<div class="">1) a metadata only system pool of 10 DWPD SSDs. 4K inodes, and a ton of small files that’ll fit in the inode.</div>
<div class="">2) a data only “hot” pool (i.e. the default pool for writes) of SSDs.</div>
<div class="">3) a data only “capacity” pool of 12 TB spinning disks.</div>
<div class=""><br class="">
</div>
<div class="">And let’s just say that you have looked back at the historical data you’ve collected and you see that over the last 6 months or so you’ve been averaging 10-12 TB of data being written into your existing filesystem per day. You want to do migrations
between pools only on the weekends if at all possible.</div>
<div class=""><br class="">
</div>
<div class="">12 * 7 = 84 TB. So if you had somewhere between 125 - 150 TB of SSDs ... 1 DWPD SSDs … then in theory you should easily be able to handle your anticipated workload without coming close to exceeding the 1 DWPD rating of the SSDs.</div>
<div class=""><br class="">
</div>
<div class="">However, as the saying goes, while in theory there’s no difference between theory and practice, in practice there is ... so am I overlooking anything here from a GPFS perspective???</div>
<div class=""><br class="">
</div>
<div class="">If anybody still wants to respond on the DWPD rating of the SSDs they use for data, I’m still listening.</div>
<div class=""><br class="">
</div>
<div class="">Thanks…</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class=""><br class="">
</div>
<div class="">P.S. I still have a couple of “outstanding issues” to respond to that I’ve posted to the list about previously:</div>
<div class=""><br class="">
</div>
<div class="">1) the long I/O’s we see occasionally in the output of “mmdiag —iohist” on our NSD servers. We’re still trying to track that down … it seems to happen only with a subset of our hardware - most of the time at least - but we’re still working to
track down what triggers it … i.e. at this point I can’t say whether it’s really the hardware or a user abusing the hardware.</div>
<div class=""><br class="">
</div>
<div class="">2) I promised to post benchmark results of 3 different metadata configs: a) RAID 1 mirrors, b) a RAID 5 stripe, c) no RAID, but GPFS metadata replication of 3. That benchmarking has been put on hold for reasons I can’t really discuss on this
mailing list at this time … but hopefully soon.</div>
<div class=""><br class="">
</div>
<div class="">I haven’t forgotten the above and will respond back on the list when it’s appropriate. Thanks...<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Mar 8, 2019, at 10:24 AM, Buterbaugh, Kevin L <<a href="mailto:Kevin.Buterbaugh@Vanderbilt.Edu" class="">Kevin.Buterbaugh@Vanderbilt.Edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi All,
<div class=""><br class="">
</div>
<div class="">This is kind of a survey if you will, so for this one it might be best if you responded directly to me and I’ll summarize the results next week.</div>
<div class=""><br class="">
</div>
<div class="">Question 1 - do you use SSDs for data? If not - i.e. if you only use SSDs for metadata (as we currently do) - thanks, that’s all! If, however, you do use SSDs for data, please see Question 2.</div>
<div class=""><br class="">
</div>
<div class="">Question 2 - what is the DWPD (daily writes per day) of the SSDs that you use for data?</div>
<div class=""><br class="">
</div>
<div class="">Question 3 - is that different than the DWPD of the SSDs for metadata?</div>
<div class=""><br class="">
</div>
<div class="">Question 4 - any pertinent information in regards to your answers above (i.e. if you’ve got a filesystem that data is uploaded to only once and never modified after that then that’s useful to know!)?</div>
<div class=""><br class="">
</div>
<div class="">Thanks…</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class=""><br class="">
<div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class="">—</div>
<div class="">Kevin Buterbaugh - Senior System Administrator</div>
<div class="">Vanderbilt University - Advanced Computing Center for Research and Education</div>
<div class=""><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" class="">Kevin.Buterbaugh@vanderbilt.edu</a> - (615)875-9633</div>
</div>
</div>
<br class="">
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>