<span style=" font-size:10pt;font-family:sans-serif">A somewhat smarter
RAID controller will "only" need to read the old values of the
single changed segment of data and the corresponding parity segment, and
know the new value of the data block. Then it can compute the new parity
segment value.</span><br><span style=" font-size:10pt;font-family:sans-serif"><br>Not necessarily the entire stripe. Still 2 reads and 2 writes +
access delay times ( guaranteed more than one full rotation time when on
spinning disks, average something like 1.7x rotation time ).<br></span><br><br><br><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:
</span><span style=" font-size:9pt;font-family:sans-serif">"Uwe
Falke" <UWEFALKE@de.ibm.com></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:
</span><span style=" font-size:9pt;font-family:sans-serif">gpfsug
main discussion list <gpfsug-discuss@spectrumscale.org></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:
</span><span style=" font-size:9pt;font-family:sans-serif">09/05/2018
04:07 PM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:
</span><span style=" font-size:9pt;font-family:sans-serif">Re:
[gpfsug-discuss] RAID type for system pool</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Sent
by: </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><br><hr noshade><br><br><br><tt><span style=" font-size:10pt">Hi, <br><br>just think that your RAID controller on parity-backed redundancy needs
to <br>read the full stripe, modify it, and write it back (including parity) -
<br>the infamous Read-Modify-Write penalty.<br>As long as your users don't bulk-create inodes and doo amend some <br>metadata, (create a file sometimes, e.g.) The writing of a 4k inode, or
<br>the update of a 32k dir block causes your controller to read a full block
<br>(let's say you use 1MiB on MD) and write back the full block plus parity
<br>(on 4+1p RAID 5 at 1MiB that'll be 1.25MiB. Overhead two orders of <br>magnitude above the payload. <br>SSDs have become better now and expensive enterprise SSDs will endure
<br>quite a lot of full rewrites, but you need to estimate the MD change rate,
<br>apply the RMW overhead and see where you end WRT lifetime (and <br>performance). <br> <br><br><br> <br>Mit freundlichen Grüßen / Kind regards<br><br> <br>Dr. Uwe Falke<br> <br>IT Specialist<br>High Performance Computing Services / Integrated Technology Services /
<br>Data Center Services<br>-------------------------------------------------------------------------------------------------------------------------------------------<br>IBM Deutschland<br>Rathausstr. 7<br>09111 Chemnitz<br>Phone: +49 371 6978 2165<br>Mobile: +49 175 575 2877<br>E-Mail: uwefalke@de.ibm.com<br>-------------------------------------------------------------------------------------------------------------------------------------------<br>IBM Deutschland Business & Technology Services GmbH / Geschäftsführung:
<br>Thomas Wolter, Sven Schooß<br>Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
<br>HRB 17122 <br><br><br><br><br>From: "Buterbaugh, Kevin L" <Kevin.Buterbaugh@Vanderbilt.Edu><br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Date: 05/09/2018 17:35<br>Subject: [gpfsug-discuss] RAID type for system
pool<br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br><br><br><br>Hi All, <br><br>We are in the process of finalizing the purchase of some new storage <br>arrays (so no sales people who might be monitoring this list need contact
<br>me) to life-cycle some older hardware. One of the things we are <br>considering is the purchase of some new SSD?s for our ?/home? filesystem
<br>and I have a question or two related to that.<br><br>Currently, the existing home filesystem has it?s metadata on SSD?s ? two
<br>RAID 1 mirrors and metadata replication set to two. However, the
<br>filesystem itself is old enough that it uses 512 byte inodes. We
have <br>analyzed our users files and know that if we create a new filesystem with
<br>4K inodes that a very significant portion of the files would now have <br>their _data_ stored in the inode as well due to the files being 3.5K or
<br>smaller (currently all data is on spinning HD RAID 1 mirrors).<br><br>Of course, if we increase the size of the inodes by a factor of 8 then
we <br>also need 8 times as much space to store those inodes. Given that
<br>Enterprise class SSDs are still very expensive and our budget is not <br>unlimited, we?re trying to get the best bang for the buck.<br><br>We have always - even back in the day when our metadata was on spinning
<br>disk and not SSD - used RAID 1 mirrors and metadata replication of two.
<br>However, we are wondering if it might be possible to switch to RAID 5?
<br>Specifically, what we are considering doing is buying 8 new SSDs and <br>creating two 3+1P RAID 5 LUNs (metadata replication would stay at two).
<br>That would give us 50% more usable space than if we configured those same
<br>8 drives as four RAID 1 mirrors.<br><br>Unfortunately, unless I?m misunderstanding something, mean that the RAID
<br>stripe size and the GPFS block size could not match. Therefore, even
<br>though we don?t need the space, would we be much better off to buy 10 SSDs
<br>and create two 4+1P RAID 5 LUNs?<br><br>I?ve searched the mailing list archives and scanned the DeveloperWorks
<br>wiki and even glanced at the GPFS documentation and haven?t found anything
<br>that says ?bad idea, Kevin?? ;-)<br><br>Expanding on this further ? if we just present those two RAID 5 LUNs to
<br>GPFS as NSDs then we can only have two NSD servers as primary for them.
So <br>another thing we?re considering is to take those RAID 5 LUNs and further
<br>sub-divide them into a total of 8 logical volumes, each of which could
be <br>a GPFS NSD and therefore would allow us to have each of our 8 NSD servers
<br>be primary for one of them. Even worse idea?!? Good idea?<br><br>Anybody have any better ideas??? ;-)<br><br>Oh, and currently we?re on GPFS 4.2.3-10, but are also planning on moving
<br>to GPFS 5.0.1-x before creating the new filesystem.<br><br>Thanks much?<br><br>?<br>Kevin Buterbaugh - Senior System Administrator<br>Vanderbilt University - Advanced Computing Center for Research and <br>Education<br>Kevin.Buterbaugh@vanderbilt.edu - (615)875-9633<br><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></span></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style=" font-size:10pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt><span style=" font-size:10pt"><br><br><br><br><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></span></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style=" font-size:10pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt><span style=" font-size:10pt"><br><br></span></tt><br><br><BR>