<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi Marc,
<div class=""><br class="">
</div>
<div class="">Thanks for the response … I understand what you’re saying, but since I’m asking for a 1 MB block size for metadata and a 4 MB block size for data and according to the chart in the mmcrfs man page both result in an 8 KB sub block size I’m still
confused as to why I’ve got a 32 KB sub block size for my non-system (i.e. data) pools? Especially when you consider that 32 KB isn’t the default even if I had chosen an 8 or 16 MB block size!</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class=""><br class="">
<div class="">
<div class="">—</div>
<div class="">Kevin Buterbaugh - Senior System Administrator</div>
<div class="">Vanderbilt University - Advanced Computing Center for Research and Education</div>
<div class=""><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" class="">Kevin.Buterbaugh@vanderbilt.edu</a> - (615)875-9633</div>
<div class=""><br class="">
</div>
</div>
<div>
<blockquote type="cite" class="">
<div class="">On Aug 1, 2018, at 12:21 PM, Marc A Kaplan <<a href="mailto:makaplan@us.ibm.com" class="">makaplan@us.ibm.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class=""><span style=" font-size:10pt;font-family:sans-serif" class="">I haven't looked into all the details but here's a clue -- notice there is only one "subblocks-per-full-block" parameter. </span><br class="">
<br class="">
<span style=" font-size:10pt;font-family:sans-serif" class="">And it is the same for both metadata blocks and datadata blocks.</span><br class="">
<span style=" font-size:10pt;font-family:sans-serif" class=""><br class="">
So maybe (MAYBE) that is a constraint somewhere...</span><br class="">
<br class="">
<span style=" font-size:10pt;font-family:sans-serif" class="">Certainly, in the currently supported code, that's what you get.<br class="">
</span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">From: </span><span style=" font-size:9pt;font-family:sans-serif" class="">"Buterbaugh, Kevin L" <<a href="mailto:Kevin.Buterbaugh@Vanderbilt.Edu" class="">Kevin.Buterbaugh@Vanderbilt.Edu</a>></span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">To: </span><span style=" font-size:9pt;font-family:sans-serif" class="">gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org" class="">gpfsug-discuss@spectrumscale.org</a>></span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">Date: </span><span style=" font-size:9pt;font-family:sans-serif" class="">08/01/2018 12:55 PM</span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">Subject: </span><span style=" font-size:9pt;font-family:sans-serif" class="">[gpfsug-discuss] Sub-block size wrong on GPFS 5 filesystem?</span><br class="">
<span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif" class="">Sent by: </span><span style=" font-size:9pt;font-family:sans-serif" class=""><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" class="">gpfsug-discuss-bounces@spectrumscale.org</a></span><br class="">
<hr noshade="" class="">
<br class="">
<br class="">
<br class="">
<span style=" font-size:12pt" class="">Hi All, </span><br class="">
<br class="">
<span style=" font-size:12pt" class="">Our production cluster is still on GPFS 4.2.3.x, but in preparation for moving to GPFS 5 I have upgraded our small (7 node) test cluster to GPFS 5.0.1-1. I am setting up a new filesystem there using hardware that we recently
life-cycled out of our production environment.</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">I “successfully” created a filesystem but I believe the sub-block size is wrong. I’m using a 4 MB filesystem block size, so according to the mmcrfs man page the sub-block size should be 8K:</span><br class="">
<br class="">
<span style=" font-size:12pt" class=""> Table 1. Block sizes and subblock sizes</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+</span><br class="">
<span style=" font-size:12pt" class="">| Block size | Subblock size |</span><br class="">
<span style=" font-size:12pt" class="">+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+</span><br class="">
<span style=" font-size:12pt" class="">| 64 KiB | 2 KiB |</span><br class="">
<span style=" font-size:12pt" class="">+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+</span><br class="">
<span style=" font-size:12pt" class="">| 128 KiB | 4 KiB |</span><br class="">
<span style=" font-size:12pt" class="">+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+</span><br class="">
<span style=" font-size:12pt" class="">| 256 KiB, 512 KiB, 1 MiB, 2 | 8 KiB |</span><br class="">
<span style=" font-size:12pt" class="">| MiB, 4 MiB | |</span><br class="">
<span style=" font-size:12pt" class="">+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+</span><br class="">
<span style=" font-size:12pt" class="">| 8 MiB, 16 MiB | 16 KiB |</span><br class="">
<span style=" font-size:12pt" class="">+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">However, it appears that it’s 8K for the system pool but 32K for the other pools:</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">flag value description</span><br class="">
<span style=" font-size:12pt" class="">------------------- ------------------------ -----------------------------------</span><br class="">
<span style=" font-size:12pt" class=""> -f 8192 Minimum fragment (subblock) size in bytes (system pool)</span><br class="">
<span style=" font-size:12pt" class=""> 32768 Minimum fragment (subblock) size in bytes (other pools)</span><br class="">
<span style=" font-size:12pt" class=""> -i 4096 Inode size in bytes</span><br class="">
<span style=" font-size:12pt" class=""> -I 32768 Indirect block size in bytes</span><br class="">
<span style=" font-size:12pt" class=""> -m 2 Default number of metadata replicas</span><br class="">
<span style=" font-size:12pt" class=""> -M 3 Maximum number of metadata replicas</span><br class="">
<span style=" font-size:12pt" class=""> -r 1 Default number of data replicas</span><br class="">
<span style=" font-size:12pt" class=""> -R 3 Maximum number of data replicas</span><br class="">
<span style=" font-size:12pt" class=""> -j scatter Block allocation type</span><br class="">
<span style=" font-size:12pt" class=""> -D nfs4 File locking semantics in effect</span><br class="">
<span style=" font-size:12pt" class=""> -k all ACL semantics in effect</span><br class="">
<span style=" font-size:12pt" class=""> -n 32 Estimated number of nodes that will mount file system</span><br class="">
<span style=" font-size:12pt" class=""> -B 1048576 Block size (system pool)</span><br class="">
<span style=" font-size:12pt" class=""> 4194304 Block size (other pools)</span><br class="">
<span style=" font-size:12pt" class=""> -Q user;group;fileset Quotas accounting enabled</span><br class="">
<span style=" font-size:12pt" class=""> user;group;fileset Quotas enforced</span><br class="">
<span style=" font-size:12pt" class=""> none Default quotas enabled</span><br class="">
<span style=" font-size:12pt" class=""> --perfileset-quota No Per-fileset quota enforcement</span><br class="">
<span style=" font-size:12pt" class=""> --filesetdf No Fileset df enabled?</span><br class="">
<span style=" font-size:12pt" class=""> -V 19.01 (5.0.1.0) File system version</span><br class="">
<span style=" font-size:12pt" class=""> --create-time Wed Aug 1 11:39:39 2018 File system creation time</span><br class="">
<span style=" font-size:12pt" class=""> -z No Is DMAPI enabled?</span><br class="">
<span style=" font-size:12pt" class=""> -L 33554432 Logfile size</span><br class="">
<span style=" font-size:12pt" class=""> -E Yes Exact mtime mount option</span><br class="">
<span style=" font-size:12pt" class=""> -S relatime Suppress atime mount option</span><br class="">
<span style=" font-size:12pt" class=""> -K whenpossible Strict replica allocation option</span><br class="">
<span style=" font-size:12pt" class=""> --fastea Yes Fast external attributes enabled?</span><br class="">
<span style=" font-size:12pt" class=""> --encryption No Encryption enabled?</span><br class="">
<span style=" font-size:12pt" class=""> --inode-limit 101095424 Maximum number of inodes</span><br class="">
<span style=" font-size:12pt" class=""> --log-replicas 0 Number of log replicas</span><br class="">
<span style=" font-size:12pt" class=""> --is4KAligned Yes is4KAligned?</span><br class="">
<span style=" font-size:12pt" class=""> --rapid-repair Yes rapidRepair enabled?</span><br class="">
<span style=" font-size:12pt" class=""> --write-cache-threshold 0 HAWC Threshold (max 65536)</span><br class="">
<span style=" font-size:12pt" class=""> --subblocks-per-full-block 128 Number of subblocks per full block</span><br class="">
<span style=" font-size:12pt" class=""> -P system;raid1;raid6 Disk storage pools in file system</span><br class="">
<span style=" font-size:12pt" class=""> --file-audit-log No File Audit Logging enabled?</span><br class="">
<span style=" font-size:12pt" class=""> --maintenance-mode No Maintenance Mode enabled?</span><br class="">
<span style=" font-size:12pt" class=""> -d test21A3nsd;test21A4nsd;test21B3nsd;test21B4nsd;test23Ansd;test23Bnsd;test23Cnsd;test24Ansd;test24Bnsd;test24Cnsd;test25Ansd;test25Bnsd;test25Cnsd Disks in file system</span><br class="">
<span style=" font-size:12pt" class=""> -A yes Automatic mount option</span><br class="">
<span style=" font-size:12pt" class=""> -o none Additional mount options</span><br class="">
<span style=" font-size:12pt" class=""> -T /gpfs5 Default mount point</span><br class="">
<span style=" font-size:12pt" class=""> --mount-priority 0 Mount priority</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">Output of mmcrfs:</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">mmcrfs gpfs5 -F ~/gpfs/gpfs5.stanza -A yes -B 4M -E yes -i 4096 -j scatter -k all -K whenpossible -m 2 -M 3 -n 32 -Q yes -r 1 -R 3 -T /gpfs5 -v yes --nofilesetdf --metadata-block-size 1M</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">The following disks of gpfs5 will be formatted on node testnsd3:</span><br class="">
<span style=" font-size:12pt" class=""> test21A3nsd: size 953609 MB</span><br class="">
<span style=" font-size:12pt" class=""> test21A4nsd: size 953609 MB</span><br class="">
<span style=" font-size:12pt" class=""> test21B3nsd: size 953609 MB</span><br class="">
<span style=" font-size:12pt" class=""> test21B4nsd: size 953609 MB</span><br class="">
<span style=" font-size:12pt" class=""> test23Ansd: size 15259744 MB</span><br class="">
<span style=" font-size:12pt" class=""> test23Bnsd: size 15259744 MB</span><br class="">
<span style=" font-size:12pt" class=""> test23Cnsd: size 1907468 MB</span><br class="">
<span style=" font-size:12pt" class=""> test24Ansd: size 15259744 MB</span><br class="">
<span style=" font-size:12pt" class=""> test24Bnsd: size 15259744 MB</span><br class="">
<span style=" font-size:12pt" class=""> test24Cnsd: size 1907468 MB</span><br class="">
<span style=" font-size:12pt" class=""> test25Ansd: size 15259744 MB</span><br class="">
<span style=" font-size:12pt" class=""> test25Bnsd: size 15259744 MB</span><br class="">
<span style=" font-size:12pt" class=""> test25Cnsd: size 1907468 MB</span><br class="">
<span style=" font-size:12pt" class="">Formatting file system ...</span><br class="">
<span style=" font-size:12pt" class="">Disks up to size 8.29 TB can be added to storage pool system.</span><br class="">
<span style=" font-size:12pt" class="">Disks up to size 16.60 TB can be added to storage pool raid1.</span><br class="">
<span style=" font-size:12pt" class="">Disks up to size 132.62 TB can be added to storage pool raid6.</span><br class="">
<span style=" font-size:12pt" class="">Creating Inode File</span><br class="">
<span style=" font-size:12pt" class=""> 8 % complete on Wed Aug 1 11:39:19 2018</span><br class="">
<span style=" font-size:12pt" class=""> 18 % complete on Wed Aug 1 11:39:24 2018</span><br class="">
<span style=" font-size:12pt" class=""> 27 % complete on Wed Aug 1 11:39:29 2018</span><br class="">
<span style=" font-size:12pt" class=""> 37 % complete on Wed Aug 1 11:39:34 2018</span><br class="">
<span style=" font-size:12pt" class=""> 48 % complete on Wed Aug 1 11:39:39 2018</span><br class="">
<span style=" font-size:12pt" class=""> 60 % complete on Wed Aug 1 11:39:44 2018</span><br class="">
<span style=" font-size:12pt" class=""> 72 % complete on Wed Aug 1 11:39:49 2018</span><br class="">
<span style=" font-size:12pt" class=""> 83 % complete on Wed Aug 1 11:39:54 2018</span><br class="">
<span style=" font-size:12pt" class=""> 95 % complete on Wed Aug 1 11:39:59 2018</span><br class="">
<span style=" font-size:12pt" class=""> 100 % complete on Wed Aug 1 11:40:01 2018</span><br class="">
<span style=" font-size:12pt" class="">Creating Allocation Maps</span><br class="">
<span style=" font-size:12pt" class="">Creating Log Files</span><br class="">
<span style=" font-size:12pt" class=""> 3 % complete on Wed Aug 1 11:40:07 2018</span><br class="">
<span style=" font-size:12pt" class=""> 28 % complete on Wed Aug 1 11:40:14 2018</span><br class="">
<span style=" font-size:12pt" class=""> 53 % complete on Wed Aug 1 11:40:19 2018</span><br class="">
<span style=" font-size:12pt" class=""> 78 % complete on Wed Aug 1 11:40:24 2018</span><br class="">
<span style=" font-size:12pt" class=""> 100 % complete on Wed Aug 1 11:40:25 2018</span><br class="">
<span style=" font-size:12pt" class="">Clearing Inode Allocation Map</span><br class="">
<span style=" font-size:12pt" class="">Clearing Block Allocation Map</span><br class="">
<span style=" font-size:12pt" class="">Formatting Allocation Map for storage pool system</span><br class="">
<span style=" font-size:12pt" class=""> 85 % complete on Wed Aug 1 11:40:32 2018</span><br class="">
<span style=" font-size:12pt" class=""> 100 % complete on Wed Aug 1 11:40:33 2018</span><br class="">
<span style=" font-size:12pt" class="">Formatting Allocation Map for storage pool raid1</span><br class="">
<span style=" font-size:12pt" class=""> 53 % complete on Wed Aug 1 11:40:38 2018</span><br class="">
<span style=" font-size:12pt" class=""> 100 % complete on Wed Aug 1 11:40:42 2018</span><br class="">
<span style=" font-size:12pt" class="">Formatting Allocation Map for storage pool raid6</span><br class="">
<span style=" font-size:12pt" class=""> 20 % complete on Wed Aug 1 11:40:47 2018</span><br class="">
<span style=" font-size:12pt" class=""> 39 % complete on Wed Aug 1 11:40:52 2018</span><br class="">
<span style=" font-size:12pt" class=""> 60 % complete on Wed Aug 1 11:40:57 2018</span><br class="">
<span style=" font-size:12pt" class=""> 79 % complete on Wed Aug 1 11:41:02 2018</span><br class="">
<span style=" font-size:12pt" class=""> 100 % complete on Wed Aug 1 11:41:08 2018</span><br class="">
<span style=" font-size:12pt" class="">Completed creation of file system /dev/gpfs5.</span><br class="">
<span style=" font-size:12pt" class="">mmcrfs: Propagating the cluster configuration data to all</span><br class="">
<span style=" font-size:12pt" class=""> affected nodes. This is an asynchronous process.</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">And contents of stanza file:</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test21A3nsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=metadataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=210</span><br class="">
<span style=" font-size:12pt" class=""> pool=system</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd3,testnsd1,testnsd2</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-15</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test21A4nsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=metadataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=210</span><br class="">
<span style=" font-size:12pt" class=""> pool=system</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd1,testnsd2,testnsd3</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-14</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test21B3nsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=metadataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=211</span><br class="">
<span style=" font-size:12pt" class=""> pool=system</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd1,testnsd2,testnsd3</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-17</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test21B4nsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=metadataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=211</span><br class="">
<span style=" font-size:12pt" class=""> pool=system</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd2,testnsd3,testnsd1</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-16</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test23Ansd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=23</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid6</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd2,testnsd3,testnsd1</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-10</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test23Bnsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=23</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid6</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd3,testnsd1,testnsd2</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-9</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test23Cnsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=23</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid1</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd1,testnsd2,testnsd3</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-5</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test24Ansd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=24</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid6</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd3,testnsd1,testnsd2</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-6</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test24Bnsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=24</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid6</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd1,testnsd2,testnsd3</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-0</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test24Cnsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=24</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid1</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd2,testnsd3,testnsd1</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-2</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test25Ansd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=25</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid6</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd1,testnsd2,testnsd3</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-6</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test25Bnsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=25</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid6</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd2,testnsd3,testnsd1</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-6</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%nsd:</span><br class="">
<span style=" font-size:12pt" class=""> nsd=test25Cnsd</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> failureGroup=25</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid1</span><br class="">
<span style=" font-size:12pt" class=""> servers=testnsd3,testnsd1,testnsd2</span><br class="">
<span style=" font-size:12pt" class=""> device=dm-3</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%pool:</span><br class="">
<span style=" font-size:12pt" class=""> pool=system</span><br class="">
<span style=" font-size:12pt" class=""> blockSize=1M</span><br class="">
<span style=" font-size:12pt" class=""> usage=metadataOnly</span><br class="">
<span style=" font-size:12pt" class=""> layoutMap=scatter</span><br class="">
<span style=" font-size:12pt" class=""> allowWriteAffinity=no</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%pool:</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid6</span><br class="">
<span style=" font-size:12pt" class=""> blockSize=4M</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> layoutMap=scatter</span><br class="">
<span style=" font-size:12pt" class=""> allowWriteAffinity=no</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">%pool:</span><br class="">
<span style=" font-size:12pt" class=""> pool=raid1</span><br class="">
<span style=" font-size:12pt" class=""> blockSize=4M</span><br class="">
<span style=" font-size:12pt" class=""> usage=dataOnly</span><br class="">
<span style=" font-size:12pt" class=""> layoutMap=scatter</span><br class="">
<span style=" font-size:12pt" class=""> allowWriteAffinity=no</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">What am I missing or what have I done wrong? Thanks…</span><br class="">
<br class="">
<span style=" font-size:12pt" class="">Kevin</span><br class="">
<span style=" font-size:12pt" class="">—</span><br class="">
<span style=" font-size:12pt" class="">Kevin Buterbaugh - Senior System Administrator</span><br class="">
<span style=" font-size:12pt" class="">Vanderbilt University - Advanced Computing Center for Research and Education</span><br class="">
<a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" class=""><span style=" font-size:12pt;color:blue" class=""><u class="">Kevin.Buterbaugh@vanderbilt.edu</u></span></a><span style=" font-size:12pt" class="">- (615)875-9633</span><br class="">
<br class="">
<br class="">
<tt class=""><span style=" font-size:10pt" class="">_______________________________________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class="">
</span></tt><a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7Cd84fdde05c65406d4d9008d5f7d32f0f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636687408760525035&sdata=YIa8UdCmnzKTXU4cWYHxn8Sn%2Fhmnkcb8e0sCnPXD8C4%3D&reserved=0" originalsrc="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" shash="qqk62KHnCHYOSs8I/2+qpHWw3SAWavCXay0m7S6+SEmx1cqpTUxs4MBxzcyHZknIaHVivxruIKOS6sgugKZAhawlvwxP/ZGZ5W3UCGiE1ZXCB6r93pAi8JVnQygtEqJxqzBn9xoF1SXQv846o8ZayyaKOdHQcvosWkjj3h0MZLg=" class=""><tt class=""><span style=" font-size:10pt" class="">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt class=""><span style=" font-size:10pt" class=""><br class="">
</span></tt><br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
gpfsug-discuss mailing list<br class="">
gpfsug-discuss at <a href="http://spectrumscale.org" class="">spectrumscale.org</a><br class="">
<a href="https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7Cd84fdde05c65406d4d9008d5f7d32f0f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636687408760535040&sdata=hqVZVIQLbxakARTspzbSkMZBHi2b6%2BIcrPLU1atNbus%3D&reserved=0" class="">https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7Cd84fdde05c65406d4d9008d5f7d32f0f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636687408760535040&sdata=hqVZVIQLbxakARTspzbSkMZBHi2b6%2BIcrPLU1atNbus%3D&reserved=0</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>