[gpfsug-discuss] Blocksize and MetaData Blocksizes - FORGET the old advice

[Marc A Kaplan]

Metadata is inodes, directories, indirect blocks (indices). 

Spectrum Scale (GPFS) Version 4.1 introduced significant improvements to 
the data structures used to represent directories.
Larger inodes supporting data and extended attributes in the inode are 
other significant relatively recent improvements.

Now small directories are stored in the inode, while for large directories 
blocks can be bigger than 32MB, and any and all directory blocks that are 
smaller than
the metadata-blocksize, are allocated just like "fragments" - so 
directories are now space efficient.

SO MUCH SO, that THE OLD ADVICE, about using smallish blocksizes for 
metadata, GOES "OUT THE WINDOW".  Period. FORGET most of what you thought
you knew about "best" or "optimal" metadata-blocksize.

The new advice is, as Sven wrote: 

Use a blocksize that optimizes IO transfer efficiency and speed.
This is true for BOTH data and metadata.

Now, IF you have system pool set up as metadata only AND system pool is on 
devices that have a different "optimal" block size than your other pools,
THEN, it may make sense to use two different blocksizes, one for data and 
another for metadata.

For example, maybe you have massively striped RAID or RAID-LIKE (GSS or 
ESS)) storage for huge files - so maybe 8MB is a good blocksize for that.
But maybe you have your metadata on SSD devices and maybe 1MB is the 
"best" blocksize for that.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20160924/f4bcd724/attachment-0002.htm>