<html><body><p><tt><font size="2">Dave Johnson at ddj@brown.edu asks:</font></tt><br>
<ul><tt><i><font size="2">When GPFS needs to add inodes to the filesystem, it seems to pre-create about 4 million of them.<br>Judging by the logs, it seems it only takes a few (13 maybe) seconds to do this.<br>However we are suspecting that this might only be to request the additional inodes and <br>that there is some background activity for some time afterwards. <br>Would someone who has knowledge of the actual internals be willing to confirm or deny this,<br>and if there is background activity, is it on all nodes in the cluster, NSD nodes, "default worker nodes"?</font></i></tt><tt><font size="2"><br></font></tt></ul><font size="2">Inodes are typically 4KB and reside ondisk in full blocks in the "inode 0 file". For every inode there is also an entry in the "inode allocation map" which indicates the inode's status (eg free, inuse). To add inodes we have to add data to both. First we determine how many inodes to add (eg always add full blocks of inodes, etc), then how many "passes" will it take to add them (the "passes" are an artifact of the inode map layout). Adding the inodes themselves involves writing blocks of free inodes. This is multi-threaded on a single node. Adding to the inode map, may involve adding more inode map "segments" or just using free space in the current segments. If adding segments these are formatted and written by multiple threads on a single node, Once the on-disk data structures are complete we update the in-memory structures to reflect that all of the new inodes are free and we update the "stripe group descriptor" and broadcast it to all the nodes that have the file system mounted.</font><br><br><font size="2">In old code - say pre 4.1 or 4.2 -- we went through another step to reread all of the inode allocation map back into memory to recompute the number of free inodes. That would have been in parallel on all the nodes that had the file system mounted. Around 4.2 or so this was changed to simply update the in-memory counters (since we know how many inodes were added, there is no need to recount them). </font><br><br><font size="2">So, adding 4M inodes involves writing a little more than 16 GB of metadata to the disk, then cycle through the in-memory data structures. Writing 16 GB in 13 seconds works out to a little more than 1 GB/s. Sounds reasonable.</font><br><br><font size="2">-Wayne</font><br><br><BR>
</body></html>