[gpfsug-discuss] How to prove that data is in inode

Billich Heinrich Rainer (ID SD) heinrich.billich at id.ethz.ch
Thu Jul 18 15:15:12 BST 2019


Hello Kums,

Thank you; I could verify that the data of small files is in the inode. In the table below you see the filesize and the result of the tsdbfs query. Up to 3k all data is in the inode.

The result of stat calls on small files is very puzzling. When data is in the inode stat reports one block of 512 bytes used, even for 3k of data. I don’t see how this would affect any of our applications, so it’s just something to note.

I append some output and the script to generate it, just for completeness.

Cheers,

Heiner

# stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel
0  indirectionLevel=DIRECT status=USERFILE. <empty file is special>
1  indirectionLevel=INODE status=USERFILE
16  indirectionLevel=INODE status=USERFILE
512  indirectionLevel=INODE status=USERFILE
1k  indirectionLevel=INODE status=USERFILE
2k  indirectionLevel=INODE status=USERFILE
3k  indirectionLevel=INODE status=USERFILE.  < up to 3k data is in inode >

4k  indirectionLevel=DIRECT status=USERFILE
16k  indirectionLevel=DIRECT status=USERFILE
64k  indirectionLevel=DIRECT status=USERFILE
1M  indirectionLevel=DIRECT status=USERFILE
2M  indirectionLevel=DIRECT status=USERFILE


Stat output

# stat -c “%n size: %s  allocated: %b*%B”

# stat -c %n size: %s  allocated: %b*%B
0 size: 0  allocated: 0*512
1 size: 1  allocated: 1*512
16 size: 16  allocated: 1*512
512 size: 512  allocated: 1*512
1k size: 1024  allocated: 1*512
2k size: 2048  allocated: 1*512
3k size: 3072  allocated: 1*512  < 3k file and  all data in inode:  stat reports 1*512 allocated/used)
4k size: 4096  allocated: 64*512
16k size: 16384  allocated: 64*512.   < as expected, 32k subblock size >
64k size: 65536  allocated: 128*512
1M size: 1048576  allocated: 2048*512
2M size: 2097152  allocated: 4096*512

The script:

# test-data-in-inode.sh
sizes="0 1 16 512 1k 2k 3k 4k 16k 64k 1M 2M"

echo create files
for s in $sizes
do
  head -c $s /dev/zero > $s
done

echo sleep 20. # give gpfs some time to update metadata
sleep 20

echo
echo "# ls -ls $sizes"
ls -ls $sizes

echo
echo "# stat -c %n size: %s  allocated: %b*%B"
stat -c "%n size: %s  allocated: %b*%B" $sizes

echo
echo '# stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel'
for f in $sizes
do
echo -n $f
stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel
done

From: <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Kumaran Rajaram <kums at us.ibm.com>
Reply to: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: Wednesday, 17 July 2019 at 14:38
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] How to prove that data is in inode


Hi,

>> How can I prove that data of a small file is stored in the inode (and not on a data nsd)?

You may use echo "inode file_inode_number" | tsdbfs fs_device | grep indirectionLevel and if it points to INODE, then the file is stored in the inodes

# 4K Inode Size
# mmlsfs gpfs3a | grep 'Inode size'
-i 4096 Inode size in bytes

# Small file
# ls -l /mnt/gpfs3a/hello.txt
-rw-r--r-- 1 root root 6 Jul 17 08:32 /mnt/gpfs3a/hello.txt

# ls -i /mnt/gpfs3a/hello.txt
91649 /mnt/gpfs3a/hello.txt

#File is inlined within Inode
# echo "inode 91649" | tsdbfs gpfs3a | grep indirectionLevel
indirectionLevel=INODE status=USERFILE

Regards,
-Kums



[Inactive hide details for "Billich  Heinrich Rainer (ID SD)" ---07/17/2019 07:49:56 AM---Hello, How can I prove that data of a]"Billich Heinrich Rainer (ID SD)" ---07/17/2019 07:49:56 AM---Hello, How can I prove that data of a small file is stored in the inode (and not on a data nsd)?

From: "Billich Heinrich Rainer (ID SD)" <heinrich.billich at id.ethz.ch>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: 07/17/2019 07:49 AM
Subject: [EXTERNAL] [gpfsug-discuss] How to prove that data is in inode
Sent by: gpfsug-discuss-bounces at spectrumscale.org

________________________________



Hello,

How can I prove that data of a small file is stored in the inode (and not on a data nsd)?

We have a filesystem with 4k inodes on Scale 5.0.2 , but it seems there is no file data in the inodes?

I would expect that 'stat' reports 'Blocks: 0'  for a small file, but I see 'Blocks:1'.

Cheers,

Heiner

I tried

[]# rm -f test; echo hello > test
[]# ls -ls test
1 -rw-r--r-- 1 root root 6 Jul 17 13:11 test
[root at testnas13ems01 test]# stat test
 File: ‘test’
 Size: 6         Blocks: 1          IO Block: 1048576 regular file
Device: 2dh/45d Inode: 353314      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2019-07-17 13:11:03.037049000 +0200
Modify: 2019-07-17 13:11:03.037331000 +0200
Change: 2019-07-17 13:11:03.037259319 +0200
Birth: -
[root at testnas13ems01 test]# du test
1 test
[root at testnas13ems01 test]# du -b test
6 test
[root at testnas13ems01 test]#

Filesystem

# mmlsfs f****
flag                value                    description
------------------- ------------------------ -----------------------------------
-f                 32768                    Minimum fragment (subblock) size in bytes
-i                 4096                     Inode size in bytes
-I                 32768                    Indirect block size in bytes
-m                 1                        Default number of metadata replicas
-M                 2                        Maximum number of metadata replicas
-r                 1                        Default number of data replicas
-R                 2                        Maximum number of data replicas
-j                 cluster                  Block allocation type
-D                 nfs4                     File locking semantics in effect
-k                 nfs4                     ACL semantics in effect
-n                 32                       Estimated number of nodes that will mount file system
-B                 1048576                  Block size
-Q                 user;group;fileset       Quotas accounting enabled
                   user;group;fileset       Quotas enforced
                   user;group;fileset       Default quotas enabled
--perfileset-quota Yes                      Per-fileset quota enforcement
--filesetdf        Yes                      Fileset df enabled?
-V                 20.01 (5.0.2.0)          Current file system version
                   15.01 (4.2.0.0)          Original file system version
--create-time      ***** 2017 File system creation time
-z                 No                       Is DMAPI enabled?
-L                 33554432                 Logfile size
-E                 Yes                      Exact mtime mount option
-S                 relatime                 Suppress atime mount option
-K                 whenpossible             Strict replica allocation option
--fastea           Yes                      Fast external attributes enabled?
--encryption       No                       Encryption enabled?
--inode-limit      1294592                  Maximum number of inodes in all inode spaces
--log-replicas     0                        Number of log replicas
--is4KAligned      Yes                      is4KAligned?
--rapid-repair     Yes                      rapidRepair enabled?
--write-cache-threshold 0                   HAWC Threshold (max 65536)
--subblocks-per-full-block 32               Number of subblocks per full block
-P                 system;data              Disk storage pools in file system
--file-audit-log   No                       File Audit Logging enabled?
--maintenance-mode No                       Maintenance Mode enabled?
-d                 ******
-A                 yes                      Automatic mount option
-o                 nfssync,nodev            Additional mount options
-T                 /****                  Default mount point
--mount-priority   0                        Mount priority

--
=======================
Heinrich Billich
ETH Zürich
Informatikdienste
Tel.: +41 44 632 72 56
heinrich.billich at id.ethz.ch
========================



_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190718/d044120a/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 106 bytes
Desc: image001.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190718/d044120a/attachment-0002.gif>


More information about the gpfsug-discuss mailing list