<font size=2 face="sans-serif">Hi,</font><br><br><font size=2 face="sans-serif">Yes, you could use "mmdf"
to obtain file-system "usage" across the NSDs (comprising the
file-system).</font><br><br><font size=2 face="sans-serif">If you want to obtain "data block
distribution corresponding to a file across the NSDs", then there
is a utility "</font><font size=2 color=blue face="sans-serif"><i>mmgetlocation</i></font><font size=2 face="sans-serif">"
in </font><font size=2 color=blue face="Courier New"><i>/usr/lpp/mmfs/samples/fpo</i></font><font size=2 face="sans-serif">that can be used to get file-data-blocks to NSD mapping. </font><br><br><font size=2 face="sans-serif"><b><u>Example: </u></b></font><br><br><font size=2 face="sans-serif"># File-system comprises of single storage
pool, all NSDs configured as dataAndMetadata, -m 1 -r 1, FS block-size=2MiB</font><br><font size=2 color=blue face="Courier New"><i># mmlsfs gpfs1b | grep
'Block size'</i></font><br><font size=2 color=blue face="Courier New"><i> -B
2097152
Block size</i></font><br><br><font size=2 face="sans-serif"># The file-system is comprised of 10
x dataAndMetadata NSDs</font><br><font size=2 color=blue face="Courier New"><i># mmlsdisk gpfs1b | grep
DMD | wc -l</i></font><br><font size=2 color=blue face="Courier New"><i>10</i></font><br><br><font size=2 face="sans-serif"># Create a sample file that is 40MiB
(20 data blocks)</font><br><font size=2 color=blue face="Courier New"><i>/mnt/sw/benchmarks/gpfsperf/gpfsperf
create seq -r 2m -n 40m /mnt/gpfs1b/temp_dir/lf.s.1</i></font><br><br><font size=2 face="sans-serif"># File size is 40 MiB</font><br><font size=2 color=blue face="Courier New"><i>(09:52:49) c25m3n07:~
# ls -lh /mnt/gpfs1b/temp_dir/lf.s.1</i></font><br><font size=2 color=blue face="Courier New"><i>-rw-r--r-- 1 root root
40M Mar 17 09:52 /mnt/gpfs1b/temp_dir/lf.s.1</i></font><br><font size=2 color=blue face="Courier New"><i>(09:52:54) c25m3n07:~
# du -sh /mnt/gpfs1b/temp_dir/lf.s.1</i></font><br><font size=2 color=blue face="Courier New"><i>40M /mnt/gpfs1b/temp_dir/lf.s.1</i></font><br><br><font size=2 face="sans-serif"># Verified through mmgetlocation that
the file data blocks is uniformly striped across all the dataAndMetadata
NSDs, with each NSD containing 2 file data blocks</font><br><font size=2 face="sans-serif"># In the output below, "DMD_NSDX"
is name of the NSDs. </font><br><font size=2 color=blue face="Courier New"><i>(09:53:00) c25m3n07:~
# /usr/lpp/mmfs/samples/fpo/mmgetlocation -f /mnt/gpfs1b/temp_dir/lf.s.1</i></font><br><br><font size=2 color=blue face="Courier New"><i>[FILE INFO]</i></font><br><font size=2 color=blue face="Courier New"><i>------------------------------------------------------------------------</i></font><br><br><font size=2 color=blue face="Courier New"><i>blockSize
2 MB</i></font><br><font size=2 color=blue face="Courier New"><i>blockGroupFactor
1</i></font><br><font size=2 color=blue face="Courier New"><i>metadataBlockSize
2M</i></font><br><font size=2 color=blue face="Courier New"><i>writeAffinityDepth 0</i></font><br><font size=2 color=blue face="Courier New"><i>flags:</i></font><br><font size=2 color=blue face="Courier New"><i>data replication:
1 max 2</i></font><br><font size=2 color=blue face="Courier New"><i>storage pool name:
system</i></font><br><font size=2 color=blue face="Courier New"><i>metadata replication:
1 max 2</i></font><br><br><font size=2 color=blue face="Courier New"><i>Chunk 0 (offset 0) is
located at disks: [ DMD_NSD09 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 1 (offset 2097152)
is located at disks: [ DMD_NSD10 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 2 (offset 4194304)
is located at disks: [ DMD_NSD01 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 3 (offset 6291456)
is located at disks: [ DMD_NSD02 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 4 (offset 8388608)
is located at disks: [ DMD_NSD03 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 5 (offset 10485760)
is located at disks: [ DMD_NSD04 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 6 (offset 12582912)
is located at disks: [ DMD_NSD05 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 7 (offset 14680064)
is located at disks: [ DMD_NSD06 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 8 (offset 16777216)
is located at disks: [ DMD_NSD07 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 9 (offset 18874368)
is located at disks: [ DMD_NSD08 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 10 (offset 20971520)
is located at disks: [ DMD_NSD09 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 11 (offset 23068672)
is located at disks: [ DMD_NSD10 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 12 (offset 25165824)
is located at disks: [ DMD_NSD01 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 13 (offset 27262976)
is located at disks: [ DMD_NSD02 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 14 (offset 29360128)
is located at disks: [ DMD_NSD03 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 15 (offset 31457280)
is located at disks: [ DMD_NSD04 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 16 (offset 33554432)
is located at disks: [ DMD_NSD05 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 17 (offset 35651584)
is located at disks: [ DMD_NSD06 c25m3n08-ib,c25m3n07-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 18 (offset 37748736)
is located at disks: [ DMD_NSD07 c25m3n07-ib,c25m3n08-ib ]</i></font><br><font size=2 color=blue face="Courier New"><i>Chunk 19 (offset 39845888)
is located at disks: [ DMD_NSD08 c25m3n08-ib,c25m3n07-ib ]</i></font><br><br><font size=2 color=blue face="Courier New"><i>[SUMMARY INFO]</i></font><br><font size=2 color=blue face="Courier New"><i>----------------------------------------------------------------------------------------------------------</i></font><br><font size=2 color=blue face="Courier New"><i>Replica num Nodename
TotalChunkst</i></font><br><br><font size=2 color=blue face="Courier New"><i>Replica 1 :
c25m3n07-ib,c25m3n08-ib: Total : 10</i></font><br><font size=2 color=blue face="Courier New"><i>Replica 1 :
c25m3n08-ib,c25m3n07-ib: Total : 10</i></font><br><br><font size=2 face="sans-serif">Best Regards,</font><br><font size=2 face="sans-serif">-Kums</font><br><br><br><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif"><Greg.Lehmann@csiro.au></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif"><gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">03/29/2017 08:00 PM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
question on viewing block distribution across NSDs</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><font size=2 color=#004080 face="Calibri">I was going to keep mmdf
in mind, not gpfs.snap. I will now also keep in mind that mmdf can have
an impact as at present we have spinning disk for metadata. The system
I am playing around on is not production yet, so I am safe for the moment.</font><br><font size=2 color=#004080 face="Calibri"> </font><br><font size=2 color=#004080 face="Calibri">Thanks again.</font><br><font size=2 color=#004080 face="Calibri"> </font><br><font size=2 face="Calibri"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org
[</font><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><font size=2 face="Calibri">mailto:gpfsug-discuss-bounces@spectrumscale.org</font></a><font size=2 face="Calibri">]
<b>On Behalf Of </b>Knister, Aaron S. (GSFC-606.2)[COMPUTER SCIENCE CORP]<b><br>Sent:</b> Thursday, 30 March 2017 9:55 AM<b><br>To:</b> gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><b><br>Subject:</b> Re: [gpfsug-discuss] question on viewing block distribution
across NSDs</font><br><font size=3 face="Times New Roman"> </font><br><font size=3 color=#004080 face="Times New Roman">I don't necessarily
think you need to run a snap prior, just the output of mmdf should be enough.
Something to keep in mind that I should have said before-- an mmdf can
be stressful on your system particularly if you have spinning disk for
your metadata. We're fortunate enough to have all flash for our metadata
and I tend to take it for granted some times :)</font><br><font size=3 face="Times New Roman"> </font><br><font size=2 face="sans-serif"><b>From:</b> </font><a href=mailto:greg.lehmann@csiro.au><font size=2 color=blue face="sans-serif"><u>greg.lehmann@csiro.au</u></font></a><font size=2 face="sans-serif"><b><br>Sent:</b> 3/29/17, 19:52<b><br>To:</b> gpfsug main discussion list<b><br>Subject:</b> Re: [gpfsug-discuss] question on viewing block distribution
across NSDs</font><br><font size=2 color=#004080 face="Calibri">Thanks. I don’t have a snap.
I’ll keep that in mind for next time I do this.</font><br><font size=2 color=#004080 face="Calibri"> </font><br><font size=2 face="Calibri"><b>From:</b> </font><a href="mailto:gpfsug-discuss-bounces@spectrumscale.org"><font size=2 color=blue face="Calibri"><u>mailto:gpfsug-discuss-bounces@spectrumscale.org</u></font></a><font size=2 face="Calibri">]
<b>On Behalf Of </b>Knister, Aaron S. (GSFC-606.2)[COMPUTER SCIENCE CORP]<b><br>Sent:</b> Thursday, 30 March 2017 9:45 AM<b><br>To:</b> gpfsug main discussion list <</font><a href="mailto:gpfsug-discuss@spectrumscale.org"><font size=2 color=blue face="Calibri"><u>gpfsug-discuss@spectrumscale.org</u></font></a><font size=2 face="Calibri">><b><br>Subject:</b> Re: [gpfsug-discuss] question on viewing block distribution
across NSDs</font><br><font size=3 face="Times New Roman"> </font><br><font size=3 color=#004080 face="Times New Roman">Hi Greg,</font><br><font size=3 color=#004080 face="Times New Roman"> </font><br><font size=3 color=#004080 face="Times New Roman">You could run an
mmdf which will show you how full each NSD is. I'm not sure how to look
back in time though to see the fs before the restripe. Do you perhaps have
a gpfs.snap you took somewhat recently before the restripe? Maybe an internaldump
in /tmp/mmfs somewhere?</font><br><font size=3 face="Times New Roman"> </font><br><font size=2 face="sans-serif"><b>From:</b> </font><a href=mailto:greg.lehmann@csiro.au><font size=2 color=blue face="sans-serif"><u>greg.lehmann@csiro.au</u></font></a><font size=2 face="sans-serif"><b><br>Sent:</b> 3/29/17, 19:21<b><br>To:</b> gpfsug main discussion list<b><br>Subject:</b> [gpfsug-discuss] question on viewing block distribution across
NSDs</font><br><font size=3 face="Times New Roman">Hi All,</font><br><font size=3 face="Times New Roman">
I added some NSDs to an existing filesystem and ran
mmrestripefs. I was sort of curious to see what the distribution looked
like before and after the restripe. Is there any way of looking at it?</font><br><font size=3 face="Times New Roman"> </font><br><font size=3 face="Times New Roman">Cheers,</font><br><font size=3 face="Times New Roman"> </font><br><font size=3 face="Times New Roman">Greg.</font><tt><font size=2>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br><BR>