<font size=2 face="sans-serif">Hi Atmane, </font><br><br><font size=2 face="sans-serif">There can be two meaning of available
free space?</font><br><font size=2 face="sans-serif">One what is available on existing filesystem.
For this you rightly referred to df -h command o/p. This is the actual
free space available in already created filesystem.</font><br><br><tt><font size=2>Filesystem
Size Used Avail Use% Mounted on<br><br>/dev/gpfs1 49T 18T
31T 38% /gpfs1<br>/dev/gpfs2 53T 13T
40T 25% /gpfs2<br>/dev/gpfs3 25T 4.9T
20T 21% /gpfs3<br>/dev/gpfs4 11T 133M
11T 1% /gpfs4<br>/dev/gpfs5 323T 34T 290T
11% /gpfs5<br></font></tt><br><font size=2 face="sans-serif">The other is free space available in
DA. For which as every one said use mmlsrecoverygroup <recovery group
name> -L</font><br><font size=2 face="sans-serif">Please note that is will give you raw
free capacity. For usable free capacity in DA you have to add RAID over
head.</font><br><font size=2 face="sans-serif">But based on your o/p you have very
little/no free space left in DA. </font><br><br><tt><font size=2>[root@server1 ~]#mmlsrecoverygroup<br><br> declustered<br> arrays
with<br>recovery group vdisks vdisks servers<br>------------------ ----------- ------ -------<br>BB1RGL
3 18 server1,server2<br>BB1RGR
3 18 server2,server1<br>--------------------------------------------------------------<br>[root@server ~]# mmlsrecoverygroup BB1RGL -L<br><br> declustered<br>recovery group arrays vdisks pdisks
format version<br>----------------- ----------- ------ ------ --------------<br>BB1RGL
3 18 119 4.2.0.1<br><br>declustered needs
replace
scrub background
activity<br> array service vdisks pdisks spares
threshold free space duration task progress
priority<br>----------- ------- ------ ------ ------ ---------
---------- -------- -------------------------<br>LOG no
1 3 0,0
1 558 GiB 14 days scrub
51% low<br>DA1 no
11 58 2,31
2 12 GiB 14 days scrub
78% low<br>DA2 no
6 58 2,31
2 4096 MiB 14 days scrub
10% low<br></font></tt><br><br><font size=2 face="sans-serif">Thanks,<br><br>Sandeep Naik<br>Elastic Storage server / GPFS Test <br>ETZ-B, Hinjewadi Pune India<br>(+91) 8600994314</font><br><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">"Kumaran Rajaram"
<kums@us.ibm.com></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org>, atmane
khiredine <a.khiredine@meteo.dz></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">09/07/2017 10:22 PM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
get free space in GSS</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><font size=2 face="Arial">Hi Atmane,</font><font size=3><br></font><font size=2 color=red face="Arial"><br>>> I can not find the free space</font><font size=3><br></font><font size=2 face="Arial"><br>Based on your output below, your setup currently has two recovery groups
BB1RGL and BB1RGR.</font><font size=3><br></font><font size=2 face="Arial"><br>Issue "</font><font size=2 color=blue face="Arial"><i>mmlsrecoverygroup
BB1RGL -L</i></font><font size=2 face="Arial">" and "</font><font size=2 color=blue face="Arial"><i>mmlsrecoverygroup
BB1RGR -L</i></font><font size=2 face="Arial">" to obtain free space
in each DA.</font><font size=3><br></font><font size=2 face="Arial"><br>Based on your "</font><font size=2 color=blue face="Arial"><i>mmlsrecoverygroup
BB1RGL -L</i></font><font size=2 face="Arial">" output below, BB1RGL
"DA1" has 12GiB and "DA2" has 4GiB free space. The
metadataOnly and dataOnly vdisk/NSD are created from DA1 and DA2. </font><font size=3><br></font><font size=2 color=blue face="Arial"><i><br> declustered needs
replace
scrub background
activity<br> array service vdisks pdisks spares
threshold free space duration task progress
priority<br>----------- ------- ------ ------ ------ ---------
---------- -------- -------------------------<br>LOG no
1 3 0,0
1 558 GiB 14 days scrub
51% low<br>DA1 no
11 58 2,31
2 12 GiB 14 days scrub
78% low<br>DA2 no
6 58 2,31
2 4096 MiB 14 days scrub
10% low</i></font><font size=3><br></font><font size=2 face="Arial"><br>In addition, you may use "</font><font size=2 color=blue face="Arial"><i>mmlsnsd</i></font><font size=2 face="Arial">"
to obtain mapping of file-system to vdisk/NSD + use "</font><font size=2 color=blue face="Arial"><i>mmdf
<fs></i></font><font size=2 face="Arial">" command to query
user or available capacity on a GPFS file system.</font><font size=3><br></font><font size=2 face="Arial"><br>Hope this helps,<br>-Kums</font><font size=3><br><br><br><br><br></font><font size=1 color=#5f5f5f face="sans-serif"><br>From: </font><font size=1 face="sans-serif">atmane
khiredine <a.khiredine@meteo.dz></font><font size=1 color=#5f5f5f face="sans-serif"><br>To: </font><font size=1 face="sans-serif">Laurence
Horrocks-Barlow <laurence@qsplace.co.uk>, "gpfsug main
discussion list" <gpfsug-discuss@spectrumscale.org></font><font size=1 color=#5f5f5f face="sans-serif"><br>Date: </font><font size=1 face="sans-serif">07/09/2017
08:27 AM</font><font size=1 color=#5f5f5f face="sans-serif"><br>Subject: </font><font size=1 face="sans-serif">Re:
[gpfsug-discuss] get free space in GSS</font><font size=1 color=#5f5f5f face="sans-serif"><br>Sent by: </font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><font size=3><br></font><hr noshade><font size=3><br><br></font><tt><font size=2><br>thank you very much for replying. I can not find the free space<br><br>Here is the output of mmlsrecoverygroup<br><br>[root@server1 ~]#mmlsrecoverygroup<br><br> declustered<br> arrays
with<br>recovery group vdisks vdisks servers<br>------------------ ----------- ------ -------<br>BB1RGL
3 18 server1,server2<br>BB1RGR
3 18 server2,server1<br>--------------------------------------------------------------<br>[root@server ~]# mmlsrecoverygroup BB1RGL -L<br><br> declustered<br>recovery group arrays vdisks pdisks
format version<br>----------------- ----------- ------ ------ --------------<br>BB1RGL
3 18 119 4.2.0.1<br><br>declustered needs
replace
scrub background
activity<br> array service vdisks pdisks spares
threshold free space duration task progress
priority<br>----------- ------- ------ ------ ------ ---------
---------- -------- -------------------------<br>LOG no
1 3 0,0
1 558 GiB 14 days scrub
51% low<br>DA1 no
11 58 2,31
2 12 GiB 14 days scrub
78% low<br>DA2 no
6 58 2,31
2 4096 MiB 14 days scrub
10% low<br><br>
declustered
checksum<br>vdisk RAID code
array vdisk size
block size granularity state remarks<br>------------------ ------------------ ----------- ----------
---------- ----------- ----- -------<br>gss0_logtip 3WayReplication LOG
128 MiB 1
MiB 512 ok logTip<br>gss0_loghome 4WayReplication DA1
40 GiB 1
MiB 512 ok log<br>BB1RGL_GPFS4_META1 4WayReplication DA1
451 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS4_DATA1 8+2p
DA1 5133 GiB
1 MiB 32 KiB ok<br>BB1RGL_GPFS1_META1 4WayReplication DA1
451 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS1_DATA1 8+2p
DA1 12
TiB 1 MiB 32 KiB ok<br>BB1RGL_GPFS3_META1 4WayReplication DA1
451 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS3_DATA1 8+2p
DA1 12 TiB
1 MiB 32 KiB ok<br>BB1RGL_GPFS2_META1 4WayReplication DA1
451 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS2_DATA1 8+2p
DA1 13 TiB
2 MiB 32 KiB ok<br>BB1RGL_GPFS2_META2 4WayReplication DA2
451 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS2_DATA2 8+2p
DA2 13 TiB
2 MiB 32 KiB ok<br>BB1RGL_GPFS1_META2 4WayReplication DA2
451 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS1_DATA2 8+2p
DA2 12
TiB 1 MiB 32 KiB ok<br>BB1RGL_GPFS5_META1 4WayReplication DA1
750 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS5_DATA1 8+2p
DA1 70 TiB
16 MiB 32 KiB ok<br>BB1RGL_GPFS5_META2 4WayReplication DA2
750 GiB 1 MiB
32 KiB ok<br>BB1RGL_GPFS5_DATA2 8+2p
DA2 90 TiB
16 MiB 32 KiB ok<br><br>config data declustered array VCD spares
actual rebuild spare space remarks<br>------------------ ------------------ ------------- ---------------------------------
----------------<br>rebuild space DA1
31 34 pdisk<br>rebuild space DA2
31 35 pdisk<br><br><br>config data max disk group fault tolerance
actual disk group fault tolerance remarks<br>------------------ --------------------------------- ---------------------------------
----------------<br>rg descriptor 1 enclosure + 1 drawer
1 enclosure + 1 drawer
limiting fault tolerance<br>system index 2 enclosure
1 enclosure
+ 1 drawer limited by rg descriptor<br><br>vdisk max disk group fault
tolerance actual disk group fault tolerance remarks<br>------------------ --------------------------------- ---------------------------------
----------------<br>gss0_logtip 2 enclosure
1 enclosure
+ 1 drawer limited by rg descriptor<br>gss0_loghome 1 enclosure + 1 drawer
1 enclosure + 1 drawer<br>BB1RGL_GPFS4_META1 1 enclosure + 1 drawer
1 enclosure + 1 drawer<br>BB1RGL_GPFS4_DATA1 2 drawer
2 drawer<br>BB1RGL_GPFS1_META1 1 enclosure + 1 drawer
1 enclosure + 1 drawer<br>BB1RGL_GPFS1_DATA1 2 drawer
2 drawer<br>BB1RGL_GPFS3_META1 1 enclosure + 1 drawer
1 enclosure + 1 drawer<br>BB1RGL_GPFS3_DATA1 2 drawer
2 drawer<br>BB1RGL_GPFS2_META1 1 enclosure + 1 drawer
1 enclosure + 1 drawer<br>BB1RGL_GPFS2_DATA1 2 drawer
2 drawer<br>BB1RGL_GPFS2_META2 3 enclosure
1 enclosure + 1 drawer
limited by rg descriptor<br>BB1RGL_GPFS2_DATA2 2 drawer
2 drawer<br>BB1RGL_GPFS1_META2 3 enclosure
1 enclosure + 1
drawer limited by rg descriptor<br>BB1RGL_GPFS1_DATA2 2 drawer
2 drawer<br>BB1RGL_GPFS5_META1 1 enclosure + 1 drawer
1 enclosure + 1 drawer<br>BB1RGL_GPFS5_DATA1 2 drawer
2 drawer<br>BB1RGL_GPFS5_META2 3 enclosure
1 enclosure + 1 drawer
limited by rg descriptor<br>BB1RGL_GPFS5_DATA2 2 drawer
2 drawer<br><br>active recovery group server
servers<br>----------------------------------------------- -------<br>server1
server1,server2<br><br><br>Atmane Khiredine<br>HPC System Administrator | Office National de la Météorologie<br>Tél : +213 21 50 73 93 # 303 | Fax : +213 21 50 79 40 | E-mail : a.khiredine@meteo.dz<br>________________________________<br>De : Laurence Horrocks-Barlow [laurence@qsplace.co.uk]<br>Envoyé : dimanche 9 juillet 2017 09:58<br>À : gpfsug main discussion list; atmane khiredine; gpfsug-discuss@spectrumscale.org<br>Objet : Re: [gpfsug-discuss] get free space in GSS<br><br>You can check the recovery groups to see if there is any remaining space.<br><br>I don't have access to my test system to confirm the syntax however if
memory serves.<br><br>Run mmlsrecoverygroup to get a list of all the recovery groups then:<br><br>mmlsrecoverygroup <YOURRECOVERYGROUP> -L<br><br>This will list all your declustered arrays and their free space.<br><br>Their might be another method, however this way has always worked well
for me.<br><br>-- Lauz<br><br><br><br>On 9 July 2017 09:00:07 BST, Atmane <a.khiredine@meteo.dz> wrote:<br><br>Dear all,<br><br>My name is Khiredine Atmane and I am a HPC system administrator at the<br>National Office of Meteorology Algeria . We have a GSS24 running<br>gss2.5.10.3<</font></tt><a href=http://2.5.10.3/><tt><font size=2 color=blue><u>http://2.5.10.3</u></font></tt></a><tt><font size=2>>-3b
and gpfs-4.2.0.3<</font></tt><a href=http://4.2.0.3/><tt><font size=2 color=blue><u>http://4.2.0.3</u></font></tt></a><tt><font size=2>>.<br><br>GSS configuration: 4 enclosures, 6 SSDs, 1 empty slots, 239 disks total,
0<br>NVRAM partitions<br><br>disks = 3Tb<br>SSD = 200 Gb<br>df -h<br>Filesystem Size Used Avail
Use% Mounted on<br><br>/dev/gpfs1 49T 18T
31T 38% /gpfs1<br>/dev/gpfs2 53T 13T
40T 25% /gpfs2<br>/dev/gpfs3 25T 4.9T
20T 21% /gpfs3<br>/dev/gpfs4 11T 133M
11T 1% /gpfs4<br>/dev/gpfs5 323T 34T 290T
11% /gpfs5<br><br>Total Is 461 To<br><br>I think we have more space<br>Could anyone make recommendation to troubleshoot find how many free space<br>in GSS ?<br>How to find the available space ?<br>Thank you!<br><br>Atmane<br><br><br><br>--<br>Sent from my Android device with K-9 Mail. Please excuse my brevity.<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org</font></tt><font size=3 color=blue><u><br></u></font><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2 color=blue><u>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</u></font></tt></a><tt><font size=2><br></font></tt><font size=3><br><br></font><tt><font size=2>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br></font></tt><br><br><BR>